[
  {
    "path": ".bazelrc",
    "content": "# Enable Bzlmod by default.\ncommon --enable_bzlmod\n\n# Use C++17.\nbuild --cxxopt=-std=c++17\nbuild --host_cxxopt=-std=c++17\n\n# Make Python protos faster by backing them with C++ protos.\n# TODO: Reenable once protobuf releases\n# https://github.com/protocolbuffers/protobuf/pull/22633\n# i.e. in version > 32.0. Or possibly switch to upb.\n# build --define=use_fast_cpp_protos=true\n\n# Options from ./configure\n# This is currently disabled because TensorFlow does not support bzlmod,\n# hence Riegeli/TensorFlow bindings are broken anyway.\n# import %workspace%/configure.bazelrc\n"
  },
  {
    "path": "CONTRIBUTING.md",
    "content": "# How to Contribute\n\nWe'd love to accept your patches and contributions to this project. There are\njust a few small guidelines you need to follow.\n\n## Contributor License Agreement\n\nContributions to this project must be accompanied by a Contributor License\nAgreement. You (or your employer) retain the copyright to your contribution,\nthis simply gives us permission to use and redistribute your contributions as\npart of the project. Head over to <https://cla.developers.google.com/> to see\nyour current agreements on file or to sign a new one.\n\nYou generally only need to submit a CLA once, so if you've already submitted one\n(even if it was for a different project), you probably don't need to do it\nagain.\n\n## Code reviews\n\nAll submissions, including submissions by project members, require review. We\nuse GitHub pull requests for this purpose. Consult\n[GitHub Help](https://help.github.com/articles/about-pull-requests/) for more\ninformation on using pull requests.\n"
  },
  {
    "path": "LICENSE",
    "content": "                                 Apache License\n                           Version 2.0, January 2004\n                        http://www.apache.org/licenses/\n\n   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n   1. Definitions.\n\n      \"License\" shall mean the terms and conditions for use, reproduction,\n      and distribution as defined by Sections 1 through 9 of this document.\n\n      \"Licensor\" shall mean the copyright owner or entity authorized by\n      the copyright owner that is granting the License.\n\n      \"Legal Entity\" shall mean the union of the acting entity and all\n      other entities that control, are controlled by, or are under common\n      control with that entity. For the purposes of this definition,\n      \"control\" means (i) the power, direct or indirect, to cause the\n      direction or management of such entity, whether by contract or\n      otherwise, or (ii) ownership of fifty percent (50%) or more of the\n      outstanding shares, or (iii) beneficial ownership of such entity.\n\n      \"You\" (or \"Your\") shall mean an individual or Legal Entity\n      exercising permissions granted by this License.\n\n      \"Source\" form shall mean the preferred form for making modifications,\n      including but not limited to software source code, documentation\n      source, and configuration files.\n\n      \"Object\" form shall mean any form resulting from mechanical\n      transformation or translation of a Source form, including but\n      not limited to compiled object code, generated documentation,\n      and conversions to other media types.\n\n      \"Work\" shall mean the work of authorship, whether in Source or\n      Object form, made available under the License, as indicated by a\n      copyright notice that is included in or attached to the work\n      (an example is provided in the Appendix below).\n\n      \"Derivative Works\" shall mean any work, whether in Source or Object\n      form, that is based on (or derived from) the Work and for which the\n      editorial revisions, annotations, elaborations, or other modifications\n      represent, as a whole, an original work of authorship. For the purposes\n      of this License, Derivative Works shall not include works that remain\n      separable from, or merely link (or bind by name) to the interfaces of,\n      the Work and Derivative Works thereof.\n\n      \"Contribution\" shall mean any work of authorship, including\n      the original version of the Work and any modifications or additions\n      to that Work or Derivative Works thereof, that is intentionally\n      submitted to Licensor for inclusion in the Work by the copyright owner\n      or by an individual or Legal Entity authorized to submit on behalf of\n      the copyright owner. For the purposes of this definition, \"submitted\"\n      means any form of electronic, verbal, or written communication sent\n      to the Licensor or its representatives, including but not limited to\n      communication on electronic mailing lists, source code control systems,\n      and issue tracking systems that are managed by, or on behalf of, the\n      Licensor for the purpose of discussing and improving the Work, but\n      excluding communication that is conspicuously marked or otherwise\n      designated in writing by the copyright owner as \"Not a Contribution.\"\n\n      \"Contributor\" shall mean Licensor and any individual or Legal Entity\n      on behalf of whom a Contribution has been received by Licensor and\n      subsequently incorporated within the Work.\n\n   2. Grant of Copyright License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      copyright license to reproduce, prepare Derivative Works of,\n      publicly display, publicly perform, sublicense, and distribute the\n      Work and such Derivative Works in Source or Object form.\n\n   3. Grant of Patent License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      (except as stated in this section) patent license to make, have made,\n      use, offer to sell, sell, import, and otherwise transfer the Work,\n      where such license applies only to those patent claims licensable\n      by such Contributor that are necessarily infringed by their\n      Contribution(s) alone or by combination of their Contribution(s)\n      with the Work to which such Contribution(s) was submitted. If You\n      institute patent litigation against any entity (including a\n      cross-claim or counterclaim in a lawsuit) alleging that the Work\n      or a Contribution incorporated within the Work constitutes direct\n      or contributory patent infringement, then any patent licenses\n      granted to You under this License for that Work shall terminate\n      as of the date such litigation is filed.\n\n   4. Redistribution. You may reproduce and distribute copies of the\n      Work or Derivative Works thereof in any medium, with or without\n      modifications, and in Source or Object form, provided that You\n      meet the following conditions:\n\n      (a) You must give any other recipients of the Work or\n          Derivative Works a copy of this License; and\n\n      (b) You must cause any modified files to carry prominent notices\n          stating that You changed the files; and\n\n      (c) You must retain, in the Source form of any Derivative Works\n          that You distribute, all copyright, patent, trademark, and\n          attribution notices from the Source form of the Work,\n          excluding those notices that do not pertain to any part of\n          the Derivative Works; and\n\n      (d) If the Work includes a \"NOTICE\" text file as part of its\n          distribution, then any Derivative Works that You distribute must\n          include a readable copy of the attribution notices contained\n          within such NOTICE file, excluding those notices that do not\n          pertain to any part of the Derivative Works, in at least one\n          of the following places: within a NOTICE text file distributed\n          as part of the Derivative Works; within the Source form or\n          documentation, if provided along with the Derivative Works; or,\n          within a display generated by the Derivative Works, if and\n          wherever such third-party notices normally appear. The contents\n          of the NOTICE file are for informational purposes only and\n          do not modify the License. You may add Your own attribution\n          notices within Derivative Works that You distribute, alongside\n          or as an addendum to the NOTICE text from the Work, provided\n          that such additional attribution notices cannot be construed\n          as modifying the License.\n\n      You may add Your own copyright statement to Your modifications and\n      may provide additional or different license terms and conditions\n      for use, reproduction, or distribution of Your modifications, or\n      for any such Derivative Works as a whole, provided Your use,\n      reproduction, and distribution of the Work otherwise complies with\n      the conditions stated in this License.\n\n   5. Submission of Contributions. Unless You explicitly state otherwise,\n      any Contribution intentionally submitted for inclusion in the Work\n      by You to the Licensor shall be under the terms and conditions of\n      this License, without any additional terms or conditions.\n      Notwithstanding the above, nothing herein shall supersede or modify\n      the terms of any separate license agreement you may have executed\n      with Licensor regarding such Contributions.\n\n   6. Trademarks. This License does not grant permission to use the trade\n      names, trademarks, service marks, or product names of the Licensor,\n      except as required for reasonable and customary use in describing the\n      origin of the Work and reproducing the content of the NOTICE file.\n\n   7. Disclaimer of Warranty. Unless required by applicable law or\n      agreed to in writing, Licensor provides the Work (and each\n      Contributor provides its Contributions) on an \"AS IS\" BASIS,\n      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n      implied, including, without limitation, any warranties or conditions\n      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n      PARTICULAR PURPOSE. You are solely responsible for determining the\n      appropriateness of using or redistributing the Work and assume any\n      risks associated with Your exercise of permissions under this License.\n\n   8. Limitation of Liability. In no event and under no legal theory,\n      whether in tort (including negligence), contract, or otherwise,\n      unless required by applicable law (such as deliberate and grossly\n      negligent acts) or agreed to in writing, shall any Contributor be\n      liable to You for damages, including any direct, indirect, special,\n      incidental, or consequential damages of any character arising as a\n      result of this License or out of the use or inability to use the\n      Work (including but not limited to damages for loss of goodwill,\n      work stoppage, computer failure or malfunction, or any and all\n      other commercial damages or losses), even if such Contributor\n      has been advised of the possibility of such damages.\n\n   9. Accepting Warranty or Additional Liability. While redistributing\n      the Work or Derivative Works thereof, You may choose to offer,\n      and charge a fee for, acceptance of support, warranty, indemnity,\n      or other liability obligations and/or rights consistent with this\n      License. However, in accepting such obligations, You may act only\n      on Your own behalf and on Your sole responsibility, not on behalf\n      of any other Contributor, and only if You agree to indemnify,\n      defend, and hold each Contributor harmless for any liability\n      incurred by, or claims asserted against, such Contributor by reason\n      of your accepting any such warranty or additional liability.\n\n   END OF TERMS AND CONDITIONS\n\n   APPENDIX: How to apply the Apache License to your work.\n\n      To apply the Apache License to your work, attach the following\n      boilerplate notice, with the fields enclosed by brackets \"[]\"\n      replaced with your own identifying information. (Don't include\n      the brackets!)  The text should be enclosed in the appropriate\n      comment syntax for the file format. We also recommend that a\n      file or class name and description of purpose be included on the\n      same \"printed page\" as the copyright notice for easier\n      identification within third-party archives.\n\n   Copyright [yyyy] [name of copyright owner]\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\n"
  },
  {
    "path": "MANIFEST.in",
    "content": "include .bazelrc\ninclude *.md\ninclude LICENSE\ninclude MANIFEST.in\ninclude WORKSPACE\ninclude configure\nrecursive-include doc *\nrecursive-include python *\nrecursive-include riegeli *\nrecursive-include third_party *\n"
  },
  {
    "path": "MODULE.bazel",
    "content": "module(\n    name = \"riegeli\",\n    repo_name = \"com_google_riegeli\",\n)\n\nbazel_dep(\n    name = \"abseil-cpp\",\n    version = \"20260107.0\",\n    repo_name = \"com_google_absl\",\n)\nbazel_dep(\n    name = \"abseil-py\",\n    version = \"2.1.0\",\n    repo_name = \"absl_py\",\n)\nbazel_dep(\n    name = \"bazel_skylib\",\n    version = \"1.7.1\",\n)\nbazel_dep(\n    name = \"boringssl\",\n    version = \"0.0.0-20240530-2db0eb3\",\n)\nbazel_dep(\n    name = \"brotli\",\n    version = \"1.1.0\",\n    repo_name = \"org_brotli\",\n)\nbazel_dep(\n    name = \"bzip2\",\n    version = \"1.0.8\",\n)\nbazel_dep(\n    name = \"highwayhash\",\n    version = \"0.0.0-20240305-5ad3bf8.bcr.1\",\n)\nbazel_dep(\n    name = \"lz4\",\n    version = \"1.9.4\",\n)\nbazel_dep(\n    name = \"platforms\",\n    version = \"0.0.9\",\n)\nbazel_dep(\n    name = \"protobuf\",\n    version = \"33.2\",\n    repo_name = \"com_google_protobuf\",\n)\nbazel_dep(\n    name = \"rules_cc\",\n    version = \"0.1.2\",\n)\nbazel_dep(\n    name = \"rules_python\",\n    version = \"0.36.0\",\n)\nbazel_dep(\n    name = \"snappy\",\n    version = \"1.2.0\",\n)\nbazel_dep(\n    name = \"xz\",\n    version = \"5.4.5.bcr.1\",\n)\nbazel_dep(\n    name = \"zlib\",\n    version = \"1.3.1.bcr.3\",\n)\nbazel_dep(\n    name = \"zstd\",\n    version = \"1.5.6\",\n    repo_name = \"net_zstd\",\n)\nbazel_dep(\n    name = \"google_cloud_cpp\",\n    version = \"3.0.0-rc1\",\n)\n\n# Configure hermetic Python toolchain\nSUPPORTED_PYTHON_VERSIONS = [\n    \"3.8\",\n    \"3.9\",\n    \"3.10\",\n    \"3.11\",\n    \"3.12\",\n]\n\nDEFAULT_PYTHON_VERSION = SUPPORTED_PYTHON_VERSIONS[-1]\n\npython = use_extension(\"@rules_python//python/extensions:python.bzl\", \"python\")\n\n[\n    python.toolchain(\n        is_default = version == DEFAULT_PYTHON_VERSION,\n        python_version = version,\n    )\n    for version in SUPPORTED_PYTHON_VERSIONS\n]\n"
  },
  {
    "path": "README.md",
    "content": "# Riegeli\n\n*Riegeli/records* is a file format for storing a sequence of string records,\ntypically serialized protocol buffers. It supports dense compression, fast\ndecoding, seeking, detection and optional skipping of data corruption, filtering\nof proto message fields for even faster decoding, and parallel encoding.\n\nSee [documentation](https://github.com/google/riegeli/blob/master/doc/index.md).\n\n# Status\n\nRiegeli file format will only change in a backward compatible way (i.e. future\nreaders will understand current files, but current readers might not understand\nfiles using future features).\n\nRiegeli C++ API might change in incompatible ways.\n"
  },
  {
    "path": "configure",
    "content": "#!/bin/bash\n# Copyright 2019 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -e\n\nPYTHON_BIN_PATH=`which python`\n\nif [[ $PYTHON_BIN_PATH ]] && $PYTHON_BIN_PATH -c \"import tensorflow\" &>/dev/null; then\n  TF_CFLAGS=$($PYTHON_BIN_PATH -c 'import tensorflow as tf; print(\" \".join(tf.sysconfig.get_compile_flags()))')\n  TF_LFLAGS=$($PYTHON_BIN_PATH -c 'import tensorflow as tf; print(\" \".join(tf.sysconfig.get_link_flags()))')\n\n  TF_HEADER_DIR=${TF_CFLAGS%% *}\n  TF_HEADER_DIR=${TF_HEADER_DIR#-I}\n  TF_SHARED_LIBRARY_DIR=${TF_LFLAGS%% *}\n  TF_SHARED_LIBRARY_DIR=${TF_SHARED_LIBRARY_DIR#-L}\n  TF_SHARED_LIBRARY_NAME=${TF_LFLAGS##* -l:}\nelse\n  TF_HEADER_DIR=\n  TF_SHARED_LIBRARY_DIR=\n  TF_SHARED_LIBRARY_NAME=\nfi\n\n{\n  printf 'build --action_env PYTHON_BIN_PATH=\"%s\"\\n' \"$PYTHON_BIN_PATH\"\n  printf 'build --action_env TF_HEADER_DIR=\"%s\"\\n' \"$TF_HEADER_DIR\"\n  printf 'build --action_env TF_SHARED_LIBRARY_DIR=\"%s\"\\n' \"$TF_SHARED_LIBRARY_DIR\"\n  printf 'build --action_env TF_SHARED_LIBRARY_NAME=\"%s\"\\n' \"$TF_SHARED_LIBRARY_NAME\"\n} >configure.bazelrc\n\necho \"Set up configure.bazelrc. Make sure to include it in your .bazelrc file.\"\n"
  },
  {
    "path": "doc/index.md",
    "content": "# Riegeli\n\n*Riegeli/records* is a file format for storing a sequence of string records,\ntypically serialized protocol buffers. It supports dense compression, fast\ndecoding, seeking, detection and optional skipping of data corruption, filtering\nof proto message fields for even faster decoding, and parallel encoding.\n\n*   [Specification of Riegeli/records file format](riegeli_records_file_format.md).\n*   [Specifying options for writing Riegeli/records files](record_writer_options.md).\n"
  },
  {
    "path": "doc/record_writer_options.md",
    "content": "# Specifying options for writing Riegeli/records files\n\nOptions for writing Riegeli/records files can be specified as a string:\n\n```data\n  options ::= option? (\",\" option?)*\n  option ::=\n    \"default\" |\n    \"transpose\" (\":\" (\"true\" | \"false\"))? |\n    \"uncompressed\" |\n    \"brotli\" (\":\" brotli_level)? |\n    \"zstd\" (\":\" zstd_level)? |\n    \"snappy\" (\":\" snappy_level)? |\n    \"window_log\" \":\" window_log |\n    \"brotli_encoder\" \":\" (\"rbrotli_or_cbrotli\" | \"cbrotli\" | \"rbrotli\") |\n    \"chunk_size\" \":\" chunk_size |\n    \"bucket_fraction\" \":\" bucket_fraction |\n    \"padding\" (\":\" padding)? |\n    \"initial_padding\" (\":\" padding)? |\n    \"final_padding\" (\":\" padding)? |\n    \"parallelism\" \":\" parallelism\n  brotli_level ::= integer in the range [0..11] (default 6)\n  zstd_level ::= integer in the range [-131072..22] (default 3)\n  snappy_level ::= integer in the range [1..2] (default 1)\n  window_log ::= \"auto\" or integer in the range [10..31]\n  chunk_size ::= \"auto\" or positive integer expressed as real with optional\n    suffix [BkKMGTPE]\n  bucket_fraction ::= real in the range [0..1]\n  padding ::= positive integer expressed as real with optional suffix [BkKMGTPE]\n    (default 64K)\n  parallelism ::= non-negative integer\n```\n\nAn empty string is the same as `default`.\n\n## `transpose`\n\nIf `true` (`transpose` is the same as `transpose:true`), records should be\nserialized proto messages (but nothing will break if they are not). A chunk of\nrecords will be processed in a way which allows for better compression.\n\nIf `false`, a chunk of records will be stored in a simpler format, directly or\nwith compression.\n\nDefault: `false`.\n\n## Compression algorithms\n\n### `uncompressed`\n\nChanges compression algorithm to Uncompressed (turns compression off).\n\n### `brotli`\n\nChanges compression algorithm to [Brotli](https://github.com/google/brotli).\nSets compression level which tunes the tradeoff between compression density and\ncompression speed (higher = better density but slower).\n\n`brotli_level` must be between 0 and 11. Default: `6`.\n\nThis is the default compression algorithm.\n\n### `zstd`\n\nChanges compression algorithm to [Zstd](https://facebook.github.io/zstd/). Sets\ncompression level which tunes the tradeoff between compression density and\ncompression speed (higher = better density but slower).\n\n`zstd_level` must be between -131072 and 22. Level 0 is currently equivalent to\n3. Default: 3.\n\n### `snappy`\n\nChanges compression algorithm to [Snappy](https://google.github.io/snappy/).\n\n`snappy_level` must be between 1 and 2. Default: 1.\n\n## `window_log`\n\nLogarithm of the LZ77 sliding window size. This tunes the tradeoff between\ncompression density and memory usage (higher = better density but more memory).\n\nSpecial value `auto` means to keep the default (`brotli`: 22, `zstd`: derived\nfrom compression level and chunk size).\n\nFor `uncompressed` and `snappy`, `window_log` must be `auto`. For `brotli`,\n`window_log` must be `auto` or between 10 and 30. For `zstd`, `window_log` must\nbe `auto` or between 10 and 30 in 32-bit build, 31 in 64-bit build.\n\nDefault: `auto`.\n\n## `chunk_size`\n\nSets the desired uncompressed size of a chunk which groups messages to be\ntransposed, compressed, and written together.\n\nA larger chunk size improves compression density; a smaller chunk size allows to\nread pieces of the file independently with finer granularity, and reduces memory\nusage of both writer and reader.\n\nSpecial value `auto` means to keep the default (compressed: 1M, uncompressed:\n4k).\n\nDefault: `auto`.\n\n## `bucket_fraction`\n\nSets the desired uncompressed size of a bucket which groups values of several\nfields of the given wire type to be compressed together, relative to the desired\nchunk size, on the scale between 0.0 (compress each field separately) to 1.0\n(put all fields of the same wire type in the same bucket.\n\nThis is meaningful if transpose and compression are enabled. A larger bucket\nsize improves compression density; a smaller bucket size makes reading with\nprojection faster, allowing to skip decompression of values of fields which are\nnot included.\n\nDefault 1.0.\n\n## `padding`\n\nIf `padding > 1`, padding is written at the beginning, when flushing, and at the\nend of the file, for the absolute position to reach a multiple of `padding`.\n\nConsequences if `padding` is a multiple of 64KB:\n\n1.  Physical concatenation of separately written files yields a valid file\n    (setting metadata in subsequent files is wasteful but harmless).\n\n2.  Even if the existing file was corrupted or truncated, data appended to it\n    will be recoverable.\n\nThe cost is that up to `padding` bytes is wasted when padding is written.\n\n`padding` is a shortcut for `set_initial_padding` with `set_final_padding`.\n\n`padding` without the parameter assumes 64KB.\n\nDefault: 1 (no padding).\n\n## `initial_padding`\n\nIf `initial_padding > 1`, padding is written at the beginning of the file, for\nthe absolute position to reach a multiple of `initial_padding`.\n\nSee `padding` for details.\n\n`initial_padding` without the parameter assumes 64KB.\n\nDefault: 1 (no padding).\n\n## `final_padding`\n\nIf `final_padding > 1`, padding is written when flushing and at the end of the\nfile, for the absolute position to reach a multiple of `final_padding`.\n\nSee `padding` for details.\n\n`final_padding` without the parameter assumes 64KB.\n\nDefault: 1 (no padding).\n\n## `parallelism`\n\nSets the maximum number of chunks being encoded in parallel in background.\nLarger parallelism can increase throughput, up to a point where it no longer\nmatters; smaller parallelism reduces memory usage.\n\nIf `parallelism > 0`, chunks are written in background and reporting writing\nerrors is delayed.\n\nDefault: 0.\n"
  },
  {
    "path": "doc/riegeli_records_file_format.md",
    "content": "# Riegeli/records file format specification\n\n## Summary\n\nFile contents are interpreted as a sequence of variable-sized *chunks,* where a\nchunk encodes some number of *records.* A record can be any byte sequence but\nRiegeli has special support for the common case where it is a serialized proto\nmessage.\n\nIn order to support seeking and recovery after data corruption, the sequence of\nchunks is interrupted by a *block header* at every multiple of the block size\nwhich is 64 KiB. After the block header the interrupted chunk continues.\n\nA record can be identified by the position of the chunk beginning and the index\nof the record within the chunk. A record can also be identified by a number\nresembling a file position, defined as the sum of the chunk beginning and the\nrecord index.\n\n## Conventions\n\nNumbers in block headers and chunk headers are encoded as unsigned Little-Endian\nintegers.\n\nHashes are 64-bit [HighwayHash](https://github.com/google/highwayhash) values\nwith the key {0x2f696c6567656952, 0x0a7364726f636572, 0x2f696c6567656952,\n0x0a7364726f636572} ('Riegeli/', 'records\\n', 'Riegeli/', 'records\\n').\n\n## Block header\n\nA block header allows to locate the chunk that the block header interrupts.\nBlock headers can interrupt a chunk at arbitrary points, including in the middle\nof the chunk header.\n\nIf a block header lies exactly between chunks, it is considered to interrupt the\nnext chunk; this includes the situation at the beginning of the file. In this\ncase the chunk formally begins at the beginning of the block, even though it\ncontains no bytes before the block header.\n\n*   Block header (24 bytes):\n    *   `header_hash` (8 bytes) — hash of the rest of the header\n        (`previous_chunk` and `next_chunk`)\n    *   `previous_chunk` (8 bytes) — distance from the beginning of the chunk\n        interrupted by this block header to the beginning of the block\n    *   `next_chunk` (8 bytes) — distance from the beginning of the block to the\n        end of the chunk interrupted by this block header\n\nIf `header_hash` does not match, then this block header is corrupted and must be\nignored. Block headers can be skipped during sequential file reading, they are\nuseful only for seeking and for error recovery.\n\n## Chunk\n\nA chunk must not begin inside nor immediately after a block header.\n\n*   Chunk header (40 bytes):\n    *   `header_hash` (8 bytes) — hash of the rest of the header (`data_size` up\n        to and including `decoded_data_size`)\n    *   `data_size` (8 bytes) — size of `data` (excluding intervening block\n        headers)\n    *   `data_hash` (8 bytes) — hash of `data`\n    *   `chunk_type` (1 byte) — determines how to interpret `data`\n    *   `num_records` (7 bytes) — number of records after decoding\n    *   `decoded_data_size` (8 bytes) — sum of record sizes after decoding\n*   `data` (`data_size` bytes) — encoded records or other data\n*   `padding` — ignored (usually filled with zeros by the encoder)\n\nIf `header_hash` does not match, header contents cannot be trusted; if skipping\nover corruption is desired, a valid chunk should be located using block headers.\nIf `data_hash` does not match, `data` is corrupted; if skipping over corruption\nis desired, the chunk must be ignored.\n\nThe size of `padding` is the minimum size which satisfies the following\nconstraints:\n\n*   The chunk (including chunk header, `data`, `padding`, and intervening block\n    headers) has at least as many bytes as `num_records`.\n*   The chunk does not end inside nor immediately after a block header.\n\nIf `num_records` is 0, `decoded_data_size` has a meaning depending on the chunk\ntype.\n\n*Rationale:*\n\n*The presence of `padding` allows to assign unique numbers resembling file\npositions to records.*\n\n*`decoded_data_size` is stored in the chunk header, instead of being implied by\nor stored in `data`, to help decoders decide how many chunks to potentially read\nahead.*\n\n## Chunk data\n\nSome parts of chunk data are compressed. The compression format is generally\nspecified as `compression_type` (byte):\n\n*   0 — none\n*   0x62 ('b') — [Brotli](https://github.com/google/brotli)\n*   0x7a ('z') — [Zstd](https://facebook.github.io/zstd/)\n*   0x73 ('s') — [Snappy](https://google.github.io/snappy/)\n\nAny compressed block is prefixed with its decompressed size (varint64) unless\n`compression_type` is 0.\n\n*Rationale:*\n\n*Knowing the decompressed size can make easier for the decoder to decompress\ndata into a preallocated array.*\n\n### File signature\n\n`chunk_type` is 0x73 ('s').\n\nA file signature chunk must be present at the beginning of the file. It may also\nbe present elsewhere, in which case it encodes no records and is ignored.\n\n`data_size`, `num_records`, and `decoded_data_size` must be 0.\n\nThis makes the first 64 bytes of a Riegeli/records file fixed:\n\n```data\n83 af 70 d1 0d 88 4a 3f 00 00 00 00 00 00 00 00\n40 00 00 00 00 00 00 00 91 ba c2 3c 92 87 e1 a9\n00 00 00 00 00 00 00 00 e1 9f 13 c0 e9 b1 c3 72\n73 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00\n```\n\n### File metadata\n\n`chunk_type` is 0x6d ('m').\n\nA file metadata chunk provides information describing the records. Metadata are\nnot necessary to read the records but might be helpful to interpret their\ncontents.\n\nIf present, metadata should be written immediately after file signature.\n\nThe chunk is encoded like a transposed chunk with a single record containing a\nserialized `RecordsMetadata` proto message, except that `chunk_type` is\ndifferent and `num_records` is 0.\n\n### Padding chunk\n\n`chunk_type` is 0x70 ('p').\n\nA padding chunk encodes no records and only occupies file space.\n\n`num_records` and `decoded_data_size` must be 0. `data` is ignored (usually\nfilled with zeros by the encoder).\n\nThis can be used for more efficient file concatenation (bringing the file offset\nmodulo `kBlockSize` to 0 allows for physical concatenation of files without\nexamining their contents), or for syncing to a file system which requires a\nparticular file offset granularity in order for the sync to be effective.\n\n### Simple chunk with records\n\n`chunk_type` is 0x72 ('r').\n\nSimple chunks store record sizes and concatenated record contents in two\nbuffers, possibly compressed.\n\nThe format:\n\n*   `compression_type` (byte) — compression type for sizes and values\n*   `compressed_sizes_size` (varint64) — size of `compressed_sizes`\n*   `compressed_sizes` (`compressed_sizes_size` bytes) - compressed buffer with\n    record sizes\n*   `compressed_values` (the rest of `data`) — compressed buffer with record\n    values\n\n`compressed_sizes`, after decompression, contains `num_records` varint64s: the\nsize of each record.\n\n`compressed_values`, after decompression, contains `decoded_data_size` bytes:\nconcatenation of record values.\n\n### Transposed chunk with records\n\n`chunk_type` is 0x74 ('t').\n\nTODO: Document this.\n\n## Properties of the file format\n\n*   Data corruption anywhere is detected whenever the hash allows this, and it\n    causes only a local data loss of up to a chunk (if chunk data are damaged)\n    or block (if chunk header is damaged).\n*   It is possible to open for append and write more records, even without\n    reading the original file contents; the original file size must be taken\n    into account though.\n*   Seeking to the chunk closest to the given file position requires a seek +\n    small read, then iterating through chunk headers in a block.\n\n## Implementation notes\n\nThe following formulas clarify how certain field values and positions can be\ncomputed.\n\nConstants for fixed sizes:\n\n```c++\nkBlockSize = 1 << 16;\nkBlockHeaderSize = 24;\nkUsableBlockSize = kBlockSize - kBlockHeaderSize;\nkChunkHeaderSize = 40;\n```\n\nConstraints for chunk boundary distances in a block header:\n\n```c++\nprevious_chunk % kBlockSize < kUsableBlockSize &&\nnext_chunk > 0 &&\n(next_chunk - 1) % kBlockSize >= kBlockHeaderSize\n```\n\nEnd position of a chunk which begins at `chunk_begin`:\n\n```c++\nNumOverheadBlocks(pos, size) =\n    (size + (pos + kUsableBlockSize - 1) % kBlockSize) / kUsableBlockSize;\nAddWithOverhead(pos, size) =\n    pos + size + NumOverheadBlocks(pos, size) * kBlockHeaderSize;\n\n// Equivalent implementation using unsigned arithmetic modulo 1 << 64:\n// RemainingInBlock(pos) = (-pos) % kBlockSize;\nRemainingInBlock(pos) = kBlockSize - 1 - (pos + kBlockSize - 1) % kBlockSize;\nSaturatingSub(a, b) = a > b ? a - b : 0;\n// 0 -> 0, 1..25 -> 25, 26 -> 26, ..., 64K -> 64K, 64K+1..64K+25 -> 64K+25 etc.\nRoundUpToPossibleChunkBoundary(pos) =\n    pos + SaturatingSub(RemainingInBlock(pos), kUsableBlockSize - 1);\n\nchunk_end = max(AddWithOverhead(chunk_begin, kChunkHeaderSize + data_size),\n                RoundUpToPossibleChunkBoundary(chunk_begin + num_records));\n```\n\nFields of a block header at `block_begin` which interrupts a chunk at\n`chunk_begin`:\n\n```c++\nprev_chunk = block_begin - chunk_begin;\nnext_chunk = chunk_end - block_begin;\n```\n"
  },
  {
    "path": "python/BUILD",
    "content": "load(\"@rules_python//python:defs.bzl\", \"py_binary\")\n\npackage(default_visibility = [\"//visibility:private\"])\n\nlicenses([\"notice\"])\n\n# These dependencies are gathered in a py_binary, instead of directly in\n# sh_binary data, so that bazel links __init__.py files to runfiles.\npy_binary(\n    name = \"dummy_binary\",\n    srcs = [\"dummy_binary.py\"],\n    srcs_version = \"PY3\",\n    deps = [\n        \"//python/riegeli\",\n        \"//python/riegeli/tensorflow:riegeli_dataset_ops\",\n    ],\n)\n\nsh_binary(\n    name = \"build_pip_package\",\n    srcs = [\"build_pip_package.sh\"],\n    data = [\n        \"MANIFEST.in\",\n        \"README.md\",\n        \"setup.py\",\n        \":dummy_binary\",\n    ],\n)\n"
  },
  {
    "path": "python/MANIFEST.in",
    "content": "recursive-include riegeli *.py\n"
  },
  {
    "path": "python/README.md",
    "content": "# Riegeli\n\n*Riegeli/records* is a file format for storing a sequence of string records,\ntypically serialized protocol buffers. It supports dense compression, fast\ndecoding, seeking, detection and optional skipping of data corruption, filtering\nof proto message fields for even faster decoding, and parallel encoding.\n\nSee [documentation](https://github.com/google/riegeli/blob/master/doc/index.md).\n\n# Status\n\nRiegeli file format will only change in a backward compatible way (i.e. future\nreaders will understand current files, but current readers might not understand\nfiles using future features).\n\nRiegeli C++ API might change in incompatible ways.\n"
  },
  {
    "path": "python/__init__.py",
    "content": ""
  },
  {
    "path": "python/build_pip_package.sh",
    "content": "#!/bin/bash\n# Copyright 2018 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# Builds a pip package for riegeli.\n#\n# Usage (where DEST is a where to write the output, e.g. ~/riegeli-dist):\n# $ bazel build -c opt python:build_pip_package\n# $ bazel-bin/python/build_pip_package --dest DEST --sdist --bdist\n\nset -e\n\nfunction is_absolute {\n  [[ \"$1\" = /* ]] || [[ \"$1\" =~ ^[a-zA-Z]:[/\\\\].* ]]\n}\n\nfunction real_path() {\n  if is_absolute \"$1\"; then\n    printf \"%s\" \"$1\"\n  else\n    printf \"%s/%s\" \"$PWD\" \"${1#./}\"\n  fi\n}\n\nfunction build_sdist() {\n  local dest=$1\n  python python/setup.py sdist --dist-dir \"$dest\"\n}\n\nfunction build_bdist() {\n  local dest=$1\n  cd bazel-bin/python/build_pip_package.runfiles/com_google_riegeli/python\n  python setup.py bdist_wheel --dist-dir \"$dest\"\n  cd -\n}\n\nfunction main() {\n  local dest=\n  local sdist=false\n  local bdist=false\n  while [[ $# -gt 0 ]]; do\n    if [[ $1 == --dest ]]; then\n      shift\n      dest=$(real_path \"$1\")\n    elif [[ $1 == --sdist ]]; then\n      sdist=true\n    elif [[ $1 == --bdist ]]; then\n      bdist=true\n    else\n      printf \"Unknown flag: %s\\n\" \"$1\" >&2\n      exit 1\n    fi\n    shift\n  done\n  if [[ -z $dest ]]; then\n    printf \"Missing required flag: --dest DIRECTORY\\n\" >&2\n    exit 1\n  fi\n  if [[ $sdist != true ]] && [[ $bdist != true ]]; then\n    printf \"Nothing to do: missing --sdist or --bdist\\n\" >&2\n    exit 1\n  fi\n  mkdir -p -- \"$dest\"\n  if [[ $sdist = true ]]; then\n    build_sdist \"$dest\"\n  fi\n  if [[ $bdist = true ]]; then\n    build_bdist \"$dest\"\n  fi\n}\n\nmain \"$@\"\n"
  },
  {
    "path": "python/dummy_binary.py",
    "content": ""
  },
  {
    "path": "python/riegeli/BUILD",
    "content": "# Riegeli, file format for storing a sequence of records.\n\nload(\"@rules_python//python:defs.bzl\", \"py_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\nexports_files([\"LICENSE\"])\n\npy_library(\n    name = \"riegeli\",\n    srcs = [\"__init__.py\"],\n    imports = [\"..\"],\n    deps = [\n        \"//python/riegeli/base:riegeli_error\",\n        \"//python/riegeli/records:record_position\",\n        \"//python/riegeli/records:record_reader\",\n        \"//python/riegeli/records:record_writer\",\n        \"//python/riegeli/records:records_metadata_py_pb2\",\n        \"//python/riegeli/records:skipped_region\",\n    ],\n)\n"
  },
  {
    "path": "python/riegeli/BUILD.tpl",
    "content": "load(\"@bazel_tools//tools/python:toolchain.bzl\", \"py_runtime_pair\")\n\nlicenses([\"restricted\"])\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\ntoolchain(\n    name = \"toolchain\",\n    toolchain = \":py_runtime_pair\",\n    toolchain_type = \"@bazel_tools//tools/python:toolchain_type\",\n)\n\n# To build Python C/C++ extension on Windows, we need to link to python import library pythonXY.lib\n# See https://docs.python.org/3/extending/windows.html\ncc_import(\n    name = \"python_lib\",\n    interface_library = select({\n        \":windows\": \":python_import_lib\",\n        # A placeholder for Unix platforms which makes --no_build happy.\n        \"//conditions:default\": \"not-existing.lib\",\n    }),\n    system_provided = 1,\n)\n\ncc_library(\n    name = \"python_headers\",\n    hdrs = [\":python_include\"],\n    deps = select({\n        \":windows\": [\":python_lib\"],\n        \"//conditions:default\": [],\n    }),\n    includes = [\"python_include\"],\n)\n\ncc_library(\n    name = \"numpy_headers\",\n    hdrs = [\":numpy_include\"],\n    includes = [\"numpy_include\"],\n)\n\nconfig_setting(\n    name = \"windows\",\n    values = {\"cpu\": \"x64_windows\"},\n    visibility = [\"//visibility:public\"],\n)\n\n%{PYTHON_RUNTIME_PAIR}\n%{PYTHON_INCLUDE_GENRULE}\n%{NUMPY_INCLUDE_GENRULE}\n%{PYTHON_IMPORT_LIB_GENRULE}\n"
  },
  {
    "path": "python/riegeli/__init__.py",
    "content": "# Copyright 2018 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\"\"\"Writes or reads Riegeli/records files.\"\"\"\n\nfrom riegeli.base import riegeli_error\nfrom riegeli.records import record_position\nfrom riegeli.records import record_reader\nfrom riegeli.records import record_writer\nfrom riegeli.records import records_metadata_pb2\nfrom riegeli.records import skipped_region\n\n__all__ = (\n    'RiegeliError',\n    'CancelledError',\n    'UnknownError',\n    'InvalidArgumentError',\n    'DeadlineExceededError',\n    'NotFoundError',\n    'AlreadyExistsError',\n    'PermissionDeniedError',\n    'UnauthenticatedError',\n    'ResourceExhaustedError',\n    'FailedPreconditionError',\n    'AbortedError',\n    'OutOfRangeError',\n    'UnimplementedError',\n    'InternalError',\n    'UnavailableError',\n    'DataLossError',\n    'FlushType',\n    'RecordPosition',\n    'SkippedRegion',\n    'RecordsMetadata',\n    'set_record_type',\n    'RecordWriter',\n    'EXISTENCE_ONLY',\n    'get_record_type',\n    'RecordReader',\n)\n\n# pylint: disable=invalid-name\nRiegeliError = riegeli_error.RiegeliError\nCancelledError = riegeli_error.CancelledError\nUnknownError = riegeli_error.UnknownError\nInvalidArgumentError = riegeli_error.InvalidArgumentError\nDeadlineExceededError = riegeli_error.DeadlineExceededError\nNotFoundError = riegeli_error.NotFoundError\nAlreadyExistsError = riegeli_error.AlreadyExistsError\nPermissionDeniedError = riegeli_error.PermissionDeniedError\nUnauthenticatedError = riegeli_error.UnauthenticatedError\nResourceExhaustedError = riegeli_error.ResourceExhaustedError\nFailedPreconditionError = riegeli_error.FailedPreconditionError\nAbortedError = riegeli_error.AbortedError\nOutOfRangeError = riegeli_error.OutOfRangeError\nUnimplementedError = riegeli_error.UnimplementedError\nInternalError = riegeli_error.InternalError\nUnavailableError = riegeli_error.UnavailableError\nDataLossError = riegeli_error.DataLossError\nRecordPosition = record_position.RecordPosition\nSkippedRegion = skipped_region.SkippedRegion\nRecordsMetadata = records_metadata_pb2.RecordsMetadata\nFlushType = record_writer.FlushType\nset_record_type = record_writer.set_record_type\nRecordWriter = record_writer.RecordWriter\nEXISTENCE_ONLY = record_reader.EXISTENCE_ONLY\nget_record_type = record_reader.get_record_type\nRecordReader = record_reader.RecordReader\n"
  },
  {
    "path": "python/riegeli/base/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\nload(\"@rules_python//python:defs.bzl\", \"py_library\")\n\npackage(\n    default_visibility = [\"//python/riegeli:__subpackages__\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"utils\",\n    srcs = [\"utils.cc\"],\n    hdrs = [\"utils.h\"],\n    data = [\":riegeli_error\"],  # Python module imported from C++.\n    # utils.cc has #define before #include to influence what the included\n    # files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n        \"@rules_python//python/cc:current_py_cc_headers\",\n    ],\n)\n\npy_library(\n    name = \"riegeli_error\",\n    srcs = [\"riegeli_error.py\"],\n)\n"
  },
  {
    "path": "python/riegeli/base/__init__.py",
    "content": ""
  },
  {
    "path": "python/riegeli/base/riegeli_error.py",
    "content": "# Copyright 2018 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n__all__ = (\n    'RiegeliError',\n    'CancelledError',\n    'UnknownError',\n    'InvalidArgumentError',\n    'DeadlineExceededError',\n    'NotFoundError',\n    'AlreadyExistsError',\n    'PermissionDeniedError',\n    'UnauthenticatedError',\n    'ResourceExhaustedError',\n    'FailedPreconditionError',\n    'AbortedError',\n    'OutOfRangeError',\n    'UnimplementedError',\n    'InternalError',\n    'UnavailableError',\n    'DataLossError',\n)\n\n\nclass RiegeliError(Exception):\n  \"\"\"Base class of errors reported by Google APIs.\n\n  Sometimes multiple error codes may apply. Services should return the most\n  specific error code that applies. For example, prefer `OutOfRangeError` over\n  `FailedPreconditionError` if both codes apply. Similarly prefer\n  `NotFoundError` or `AlreadyExistsError` over `FailedPreconditionError`.\n\n  Attributes:\n    code: Error code classifying the error, matching C++ StatusCode.\n  \"\"\"\n\n\nclass CancelledError(RiegeliError):\n  \"\"\"The operation was cancelled, typically by the caller.\"\"\"\n\n  code = 1\n\n\nclass UnknownError(RiegeliError):\n  \"\"\"Unknown error.\n\n  For example, this error may be returned when a Status value received from\n  another address space belongs to an error-space that is not known in this\n  address space. Also errors raised by APIs that do not return enough error\n  information may be converted to this error.\n  \"\"\"\n\n  code = 2\n\n\nclass InvalidArgumentError(RiegeliError):\n  \"\"\"The client specified an invalid argument.\n\n  Note that this differs from `FailedPreconditionError`. `InvalidArgumentError`\n  indicates arguments that are problematic regardless of the state of the system\n  (e.g., a malformed file name).\n  \"\"\"\n\n  code = 3\n\n\nclass DeadlineExceededError(RiegeliError):\n  \"\"\"The deadline expired before the operation could complete.\n\n  For operations that change the state of the system, this error may be returned\n  even if the operation has completed successfully. or example, a successful\n  response from a server could have been delayed long enough for the deadline to\n  expire.\n  \"\"\"\n\n  code = 4\n\n\nclass NotFoundError(RiegeliError):\n  \"\"\"Some requested entity (e.g., file or directory) was not found.\n\n  Note to server developers: if a request is denied for an entire class of\n  users, such as gradual feature rollout or undocumented allowlist,\n  `NotFoundError` may be used. If a request is denied for some users within a\n  class of users, such as user-based access control, `PermissionDeniedError`\n  must be used.\n  \"\"\"\n\n  code = 5\n\n\nclass AlreadyExistsError(RiegeliError):\n  \"\"\"The entity that a client attempted to create already exists.\"\"\"\n\n  code = 6\n\n\nclass PermissionDeniedError(RiegeliError):\n  \"\"\"The caller does not have permission to execute the specified operation.\n\n  `PermissionDeniedError` must not be used for rejections caused by exhausting\n  some resource (use `ResourceExhaustedError` instead for those errors).\n  `PermissionDeniedError` must not be used if the caller can not be identified\n  (use `UnauthenticatedError` instead for those errors). This error code does\n  not imply the request is valid or the requested entity exists or satisfies\n  other pre-conditions.\n  \"\"\"\n\n  code = 7\n\n\nclass UnauthenticatedError(RiegeliError):\n  \"\"\"No valid authentication credentials for the operation.\"\"\"\n\n  code = 16\n\n\nclass ResourceExhaustedError(RiegeliError):\n  \"\"\"Some resource has been exhausted.\n\n  Perhaps a per-user quota, or perhaps the entire file system is out of\n  space.\n  \"\"\"\n\n  code = 8\n\n\nclass FailedPreconditionError(RiegeliError):\n  \"\"\"Failed precondition.\n\n  The operation was rejected because the system is not in a state required for\n  the operation's execution. For example, the directory to be deleted is\n  non-empty, an rmdir operation is applied to a non-directory, etc.\n\n  A litmus test that may help a service implementor in deciding between\n  `FailedPreconditionError`, `AbortedError`, and `UnavailableError`:\n   (a) Use `UnavailableError` if the client can retry just the failing call.\n   (b) Use `AbortedError` if the client should retry at a higher-level (e.g.,\n       when a client-specified test-and-set fails, indicating the client should\n       restart a read-modify-write sequence).\n   (c) Use `FailedPreconditionError` if the client should not retry until the\n       system state has been explicitly fixed. E.g., if an \"rmdir\" fails because\n       the directory is non-empty, `FailedPreconditionError` should be returned\n       since the client should not retry unless the files are deleted from the\n       directory.\n  \"\"\"\n\n  code = 9\n\n\nclass AbortedError(RiegeliError):\n  \"\"\"The operation was aborted.\n\n  Typically due to a concurrency issue such as a sequencer check failure or\n  transaction abort.\n\n  See litmus test at `FailedPreconditionError` for deciding between\n  `FailedPreconditionError`, `AbortedError`, and `UnavailableError`.\n  \"\"\"\n\n  code = 10\n\n\nclass OutOfRangeError(RiegeliError):\n  \"\"\"The operation was attempted past the valid range.\n\n  E.g., seeking or reading past end-of-file.\n\n  Unlike `InvalidArgumentError`, this error indicates a problem that may be\n  fixed if the system state changes. For example, a 32-bit file system will\n  generate `InvalidArgumentError` if asked to read at an offset that is not in\n  the range [0,2^32-1], but it will generate `OutOfRangeError` if asked to read\n  from an offset past the current file size.\n\n  There is a fair bit of overlap between `FailedPreconditionError` and\n  `OutOfRangeError`. We recommend using `OutOfRangeError` (the more specific\n  error) when it applies so that callers who are iterating through a space can\n  easily look for an `OutOfRangeError` error to detect when they are done.\n  \"\"\"\n\n  code = 11\n\n\nclass UnimplementedError(RiegeliError):\n  \"\"\"The operation is not implemented.\n\n  Or is not supported/enabled in this service.\n  \"\"\"\n\n  code = 12\n\n\nclass InternalError(RiegeliError):\n  \"\"\"Internal errors.\n\n  This means that some invariants expected by the underlying system have been\n  broken. This error code is reserved for serious errors.\n  \"\"\"\n\n  code = 13\n\n\nclass UnavailableError(RiegeliError):\n  \"\"\"The service is currently unavailable.\n\n  This is most likely a transient condition, which can be corrected by retrying\n  with a backoff.\n\n  See litmus test at `FailedPreconditionError` for deciding between\n  `FailedPreconditionError`, `AbortedError`, and `UnavailableError`.\n  \"\"\"\n\n  code = 14\n\n\nclass DataLossError(RiegeliError):\n  \"\"\"Unrecoverable data loss or corruption.\"\"\"\n\n  code = 15\n"
  },
  {
    "path": "python/riegeli/base/utils.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// From https://docs.python.org/3/c-api/intro.html:\n// Since Python may define some pre-processor definitions which affect the\n// standard headers on some systems, you must include Python.h before any\n// standard headers are included.\n#define PY_SSIZE_T_CLEAN\n#include <Python.h>\n// clang-format: do not reorder the above include.\n\n#include \"python/riegeli/base/utils.h\"\n// clang-format: do not reorder the above include.\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/types.h\"\n\nnamespace riegeli::python {\n\nException& Exception::operator=(const Exception& that) noexcept {\n  PythonLock lock;\n  Py_XINCREF(that.type_.get());\n  type_.reset(that.type_.get());\n  Py_XINCREF(that.value_.get());\n  value_.reset(that.value_.get());\n  Py_XINCREF(that.traceback_.get());\n  traceback_.reset(that.traceback_.get());\n  return *this;\n}\n\nException Exception::Fetch() {\n  PythonLock::AssertHeld();\n  PyObject* type;\n  PyObject* value;\n  PyObject* traceback;\n  PyErr_Fetch(&type, &value, &traceback);\n  PyErr_NormalizeException(&type, &value, &traceback);\n  return Exception(type, value, traceback);\n}\n\nPyObject* Exception::Restore() const& {\n  PythonLock::AssertHeld();\n  Py_XINCREF(type_.get());\n  Py_XINCREF(value_.get());\n  Py_XINCREF(traceback_.get());\n  PyErr_Restore(type_.get(), value_.get(), traceback_.get());\n  return nullptr;\n}\n\nPyObject* Exception::Restore() && {\n  PythonLock::AssertHeld();\n  PyErr_Restore(type_.release(), value_.release(), traceback_.release());\n  return nullptr;\n}\n\nstd::string Exception::message() const {\n  if (ok()) return \"OK\";\n  PythonLock lock;\n  RIEGELI_ASSERT(PyExceptionClass_Check(type_.get()))\n      << \"Expected an exception class, not \" << Py_TYPE(type_.get())->tp_name;\n  std::string message = PyExceptionClass_Name(type_.get());\n  if (value_ == nullptr) return message;\n  const PythonPtr str_result(PyObject_Str(value_.get()));\n  if (ABSL_PREDICT_FALSE(str_result == nullptr)) {\n    PyErr_Clear();\n    absl::StrAppend(&message, \": <str() failed>\");\n    return message;\n  }\n  StrOrBytes str;\n  if (ABSL_PREDICT_FALSE(!str.FromPython(str_result.get()))) {\n    PyErr_Clear();\n    absl::StrAppend(&message, \": <StrOrBytes::FromPython() failed>\");\n    return message;\n  }\n  if (!absl::string_view(str).empty()) {\n    absl::StrAppend(&message, \": \", absl::string_view(str));\n  }\n  return message;\n}\n\nvoid SetRiegeliError(const absl::Status& status) {\n  RIEGELI_ASSERT(!status.ok())\n      << \"Failed precondition of SetRiegeliError(): status not failed\";\n  PythonLock::AssertHeld();\n  PythonPtr message = StringToPython(status.message());\n  if (ABSL_PREDICT_FALSE(message == nullptr)) return;\n  PyObject* type;\n  switch (status.code()) {\n#define HANDLE_CODE(name)                                     \\\n  case absl::StatusCode::k##name: {                           \\\n    static constexpr ImportedConstant k##name##Error(         \\\n        \"riegeli.base.riegeli_error\", #name \"Error\");         \\\n    if (ABSL_PREDICT_FALSE(!k##name##Error.Verify())) return; \\\n    type = k##name##Error.get();                              \\\n  } break\n\n    // clang-format off\n    HANDLE_CODE(Cancelled);\n    default:\n    HANDLE_CODE(Unknown);\n    HANDLE_CODE(InvalidArgument);\n    HANDLE_CODE(DeadlineExceeded);\n    HANDLE_CODE(NotFound);\n    HANDLE_CODE(AlreadyExists);\n    HANDLE_CODE(PermissionDenied);\n    HANDLE_CODE(Unauthenticated);\n    HANDLE_CODE(ResourceExhausted);\n    HANDLE_CODE(FailedPrecondition);\n    HANDLE_CODE(Aborted);\n    HANDLE_CODE(OutOfRange);\n    HANDLE_CODE(Unimplemented);\n    HANDLE_CODE(Internal);\n    HANDLE_CODE(Unavailable);\n    HANDLE_CODE(DataLoss);\n      // clang-format on\n\n#undef HANDLE_CODE\n  }\n\n  Py_INCREF(type);\n  PyErr_Restore(type, message.release(), nullptr);\n}\n\nnamespace py_internal {\n\nnamespace {\n\n// A linked list of all objects of type `StaticObject` which have `value_`\n// allocated, chained by their `next_` fields. This is used to free the objects\n// on Python interpreter shutdown.\nconst StaticObject* all_static_objects = nullptr;\n\n}  // namespace\n\nvoid FreeStaticObjectsImpl() {\n  const StaticObject* static_object =\n      std::exchange(all_static_objects, nullptr);\n  while (static_object != nullptr) {\n    static_object->value_ = nullptr;\n    static_object = std::exchange(static_object->next_, nullptr);\n  }\n}\n\n// `extern \"C\"` sets the C calling convention for compatibility with the Python\n// API. `static` avoids making symbols public, as `extern \"C\"` trumps anonymous\n// namespace.\nextern \"C\" {\nstatic void FreeStaticObjects() { FreeStaticObjectsImpl(); }\n}  // extern \"C\"\n\nvoid StaticObject::RegisterThis() const {\n  PythonLock::AssertHeld();\n  if (all_static_objects == nullptr) {\n    // This is the first registered `StaticObject` since `Py_Initialize()`.\n    Py_AtExit(FreeStaticObjects);\n  }\n  next_ = std::exchange(all_static_objects, this);\n}\n\nbool ImportedCapsuleBase::ImportValue() const {\n  // For some reason `PyImport_ImportModule()` is sometimes required before\n  // `PyCapsule_Import()` for a module with a nested name.\n  const size_t dot = absl::string_view(capsule_name_).rfind('.');\n  RIEGELI_ASSERT_NE(dot, absl::string_view::npos)\n      << \"Capsule name does not contain a dot: \" << capsule_name_;\n  RIEGELI_CHECK(\n      PyImport_ImportModule(std::string(capsule_name_, dot).c_str()) != nullptr)\n      << Exception::Fetch().message();\n  value_ = PyCapsule_Import(capsule_name_, false);\n  return value_ != nullptr;\n}\n\n}  // namespace py_internal\n\nbool Identifier::AllocateValue() const {\n  value_ = StringToPython(name_).release();\n  if (ABSL_PREDICT_FALSE(value_ == nullptr)) return false;\n  PyUnicode_InternInPlace(&value_);\n  RegisterThis();\n  return true;\n}\n\nbool ImportedConstant::AllocateValue() const {\n  const PythonPtr module_name = StringToPython(module_name_);\n  if (ABSL_PREDICT_FALSE(module_name == nullptr)) return false;\n  const PythonPtr module(PyImport_Import(module_name.get()));\n  if (ABSL_PREDICT_FALSE(module == nullptr)) return false;\n  const PythonPtr attr_name = StringToPython(attr_name_);\n  if (ABSL_PREDICT_FALSE(attr_name == nullptr)) return false;\n  value_ = PyObject_GetAttr(module.get(), attr_name.get());\n  if (ABSL_PREDICT_FALSE(value_ == nullptr)) return false;\n  RegisterThis();\n  return true;\n}\n\nbool ExportCapsule(PyObject* module, const char* capsule_name,\n                   const void* ptr) {\n  PythonPtr capsule(\n      PyCapsule_New(const_cast<void*>(ptr), capsule_name, nullptr));\n  if (ABSL_PREDICT_FALSE(capsule == nullptr)) return false;\n  const size_t dot = absl::string_view(capsule_name).rfind('.');\n  RIEGELI_ASSERT_NE(dot, absl::string_view::npos)\n      << \"Capsule name does not contain a dot: \" << capsule_name;\n  RIEGELI_ASSERT(PyModule_Check(module))\n      << \"Expected a module, not \" << Py_TYPE(module)->tp_name;\n  RIEGELI_ASSERT_EQ(absl::string_view(PyModule_GetName(module)),\n                    absl::string_view(capsule_name, dot))\n      << \"Module name mismatch\";\n  if (ABSL_PREDICT_FALSE(PyModule_AddObject(module, capsule_name + dot + 1,\n                                            capsule.release()) < 0)) {\n    return false;\n  }\n  return true;\n}\n\nMemoryView::~MemoryView() {\n  if (object_ != nullptr && Py_REFCNT(object_.get()) > 1) {\n    PyObject* value;\n    PyObject* type;\n    PyObject* traceback;\n    PyErr_Fetch(&value, &type, &traceback);\n    ReleaseInternal();\n    PyErr_Restore(value, type, traceback);\n  }\n}\n\nPyObject* MemoryView::ToPython(absl::string_view value) {\n  RIEGELI_ASSERT_EQ(object_, nullptr)\n      << \"Failed precondition of MemoryView::ToPython(): \"\n         \"called more than once\";\n  object_.reset(PyMemoryView_FromMemory(const_cast<char*>(value.data()),\n                                        IntCast<Py_ssize_t>(value.size()),\n                                        PyBUF_READ));\n  return object_.get();\n}\n\nPyObject* MemoryView::MutableToPython(absl::Span<char> value) {\n  RIEGELI_ASSERT_EQ(object_, nullptr)\n      << \"Failed precondition of MemoryView::MutableToPython(): \"\n         \"called more than once\";\n  object_.reset(PyMemoryView_FromMemory(\n      value.data(), IntCast<Py_ssize_t>(value.size()), PyBUF_WRITE));\n  return object_.get();\n}\n\nbool MemoryView::Release() {\n  bool release_ok = true;\n  if (object_ != nullptr && Py_REFCNT(object_.get()) > 1) {\n    release_ok = ReleaseInternal();\n  }\n  object_.reset();\n  return release_ok;\n}\n\ninline bool MemoryView::ReleaseInternal() {\n  static constexpr Identifier id_release(\"release\");\n  const PythonPtr release_result(\n      PyObject_CallMethodObjArgs(object_.get(), id_release.get(), nullptr));\n  return release_result != nullptr;\n}\n\nbool StrOrBytes::FromPython(PyObject* object ABSL_ATTRIBUTE_LIFETIME_BOUND) {\n  RIEGELI_ASSERT_EQ(data_.data(), nullptr)\n      << \"Failed precondition of StrOrBytes::FromPython(): \"\n         \"called more than once\";\n  if (PyUnicode_Check(object)) {\n    Py_ssize_t length;\n    const char* data = PyUnicode_AsUTF8AndSize(object, &length);\n    if (ABSL_PREDICT_FALSE(data == nullptr)) return false;\n    data_ = absl::string_view(data, IntCast<size_t>(length));\n    return true;\n  } else if (ABSL_PREDICT_FALSE(!PyBytes_Check(object))) {\n    PyErr_Format(PyExc_TypeError, \"Expected str or bytes, not %s\",\n                 Py_TYPE(object)->tp_name);\n    return false;\n  }\n  data_ = absl::string_view(PyBytes_AS_STRING(object),\n                            IntCast<size_t>(PyBytes_GET_SIZE(object)));\n  return true;\n}\n\nPythonPtr ChainToPython(const Chain& value) {\n  PythonPtr bytes(\n      PyBytes_FromStringAndSize(nullptr, IntCast<Py_ssize_t>(value.size())));\n  if (ABSL_PREDICT_FALSE(bytes == nullptr)) return nullptr;\n  value.CopyTo(PyBytes_AS_STRING(bytes.get()));\n  return bytes;\n}\n\nstd::optional<Chain> ChainFromPython(PyObject* object) {\n  Py_buffer buffer;\n  if (ABSL_PREDICT_FALSE(PyObject_GetBuffer(object, &buffer, PyBUF_CONTIG_RO) <\n                         0)) {\n    return std::nullopt;\n  }\n  Chain result(absl::string_view(static_cast<const char*>(buffer.buf),\n                                 IntCast<size_t>(buffer.len)));\n  PyBuffer_Release(&buffer);\n  return result;\n}\n\nPythonPtr SizeToPython(size_t value) {\n  if (ABSL_PREDICT_FALSE(value >\n                         std::numeric_limits<unsigned long long>::max())) {\n    PyErr_Format(PyExc_OverflowError, \"Size out of range: %zu\", value);\n    return nullptr;\n  }\n  return PythonPtr(\n      PyLong_FromUnsignedLongLong(IntCast<unsigned long long>(value)));\n}\n\nstd::optional<size_t> SizeFromPython(PyObject* object) {\n  const PythonPtr index(PyNumber_Index(object));\n  if (ABSL_PREDICT_FALSE(index == nullptr)) return std::nullopt;\n  RIEGELI_ASSERT(PyLong_Check(index.get()))\n      << \"PyNumber_Index() returned an unexpected type: \"\n      << Py_TYPE(index.get())->tp_name;\n  unsigned long long index_value = PyLong_AsUnsignedLongLong(index.get());\n  if (ABSL_PREDICT_FALSE(index_value == static_cast<unsigned long long>(-1)) &&\n      PyErr_Occurred()) {\n    return std::nullopt;\n  }\n  if (ABSL_PREDICT_FALSE(index_value > std::numeric_limits<size_t>::max())) {\n    PyErr_Format(PyExc_OverflowError, \"Size out of range: %llu\", index_value);\n    return std::nullopt;\n  }\n  return IntCast<size_t>(index_value);\n}\n\nPythonPtr PositionToPython(Position value) {\n  if (ABSL_PREDICT_FALSE(value >\n                         std::numeric_limits<unsigned long long>::max())) {\n    PyErr_Format(PyExc_OverflowError, \"Position out of range: %ju\",\n                 uintmax_t{value});\n    return nullptr;\n  }\n  return PythonPtr(\n      PyLong_FromUnsignedLongLong(IntCast<unsigned long long>(value)));\n}\n\nstd::optional<Position> PositionFromPython(PyObject* object) {\n  const PythonPtr index(PyNumber_Index(object));\n  if (ABSL_PREDICT_FALSE(index == nullptr)) return std::nullopt;\n  RIEGELI_ASSERT(PyLong_Check(index.get()))\n      << \"PyNumber_Index() returned an unexpected type: \"\n      << Py_TYPE(index.get())->tp_name;\n  const unsigned long long index_value = PyLong_AsUnsignedLongLong(index.get());\n  if (ABSL_PREDICT_FALSE(index_value == static_cast<unsigned long long>(-1)) &&\n      PyErr_Occurred()) {\n    return std::nullopt;\n  }\n  if (ABSL_PREDICT_FALSE(index_value > std::numeric_limits<Position>::max())) {\n    PyErr_Format(PyExc_OverflowError, \"Position out of range: %llu\",\n                 index_value);\n    return std::nullopt;\n  }\n  return IntCast<Position>(index_value);\n}\n\nPythonPtr PartialOrderingToPython(PartialOrdering ordering) {\n  if (ordering == PartialOrdering::unordered) {\n    return Py_INCREF(Py_None), PythonPtr(Py_None);\n  }\n  return PythonPtr(PyLong_FromLong(ordering < 0 ? -1 : ordering == 0 ? 0 : 1));\n}\n\nstd::optional<PartialOrdering> PartialOrderingFromPython(PyObject* object) {\n  if (object == Py_None) return PartialOrdering::unordered;\n  const long long_value = PyLong_AsLong(object);\n  if (ABSL_PREDICT_FALSE(long_value == -1) && PyErr_Occurred()) {\n    return std::nullopt;\n  }\n  return riegeli::Compare(long_value, 0);\n}\n\n}  // namespace riegeli::python\n"
  },
  {
    "path": "python/riegeli/base/utils.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef PYTHON_RIEGELI_BASE_UTILS_H_\n#define PYTHON_RIEGELI_BASE_UTILS_H_\n\n// From https://docs.python.org/3/c-api/intro.html:\n// Since Python may define some pre-processor definitions which affect the\n// standard headers on some systems, you must include Python.h before any\n// standard headers are included.\n#include <Python.h>\n// clang-format: do not reorder the above include.\n\n#include <stddef.h>\n\n#include <memory>\n#include <new>\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/types.h\"\n\nnamespace riegeli::python {\n\n// Ensures that Python GIL is locked. Reentrant.\n//\n// Same as `PyGILState_Ensure()` / `PyGILState_Release()`.\nclass PythonLock {\n public:\n  static void AssertHeld() {\n    RIEGELI_ASSERT(PyGILState_Check()) << \"Python GIL was assumed to be held\";\n  }\n\n  PythonLock() { gstate_ = PyGILState_Ensure(); }\n\n  PythonLock(const PythonLock&) = delete;\n  PythonLock& operator=(const PythonLock&) = delete;\n\n  ~PythonLock() { PyGILState_Release(gstate_); }\n\n private:\n  PyGILState_STATE gstate_;\n};\n\n// Unlocks Python GIL, allowing non-Python threads to run.\n//\n// Same as `Py_BEGIN_ALLOW_THREADS` / `Py_END_ALLOW_THREADS`.\nclass PythonUnlock {\n public:\n  PythonUnlock() {\n    PythonLock::AssertHeld();\n    tstate_ = PyEval_SaveThread();\n  }\n\n  PythonUnlock(const PythonUnlock&) = delete;\n  PythonUnlock& operator=(const PythonUnlock&) = delete;\n\n  ~PythonUnlock() { PyEval_RestoreThread(tstate_); }\n\n private:\n  PyThreadState* tstate_;\n};\n\n// Apply a function with Python GIL unlocked, allowing non-Python threads to\n// run.\n//\n// Same as `Py_BEGIN_ALLOW_THREADS` / `Py_END_ALLOW_THREADS`.\ntemplate <typename Function>\nstd::invoke_result_t<Function> PythonUnlocked(Function&& f) {\n  PythonUnlock unlock;\n  return std::forward<Function>(f)();\n}\n\n// Owned `PyObject` which assumes that Python GIL is held.\n\nstruct Deleter {\n  template <typename T>\n  void operator()(T* ptr) const {\n    PythonLock::AssertHeld();\n    Py_DECREF(ptr);\n  }\n};\n\nusing PythonPtr = std::unique_ptr<PyObject, Deleter>;\n\n// Owned `PyObject` which does not assume that Python GIL is held.\n\nstruct LockingDeleter {\n  template <typename T>\n  void operator()(T* ptr) const {\n    PythonLock lock;\n    Py_DECREF(ptr);\n  }\n};\n\nusing PythonPtrLocking = std::unique_ptr<PyObject, LockingDeleter>;\n\n// Allows a C++ object to be safely embedded in a Python object allocated with\n// `PyType_GenericAlloc()`.\n//\n// `PythonWrapped<T>` is similar to `std::optional<T>`, but:\n//  * `PythonWrapped<T>` is POD.\n//  * `PythonWrapped<T>` supports only a subset of `std::optional<T>` API.\n//  * `PythonWrapped<T>` filled with zero bytes is valid and absent\n//    (`PyType_GenericAlloc()` fills the Python object with zero bytes).\n//  * `PythonWrapped<T>` should be explicitly `reset()` in the implementation of\n//    `tp_dealloc` (there is no C++ destructor).\ntemplate <typename T>\nclass PythonWrapped {\n public:\n  static_assert(alignof(T) <= alignof(max_align_t),\n                \"PythonWrapped does not support overaligned types\");\n\n  template <typename... Args>\n  ABSL_ATTRIBUTE_REINITIALIZES void emplace(Args&&... args) {\n    if (has_value_) {\n      get()->~T();\n    } else {\n      has_value_ = true;\n    }\n    new (storage_) T(std::forward<Args>(args)...);\n  }\n\n  ABSL_ATTRIBUTE_REINITIALIZES void reset() {\n    if (has_value_) {\n      get()->~T();\n      has_value_ = false;\n    }\n  }\n\n  bool has_value() const { return has_value_; }\n\n  T* get() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT(has_value_) << \"Object uninitialized\";\n    return std::launder(reinterpret_cast<T*>(storage_));\n  }\n  const T* get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT(has_value_) << \"Object uninitialized\";\n    return std::launder(reinterpret_cast<const T*>(storage_));\n  }\n  T& operator*() ABSL_ATTRIBUTE_LIFETIME_BOUND { return *get(); }\n  const T& operator*() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return *get(); }\n  T* operator->() ABSL_ATTRIBUTE_LIFETIME_BOUND { return get(); }\n  const T* operator->() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return get(); }\n\n  bool Verify() const {\n    PythonLock::AssertHeld();\n    if (ABSL_PREDICT_FALSE(!has_value())) {\n      PyErr_SetString(PyExc_ValueError, \"Object uninitialized\");\n      return false;\n    }\n    return true;\n  }\n\n private:\n  bool has_value_;\n  alignas(T) char storage_[sizeof(T)];\n};\n\n// Represents an optional Python exception being raised.\nclass Exception {\n public:\n  // No exception.\n  Exception() = default;\n\n  Exception(const Exception& that) noexcept;\n  Exception& operator=(const Exception& that) noexcept;\n\n  Exception(Exception&& that) = default;\n  Exception& operator=(Exception&& that) = default;\n\n  // Fetches the active Python exception.\n  static Exception Fetch();\n\n  // Restores the active Python exception.\n  PyObject* Restore() const&;\n  PyObject* Restore() &&;\n\n  bool ok() const { return type_ == nullptr; }\n\n  std::string message() const;\n\n  // For implementing `tp_traverse` of objects containing `Exception`.\n  int Traverse(visitproc visit, void* arg);\n\n private:\n  // Steals references.\n  explicit Exception(PyObject* type, PyObject* value, PyObject* traceback)\n      : type_(type), value_(value), traceback_(traceback) {}\n\n  PythonPtrLocking type_;\n  PythonPtrLocking value_;\n  PythonPtrLocking traceback_;\n};\n\n// Translate a failed status to the active Python exception, a class extending\n// `RiegeliError`.\nvoid SetRiegeliError(const absl::Status& status);\n\nnamespace py_internal {\n\n// Lazily initialized pointer to a Python object, persisting until interpreter\n// shutdown.\nclass StaticObject {\n protected:\n  mutable PyObject* value_ = nullptr;\n  mutable const StaticObject* next_ = nullptr;\n\n  // Register this object in a global list of static objects. This must be\n  // called when `value_` is allocated.\n  void RegisterThis() const;\n\n private:\n  friend void FreeStaticObjectsImpl();\n};\n\n// Template parameter independent part of `ImportedCapsule`.\nclass ImportedCapsuleBase {\n public:\n  // Forces importing the value, returning `false` on failures (with Python\n  // exception set).\n  //\n  // If `Verify()` returns `true`, `get()` does not die.\n  bool Verify() const {\n    PythonLock::AssertHeld();\n    if (ABSL_PREDICT_FALSE(value_ == nullptr)) return ImportValue();\n    return true;\n  }\n\n protected:\n  explicit constexpr ImportedCapsuleBase(const char* capsule_name)\n      : capsule_name_(capsule_name) {}\n\n  ~ImportedCapsuleBase() = default;\n\n  bool ImportValue() const;\n\n  mutable void* value_ = nullptr;\n\n private:\n  const char* capsule_name_;\n};\n\n}  // namespace py_internal\n\n// Creates a Python string (type `str`) which persists until interpreter\n// shutdown. This is useful for attribute or method names in\n// `PyObject_GetAttr()` or `PyObject_CallMethodObjArgs()`.\n//\n// An instance of `Identifier` should be allocated statically:\n// ```\n//   static constexpr Identifier id_write(\"write\");\n// ```\n//\n// Then `id_write.get()` is a borrowed reference to the Python object.\nclass Identifier : public py_internal::StaticObject {\n public:\n  explicit constexpr Identifier(absl::string_view name) : name_(name) {}\n\n  // Forces allocating the value, returning `false` on failures (with Python\n  // exception set).\n  //\n  // If `Verify()` returns `true`, `get()` does not die.\n  bool Verify() const {\n    PythonLock::AssertHeld();\n    if (ABSL_PREDICT_FALSE(value_ == nullptr)) return AllocateValue();\n    return true;\n  }\n\n  // Returns the value, allocating it on the first call. Dies on failure\n  // (use `Verify()` to prevent this).\n  PyObject* get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    PythonLock::AssertHeld();\n    if (ABSL_PREDICT_FALSE(value_ == nullptr)) {\n      RIEGELI_CHECK(AllocateValue()) << Exception::Fetch().message();\n    }\n    return value_;\n  }\n\n private:\n  bool AllocateValue() const;\n\n  absl::string_view name_;\n};\n\n// Imports a Python module and gets its attribute, which persists until\n// interpreter shutdown.\n//\n// An instance of `ImportedConstant` should be allocated statically:\n// ```\n//   static constexpr ImportedConstant kRiegeliError(\n//        \"riegeli.base.riegeli_error\", \"RiegeliError\");\n// ```\n//\n// Then `kRiegeliError.get()` is a borrowed reference to the Python object.\nclass ImportedConstant : public py_internal::StaticObject {\n public:\n  explicit constexpr ImportedConstant(absl::string_view module_name,\n                                      absl::string_view attr_name)\n      : module_name_(module_name), attr_name_(attr_name) {}\n\n  // Forces importing the value, returning `false` on failures (with Python\n  // exception set).\n  //\n  // If `Verify()` returns `true`, `get()` does not die.\n  bool Verify() const {\n    PythonLock::AssertHeld();\n    if (ABSL_PREDICT_FALSE(value_ == nullptr)) return AllocateValue();\n    return true;\n  }\n\n  // Returns the value, importing it on the first call. Dies on failure\n  // (use `Verify()` to prevent this).\n  PyObject* get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    PythonLock::AssertHeld();\n    if (ABSL_PREDICT_FALSE(value_ == nullptr)) {\n      RIEGELI_CHECK(AllocateValue()) << Exception::Fetch().message();\n    }\n    return value_;\n  }\n\n private:\n  bool AllocateValue() const;\n\n  absl::string_view module_name_;\n  absl::string_view attr_name_;\n};\n\n// Exports a Python capsule containing a C++ pointer, which should be valid\n// forever, by adding it to the given module.\n//\n// `capsule_name` must be \"module_name.attr_name\" with `module_name`\n// corresponding to `PyModule_GetName(module)`.\n//\n// Returns `false` on failure (with Python exception set).\nbool ExportCapsule(PyObject* module, const char* capsule_name, const void* ptr);\n\n// Imports a Python capsule and gets its stored pointer, which persists forever.\n//\n// `capsule_name must` be \"module_name.attr_name\".\n//\n// An instance of `ImportedCapsule` should be allocated statically:\n// ```\n//   static constexpr ImportedCapsule<RecordPositionApi> kRecordPositionApi(\n//       \"riegeli.records.record_position._CPPAPI\");\n// ```\n//\n// Then `kRecordPositionApi.get()` is a pointer stored in the capsule.\ntemplate <typename T>\nclass ImportedCapsule : public py_internal::ImportedCapsuleBase {\n public:\n  explicit constexpr ImportedCapsule(const char* capsule_name)\n      : ImportedCapsuleBase(capsule_name) {}\n\n  // Returns the value, importing it on the first call. Dies on failure\n  // (use `Verify()` to prevent this).\n  const T* get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    PythonLock::AssertHeld();\n    if (ABSL_PREDICT_FALSE(value_ == nullptr)) {\n      RIEGELI_CHECK(ImportValue()) << Exception::Fetch().message();\n    }\n    return static_cast<const T*>(value_);\n  }\n\n  const T& operator*() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return *get(); }\n  const T* operator->() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return get(); }\n};\n\n// Converts C++ `long` to a Python `int` object.\n//\n// Returns `nullptr` on failure (with Python exception set).\ninline PythonPtr IntToPython(long value) {\n  return PythonPtr(PyLong_FromLong(value));\n}\n\n// Converts C++ `absl::string_view` to a Python `bytes` object.\n//\n// Returns `nullptr` on failure (with Python exception set).\ninline PythonPtr BytesToPython(absl::string_view value) {\n  return PythonPtr(PyBytes_FromStringAndSize(\n      value.data(), IntCast<Py_ssize_t>(value.size())));\n}\n\n// Converts C++ array of bytes to a Python `memoryview` object.\n//\n// Memory is shared. The C++ memory must be valid as long as the Python object\n// is needed.\nclass MemoryView {\n public:\n  MemoryView() = default;\n\n  MemoryView(const MemoryView&) = delete;\n  MemoryView& operator=(const MemoryView&) = delete;\n\n  // Calls `Release()`, ignoring its result, without disturbing the Python\n  // exception state.\n  ~MemoryView();\n\n  // Creates and returns a read-only `memoryview` object.\n  //\n  // Returns `nullptr` on failure (with Python exception set).\n  //\n  // `ToPython()` or `MutableToPython()` must be called at most once for each\n  // `MemoryView` object.\n  PyObject* ToPython(absl::string_view value);\n\n  // Creates and returns a mutable `memoryview` object.\n  //\n  // Returns `nullptr` on failure (with Python exception set).\n  //\n  // `ToPython()` or `MutableToPython()` must be called at most once for each\n  // `MemoryView` object.\n  PyObject* MutableToPython(absl::Span<char> value);\n\n  // If a reference to the `memoryview` has been stored elsewhere, calls\n  // `memoryview.release()` to mark the `memoryview` as invalid.\n  //\n  // Returns `false` on failure (with Python exception set).\n  bool Release();\n\n private:\n  bool ReleaseInternal();\n\n  PythonPtr object_;\n};\n\n// Refers to internals of a Python `bytes`-like object, using the buffer\n// protocol.\nclass BytesLike {\n public:\n  BytesLike() noexcept { buffer_.obj = nullptr; }\n\n  BytesLike(const BytesLike&) = delete;\n  BytesLike& operator=(const BytesLike&) = delete;\n\n  ~BytesLike() {\n    PythonLock::AssertHeld();\n    if (buffer_.obj != nullptr) PyBuffer_Release(&buffer_);\n  }\n\n  // Converts from a Python object.\n  //\n  // Returns `false` on failure (with Python exception set).\n  //\n  // Must be called at most once for each `BytesLike` object.\n  bool FromPython(PyObject* object) {\n    RIEGELI_ASSERT_EQ(buffer_.obj, nullptr)\n        << \"Failed precondition of BytesLike::FromPython(): \"\n           \"called more than once\";\n    return PyObject_GetBuffer(object, &buffer_, PyBUF_CONTIG_RO) == 0;\n  }\n\n  // Returns the binary contents.\n  /*implicit*/ operator absl::string_view() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return absl::string_view(static_cast<const char*>(buffer_.buf),\n                             IntCast<size_t>(buffer_.len));\n  }\n\n private:\n  Py_buffer buffer_;\n};\n\n// Converts C++ `absl::string_view` to a Python `str` object. Unicode is\n// converted from UTF-8.\n//\n// Returns `nullptr` on failure (with Python exception set).\ninline PythonPtr StringToPython(absl::string_view value) {\n  return PythonPtr(PyUnicode_FromStringAndSize(\n      value.data(), IntCast<Py_ssize_t>(value.size())));\n}\n\n// Refers to internals of a Python object representing text. Valid Python\n// objects are `str` or `bytes`. Unicode is converted to UTF-8.\nclass StrOrBytes {\n public:\n  StrOrBytes() noexcept {}\n\n  StrOrBytes(const StrOrBytes&) = delete;\n  StrOrBytes& operator=(const StrOrBytes&) = delete;\n\n  // Converts from a Python object.\n  //\n  // Returns `false` on failure (with Python exception set).\n  //\n  // Must be called at most once for each `StrOrBytes` object.\n  bool FromPython(PyObject* object ABSL_ATTRIBUTE_LIFETIME_BOUND);\n\n  // Returns the text contents.\n  /*implicit*/ operator absl::string_view() const { return data_; }\n\n private:\n  absl::string_view data_;\n};\n\n// Converts C++ `Chain` to a Python `bytes` object.\n//\n// Returns `nullptr` on failure (with Python exception set).\nPythonPtr ChainToPython(const Chain& value);\n\n// Converts a Python `bytes`-like object to C++ `Chain`, using the buffer\n// protocol.\n//\n// Returns `std::nullopt` on failure (with Python exception set).\nstd::optional<Chain> ChainFromPython(PyObject* object);\n\n// Converts C++ `size_t` to a Python `int` object.\n//\n// Returns `nullptr` on failure (with Python exception set).\nPythonPtr SizeToPython(size_t value);\n\n// Converts a Python object to C++ `size_t`. Valid Python objects are the same\n// as for slicing: `int` or objects supporting `__index__()`.\n//\n// Returns `std::nullopt` on failure (with Python exception set).\nstd::optional<size_t> SizeFromPython(PyObject* object);\n\n// Converts C++ `Position` to a Python `int` object.\n//\n// Returns `nullptr` on failure (with Python exception set).\nPythonPtr PositionToPython(Position value);\n\n// Converts a Python object to C++ `Position`. Valid Python objects are the same\n// as for slicing: `int` or objects supporting `__index__()`.\n//\n// Returns `std::nullopt` on failure (with Python exception set).\nstd::optional<Position> PositionFromPython(PyObject* object);\n\n// Converts C++ `PartialOrdering` to a Python `None` (for `unordered`) or `int`\n// object (-1 for `less`, 0 for `equivalent`, or 1 for `greater`).\n//\n// Returns `nullptr` on failure (with Python exception set).\nPythonPtr PartialOrderingToPython(PartialOrdering ordering);\n\n// Converts a Python object to C++ `PartialOrdering`. Valid Python objects are\n// `int` (compared with 0) or `None`.\n//\n// Returns `std::nullopt` on failure (with Python exception set).\nstd::optional<PartialOrdering> PartialOrderingFromPython(PyObject* object);\n\n// Implementation details follow.\n\ninline Exception::Exception(const Exception& that) noexcept { *this = that; }\n\ninline int Exception::Traverse(visitproc visit, void* arg) {\n  Py_VISIT(type_.get());\n  Py_VISIT(value_.get());\n  Py_VISIT(traceback_.get());\n  return 0;\n}\n\n}  // namespace riegeli::python\n\n#endif  // PYTHON_RIEGELI_BASE_UTILS_H_\n"
  },
  {
    "path": "python/riegeli/bytes/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//python/riegeli:__subpackages__\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"python_reader\",\n    srcs = [\"python_reader.cc\"],\n    hdrs = [\"python_reader.h\"],\n    # python_reader.cc has #define before #include to influence what the\n    # included files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \"//python/riegeli/base:utils\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:global\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:bits\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n        \"@rules_python//python/cc:current_py_cc_headers\",\n    ],\n)\n\ncc_library(\n    name = \"python_writer\",\n    srcs = [\"python_writer.cc\"],\n    hdrs = [\"python_writer.h\"],\n    # python_writer.cc has #define before #include to influence what the\n    # included files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \"//python/riegeli/base:utils\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:global\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:bits\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@rules_python//python/cc:current_py_cc_headers\",\n    ],\n)\n"
  },
  {
    "path": "python/riegeli/bytes/python_reader.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// From https://docs.python.org/3/c-api/intro.html:\n// Since Python may define some pre-processor definitions which affect the\n// standard headers on some systems, you must include Python.h before any\n// standard headers are included.\n#define PY_SSIZE_T_CLEAN\n#include <Python.h>\n// clang-format: do not reorder the above include.\n\n#include \"python/riegeli/bytes/python_reader.h\"\n// clang-format: do not reorder the above include.\n\n#include <stddef.h>\n\n#include <cstring>\n#include <limits>\n#include <optional>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/bits.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"python/riegeli/base/utils.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/global.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n\nnamespace riegeli::python {\n\nPythonReader::PythonReader(PyObject* src, Options options)\n    : BufferedReader(options.buffer_options()), owns_src_(options.owns_src()) {\n  PythonLock::AssertHeld();\n  Py_INCREF(src);\n  src_.reset(src);\n  if (options.assumed_pos() != std::nullopt) {\n    set_limit_pos(*options.assumed_pos());\n    // `supports_random_access_` is left as `false`.\n    random_access_status_ = Global([] {\n      return absl::UnimplementedError(\n          \"PythonReader::Options::assumed_pos() excludes random access\");\n    });\n  } else {\n    static constexpr Identifier id_seekable(\"seekable\");\n    const PythonPtr seekable_result(\n        PyObject_CallMethodObjArgs(src_.get(), id_seekable.get(), nullptr));\n    if (ABSL_PREDICT_FALSE(seekable_result == nullptr)) {\n      FailOperation(\"seekable()\");\n      return;\n    }\n    const int seekable_is_true = PyObject_IsTrue(seekable_result.get());\n    if (ABSL_PREDICT_FALSE(seekable_is_true < 0)) {\n      FailOperation(\"PyObject_IsTrue() after seekable()\");\n      return;\n    }\n    if (seekable_is_true == 0) {\n      // Random access is not supported. Assume 0 as the initial position.\n      // `supports_random_access_` is left as `false`.\n      random_access_status_ = Global([] {\n        return absl::UnimplementedError(\n            \"seekable() is False which excludes random access\");\n      });\n      return;\n    }\n    static constexpr Identifier id_tell(\"tell\");\n    const PythonPtr tell_result(\n        PyObject_CallMethodObjArgs(src_.get(), id_tell.get(), nullptr));\n    if (ABSL_PREDICT_FALSE(tell_result == nullptr)) {\n      FailOperation(\"tell()\");\n      return;\n    }\n    const std::optional<Position> file_pos =\n        PositionFromPython(tell_result.get());\n    if (ABSL_PREDICT_FALSE(file_pos == std::nullopt)) {\n      FailOperation(\"PositionFromPython() after tell()\");\n      return;\n    }\n    set_limit_pos(*file_pos);\n    supports_random_access_ = true;\n  }\n  BeginRun();\n}\n\nvoid PythonReader::Done() {\n  BufferedReader::Done();\n  random_access_status_ = absl::OkStatus();\n  if (owns_src_ && src_ != nullptr) {\n    PythonLock lock;\n    static constexpr Identifier id_close(\"close\");\n    const PythonPtr close_result(\n        PyObject_CallMethodObjArgs(src_.get(), id_close.get(), nullptr));\n    if (ABSL_PREDICT_FALSE(close_result == nullptr)) FailOperation(\"close()\");\n  }\n}\n\ninline bool PythonReader::FailOperation(absl::string_view operation) {\n  RIEGELI_ASSERT(is_open())\n      << \"Failed precondition of PythonReader::FailOperation(): \"\n         \"Object closed\";\n  PythonLock::AssertHeld();\n  if (ABSL_PREDICT_FALSE(!ok())) {\n    // Ignore this error because `PythonReader` already failed.\n    PyErr_Clear();\n    return false;\n  }\n  exception_ = Exception::Fetch();\n  return Fail(absl::UnknownError(\n      absl::StrCat(operation, \" failed: \", exception_.message())));\n}\n\nbool PythonReader::ReadInternal(size_t min_length, size_t max_length,\n                                char* dest) {\n  RIEGELI_ASSERT_GT(min_length, 0u)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"nothing to read\";\n  RIEGELI_ASSERT_GE(max_length, min_length)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"max_length < min_length\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::ReadInternal()\";\n  PythonLock lock;\n  // Find a read function to use, preferring in order: `readinto1()`,\n  // `readinto()`, `read1()`, `read()`.\n  if (ABSL_PREDICT_FALSE(read_function_ == nullptr)) {\n    static constexpr Identifier id_readinto1(\"readinto1\");\n    read_function_.reset(PyObject_GetAttr(src_.get(), id_readinto1.get()));\n    read_function_name_ = \"readinto1()\";\n    if (read_function_ == nullptr) {\n      if (ABSL_PREDICT_FALSE(!PyErr_ExceptionMatches(PyExc_AttributeError))) {\n        return FailOperation(read_function_name_);\n      }\n      PyErr_Clear();\n      static constexpr Identifier id_readinto(\"readinto\");\n      read_function_.reset(PyObject_GetAttr(src_.get(), id_readinto.get()));\n      read_function_name_ = \"readinto()\";\n      if (read_function_ == nullptr) {\n        if (ABSL_PREDICT_FALSE(!PyErr_ExceptionMatches(PyExc_AttributeError))) {\n          return FailOperation(read_function_name_);\n        }\n        PyErr_Clear();\n        use_bytes_ = true;\n        static constexpr Identifier id_read1(\"read1\");\n        read_function_.reset(PyObject_GetAttr(src_.get(), id_read1.get()));\n        read_function_name_ = \"read1()\";\n        if (read_function_ == nullptr) {\n          if (ABSL_PREDICT_FALSE(\n                  !PyErr_ExceptionMatches(PyExc_AttributeError))) {\n            return FailOperation(read_function_name_);\n          }\n          PyErr_Clear();\n          static constexpr Identifier id_read(\"read\");\n          read_function_.reset(PyObject_GetAttr(src_.get(), id_read.get()));\n          read_function_name_ = \"read()\";\n          if (ABSL_PREDICT_FALSE(read_function_ == nullptr)) {\n            return FailOperation(read_function_name_);\n          }\n        }\n      }\n    }\n  }\n  for (;;) {\n    if (ABSL_PREDICT_FALSE(limit_pos() ==\n                           std::numeric_limits<Position>::max())) {\n      return FailOverflow();\n    }\n    const size_t length_to_read = UnsignedMin(\n        max_length, std::numeric_limits<Position>::max() - limit_pos(),\n        absl::bit_floor(size_t{std::numeric_limits<Py_ssize_t>::max()}));\n    size_t length_read;\n    if (!use_bytes_) {\n      PythonPtr read_result;\n      {\n        // Prefer using `readinto1()` or `readinto()` to avoid copying memory.\n        MemoryView memory_view;\n        PyObject* const memory_view_object =\n            memory_view.MutableToPython(absl::MakeSpan(dest, length_to_read));\n        if (ABSL_PREDICT_FALSE(memory_view_object == nullptr)) {\n          return FailOperation(\"MemoryView::MutableToPython()\");\n        }\n        read_result.reset(PyObject_CallFunctionObjArgs(\n            read_function_.get(), memory_view_object, nullptr));\n        if (ABSL_PREDICT_FALSE(read_result == nullptr)) {\n          return FailOperation(read_function_name_);\n        }\n        if (ABSL_PREDICT_FALSE(!memory_view.Release())) {\n          return FailOperation(\"MemoryView::Release()\");\n        }\n      }\n      const std::optional<size_t> length_read_opt =\n          SizeFromPython(read_result.get());\n      if (ABSL_PREDICT_FALSE(length_read_opt == std::nullopt)) {\n        return FailOperation(\n            absl::StrCat(\"SizeFromPython() after \", read_function_name_));\n      }\n      length_read = *length_read_opt;\n      if (ABSL_PREDICT_FALSE(length_read == 0)) return false;\n      if (ABSL_PREDICT_FALSE(length_read > max_length)) {\n        return Fail(absl::InternalError(\n            absl::StrCat(read_function_name_, \" read more than requested\")));\n      }\n    } else {\n      const PythonPtr length(SizeToPython(length_to_read));\n      if (ABSL_PREDICT_FALSE(length == nullptr)) {\n        return FailOperation(\"SizeToPython()\");\n      }\n      const PythonPtr read_result(PyObject_CallFunctionObjArgs(\n          read_function_.get(), length.get(), nullptr));\n      if (ABSL_PREDICT_FALSE(read_result == nullptr)) {\n        return FailOperation(read_function_name_);\n      }\n      Py_buffer buffer;\n      if (ABSL_PREDICT_FALSE(PyObject_GetBuffer(read_result.get(), &buffer,\n                                                PyBUF_CONTIG_RO) < 0)) {\n        return FailOperation(\n            absl::StrCat(\"PyObject_GetBuffer() after \", read_function_name_));\n      }\n      if (ABSL_PREDICT_FALSE(buffer.len == 0)) {\n        PyBuffer_Release(&buffer);\n        return false;\n      }\n      if (ABSL_PREDICT_FALSE(IntCast<size_t>(buffer.len) > max_length)) {\n        PyBuffer_Release(&buffer);\n        return Fail(absl::InternalError(\n            absl::StrCat(read_function_name_, \" read more than requested\")));\n      }\n      std::memcpy(dest, buffer.buf, IntCast<size_t>(buffer.len));\n      length_read = IntCast<size_t>(buffer.len);\n      PyBuffer_Release(&buffer);\n    }\n    move_limit_pos(length_read);\n    if (length_read >= min_length) return true;\n    dest += length_read;\n    min_length -= length_read;\n    max_length -= length_read;\n  }\n}\n\nbool PythonReader::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!PythonReader::SupportsRandomAccess())) {\n    if (ABSL_PREDICT_FALSE(new_pos < start_pos())) {\n      if (ok()) Fail(random_access_status_);\n      return false;\n    }\n    return BufferedReader::SeekBehindBuffer(new_pos);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  PythonLock lock;\n  if (new_pos > limit_pos()) {\n    // Seeking forwards.\n    const std::optional<Position> size = SizeInternal();\n    if (ABSL_PREDICT_FALSE(size == std::nullopt)) return false;\n    if (ABSL_PREDICT_FALSE(new_pos > *size)) {\n      // File ends.\n      set_limit_pos(*size);\n      return false;\n    }\n  }\n  set_limit_pos(new_pos);\n  const PythonPtr file_pos = PositionToPython(limit_pos());\n  if (ABSL_PREDICT_FALSE(file_pos == nullptr)) {\n    return FailOperation(\"PositionToPython()\");\n  }\n  static constexpr Identifier id_seek(\"seek\");\n  const PythonPtr seek_result(PyObject_CallMethodObjArgs(\n      src_.get(), id_seek.get(), file_pos.get(), nullptr));\n  if (ABSL_PREDICT_FALSE(seek_result == nullptr)) {\n    return FailOperation(\"seek()\");\n  }\n  return true;\n}\n\ninline std::optional<Position> PythonReader::SizeInternal() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of PythonReader::SizeInternal()\";\n  RIEGELI_ASSERT(PythonReader::SupportsRandomAccess())\n      << \"Failed precondition of PythonReader::SizeInternal(): \"\n         \"random access not supported\";\n  PythonLock::AssertHeld();\n  absl::string_view operation;\n  const PythonPtr file_pos = PositionToPython(0);\n  if (ABSL_PREDICT_FALSE(file_pos == nullptr)) {\n    FailOperation(\"PositionToPython()\");\n    return std::nullopt;\n  }\n  const PythonPtr whence = IntToPython(2);  // `io.SEEK_END`\n  if (ABSL_PREDICT_FALSE(whence == nullptr)) {\n    FailOperation(\"IntToPython()\");\n    return std::nullopt;\n  }\n  static constexpr Identifier id_seek(\"seek\");\n  PythonPtr result(PyObject_CallMethodObjArgs(\n      src_.get(), id_seek.get(), file_pos.get(), whence.get(), nullptr));\n  if (result.get() == Py_None) {\n    // Python2 `file.seek()` returns `None`, so `tell()` is needed to get the\n    // new position. Python2 is dead, but some classes still behave like that.\n    static constexpr Identifier id_tell(\"tell\");\n    result.reset(\n        PyObject_CallMethodObjArgs(src_.get(), id_tell.get(), nullptr));\n    operation = \"tell()\";\n  } else {\n    // `io.IOBase.seek()` returns the new position.\n    operation = \"seek()\";\n  }\n  if (ABSL_PREDICT_FALSE(result == nullptr)) {\n    FailOperation(operation);\n    return std::nullopt;\n  }\n  const std::optional<Position> size = PositionFromPython(result.get());\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n    FailOperation(absl::StrCat(\"PositionFromPython() after \", operation));\n    return std::nullopt;\n  }\n  return *size;\n}\n\nstd::optional<Position> PythonReader::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!PythonReader::SupportsRandomAccess())) {\n    if (ok()) Fail(random_access_status_);\n    return std::nullopt;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  PythonLock lock;\n  const std::optional<Position> size = SizeInternal();\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) return std::nullopt;\n  const PythonPtr file_pos = PositionToPython(limit_pos());\n  if (ABSL_PREDICT_FALSE(file_pos == nullptr)) {\n    FailOperation(\"PositionToPython()\");\n    return std::nullopt;\n  }\n  static constexpr Identifier id_seek(\"seek\");\n  const PythonPtr seek_result(PyObject_CallMethodObjArgs(\n      src_.get(), id_seek.get(), file_pos.get(), nullptr));\n  if (ABSL_PREDICT_FALSE(seek_result == nullptr)) {\n    FailOperation(\"seek()\");\n    return std::nullopt;\n  }\n  return *size;\n}\n\n}  // namespace riegeli::python\n"
  },
  {
    "path": "python/riegeli/bytes/python_reader.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef PYTHON_RIEGELI_BYTES_PYTHON_READER_H_\n#define PYTHON_RIEGELI_BYTES_PYTHON_READER_H_\n\n// From https://docs.python.org/3/c-api/intro.html:\n// Since Python may define some pre-processor definitions which affect the\n// standard headers on some systems, you must include Python.h before any\n// standard headers are included.\n#include <Python.h>\n// clang-format: do not reorder the above include.\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"python/riegeli/base/utils.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n\nnamespace riegeli::python {\n\n// A `Reader` which reads from a Python binary I/O stream.\n//\n// The stream must support:\n//  * `close()`          - for `Close()` if `Options::owns_src()`\n//  * `readinto1(memoryview)` or\n//    `readinto(memoryview)` or\n//    `read1(int)` or\n//    `read(int)`\n//  * `seekable()`\n//  * `seek(int[, int])` - for `Seek()` or `Size()`\n//  * `tell()`           - for `Seek()` or `Size()`\n//\n// `PythonReader` supports random access if\n// `Options::assumed_pos() == std::nullopt` and the stream supports random\n// access (this is checked by calling `seekable()`).\n//\n// Warning: if random access is not supported and the stream is not owned,\n// it will have an unpredictable amount of extra data consumed because of\n// buffering.\nclass PythonReader : public BufferedReader {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // If `true`, `PythonReader::Close()` closes the stream.\n    //\n    // Default: `false`.\n    Options& set_owns_src(bool owns_src) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      owns_src_ = owns_src;\n      return *this;\n    }\n    Options&& set_owns_src(bool owns_src) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_owns_src(owns_src));\n    }\n    bool owns_src() const { return owns_src_; }\n\n    // If `std::nullopt`, the current position reported by `pos()` corresponds\n    // to the current stream position if possible, otherwise 0 is assumed as the\n    // initial position. Random access is supported if the stream supports\n    // random access.\n    //\n    // If not `std::nullopt`, this position is assumed initially, to be reported\n    // by `pos()`. It does not need to correspond to the current stream\n    // position. Random access is not supported.\n    //\n    // Default: `std::nullopt`.\n    Options& set_assumed_pos(std::optional<Position> assumed_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      assumed_pos_ = assumed_pos;\n      return *this;\n    }\n    Options&& set_assumed_pos(std::optional<Position> assumed_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_pos(assumed_pos));\n    }\n    std::optional<Position> assumed_pos() const { return assumed_pos_; }\n\n   private:\n    bool owns_src_ = false;\n    std::optional<Position> assumed_pos_;\n  };\n\n  // Creates a closed `PythonReader`.\n  explicit PythonReader(Closed) noexcept : BufferedReader(kClosed) {}\n\n  // Will read from `src`.\n  explicit PythonReader(PyObject* src, Options options = Options());\n\n  PythonReader(PythonReader&& that) noexcept;\n  PythonReader& operator=(PythonReader&& that) noexcept;\n\n  // Returns a borrowed reference to the stream being read from.\n  PyObject* src() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.get(); }\n\n  const Exception& exception() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return exception_;\n  }\n\n  bool ToleratesReadingAhead() override {\n    return BufferedReader::ToleratesReadingAhead() ||\n           PythonReader::SupportsRandomAccess();\n  }\n  bool SupportsRandomAccess() override { return supports_random_access_; }\n\n  // For implementing `tp_traverse` of objects containing `PythonReader`.\n  int Traverse(visitproc visit, void* arg);\n\n protected:\n  void Done() override;\n  bool ReadInternal(size_t min_length, size_t max_length, char* dest) override;\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n\n private:\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation);\n  std::optional<Position> SizeInternal();\n\n  PythonPtrLocking src_;\n  bool owns_src_ = false;\n  bool supports_random_access_ = false;\n  absl::Status random_access_status_;\n  Exception exception_;\n  PythonPtrLocking read_function_;\n  absl::string_view read_function_name_;\n  bool use_bytes_ = false;\n};\n\ninline PythonReader::PythonReader(PythonReader&& that) noexcept\n    : BufferedReader(static_cast<BufferedReader&&>(that)),\n      src_(std::move(that.src_)),\n      owns_src_(that.owns_src_),\n      supports_random_access_(\n          std::exchange(that.supports_random_access_, false)),\n      random_access_status_(std::move(that.random_access_status_)),\n      exception_(std::move(that.exception_)),\n      read_function_(std::move(that.read_function_)),\n      read_function_name_(that.read_function_name_),\n      use_bytes_(that.use_bytes_) {}\n\ninline PythonReader& PythonReader::operator=(PythonReader&& that) noexcept {\n  BufferedReader::operator=(static_cast<BufferedReader&&>(that));\n  src_ = std::move(that.src_);\n  owns_src_ = that.owns_src_;\n  supports_random_access_ = std::exchange(that.supports_random_access_, false);\n  random_access_status_ = std::move(that.random_access_status_);\n  exception_ = std::move(that.exception_);\n  read_function_ = std::move(that.read_function_);\n  read_function_name_ = that.read_function_name_;\n  use_bytes_ = that.use_bytes_;\n  return *this;\n}\n\ninline int PythonReader::Traverse(visitproc visit, void* arg) {\n  Py_VISIT(src_.get());\n  Py_VISIT(read_function_.get());\n  return exception_.Traverse(visit, arg);\n}\n\n}  // namespace riegeli::python\n\n#endif  // PYTHON_RIEGELI_BYTES_PYTHON_READER_H_\n"
  },
  {
    "path": "python/riegeli/bytes/python_writer.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// From https://docs.python.org/3/c-api/intro.html:\n// Since Python may define some pre-processor definitions which affect the\n// standard headers on some systems, you must include Python.h before any\n// standard headers are included.\n#define PY_SSIZE_T_CLEAN\n#include <Python.h>\n// clang-format: do not reorder the above include.\n\n#include \"python/riegeli/bytes/python_writer.h\"\n// clang-format: do not reorder the above include.\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/bits.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"python/riegeli/base/utils.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/global.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n\nnamespace riegeli::python {\n\nPythonWriter::PythonWriter(PyObject* dest, Options options)\n    : BufferedWriter(options.buffer_options()),\n      owns_dest_(options.owns_dest()) {\n  PythonLock::AssertHeld();\n  Py_INCREF(dest);\n  dest_.reset(dest);\n  if (options.assumed_pos() != std::nullopt) {\n    set_start_pos(*options.assumed_pos());\n    // `supports_random_access_` is left as `false`.\n    random_access_status_ = Global([] {\n      return absl::UnimplementedError(\n          \"PythonWriter::Options::assumed_pos() excludes random access\");\n    });\n  } else {\n    static constexpr Identifier id_seekable(\"seekable\");\n    const PythonPtr seekable_result(\n        PyObject_CallMethodObjArgs(dest_.get(), id_seekable.get(), nullptr));\n    if (ABSL_PREDICT_FALSE(seekable_result == nullptr)) {\n      FailOperation(\"seekable()\");\n      return;\n    }\n    const int seekable_is_true = PyObject_IsTrue(seekable_result.get());\n    if (ABSL_PREDICT_FALSE(seekable_is_true < 0)) {\n      FailOperation(\"PyObject_IsTrue() after seekable()\");\n      return;\n    }\n    if (seekable_is_true == 0) {\n      // Random access is not supported. Assume 0 as the initial position.\n      // `supports_random_access_` is left as `false`.\n      random_access_status_ = Global([] {\n        return absl::UnimplementedError(\n            \"seekable() is False which excludes random access\");\n      });\n      return;\n    }\n    static constexpr Identifier id_tell(\"tell\");\n    const PythonPtr tell_result(\n        PyObject_CallMethodObjArgs(dest_.get(), id_tell.get(), nullptr));\n    if (ABSL_PREDICT_FALSE(tell_result == nullptr)) {\n      FailOperation(\"tell()\");\n      return;\n    }\n    const std::optional<Position> file_pos =\n        PositionFromPython(tell_result.get());\n    if (ABSL_PREDICT_FALSE(file_pos == std::nullopt)) {\n      FailOperation(\"PositionFromPython() after tell()\");\n      return;\n    }\n    set_start_pos(*file_pos);\n    supports_random_access_ = true;\n  }\n  BeginRun();\n}\n\nvoid PythonWriter::Done() {\n  BufferedWriter::Done();\n  random_access_status_ = absl::OkStatus();\n  if (owns_dest_ && dest_ != nullptr) {\n    PythonLock lock;\n    static constexpr Identifier id_close(\"close\");\n    const PythonPtr close_result(\n        PyObject_CallMethodObjArgs(dest_.get(), id_close.get(), nullptr));\n    if (ABSL_PREDICT_FALSE(close_result == nullptr)) FailOperation(\"close()\");\n  }\n}\n\ninline bool PythonWriter::FailOperation(absl::string_view operation) {\n  RIEGELI_ASSERT(is_open())\n      << \"Failed precondition of PythonWriter::FailOperation(): \"\n         \"Object closed\";\n  PythonLock::AssertHeld();\n  if (ABSL_PREDICT_FALSE(!ok())) {\n    // Ignore this error because `PythonWriter` already failed.\n    PyErr_Clear();\n    return false;\n  }\n  exception_ = Exception::Fetch();\n  return Fail(absl::UnknownError(\n      absl::StrCat(operation, \" failed: \", exception_.message())));\n}\n\nbool PythonWriter::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of BufferedWriter::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedWriter::WriteInternal()\";\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  PythonLock lock;\n  if (ABSL_PREDICT_FALSE(write_function_ == nullptr)) {\n    static constexpr Identifier id_write(\"write\");\n    write_function_.reset(PyObject_GetAttr(dest_.get(), id_write.get()));\n    if (ABSL_PREDICT_FALSE(write_function_ == nullptr)) {\n      return FailOperation(\"write()\");\n    }\n  }\n  do {\n    const size_t length_to_write = UnsignedMin(\n        src.size(),\n        absl::bit_floor(size_t{std::numeric_limits<Py_ssize_t>::max()}));\n    size_t length_written;\n    {\n      PythonPtr write_result;\n      if (!use_bytes_) {\n        // Prefer passing a `memoryview` to avoid copying memory.\n        MemoryView memory_view;\n        PyObject* const memory_view_object = memory_view.ToPython(\n            absl::string_view(src.data(), length_to_write));\n        if (ABSL_PREDICT_FALSE(memory_view_object == nullptr)) {\n          return FailOperation(\"MemoryView::ToPython()\");\n        }\n        write_result.reset(PyObject_CallFunctionObjArgs(\n            write_function_.get(), memory_view_object, nullptr));\n        if (ABSL_PREDICT_FALSE(write_result == nullptr)) {\n          if (!PyErr_ExceptionMatches(PyExc_TypeError)) {\n            return FailOperation(\"write()\");\n          }\n          PyErr_Clear();\n          use_bytes_ = true;\n        }\n        if (ABSL_PREDICT_FALSE(!memory_view.Release())) {\n          return FailOperation(\"MemoryView::Release()\");\n        }\n      }\n      if (use_bytes_) {\n        // `write()` does not support `memoryview`. Use `bytes`.\n        const PythonPtr bytes = BytesToPython(src.substr(0, length_to_write));\n        if (ABSL_PREDICT_FALSE(bytes == nullptr)) {\n          return FailOperation(\"BytesToPython()\");\n        }\n        write_result.reset(PyObject_CallFunctionObjArgs(write_function_.get(),\n                                                        bytes.get(), nullptr));\n        if (ABSL_PREDICT_FALSE(write_result == nullptr)) {\n          return FailOperation(\"write()\");\n        }\n      }\n      if (write_result.get() == Py_None) {\n        // Python2 `file.write()` returns `None`, and would raise an exception\n        // if less than the full length had been written. Python2 is dead, but\n        // some classes still behave like that.\n        length_written = length_to_write;\n      } else {\n        // `io.IOBase.write()` returns the length written.\n        const std::optional<size_t> length_written_opt =\n            SizeFromPython(write_result.get());\n        if (ABSL_PREDICT_FALSE(length_written_opt == std::nullopt)) {\n          return FailOperation(\"SizeFromPython() after write()\");\n        }\n        length_written = *length_written_opt;\n      }\n    }\n    if (ABSL_PREDICT_FALSE(length_written > length_to_write)) {\n      return Fail(absl::InternalError(\"write() wrote more than requested\"));\n    }\n    move_start_pos(length_written);\n    src.remove_prefix(length_written);\n  } while (!src.empty());\n  return true;\n}\n\nbool PythonWriter::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!BufferedWriter::FlushImpl(flush_type))) return false;\n  switch (flush_type) {\n    case FlushType::kFromObject:\n      if (!owns_dest_) return true;\n      ABSL_FALLTHROUGH_INTENDED;\n    case FlushType::kFromProcess:\n    case FlushType::kFromMachine:\n      PythonLock lock;\n      static constexpr Identifier id_flush(\"flush\");\n      const PythonPtr flush_result(\n          PyObject_CallMethodObjArgs(dest_.get(), id_flush.get(), nullptr));\n      if (ABSL_PREDICT_FALSE(flush_result == nullptr)) {\n        return FailOperation(\"flush()\");\n      }\n      return true;\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown flush type: \" << static_cast<int>(flush_type);\n}\n\nbool PythonWriter::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of BufferedWriter::SeekBehindBuffer(): \"\n         \"position unchanged, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!PythonWriter::SupportsRandomAccess())) {\n    if (ok()) Fail(random_access_status_);\n    return false;\n  }\n  PythonLock lock;\n  if (new_pos > start_pos()) {\n    // Seeking forwards.\n    const std::optional<Position> size = SizeInternal();\n    if (ABSL_PREDICT_FALSE(size == std::nullopt)) return false;\n    if (ABSL_PREDICT_FALSE(new_pos > *size)) {\n      // File ends.\n      set_start_pos(*size);\n      return false;\n    }\n  }\n  set_start_pos(new_pos);\n  const PythonPtr file_pos = PositionToPython(start_pos());\n  if (ABSL_PREDICT_FALSE(file_pos == nullptr)) {\n    return FailOperation(\"PositionToPython()\");\n  }\n  static constexpr Identifier id_seek(\"seek\");\n  const PythonPtr seek_result(PyObject_CallMethodObjArgs(\n      dest_.get(), id_seek.get(), file_pos.get(), nullptr));\n  if (ABSL_PREDICT_FALSE(seek_result == nullptr)) {\n    return FailOperation(\"seek()\");\n  }\n  return true;\n}\n\ninline std::optional<Position> PythonWriter::SizeInternal() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of PythonWriter::SizeInternal()\";\n  RIEGELI_ASSERT(PythonWriter::SupportsRandomAccess())\n      << \"Failed precondition of PythonWriter::SizeInternal(): \"\n         \"random access not supported\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of PythonWriter::SizeInternal(): \"\n         \"buffer not empty\";\n  PythonLock::AssertHeld();\n  absl::string_view operation;\n  const PythonPtr file_pos = PositionToPython(0);\n  if (ABSL_PREDICT_FALSE(file_pos == nullptr)) {\n    FailOperation(\"PositionToPython()\");\n    return std::nullopt;\n  }\n  const PythonPtr whence = IntToPython(2);  // `io.SEEK_END`\n  if (ABSL_PREDICT_FALSE(whence == nullptr)) {\n    FailOperation(\"IntToPython()\");\n    return std::nullopt;\n  }\n  static constexpr Identifier id_seek(\"seek\");\n  PythonPtr result(PyObject_CallMethodObjArgs(\n      dest_.get(), id_seek.get(), file_pos.get(), whence.get(), nullptr));\n  if (result.get() == Py_None) {\n    // Python2 `file.seek()` returns `None`. Python2 is dead, but some classes\n    // still behave like that.\n    static constexpr Identifier id_tell(\"tell\");\n    result.reset(\n        PyObject_CallMethodObjArgs(dest_.get(), id_tell.get(), nullptr));\n    operation = \"tell()\";\n  } else {\n    // `io.IOBase.seek()` returns the new position.\n    operation = \"seek()\";\n  }\n  if (ABSL_PREDICT_FALSE(result == nullptr)) {\n    FailOperation(operation);\n    return std::nullopt;\n  }\n  const std::optional<Position> size = PositionFromPython(result.get());\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n    FailOperation(absl::StrCat(\"PositionFromPython() after \", operation));\n    return std::nullopt;\n  }\n  return *size;\n}\n\nstd::optional<Position> PythonWriter::SizeBehindBuffer() {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::SizeBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!PythonWriter::SupportsRandomAccess())) {\n    if (ok()) Fail(random_access_status_);\n    return std::nullopt;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  PythonLock lock;\n  const std::optional<Position> size = SizeInternal();\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) return std::nullopt;\n  const PythonPtr file_pos = PositionToPython(start_pos());\n  if (ABSL_PREDICT_FALSE(file_pos == nullptr)) {\n    FailOperation(\"PositionToPython()\");\n    return std::nullopt;\n  }\n  static constexpr Identifier id_seek(\"seek\");\n  const PythonPtr seek_result(PyObject_CallMethodObjArgs(\n      dest_.get(), id_seek.get(), file_pos.get(), nullptr));\n  if (ABSL_PREDICT_FALSE(seek_result == nullptr)) {\n    FailOperation(\"seek()\");\n    return std::nullopt;\n  }\n  return *size;\n}\n\nbool PythonWriter::TruncateBehindBuffer(Position new_size) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::TruncateBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!PythonWriter::SupportsRandomAccess())) {\n    if (ok()) Fail(random_access_status_);\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  PythonLock lock;\n  const std::optional<Position> size = SizeInternal();\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) return false;\n  if (ABSL_PREDICT_FALSE(new_size > *size)) {\n    // File ends.\n    set_start_pos(*size);\n    return false;\n  }\n  {\n    const PythonPtr file_pos = PositionToPython(new_size);\n    if (ABSL_PREDICT_FALSE(file_pos == nullptr)) {\n      return FailOperation(\"PositionToPython()\");\n    }\n    static constexpr Identifier id_seek(\"seek\");\n    const PythonPtr seek_result(PyObject_CallMethodObjArgs(\n        dest_.get(), id_seek.get(), file_pos.get(), nullptr));\n    if (ABSL_PREDICT_FALSE(seek_result == nullptr)) {\n      return FailOperation(\"seek()\");\n    }\n  }\n  set_start_pos(new_size);\n  static constexpr Identifier id_truncate(\"truncate\");\n  const PythonPtr truncate_result(\n      PyObject_CallMethodObjArgs(dest_.get(), id_truncate.get(), nullptr));\n  if (ABSL_PREDICT_FALSE(truncate_result == nullptr)) {\n    return FailOperation(\"truncate()\");\n  }\n  return true;\n}\n\n}  // namespace riegeli::python\n"
  },
  {
    "path": "python/riegeli/bytes/python_writer.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef PYTHON_RIEGELI_BYTES_PYTHON_WRITER_H_\n#define PYTHON_RIEGELI_BYTES_PYTHON_WRITER_H_\n\n// From https://docs.python.org/3/c-api/intro.html:\n// Since Python may define some pre-processor definitions which affect the\n// standard headers on some systems, you must include Python.h before any\n// standard headers are included.\n#include <Python.h>\n// clang-format: do not reorder the above include.\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"python/riegeli/base/utils.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n\nnamespace riegeli::python {\n\n// A `Writer` which writes to a Python binary I/O stream.\n//\n// The stream must support:\n//  * `close()`          - for `Close()` if `Options::owns_dest()`\n//  * `write(bytes)`\n//  * `flush()`          - for `Flush()`\n//  * `seekable()`\n//  * `seek(int[, int])` - for `Seek()`, `Size()`, or `Truncate()`\n//  * `tell()`           - for `Seek()`, `Size()`, or `Truncate()`\n//  * `truncate()`       - for `Truncate()`\n//\n// `PythonWriter` supports random access if\n// `Options::assumed_pos() == std::nullopt` and the stream supports random\n// access (this is checked by calling `seekable()`).\nclass PythonWriter : public BufferedWriter {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // If `true`, `PythonWriter::Close()` closes the stream, and\n    // `PythonWriter::Flush(flush_type)` flushes the stream even if `flush_type`\n    // is `FlushType::kFromObject`.\n    //\n    // Default: `false`.\n    Options& set_owns_dest(bool owns_dest) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      owns_dest_ = owns_dest;\n      return *this;\n    }\n    Options&& set_owns_dest(bool owns_dest) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_owns_dest(owns_dest));\n    }\n    bool owns_dest() const { return owns_dest_; }\n\n    // If `std::nullopt`, the current position reported by `pos()` corresponds\n    // to the current stream position if possible, otherwise 0 is assumed as the\n    // initial position. Random access is supported if the stream supports\n    // random access.\n    //\n    // If not `std::nullopt`, this position is assumed initially, to be reported\n    // by `pos()`. It does not need to correspond to the current stream\n    // position. Random access is not supported.\n    //\n    // Default: `std::nullopt`.\n    Options& set_assumed_pos(std::optional<Position> assumed_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      assumed_pos_ = assumed_pos;\n      return *this;\n    }\n    Options&& set_assumed_pos(std::optional<Position> assumed_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_pos(assumed_pos));\n    }\n    std::optional<Position> assumed_pos() const { return assumed_pos_; }\n\n   private:\n    bool owns_dest_ = false;\n    std::optional<Position> assumed_pos_;\n  };\n\n  // Creates a closed `PythonWriter`.\n  explicit PythonWriter(Closed) noexcept : BufferedWriter(kClosed) {}\n\n  // Will write to `dest`.\n  explicit PythonWriter(PyObject* dest, Options options = Options());\n\n  PythonWriter(PythonWriter&& that) noexcept;\n  PythonWriter& operator=(PythonWriter&& that) noexcept;\n\n  // Returns a borrowed reference to the stream being written to.\n  PyObject* dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.get(); }\n\n  const Exception& exception() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return exception_;\n  }\n\n  bool SupportsRandomAccess() override { return supports_random_access_; }\n\n  // For implementing `tp_traverse` of objects containing `PythonWriter`.\n  int Traverse(visitproc visit, void* arg);\n\n protected:\n  void Done() override;\n  bool WriteInternal(absl::string_view src) override;\n  bool FlushImpl(FlushType flush_type) override;\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::optional<Position> SizeBehindBuffer() override;\n  bool TruncateBehindBuffer(Position new_size) override;\n\n private:\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation);\n  std::optional<Position> SizeInternal();\n\n  PythonPtrLocking dest_;\n  bool owns_dest_ = false;\n  bool supports_random_access_ = false;\n  absl::Status random_access_status_;\n  Exception exception_;\n  PythonPtrLocking write_function_;\n  bool use_bytes_ = false;\n};\n\ninline PythonWriter::PythonWriter(PythonWriter&& that) noexcept\n    : BufferedWriter(static_cast<BufferedWriter&&>(that)),\n      dest_(std::move(that.dest_)),\n      owns_dest_(that.owns_dest_),\n      supports_random_access_(\n          std::exchange(that.supports_random_access_, false)),\n      random_access_status_(std::move(that.random_access_status_)),\n      exception_(std::move(that.exception_)),\n      write_function_(std::move(that.write_function_)),\n      use_bytes_(that.use_bytes_) {}\n\ninline PythonWriter& PythonWriter::operator=(PythonWriter&& that) noexcept {\n  BufferedWriter::operator=(static_cast<BufferedWriter&&>(that));\n  dest_ = std::move(that.dest_);\n  owns_dest_ = that.owns_dest_;\n  supports_random_access_ = std::exchange(that.supports_random_access_, false);\n  random_access_status_ = std::move(that.random_access_status_);\n  exception_ = std::move(that.exception_);\n  write_function_ = std::move(that.write_function_);\n  use_bytes_ = that.use_bytes_;\n  return *this;\n}\n\ninline int PythonWriter::Traverse(visitproc visit, void* arg) {\n  Py_VISIT(dest_.get());\n  Py_VISIT(write_function_.get());\n  return exception_.Traverse(visit, arg);\n}\n\n}  // namespace riegeli::python\n\n#endif  // PYTHON_RIEGELI_BYTES_PYTHON_WRITER_H_\n"
  },
  {
    "path": "python/riegeli/py_extension.bzl",
    "content": "\"\"\"Supports writing Python modules in C++.\"\"\"\n\nload(\"@rules_cc//cc:defs.bzl\", \"cc_binary\", \"cc_library\")\nload(\"@rules_python//python:defs.bzl\", \"py_library\")\n\ndef py_extension(\n        name = None,\n        srcs = None,\n        hdrs = None,\n        data = None,\n        features = None,\n        visibility = None,\n        deps = None):\n    \"\"\"Creates a Python module implemented in C++.\n\n    Python modules can depend on a py_extension. Other py_extensions can depend\n    on a generated C++ library named with \"_cc\" suffix.\n\n    Args:\n      name: Name for this target.\n      srcs: C++ source files.\n      hdrs: C++ header files, for other py_extensions which depend on this.\n      data: Files needed at runtime. This may include Python libraries.\n      features: Passed to cc_library.\n      visibility: Controls which rules can depend on this.\n      deps: Other C++ libraries that this library depends upon.\n    \"\"\"\n\n    cc_library_name = name + \"_cc\"\n    cc_binary_name = name + \".so\"\n    cc_library(\n        name = cc_library_name,\n        srcs = srcs,\n        hdrs = hdrs,\n        data = data,\n        features = features,\n        visibility = visibility,\n        deps = deps,\n        alwayslink = True,\n    )\n    cc_binary(\n        name = cc_binary_name,\n        linkshared = True,\n        linkstatic = True,\n        visibility = [\"//visibility:private\"],\n        deps = [cc_library_name],\n    )\n\n    py_library(\n        name = name,\n        data = [cc_binary_name],\n        visibility = visibility,\n    )\n"
  },
  {
    "path": "python/riegeli/python_configure.bzl",
    "content": "\"\"\"Repository rule for Python autoconfiguration.\n\n`python_configure` depends on the following environment variables:\n\n  * `PYTHON_BIN_PATH`: location of python binary.\n  * `PYTHON_LIB_PATH`: Location of python libraries.\n\"\"\"\n\n_BAZEL_SH = \"BAZEL_SH\"\n_PYTHON_BIN_PATH = \"PYTHON_BIN_PATH\"\n_PYTHON_LIB_PATH = \"PYTHON_LIB_PATH\"\n_TF_PYTHON_CONFIG_REPO = \"TF_PYTHON_CONFIG_REPO\"\n\ndef _tpl(repository_ctx, tpl, substitutions = {}, out = None):\n    if not out:\n        out = tpl\n    repository_ctx.template(\n        out,\n        Label(\"//python/riegeli:{}.tpl\".format(tpl)),\n        substitutions,\n    )\n\ndef _fail(msg):\n    \"\"\"Outputs failure message when auto configuration fails.\"\"\"\n    red = \"\\033[0;31m\"\n    no_color = \"\\033[0m\"\n    fail(\"{}Python Configuration Error:{} {}\\n\".format(red, no_color, msg))\n\ndef _is_windows(repository_ctx):\n    \"\"\"Returns true if the host operating system is Windows.\"\"\"\n    os_name = repository_ctx.os.name.lower()\n    return \"windows\" in os_name\n\ndef _execute(\n        repository_ctx,\n        cmdline,\n        error_msg = None,\n        error_details = None,\n        empty_stdout_fine = False):\n    \"\"\"Executes an arbitrary shell command.\n\n    Args:\n      repository_ctx: the repository_ctx object\n      cmdline: list of strings, the command to execute\n      error_msg: string, a summary of the error if the command fails\n      error_details: string, details about the error or steps to fix it\n      empty_stdout_fine: bool, if True, an empty stdout result is fine,\n        otherwise it's an error\n    Return:\n      the result of repository_ctx.execute(cmdline)\n    \"\"\"\n    result = repository_ctx.execute(cmdline)\n    if result.stderr or not (empty_stdout_fine or result.stdout):\n        _fail(\"\\n\".join([\n            error_msg.strip() if error_msg else \"Repository command failed\",\n            result.stderr.strip(),\n            error_details if error_details else \"\",\n        ]))\n    return result\n\ndef _read_dir(repository_ctx, src_dir):\n    \"\"\"Returns a string with all files in a directory.\n\n    Finds all files inside a directory, traversing subfolders and following\n    symlinks. The returned string contains the full path of all files\n    separated by line breaks.\n    \"\"\"\n    if _is_windows(repository_ctx):\n        src_dir = src_dir.replace(\"/\", \"\\\\\")\n        find_result = _execute(\n            repository_ctx,\n            [\"cmd.exe\", \"/c\", \"dir\", src_dir, \"/b\", \"/s\", \"/a-d\"],\n            empty_stdout_fine = True,\n        )\n\n        # src_files will be used in genrule.outs where the paths must\n        # use forward slashes.\n        result = find_result.stdout.replace(\"\\\\\", \"/\")\n    else:\n        find_result = _execute(\n            repository_ctx,\n            [\"find\", src_dir, \"-follow\", \"-type\", \"f\"],\n            empty_stdout_fine = True,\n        )\n        result = find_result.stdout\n    return result\n\ndef _genrule(src_dir, genrule_name, command, outs):\n    \"\"\"Returns a string with a genrule.\n\n    Genrule executes the given command and produces the given outputs.\n    \"\"\"\n    return (\n        \"genrule(\\n\" +\n        '    name = \"{}\",\\n' +\n        \"    outs = [\\n\" +\n        \"{}\\n\" +\n        \"    ],\\n\" +\n        '    cmd = \"\"\"\\n' +\n        \"{}\\n\" +\n        '   \"\"\",\\n' +\n        \")\\n\"\n    ).format(genrule_name, outs, command)\n\ndef _norm_path(path):\n    \"\"\"Returns a path with '/' and removes the trailing slash.\"\"\"\n    return path.replace(\"\\\\\", \"/\").rstrip(\"/\")\n\ndef _symlink_genrule_for_dir(\n        repository_ctx,\n        src_dir,\n        dest_dir,\n        genrule_name,\n        src_files = [],\n        dest_files = []):\n    \"\"\"Returns a genrule to symlink (or copy if on Windows) a set of files.\n\n    If src_dir is passed, files will be read from the given directory; otherwise\n    we assume files are in src_files and dest_files\n    \"\"\"\n    if src_dir != None:\n        src_dir = _norm_path(src_dir)\n        dest_dir = _norm_path(dest_dir)\n        files = \"\\n\".join(\n            sorted(_read_dir(repository_ctx, src_dir).splitlines()),\n        )\n\n        # Create a list with the src_dir stripped to use for outputs.\n        dest_files = files.replace(src_dir, \"\").splitlines()\n        src_files = files.splitlines()\n    command = []\n    outs = []\n    for i in range(len(dest_files)):\n        if dest_files[i] != \"\":\n            # If we have only one file to link we do not want to use the\n            # dest_dir, as $(@D) will include the full path to the file.\n            dest = \"$(@D)/{}{}\".format(\n                dest_dir if len(dest_files) != 1 else \"\",\n                dest_files[i],\n            )\n\n            # Copy the headers to create a sandboxable setup.\n            cmd = \"cp -f\"\n            command.append('{} \"{}\" \"{}\"'.format(cmd, src_files[i], dest))\n            outs.append('        \"{}{}\",'.format(dest_dir, dest_files[i]))\n    genrule = _genrule(\n        src_dir,\n        genrule_name,\n        \" && \".join(command),\n        \"\\n\".join(outs),\n    )\n    return genrule\n\ndef _get_python_bin(repository_ctx):\n    \"\"\"Gets the python bin path.\"\"\"\n    python_bin = repository_ctx.os.environ.get(_PYTHON_BIN_PATH)\n    if python_bin != None:\n        return python_bin\n    python_bin_path = repository_ctx.which(\"python\")\n    if python_bin_path != None:\n        return str(python_bin_path)\n    _fail((\"Cannot find python in PATH, please make sure \" +\n           \"python is installed and add its directory in PATH, \" +\n           \"or --define {}='/something/else'.\\nPATH={}\").format(\n        _PYTHON_BIN_PATH,\n        repository_ctx.os.environ.get(\"PATH\", \"\"),\n    ))\n\ndef _get_bash_bin(repository_ctx):\n    \"\"\"Gets the bash bin path.\"\"\"\n    bash_bin = repository_ctx.os.environ.get(_BAZEL_SH)\n    if bash_bin != None:\n        return bash_bin\n    bash_bin_path = repository_ctx.which(\"bash\")\n    if bash_bin_path != None:\n        return str(bash_bin_path)\n    _fail((\"Cannot find bash in PATH, please make sure \" +\n           \"bash is installed and add its directory in PATH, \" +\n           \"or --define {}='/path/to/bash'.\\nPATH={}\").format(\n        _BAZEL_SH,\n        repository_ctx.os.environ.get(\"PATH\", \"\"),\n    ))\n\ndef _get_python_runtime_pair(repository_ctx, python_bin):\n    \"\"\"Builds a py_runtime_pair definition.\"\"\"\n    return (\n        \"py_runtime_pair(\\n\" +\n        '    name = \"py_runtime_pair\",\\n' +\n        \"    py2_runtime = None,\\n\" +\n        \"    py3_runtime = \\\":py3_runtime\\\",\\n\" +\n        \")\\n\" +\n        \"\\n\" +\n        \"py_runtime(\\n\" +\n        '    name = \"py3_runtime\",\\n' +\n        '    interpreter_path = \"{}\",\\n' +\n        '    python_version = \"PY3\",\\n' +\n        \")\\n\"\n    ).format(python_bin)\n\ndef _get_python_lib(repository_ctx, python_bin):\n    \"\"\"Gets the python lib path.\"\"\"\n    python_lib = repository_ctx.os.environ.get(_PYTHON_LIB_PATH)\n    if python_lib != None:\n        return python_lib\n    print_lib = (\"<<END\\n\" +\n                 \"import site\\n\" +\n                 \"import os\\n\" +\n                 \"\\n\" +\n                 \"try:\\n\" +\n                 \"  input = raw_input\\n\" +\n                 \"except NameError:\\n\" +\n                 \"  pass\\n\" +\n                 \"\\n\" +\n                 \"python_paths = []\\n\" +\n                 \"if os.getenv('PYTHONPATH') is not None:\\n\" +\n                 \"  python_paths = os.getenv('PYTHONPATH').split(':')\\n\" +\n                 \"try:\\n\" +\n                 \"  library_paths = site.getsitepackages()\\n\" +\n                 \"except AttributeError:\\n\" +\n                 \"  from distutils.sysconfig import get_python_lib\\n\" +\n                 \"  library_paths = [get_python_lib()]\\n\" +\n                 \"all_paths = set(python_paths + library_paths)\\n\" +\n                 \"paths = []\\n\" +\n                 \"for path in all_paths:\\n\" +\n                 \"  if os.path.isdir(path):\\n\" +\n                 \"    paths.append(path)\\n\" +\n                 \"if len(paths) >= 1:\\n\" +\n                 \"  print(paths[0])\\n\" +\n                 \"END\")\n    cmd = \"{} - {}\".format(python_bin, print_lib)\n    result = repository_ctx.execute([_get_bash_bin(repository_ctx), \"-c\", cmd])\n    return result.stdout.strip(\"\\n\")\n\ndef _check_python_lib(repository_ctx, python_lib):\n    \"\"\"Checks the python lib path.\"\"\"\n    cmd = 'test -d \"{}\" -a -x \"{}\"'.format(python_lib, python_lib)\n    result = repository_ctx.execute([_get_bash_bin(repository_ctx), \"-c\", cmd])\n    if result.return_code == 1:\n        _fail(\"Invalid python library path: {}\".format(python_lib))\n\ndef _check_python_bin(repository_ctx, python_bin):\n    \"\"\"Checks the python bin path.\"\"\"\n    cmd = '[[ -x \"{}\" ]] && [[ ! -d \"{}\" ]]'.format(python_bin, python_bin)\n    result = repository_ctx.execute([_get_bash_bin(repository_ctx), \"-c\", cmd])\n    if result.return_code == 1:\n        _fail((\"--define {}='{}' is not executable. \" +\n               \"Is it the python binary?\").format(\n            _PYTHON_BIN_PATH,\n            python_bin,\n        ))\n\ndef _get_python_include(repository_ctx, python_bin):\n    \"\"\"Gets the python include path.\"\"\"\n    result = _execute(\n        repository_ctx,\n        [\n            python_bin,\n            \"-c\",\n            \"import importlib; \" +\n            \"import importlib.util; \" +\n            \"print(importlib.import_module('distutils.sysconfig').get_python_inc() \" +\n            \"if importlib.util.find_spec('distutils.sysconfig') \" +\n            \"else importlib.import_module('sysconfig').get_path('include'))\",\n        ],\n        error_msg = \"Problem getting python include path.\",\n        error_details = (\"Is the Python binary path set up right? \" +\n                         \"(See ./configure or {}.) \" +\n                         \"Is distutils installed?\").format(_PYTHON_BIN_PATH),\n    )\n    return result.stdout.splitlines()[0]\n\ndef _get_python_import_lib_name(repository_ctx, python_bin):\n    \"\"\"Gets Python import library name (pythonXY.lib) on Windows.\"\"\"\n    result = _execute(\n        repository_ctx,\n        [\n            python_bin,\n            \"-c\",\n            \"import sys; \" +\n            'print(\"python{}{}.lib\".format(' +\n            \"sys.version_info.major, sys.version_info.minor))\",\n        ],\n        error_msg = \"Problem getting python import library.\",\n        error_details = (\"Is the Python binary path set up right? \" +\n                         \"(See ./configure or {}.) \").format(_PYTHON_BIN_PATH),\n    )\n    return result.stdout.splitlines()[0]\n\ndef _get_numpy_include(repository_ctx, python_bin):\n    \"\"\"Gets the numpy include path.\"\"\"\n    return _execute(\n        repository_ctx,\n        [\n            python_bin,\n            \"-c\",\n            \"import numpy; print(numpy.get_include())\",\n        ],\n        error_msg = \"Problem getting numpy include path.\",\n        error_details = \"Is numpy installed?\",\n    ).stdout.splitlines()[0]\n\ndef _create_local_python_repository(repository_ctx):\n    \"\"\"Creates the repository containing files set up to build with Python.\"\"\"\n    python_bin = _get_python_bin(repository_ctx)\n    _check_python_bin(repository_ctx, python_bin)\n    python_runtime_pair = _get_python_runtime_pair(repository_ctx, python_bin)\n    python_lib = _get_python_lib(repository_ctx, python_bin)\n    _check_python_lib(repository_ctx, python_lib)\n    python_include = _get_python_include(repository_ctx, python_bin)\n    numpy_include = _get_numpy_include(repository_ctx, python_bin) + \"/numpy\"\n    python_include_rule = _symlink_genrule_for_dir(\n        repository_ctx,\n        python_include,\n        \"python_include\",\n        \"python_include\",\n    )\n    python_import_lib_genrule = \"\"\n\n    # To build Python C/C++ extension on Windows, we need to link to python\n    # import library pythonXY.lib\n    # See https://docs.python.org/3/extending/windows.html\n    if _is_windows(repository_ctx):\n        python_include = _norm_path(python_include)\n        python_import_lib_name = _get_python_import_lib_name(\n            repository_ctx,\n            python_bin,\n        )\n        python_import_lib_src = \"{}/libs/{}\".format(\n            python_include.rsplit(\"/\", 1)[0],\n            python_import_lib_name,\n        )\n        python_import_lib_genrule = _symlink_genrule_for_dir(\n            repository_ctx,\n            None,\n            \"\",\n            \"python_import_lib\",\n            [python_import_lib_src],\n            [python_import_lib_name],\n        )\n    numpy_include_rule = _symlink_genrule_for_dir(\n        repository_ctx,\n        numpy_include,\n        \"numpy_include/numpy\",\n        \"numpy_include\",\n    )\n    _tpl(repository_ctx, \"BUILD\", {\n        \"%{PYTHON_RUNTIME_PAIR}\": python_runtime_pair,\n        \"%{PYTHON_INCLUDE_GENRULE}\": python_include_rule,\n        \"%{PYTHON_IMPORT_LIB_GENRULE}\": python_import_lib_genrule,\n        \"%{NUMPY_INCLUDE_GENRULE}\": numpy_include_rule,\n    })\n\ndef _create_remote_python_repository(repository_ctx, remote_config_repo):\n    \"\"\"Creates pointers to a remotely configured repo set up to build with Python.\n    \"\"\"\n    repository_ctx.template(\"BUILD\", Label(remote_config_repo + \":BUILD\"), {})\n\ndef _python_autoconf_impl(repository_ctx):\n    \"\"\"Implementation of the python_autoconf repository rule.\"\"\"\n    if _TF_PYTHON_CONFIG_REPO in repository_ctx.os.environ:\n        _create_remote_python_repository(\n            repository_ctx,\n            repository_ctx.os.environ[_TF_PYTHON_CONFIG_REPO],\n        )\n    else:\n        _create_local_python_repository(repository_ctx)\n\npython_configure = repository_rule(\n    implementation = _python_autoconf_impl,\n    environ = [\n        _BAZEL_SH,\n        _PYTHON_BIN_PATH,\n        _PYTHON_LIB_PATH,\n        _TF_PYTHON_CONFIG_REPO,\n    ],\n)\n\"\"\"Detects and configures the local Python.\n\nAdd the following to your WORKSPACE FILE:\n\n```python\npython_configure(name = \"local_config_python\")\n```\n\nArgs:\n  name: A unique name for this workspace rule.\n\"\"\"\n"
  },
  {
    "path": "python/riegeli/records/BUILD",
    "content": "load(\"@com_google_protobuf//bazel:proto_library.bzl\", \"proto_library\")\nload(\"@com_google_protobuf//bazel:py_proto_library.bzl\", \"py_proto_library\")\nload(\"@rules_python//python:defs.bzl\", \"py_library\")\nload(\"//python/riegeli:py_extension.bzl\", \"py_extension\")\n\npackage(\n    default_visibility = [\"//python/riegeli:__subpackages__\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\npy_extension(\n    name = \"record_reader\",\n    srcs = [\"record_reader.cc\"],\n    # Python modules imported from C++.\n    data = [\n        \":records_metadata_py_pb2\",\n        \":skipped_region\",\n        \"@com_google_protobuf//:protobuf_python\",\n    ],\n    # record_reader.cc has #define before #include to influence what the\n    # included files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \":record_position_cc\",\n        \"//python/riegeli/base:utils\",\n        \"//python/riegeli/bytes:python_reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:types\",\n        \"//riegeli/chunk_encoding:field_projection\",\n        \"//riegeli/records:record_position\",\n        \"//riegeli/records:record_reader\",\n        \"//riegeli/records:skipped_region\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@rules_python//python/cc:current_py_cc_headers\",\n    ],\n)\n\npy_extension(\n    name = \"record_writer\",\n    srcs = [\"record_writer.cc\"],\n    # record_writer.cc has #define before #include to influence what the\n    # included files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \":record_position_cc\",\n        \"//python/riegeli/base:utils\",\n        \"//python/riegeli/bytes:python_writer\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:types\",\n        \"//riegeli/records:record_writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@rules_python//python/cc:current_py_cc_headers\",\n    ],\n)\n\npy_extension(\n    name = \"record_position\",\n    srcs = [\"record_position.cc\"],\n    hdrs = [\"record_position.h\"],\n    # record_position.cc has #define before #include to influence what the\n    # included files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \"//python/riegeli/base:utils\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:types\",\n        \"//riegeli/records:record_position\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/hash\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@rules_python//python/cc:current_py_cc_headers\",\n    ],\n)\n\npy_library(\n    name = \"skipped_region\",\n    srcs = [\"skipped_region.py\"],\n)\n\nproto_library(\n    name = \"records_metadata_proto\",\n    srcs = [\"records_metadata.proto\"],\n    deps = [\"@com_google_protobuf//:descriptor_proto\"],\n)\n\npy_proto_library(\n    name = \"records_metadata_py_pb2\",\n    deps = [\":records_metadata_proto\"],\n)\n"
  },
  {
    "path": "python/riegeli/records/__init__.py",
    "content": ""
  },
  {
    "path": "python/riegeli/records/examples/BUILD",
    "content": "load(\"@rules_python//python:defs.bzl\", \"py_binary\")\n\npackage(features = [\"header_modules\"])\n\nlicenses([\"notice\"])\n\npy_binary(\n    name = \"write_read_records\",\n    srcs = [\"write_read_records.py\"],\n    deps = [\n        \"//python/riegeli\",\n        \"//python/riegeli/records/tests:records_test_py_pb2\",\n    ],\n)\n"
  },
  {
    "path": "python/riegeli/records/examples/__init__.py",
    "content": ""
  },
  {
    "path": "python/riegeli/records/examples/write_read_records.py",
    "content": "# Copyright 2019 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\"\"\"Simple example which writes and reads a Riegeli/records file.\"\"\"\n\nimport io\n\nimport riegeli\nfrom riegeli.records.tests import records_test_pb2\n\n\ndef sample_string(i, size):\n  piece = f'{i} '.encode()\n  result = piece * -(-size // len(piece))  # len(result) >= size\n  return result[:size]\n\n\ndef sample_message(i, size):\n  return records_test_pb2.SimpleMessage(id=i, payload=sample_string(i, size))\n\n\ndef write_records(filename):\n  print('Writing', filename)\n  metadata = riegeli.RecordsMetadata()\n  riegeli.set_record_type(metadata, records_test_pb2.SimpleMessage)\n  with riegeli.RecordWriter(\n      io.FileIO(filename, mode='wb'), options='transpose', metadata=metadata\n  ) as writer:\n    writer.write_messages(sample_message(i, 100) for i in range(100))\n\n\ndef read_records(filename):\n  print('Reading', filename)\n  with riegeli.RecordReader(\n      io.FileIO(filename, mode='rb'),\n      field_projection=[[\n          records_test_pb2.SimpleMessage.DESCRIPTOR.fields_by_name['id'].number\n      ]],\n  ) as reader:\n    print(\n        ' '.join(\n            str(record.id)\n            for record in reader.read_messages(records_test_pb2.SimpleMessage)\n        )\n    )\n\n\ndef main():\n  filename = '/tmp/riegeli_example'\n  write_records(filename)\n  read_records(filename)\n\n\nif __name__ == '__main__':\n  main()\n"
  },
  {
    "path": "python/riegeli/records/record_position.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// From https://docs.python.org/3/c-api/intro.html:\n// Since Python may define some pre-processor definitions which affect the\n// standard headers on some systems, you must include Python.h before any\n// standard headers are included.\n#define PY_SSIZE_T_CLEAN\n#include <Python.h>\n// clang-format: do not reorder the above include.\n\n#include \"python/riegeli/records/record_position.h\"\n// clang-format: do not reorder the above include.\n\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/hash/hash.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"python/riegeli/base/utils.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/records/record_position.h\"\n\nnamespace riegeli::python {\n\nnamespace {\n\nstruct PyRecordPositionObject {\n  // clang-format off\n  PyObject_HEAD\n  static_assert(true, \"\");  // clang-format workaround.\n  // clang-format on\n\n  PythonWrapped<FutureRecordPosition> record_position;\n};\n\nextern PyTypeObject PyRecordPosition_Type;\n\n// `extern \"C\"` sets the C calling convention for compatibility with the Python\n// API. `static` avoids making symbols public, as `extern \"C\"` trumps anonymous\n// namespace.\nextern \"C\" {\n\nstatic void RecordPositionDestructor(PyRecordPositionObject* self) {\n  PythonUnlocked([&] { self->record_position.reset(); });\n  Py_TYPE(self)->tp_free(self);\n}\n\nstatic PyRecordPositionObject* RecordPositionNew(PyTypeObject* cls,\n                                                 PyObject* args,\n                                                 PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"chunk_begin\", \"record_index\",\n                                             nullptr};\n  PyObject* chunk_begin_arg;\n  PyObject* record_index_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"OO:RecordPosition\", const_cast<char**>(keywords),\n          &chunk_begin_arg, &record_index_arg))) {\n    return nullptr;\n  }\n  const std::optional<Position> chunk_begin =\n      PositionFromPython(chunk_begin_arg);\n  if (ABSL_PREDICT_FALSE(chunk_begin == std::nullopt)) return nullptr;\n  const std::optional<Position> record_index =\n      PositionFromPython(record_index_arg);\n  if (ABSL_PREDICT_FALSE(record_index == std::nullopt)) return nullptr;\n  if (ABSL_PREDICT_FALSE(*chunk_begin > std::numeric_limits<uint64_t>::max()) ||\n      ABSL_PREDICT_FALSE(*record_index >\n                         std::numeric_limits<uint64_t>::max() - *chunk_begin)) {\n    PyErr_Format(PyExc_OverflowError, \"RecordPosition overflow: %llu/%llu\",\n                 static_cast<unsigned long long>(*chunk_begin),\n                 static_cast<unsigned long long>(*record_index));\n    return nullptr;\n  }\n  std::unique_ptr<PyRecordPositionObject, Deleter> self(\n      reinterpret_cast<PyRecordPositionObject*>(cls->tp_alloc(cls, 0)));\n  if (ABSL_PREDICT_FALSE(self == nullptr)) return nullptr;\n  self->record_position.emplace(RecordPosition(\n      IntCast<uint64_t>(*chunk_begin), IntCast<uint64_t>(*record_index)));\n  return self.release();\n}\n\nstatic PyObject* RecordPositionChunkBegin(PyRecordPositionObject* self,\n                                          void* closure) {\n  const RecordPosition pos =\n      PythonUnlocked([&] { return self->record_position->get(); });\n  return PositionToPython(pos.chunk_begin()).release();\n}\n\nstatic PyObject* RecordPositionRecordIndex(PyRecordPositionObject* self,\n                                           void* closure) {\n  const RecordPosition pos =\n      PythonUnlocked([&] { return self->record_position->get(); });\n  return PositionToPython(pos.record_index()).release();\n}\n\nstatic PyObject* RecordPositionNumeric(PyRecordPositionObject* self,\n                                       void* closure) {\n  const RecordPosition pos =\n      PythonUnlocked([&] { return self->record_position->get(); });\n  return PositionToPython(pos.numeric()).release();\n}\n\nstatic PyObject* RecordPositionCompare(PyObject* a, PyObject* b, int op) {\n  if (ABSL_PREDICT_FALSE(!PyObject_TypeCheck(a, &PyRecordPosition_Type)) ||\n      ABSL_PREDICT_FALSE(!PyObject_TypeCheck(b, &PyRecordPosition_Type))) {\n    Py_INCREF(Py_NotImplemented);\n    return Py_NotImplemented;\n  }\n  RecordPosition a_pos;\n  RecordPosition b_pos;\n  PythonUnlocked([&] {\n    a_pos =\n        reinterpret_cast<PyRecordPositionObject*>(a)->record_position->get();\n    b_pos =\n        reinterpret_cast<PyRecordPositionObject*>(b)->record_position->get();\n  });\n  switch (op) {\n    case Py_EQ:\n      return PyBool_FromLong(a_pos == b_pos);\n    case Py_NE:\n      return PyBool_FromLong(a_pos != b_pos);\n    case Py_LT:\n      return PyBool_FromLong(a_pos < b_pos);\n    case Py_GT:\n      return PyBool_FromLong(a_pos > b_pos);\n    case Py_LE:\n      return PyBool_FromLong(a_pos <= b_pos);\n    case Py_GE:\n      return PyBool_FromLong(a_pos >= b_pos);\n    default:\n      Py_INCREF(Py_NotImplemented);\n      return Py_NotImplemented;\n  }\n}\n\nstatic Py_hash_t RecordPositionHash(PyRecordPositionObject* self) {\n  const RecordPosition pos =\n      PythonUnlocked([&] { return self->record_position->get(); });\n  Py_hash_t hash = static_cast<Py_hash_t>(absl::Hash<RecordPosition>()(pos));\n  if (ABSL_PREDICT_FALSE(hash == -1)) hash = -2;\n  return hash;\n}\n\nstatic PyObject* RecordPositionStr(PyRecordPositionObject* self) {\n  const RecordPosition pos =\n      PythonUnlocked([&] { return self->record_position->get(); });\n  return StringToPython(pos.ToString()).release();\n}\n\nstatic PyRecordPositionObject* RecordPositionFromStr(PyTypeObject* cls,\n                                                     PyObject* args,\n                                                     PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"serialized\", nullptr};\n  PyObject* serialized_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:from_str\", const_cast<char**>(keywords),\n          &serialized_arg))) {\n    return nullptr;\n  }\n  StrOrBytes serialized;\n  if (ABSL_PREDICT_FALSE(!serialized.FromPython(serialized_arg))) {\n    return nullptr;\n  }\n  RecordPosition pos;\n  if (ABSL_PREDICT_FALSE(!pos.FromString(serialized))) {\n    PyErr_SetString(PyExc_ValueError, \"RecordPosition.from_str() failed\");\n    return nullptr;\n  }\n  std::unique_ptr<PyRecordPositionObject, Deleter> self(\n      reinterpret_cast<PyRecordPositionObject*>(cls->tp_alloc(cls, 0)));\n  if (ABSL_PREDICT_FALSE(self == nullptr)) return nullptr;\n  self->record_position.emplace(pos);\n  return self.release();\n}\n\nstatic PyObject* RecordPositionRepr(PyRecordPositionObject* self) {\n  const RecordPosition pos =\n      PythonUnlocked([&] { return self->record_position->get(); });\n  return StringToPython(absl::StrCat(\"RecordPosition(\", pos.chunk_begin(), \", \",\n                                     pos.record_index(), \")\"))\n      .release();\n}\n\nstatic PyObject* RecordPositionToBytes(PyRecordPositionObject* self,\n                                       PyObject* args) {\n  const RecordPosition pos =\n      PythonUnlocked([&] { return self->record_position->get(); });\n  return BytesToPython(pos.ToBytes()).release();\n}\n\nstatic PyRecordPositionObject* RecordPositionFromBytes(PyTypeObject* cls,\n                                                       PyObject* args,\n                                                       PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"serialized\", nullptr};\n  PyObject* serialized_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:from_bytes\", const_cast<char**>(keywords),\n          &serialized_arg))) {\n    return nullptr;\n  }\n  BytesLike serialized;\n  if (ABSL_PREDICT_FALSE(!serialized.FromPython(serialized_arg))) {\n    return nullptr;\n  }\n  RecordPosition pos;\n  if (ABSL_PREDICT_FALSE(!pos.FromBytes(serialized))) {\n    PyErr_SetString(PyExc_ValueError, \"RecordPosition.from_bytes() failed\");\n    return nullptr;\n  }\n  std::unique_ptr<PyRecordPositionObject, Deleter> self(\n      reinterpret_cast<PyRecordPositionObject*>(cls->tp_alloc(cls, 0)));\n  if (ABSL_PREDICT_FALSE(self == nullptr)) return nullptr;\n  self->record_position.emplace(pos);\n  return self.release();\n}\n\n}  // extern \"C\"\n\nconst PyMethodDef RecordPositionMethods[] = {\n    {\"from_str\", reinterpret_cast<PyCFunction>(RecordPositionFromStr),\n     METH_VARARGS | METH_KEYWORDS | METH_CLASS,\n     R\"doc(\nfrom_str(type, serialized: str | bytes) -> RecordPosition\n\nParses RecordPosition from its text format.\n\nArgs:\n  serialized: Text string to parse.\n)doc\"},\n    {\"to_bytes\", reinterpret_cast<PyCFunction>(RecordPositionToBytes),\n     METH_NOARGS,\n     R\"doc(\nto_bytes(self) -> bytes\n\nReturns the RecordPosition serialized to its binary format.\n\nSerialized byte strings have the same natural order as the corresponding\npositions.\n)doc\"},\n    {\"from_bytes\", reinterpret_cast<PyCFunction>(RecordPositionFromBytes),\n     METH_VARARGS | METH_KEYWORDS | METH_CLASS, R\"doc(\nfrom_bytes(\n    type, serialized: bytes | bytearray | memoryview) -> RecordPosition\n\nParses RecordPosition from its binary format.\n\nSerialized byte strings have the same natural order as the corresponding\npositions.\n\nArgs:\n  serialized: Byte string to parse.\n)doc\"},\n    {nullptr, nullptr, 0, nullptr},\n};\n\nconst PyGetSetDef RecordPositionGetSet[] = {\n    {const_cast<char*>(\"chunk_begin\"),\n     reinterpret_cast<getter>(RecordPositionChunkBegin), nullptr,\n     const_cast<char*>(R\"doc(\nchunk_begin: int\n\nFile position of the beginning of the chunk containing the given record.\n)doc\"),\n     nullptr},\n    {const_cast<char*>(\"record_index\"),\n     reinterpret_cast<getter>(RecordPositionRecordIndex), nullptr,\n     const_cast<char*>(R\"doc(\nrecord_index: int\n\nIndex of the record within the chunk.\n)doc\"),\n     nullptr},\n    {const_cast<char*>(\"numeric\"),\n     reinterpret_cast<getter>(RecordPositionNumeric), nullptr,\n     const_cast<char*>(R\"doc(\nnumeric: int\n\nConverts RecordPosition to an integer scaled between 0 and file size.\n\nDistinct RecordPositions of a valid file have distinct numeric values.\n)doc\"),\n     nullptr},\n    {nullptr, nullptr, nullptr, nullptr, nullptr}};\n\nPyTypeObject PyRecordPosition_Type = {\n    // clang-format off\n    PyVarObject_HEAD_INIT(&PyType_Type, 0)\n    // clang-format on\n    \"riegeli.records.record_position.RecordPosition\",        // tp_name\n    sizeof(PyRecordPositionObject),                          // tp_basicsize\n    0,                                                       // tp_itemsize\n    reinterpret_cast<destructor>(RecordPositionDestructor),  // tp_dealloc\n#if PY_VERSION_HEX >= 0x03080000\n    0,  // tp_vectorcall_offset\n#else\n    nullptr,  // tp_print\n#endif\n    nullptr,                                          // tp_getattr\n    nullptr,                                          // tp_setattr\n    nullptr,                                          // tp_as_async\n    reinterpret_cast<reprfunc>(RecordPositionRepr),   // tp_repr\n    nullptr,                                          // tp_as_number\n    nullptr,                                          // tp_as_sequence\n    nullptr,                                          // tp_as_mapping\n    reinterpret_cast<hashfunc>(RecordPositionHash),   // tp_hash\n    nullptr,                                          // tp_call\n    reinterpret_cast<reprfunc>(RecordPositionStr),    // tp_str\n    nullptr,                                          // tp_getattro\n    nullptr,                                          // tp_setattro\n    nullptr,                                          // tp_as_buffer\n    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,         // tp_flags\n    R\"doc(\nRecordPosition(chunk_begin: int, record_index: int) -> RecordPosition\n\nRepresents a position in a Riegeli/records file.\n\nThere are two ways of expressing positions, both strictly monotonic:\n * RecordPosition - Faster for seeking.\n * int            - Scaled between 0 and file size.\n\nRecordPosition can be converted to int by the numeric property.\n\nWorking with RecordPosition is recommended, unless it is needed to seek to an\napproximate position interpolated along the file, e.g. for splitting the file\ninto shards, or unless the position must be expressed as an integer from the\nrange [0, file_size] in order to fit into a preexisting API.\n\nBoth RecordReader and RecordWriter return positions. A position from\nRecordWriter can act as a future: accessing its contents for the first time\nmight block, waiting for pending operations to complete.\n)doc\",                                                // tp_doc\n    nullptr,                                          // tp_traverse\n    nullptr,                                          // tp_clear\n    RecordPositionCompare,                            // tp_richcompare\n    0,                                                // tp_weaklistoffset\n    nullptr,                                          // tp_iter\n    nullptr,                                          // tp_iternext\n    const_cast<PyMethodDef*>(RecordPositionMethods),  // tp_methods\n    nullptr,                                          // tp_members\n    const_cast<PyGetSetDef*>(RecordPositionGetSet),   // tp_getset\n    nullptr,                                          // tp_base\n    nullptr,                                          // tp_dict\n    nullptr,                                          // tp_descr_get\n    nullptr,                                          // tp_descr_set\n    0,                                                // tp_dictoffset\n    nullptr,                                          // tp_init\n    nullptr,                                          // tp_alloc\n    reinterpret_cast<newfunc>(RecordPositionNew),     // tp_new\n    nullptr,                                          // tp_free\n    nullptr,                                          // tp_is_gc\n    nullptr,                                          // tp_bases\n    nullptr,                                          // tp_mro\n    nullptr,                                          // tp_cache\n    nullptr,                                          // tp_subclasses\n    nullptr,                                          // tp_weaklist\n    nullptr,                                          // tp_del\n    0,                                                // tp_version_tag\n    nullptr,                                          // tp_finalize\n};\n\nPythonPtr RecordPositionToPython(FutureRecordPosition value) {\n  PythonPtr self(PyRecordPosition_Type.tp_alloc(&PyRecordPosition_Type, 0));\n  if (ABSL_PREDICT_FALSE(self == nullptr)) return nullptr;\n  reinterpret_cast<PyRecordPositionObject*>(self.get())\n      ->record_position.emplace(std::move(value));\n  return self;\n}\n\nstd::optional<RecordPosition> RecordPositionFromPython(PyObject* object) {\n  if (ABSL_PREDICT_FALSE(!PyObject_TypeCheck(object, &PyRecordPosition_Type))) {\n    PyErr_Format(PyExc_TypeError, \"Expected RecordPosition, not %s\",\n                 Py_TYPE(object)->tp_name);\n    return std::nullopt;\n  }\n  return PythonUnlocked([&] {\n    return reinterpret_cast<PyRecordPositionObject*>(object)\n        ->record_position->get();\n  });\n}\n\nconst char* const kModuleName = \"riegeli.records.record_position\";\nconst char kModuleDoc[] =\n    R\"doc(Represents a position in a Riegeli/records file.)doc\";\n\nPyModuleDef kModuleDef = {\n    PyModuleDef_HEAD_INIT,\n    kModuleName,  // m_name\n    kModuleDoc,   // m_doc\n    -1,           // m_size\n    nullptr,      // m_methods\n    nullptr,      // m_slots\n    nullptr,      // m_traverse\n    nullptr,      // m_clear\n    nullptr,      // m_free\n};\n\nPyObject* InitModule() {\n  if (ABSL_PREDICT_FALSE(PyType_Ready(&PyRecordPosition_Type) < 0)) {\n    return nullptr;\n  }\n  PythonPtr module(PyModule_Create(&kModuleDef));\n  if (ABSL_PREDICT_FALSE(module == nullptr)) return nullptr;\n  Py_INCREF(&PyRecordPosition_Type);\n  if (ABSL_PREDICT_FALSE(PyModule_AddObject(module.get(), \"RecordPosition\",\n                                            reinterpret_cast<PyObject*>(\n                                                &PyRecordPosition_Type)) < 0)) {\n    return nullptr;\n  }\n  static constexpr RecordPositionApi kRecordPositionApi = {\n      RecordPositionToPython,\n      RecordPositionFromPython,\n  };\n  if (ABSL_PREDICT_FALSE(!ExportCapsule(\n          module.get(), kRecordPositionCapsuleName, &kRecordPositionApi))) {\n    return nullptr;\n  }\n  return module.release();\n}\n\n}  // namespace\n\nPyMODINIT_FUNC PyInit_record_position() { return InitModule(); }\n\n}  // namespace riegeli::python\n"
  },
  {
    "path": "python/riegeli/records/record_position.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef PYTHON_RIEGELI_RECORDS_RECORD_POSITION_H_\n#define PYTHON_RIEGELI_RECORDS_RECORD_POSITION_H_\n\n// From https://docs.python.org/3/c-api/intro.html:\n// Since Python may define some pre-processor definitions which affect the\n// standard headers on some systems, you must include Python.h before any\n// standard headers are included.\n#include <Python.h>\n// clang-format: do not reorder the above include.\n\n#include <optional>\n\n#include \"python/riegeli/base/utils.h\"\n#include \"riegeli/records/record_position.h\"\n\nnamespace riegeli::python {\n\n// Access the API thus:\n// ```\n// static constexpr ImportedCapsule<RecordPositionApi> kRecordPositionApi(\n//    kRecordPositionCapsuleName);\n// ```\n\nstruct RecordPositionApi {\n  PythonPtr (*RecordPositionToPython)(FutureRecordPosition value);\n  std::optional<RecordPosition> (*RecordPositionFromPython)(PyObject* object);\n};\n\ninline constexpr const char* kRecordPositionCapsuleName =\n    \"riegeli.records.record_position._CPPAPI\";\n\n}  // namespace riegeli::python\n\n#endif  // PYTHON_RIEGELI_RECORDS_RECORD_POSITION_H_\n"
  },
  {
    "path": "python/riegeli/records/record_reader.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// From https://docs.python.org/3/c-api/intro.html:\n// Since Python may define some pre-processor definitions which affect the\n// standard headers on some systems, you must include Python.h before any\n// standard headers are included.\n#define PY_SSIZE_T_CLEAN\n#include <Python.h>\n// clang-format: do not reorder the above include.\n\n#include <stddef.h>\n\n#include <functional>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"python/riegeli/base/utils.h\"\n#include \"python/riegeli/bytes/python_reader.h\"\n#include \"python/riegeli/records/record_position.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/chunk_encoding/field_projection.h\"\n#include \"riegeli/records/record_position.h\"\n#include \"riegeli/records/record_reader.h\"\n#include \"riegeli/records/skipped_region.h\"\n\nnamespace riegeli::python {\n\nnamespace {\n\nconstexpr ImportedCapsule<RecordPositionApi> kRecordPositionApi(\n    kRecordPositionCapsuleName);\n\n// `extern \"C\"` sets the C calling convention for compatibility with the Python\n// API. `static` avoids making symbols public, as `extern \"C\"` trumps anonymous\n// namespace.\nextern \"C\" {\n\nstatic PyObject* GetRecordType(PyObject* self, PyObject* args,\n                               PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"metadata\", nullptr};\n  PyObject* metadata_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:get_record_type\", const_cast<char**>(keywords),\n          &metadata_arg))) {\n    return nullptr;\n  }\n  // record_type_name = metadata.record_type_name\n  static constexpr Identifier id_record_type_name(\"record_type_name\");\n  const PythonPtr record_type_name(\n      PyObject_GetAttr(metadata_arg, id_record_type_name.get()));\n  if (ABSL_PREDICT_FALSE(record_type_name == nullptr)) return nullptr;\n  // if not record_type_name: return None\n  const int record_type_name_is_true = PyObject_IsTrue(record_type_name.get());\n  if (ABSL_PREDICT_FALSE(record_type_name_is_true < 0)) return nullptr;\n  if (record_type_name_is_true == 0) Py_RETURN_NONE;\n  // file_descriptors = metadata.file_descriptor\n  static constexpr Identifier id_file_descriptor(\"file_descriptor\");\n  const PythonPtr file_descriptors(\n      PyObject_GetAttr(metadata_arg, id_file_descriptor.get()));\n  if (ABSL_PREDICT_FALSE(file_descriptors == nullptr)) return nullptr;\n  // if not file_descriptors: return None\n  const int file_descriptors_is_true = PyObject_IsTrue(file_descriptors.get());\n  if (ABSL_PREDICT_FALSE(file_descriptors_is_true < 0)) return nullptr;\n  if (file_descriptors_is_true == 0) Py_RETURN_NONE;\n  // pool = DescriptorPool()\n  static constexpr ImportedConstant kDescriptorPool(\n      \"google.protobuf.descriptor_pool\", \"DescriptorPool\");\n  if (ABSL_PREDICT_FALSE(!kDescriptorPool.Verify())) return nullptr;\n  const PythonPtr pool(\n      PyObject_CallFunctionObjArgs(kDescriptorPool.get(), nullptr));\n  if (ABSL_PREDICT_FALSE(pool == nullptr)) return nullptr;\n  // for file_descriptor in file_descriptors:\n  //   pool.Add(file_descriptor)\n  const PythonPtr iter(PyObject_GetIter(file_descriptors.get()));\n  if (ABSL_PREDICT_FALSE(iter == nullptr)) return nullptr;\n  while (const PythonPtr file_descriptor{PyIter_Next(iter.get())}) {\n    static constexpr Identifier id_Add(\"Add\");\n    const PythonPtr add_result(PyObject_CallMethodObjArgs(\n        pool.get(), id_Add.get(), file_descriptor.get(), nullptr));\n    if (ABSL_PREDICT_FALSE(add_result == nullptr)) return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(PyErr_Occurred() != nullptr)) return nullptr;\n  // message_descriptor = pool.FindMessageTypeByName(record_type_name)\n  static constexpr Identifier id_FindMessageTypeByName(\"FindMessageTypeByName\");\n  const PythonPtr message_descriptor(\n      PyObject_CallMethodObjArgs(pool.get(), id_FindMessageTypeByName.get(),\n                                 record_type_name.get(), nullptr));\n  if (ABSL_PREDICT_FALSE(message_descriptor == nullptr)) return nullptr;\n  // return GetMessageClass(message_descriptor)\n  const PythonPtr message_factory(\n      PyImport_ImportModule(\"google.protobuf.message_factory\"));\n  if (ABSL_PREDICT_FALSE(message_factory == nullptr)) return nullptr;\n  static constexpr Identifier id_GetMessageClass(\"GetMessageClass\");\n  return PyObject_CallMethodObjArgs(message_factory.get(),\n                                    id_GetMessageClass.get(),\n                                    message_descriptor.get(), nullptr);\n}\n\n}  // extern \"C\"\n\nstruct PyRecordReaderObject {\n  // clang-format off\n  PyObject_HEAD\n  static_assert(true, \"\");  // clang-format workaround.\n  // clang-format on\n\n  PythonWrapped<RecordReader<PythonReader>> record_reader;\n  PyObject* recovery;\n  PythonWrapped<Exception> recovery_exception;\n};\n\nextern PyTypeObject PyRecordReader_Type;\n\nstruct PyRecordIterObject {\n  // clang-format off\n  PyObject_HEAD\n  static_assert(true, \"\");  // clang-format workaround.\n  // clang-format on\n\n  PyObject* (*read_record)(PyRecordReaderObject* self, PyObject* args);\n  PyRecordReaderObject* record_reader;\n  PyObject* args;\n};\n\nextern PyTypeObject PyRecordIter_Type;\n\nbool RecordReaderHasException(PyRecordReaderObject* self) {\n  return self->recovery_exception.has_value() || !self->record_reader->ok();\n}\n\nvoid SetExceptionFromRecordReader(PyRecordReaderObject* self) {\n  if (self->recovery_exception.has_value()) {\n    self->recovery_exception->Restore();\n    return;\n  }\n  RIEGELI_ASSERT(!self->record_reader->ok())\n      << \"Failed precondition of SetExceptionFromRecordReader(): \"\n         \"RecordReader OK\";\n  if (!self->record_reader->src().exception().ok()) {\n    self->record_reader->src().exception().Restore();\n    return;\n  }\n  SetRiegeliError(self->record_reader->status());\n}\n\nstd::optional<int> VerifyFieldNumber(long field_number_value) {\n  static_assert(Field::kExistenceOnly == 0,\n                \"VerifyFieldNumber() assumes that Field::kExistenceOnly == 0\");\n  if (ABSL_PREDICT_FALSE(field_number_value < Field::kExistenceOnly ||\n                         field_number_value > (1 << 29) - 1)) {\n    PyErr_Format(PyExc_OverflowError, \"Field number out of range: %ld\",\n                 field_number_value);\n    return std::nullopt;\n  }\n  return IntCast<int>(field_number_value);\n}\n\nstd::optional<int> FieldNumberFromPython(PyObject* object) {\n  if (ABSL_PREDICT_FALSE(!PyLong_Check(object))) {\n    PyErr_Format(PyExc_TypeError, \"Expected int, not %s\",\n                 Py_TYPE(object)->tp_name);\n    return std::nullopt;\n  }\n  const long field_number_value = PyLong_AsLong(object);\n  if (ABSL_PREDICT_FALSE(field_number_value == -1) && PyErr_Occurred()) {\n    return std::nullopt;\n  }\n  return VerifyFieldNumber(field_number_value);\n}\n\nstd::optional<FieldProjection> FieldProjectionFromPython(PyObject* object) {\n  FieldProjection field_projection;\n  const PythonPtr field_iter(PyObject_GetIter(object));\n  if (ABSL_PREDICT_FALSE(field_iter == nullptr)) return std::nullopt;\n  while (const PythonPtr field_object{PyIter_Next(field_iter.get())}) {\n    Field field;\n    const PythonPtr field_number_iter(PyObject_GetIter(field_object.get()));\n    if (ABSL_PREDICT_FALSE(field_number_iter == nullptr)) return std::nullopt;\n    while (const PythonPtr field_number_object{\n        PyIter_Next(field_number_iter.get())}) {\n      const std::optional<int> field_number =\n          FieldNumberFromPython(field_number_object.get());\n      if (ABSL_PREDICT_FALSE(field_number == std::nullopt)) return std::nullopt;\n      field.AddFieldNumber(*field_number);\n    }\n    if (ABSL_PREDICT_FALSE(PyErr_Occurred() != nullptr)) return std::nullopt;\n    field_projection.AddField(std::move(field));\n  }\n  if (ABSL_PREDICT_FALSE(PyErr_Occurred() != nullptr)) return std::nullopt;\n  return field_projection;\n}\n\n// `extern \"C\"` sets the C calling convention for compatibility with the Python\n// API. `static` avoids making symbols public, as `extern \"C\"` trumps anonymous\n// namespace.\nextern \"C\" {\n\nstatic void RecordReaderDestructor(PyRecordReaderObject* self) {\n  PyObject_GC_UnTrack(self);\n#if PY_VERSION_HEX < 0x030D0000  // < 3.13\n  Py_TRASHCAN_BEGIN(self, RecordReaderDestructor);\n#endif\n  PythonUnlocked([&] { self->record_reader.reset(); });\n  Py_XDECREF(self->recovery);\n  self->recovery_exception.reset();\n  Py_TYPE(self)->tp_free(self);\n#if PY_VERSION_HEX < 0x030D0000  // < 3.13\n  Py_TRASHCAN_END;\n#endif\n}\n\nstatic int RecordReaderTraverse(PyRecordReaderObject* self, visitproc visit,\n                                void* arg) {\n  Py_VISIT(self->recovery);\n  if (self->recovery_exception.has_value()) {\n    const int recovery_exception_result =\n        self->recovery_exception->Traverse(visit, arg);\n    if (ABSL_PREDICT_FALSE(recovery_exception_result != 0)) {\n      return recovery_exception_result;\n    }\n  }\n  if (self->record_reader.has_value()) {\n    return self->record_reader->src().Traverse(visit, arg);\n  }\n  return 0;\n}\n\nstatic int RecordReaderClear(PyRecordReaderObject* self) {\n  PythonUnlocked([&] { self->record_reader.reset(); });\n  Py_CLEAR(self->recovery);\n  self->recovery_exception.reset();\n  return 0;\n}\n\nstatic int RecordReaderInit(PyRecordReaderObject* self, PyObject* args,\n                            PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"src\",\n                                             \"owns_src\",\n                                             \"assumed_pos\",\n                                             \"min_buffer_size\",\n                                             \"max_buffer_size\",\n                                             \"buffer_size\",\n                                             \"field_projection\",\n                                             \"recovery\",\n                                             nullptr};\n  PyObject* src_arg;\n  PyObject* owns_src_arg = nullptr;\n  PyObject* assumed_pos_arg = nullptr;\n  PyObject* min_buffer_size_arg = nullptr;\n  PyObject* max_buffer_size_arg = nullptr;\n  PyObject* buffer_size_arg = nullptr;\n  PyObject* field_projection_arg = nullptr;\n  PyObject* recovery_arg = nullptr;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O|$OOOOOOO:RecordReader\", const_cast<char**>(keywords),\n          &src_arg, &owns_src_arg, &assumed_pos_arg, &min_buffer_size_arg,\n          &max_buffer_size_arg, &buffer_size_arg, &field_projection_arg,\n          &recovery_arg))) {\n    return -1;\n  }\n\n  PythonReader::Options python_reader_options;\n  python_reader_options.set_owns_src(true);\n  if (owns_src_arg != nullptr) {\n    const int owns_src_is_true = PyObject_IsTrue(owns_src_arg);\n    if (ABSL_PREDICT_FALSE(owns_src_is_true < 0)) return -1;\n    python_reader_options.set_owns_src(owns_src_is_true != 0);\n  }\n  if (assumed_pos_arg != nullptr && assumed_pos_arg != Py_None) {\n    const std::optional<Position> assumed_pos =\n        PositionFromPython(assumed_pos_arg);\n    if (ABSL_PREDICT_FALSE(assumed_pos == std::nullopt)) return -1;\n    python_reader_options.set_assumed_pos(*assumed_pos);\n  }\n  if (buffer_size_arg != nullptr && buffer_size_arg != Py_None) {\n    min_buffer_size_arg = buffer_size_arg;\n    max_buffer_size_arg = buffer_size_arg;\n  }\n  if (min_buffer_size_arg != nullptr) {\n    const std::optional<size_t> min_buffer_size =\n        SizeFromPython(min_buffer_size_arg);\n    if (ABSL_PREDICT_FALSE(min_buffer_size == std::nullopt)) return -1;\n    python_reader_options.set_min_buffer_size(*min_buffer_size);\n  }\n  if (max_buffer_size_arg != nullptr) {\n    const std::optional<size_t> max_buffer_size =\n        SizeFromPython(max_buffer_size_arg);\n    if (ABSL_PREDICT_FALSE(max_buffer_size == std::nullopt)) return -1;\n    python_reader_options.set_max_buffer_size(*max_buffer_size);\n  }\n\n  RecordReaderBase::Options record_reader_options;\n  if (field_projection_arg != nullptr && field_projection_arg != Py_None) {\n    std::optional<FieldProjection> field_projection =\n        FieldProjectionFromPython(field_projection_arg);\n    if (ABSL_PREDICT_FALSE(field_projection == std::nullopt)) return -1;\n    record_reader_options.set_field_projection(*std::move(field_projection));\n  }\n  if (recovery_arg != nullptr && recovery_arg != Py_None) {\n    Py_INCREF(recovery_arg);\n    Py_XDECREF(self->recovery);\n    self->recovery = recovery_arg;\n    record_reader_options.set_recovery([self](\n                                           const SkippedRegion& skipped_region,\n                                           RecordReaderBase& record_reader) {\n      PythonLock lock;\n      const PythonPtr begin_object = PositionToPython(skipped_region.begin());\n      if (ABSL_PREDICT_FALSE(begin_object == nullptr)) {\n        self->recovery_exception.emplace(Exception::Fetch());\n        return false;\n      }\n      const PythonPtr end_object = PositionToPython(skipped_region.end());\n      if (ABSL_PREDICT_FALSE(end_object == nullptr)) {\n        self->recovery_exception.emplace(Exception::Fetch());\n        return false;\n      }\n      const PythonPtr message_object = StringToPython(skipped_region.message());\n      if (ABSL_PREDICT_FALSE(message_object == nullptr)) {\n        self->recovery_exception.emplace(Exception::Fetch());\n        return false;\n      }\n      static constexpr ImportedConstant kSkippedRegion(\n          \"riegeli.records.skipped_region\", \"SkippedRegion\");\n      if (ABSL_PREDICT_FALSE(!kSkippedRegion.Verify())) {\n        self->recovery_exception.emplace(Exception::Fetch());\n        return false;\n      }\n      const PythonPtr skipped_region_object(PyObject_CallFunctionObjArgs(\n          kSkippedRegion.get(), begin_object.get(), end_object.get(),\n          message_object.get(), nullptr));\n      if (ABSL_PREDICT_FALSE(skipped_region_object == nullptr)) {\n        self->recovery_exception.emplace(Exception::Fetch());\n        return false;\n      }\n      const PythonPtr recovery_result(PyObject_CallFunctionObjArgs(\n          self->recovery, skipped_region_object.get(), nullptr));\n      if (ABSL_PREDICT_FALSE(recovery_result == nullptr)) {\n        if (PyErr_ExceptionMatches(PyExc_StopIteration)) {\n          PyErr_Clear();\n        } else {\n          self->recovery_exception.emplace(Exception::Fetch());\n        }\n        return false;\n      }\n      return true;\n    });\n  }\n\n  PythonReader python_reader(src_arg, std::move(python_reader_options));\n  PythonUnlocked([&] {\n    self->record_reader.emplace(std::move(python_reader),\n                                std::move(record_reader_options));\n  });\n  if (ABSL_PREDICT_FALSE(!self->record_reader->ok())) {\n    self->record_reader->src().Close();\n    SetExceptionFromRecordReader(self);\n    return -1;\n  }\n  return 0;\n}\n\nstatic PyObject* RecordReaderSrc(PyRecordReaderObject* self, void* closure) {\n  PyObject* const src = ABSL_PREDICT_FALSE(!self->record_reader.has_value())\n                            ? Py_None\n                            : self->record_reader->src().src();\n  Py_INCREF(src);\n  return src;\n}\n\nstatic PyObject* RecordReaderRepr(PyRecordReaderObject* self) {\n  const PythonPtr format = StringToPython(\"<RecordReader src={!r}>\");\n  if (ABSL_PREDICT_FALSE(format == nullptr)) return nullptr;\n  // return format.format(self.src)\n  PyObject* const src = ABSL_PREDICT_FALSE(!self->record_reader.has_value())\n                            ? Py_None\n                            : self->record_reader->src().src();\n  static constexpr Identifier id_format(\"format\");\n  return PyObject_CallMethodObjArgs(format.get(), id_format.get(), src,\n                                    nullptr);\n}\n\nstatic PyObject* RecordReaderEnter(PyObject* self, PyObject* args) {\n  // return self\n  Py_INCREF(self);\n  return self;\n}\n\nstatic PyObject* RecordReaderExit(PyRecordReaderObject* self, PyObject* args) {\n  PyObject* exc_type;\n  PyObject* exc_value;\n  PyObject* traceback;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTuple(args, \"OOO:__exit__\", &exc_type,\n                                           &exc_value, &traceback))) {\n    return nullptr;\n  }\n  // self.close(), suppressing exceptions if exc_type != None.\n  if (ABSL_PREDICT_TRUE(self->record_reader.has_value())) {\n    const bool close_ok =\n        PythonUnlocked([&] { return self->record_reader->Close(); });\n    if (ABSL_PREDICT_FALSE(!close_ok) && exc_type == Py_None) {\n      SetExceptionFromRecordReader(self);\n      return nullptr;\n    }\n  }\n  Py_RETURN_FALSE;\n}\n\nstatic PyObject* RecordReaderClose(PyRecordReaderObject* self, PyObject* args) {\n  if (ABSL_PREDICT_TRUE(self->record_reader.has_value())) {\n    const bool close_ok =\n        PythonUnlocked([&] { return self->record_reader->Close(); });\n    if (ABSL_PREDICT_FALSE(!close_ok)) {\n      SetExceptionFromRecordReader(self);\n      return nullptr;\n    }\n  }\n  Py_RETURN_NONE;\n}\n\nstatic PyObject* RecordReaderCheckFileFormat(PyRecordReaderObject* self,\n                                             PyObject* args) {\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  const bool check_file_format_ok =\n      PythonUnlocked([&] { return self->record_reader->CheckFileFormat(); });\n  if (ABSL_PREDICT_FALSE(!check_file_format_ok)) {\n    if (ABSL_PREDICT_FALSE(RecordReaderHasException(self))) {\n      SetExceptionFromRecordReader(self);\n      return nullptr;\n    }\n    Py_RETURN_FALSE;\n  }\n  Py_RETURN_TRUE;\n}\n\nstatic PyObject* RecordReaderReadMetadata(PyRecordReaderObject* self,\n                                          PyObject* args) {\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  Chain metadata;\n  const bool read_serialized_metadata_ok = PythonUnlocked(\n      [&] { return self->record_reader->ReadSerializedMetadata(metadata); });\n  if (ABSL_PREDICT_FALSE(!read_serialized_metadata_ok)) {\n    if (ABSL_PREDICT_FALSE(RecordReaderHasException(self))) {\n      SetExceptionFromRecordReader(self);\n      return nullptr;\n    }\n    Py_RETURN_NONE;\n  }\n  const PythonPtr serialized_metadata = ChainToPython(metadata);\n  if (ABSL_PREDICT_FALSE(serialized_metadata == nullptr)) return nullptr;\n  // return RecordsMetadata.FromString(serialized_metadata)\n  static constexpr ImportedConstant kRecordsMetadata(\n      \"riegeli.records.records_metadata_pb2\", \"RecordsMetadata\");\n  if (ABSL_PREDICT_FALSE(!kRecordsMetadata.Verify())) return nullptr;\n  static constexpr ImportedConstant kDecodeError(\"google.protobuf.message\",\n                                                 \"DecodeError\");\n  if (ABSL_PREDICT_FALSE(!kDecodeError.Verify())) return nullptr;\n  static constexpr Identifier id_FromString(\"FromString\");\n  PythonPtr metadata_object(\n      PyObject_CallMethodObjArgs(kRecordsMetadata.get(), id_FromString.get(),\n                                 serialized_metadata.get(), nullptr));\n  if (ABSL_PREDICT_FALSE(metadata_object == nullptr)) {\n    if (self->record_reader->recovery() != nullptr &&\n        PyErr_ExceptionMatches(kDecodeError.get())) {\n      const Exception exception = Exception::Fetch();\n      if (self->record_reader->recovery()(\n              SkippedRegion(self->record_reader->last_pos().chunk_begin(),\n                            self->record_reader->pos().numeric(),\n                            exception.message()),\n              *self->record_reader)) {\n        // Recovered metadata decoding, assume empty `RecordsMetadata`.\n        return PyObject_CallFunctionObjArgs(kRecordsMetadata.get(), nullptr);\n      }\n      if (ABSL_PREDICT_FALSE(self->recovery_exception.has_value())) {\n        self->recovery_exception->Restore();\n        return nullptr;\n      }\n      Py_RETURN_NONE;\n    }\n    return nullptr;\n  }\n  return metadata_object.release();\n}\n\nstatic PyObject* RecordReaderReadSerializedMetadata(PyRecordReaderObject* self,\n                                                    PyObject* args) {\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  Chain metadata;\n  const bool read_serialized_metadata_ok = PythonUnlocked(\n      [&] { return self->record_reader->ReadSerializedMetadata(metadata); });\n  if (ABSL_PREDICT_FALSE(!read_serialized_metadata_ok)) {\n    if (ABSL_PREDICT_FALSE(RecordReaderHasException(self))) {\n      SetExceptionFromRecordReader(self);\n      return nullptr;\n    }\n    Py_RETURN_NONE;\n  }\n  return ChainToPython(metadata).release();\n}\n\nstatic PyObject* RecordReaderReadRecord(PyRecordReaderObject* self,\n                                        PyObject* args) {\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  Chain record;\n  const bool read_record_ok =\n      PythonUnlocked([&] { return self->record_reader->ReadRecord(record); });\n  if (ABSL_PREDICT_FALSE(!read_record_ok)) {\n    if (ABSL_PREDICT_FALSE(RecordReaderHasException(self))) {\n      SetExceptionFromRecordReader(self);\n      return nullptr;\n    }\n    Py_RETURN_NONE;\n  }\n  return ChainToPython(record).release();\n}\n\nstatic PyObject* RecordReaderReadMessage(PyRecordReaderObject* self,\n                                         PyObject* args, PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"message_type\", nullptr};\n  PyObject* message_type_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:read_message\", const_cast<char**>(keywords),\n          &message_type_arg))) {\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  absl::string_view record;\n  for (;;) {\n    const bool read_record_ok =\n        PythonUnlocked([&] { return self->record_reader->ReadRecord(record); });\n    if (ABSL_PREDICT_FALSE(!read_record_ok)) {\n      if (ABSL_PREDICT_FALSE(RecordReaderHasException(self))) {\n        SetExceptionFromRecordReader(self);\n        return nullptr;\n      }\n      Py_RETURN_NONE;\n    }\n    MemoryView memory_view;\n    PyObject* const record_object = memory_view.ToPython(record);\n    if (ABSL_PREDICT_FALSE(record_object == nullptr)) return nullptr;\n    static constexpr ImportedConstant kDecodeError(\"google.protobuf.message\",\n                                                   \"DecodeError\");\n    if (ABSL_PREDICT_FALSE(!kDecodeError.Verify())) return nullptr;\n    // return message_type.FromString(record)\n    static constexpr Identifier id_FromString(\"FromString\");\n    PythonPtr message(PyObject_CallMethodObjArgs(\n        message_type_arg, id_FromString.get(), record_object, nullptr));\n    if (ABSL_PREDICT_FALSE(message == nullptr)) {\n      if (self->record_reader->recovery() != nullptr &&\n          PyErr_ExceptionMatches(kDecodeError.get())) {\n        const Exception exception = Exception::Fetch();\n        if (ABSL_PREDICT_FALSE(!memory_view.Release())) return nullptr;\n        if (self->record_reader->recovery()(\n                SkippedRegion(self->record_reader->last_pos().numeric(),\n                              self->record_reader->pos().numeric(),\n                              exception.message()),\n                *self->record_reader)) {\n          continue;\n        }\n        if (ABSL_PREDICT_FALSE(self->recovery_exception.has_value())) {\n          self->recovery_exception->Restore();\n          return nullptr;\n        }\n        Py_RETURN_NONE;\n      }\n      return nullptr;\n    }\n    if (ABSL_PREDICT_FALSE(!memory_view.Release())) return nullptr;\n    return message.release();\n  }\n}\n\nstatic PyRecordIterObject* RecordReaderReadRecords(PyRecordReaderObject* self,\n                                                   PyObject* args) {\n  std::unique_ptr<PyRecordIterObject, Deleter> iter(\n      PyObject_GC_New(PyRecordIterObject, &PyRecordIter_Type));\n  if (ABSL_PREDICT_FALSE(iter == nullptr)) return nullptr;\n  iter->read_record = [](PyRecordReaderObject* self, PyObject* args) {\n    return RecordReaderReadRecord(self, args);\n  };\n  Py_INCREF(self);\n  iter->record_reader = self;\n  iter->args = nullptr;\n  return iter.release();\n}\n\nstatic PyRecordIterObject* RecordReaderReadMessages(PyRecordReaderObject* self,\n                                                    PyObject* args,\n                                                    PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"message_type\", nullptr};\n  PyObject* message_type_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:read_messages\", const_cast<char**>(keywords),\n          &message_type_arg))) {\n    return nullptr;\n  }\n  std::unique_ptr<PyRecordIterObject, Deleter> iter(\n      PyObject_GC_New(PyRecordIterObject, &PyRecordIter_Type));\n  if (ABSL_PREDICT_FALSE(iter == nullptr)) return nullptr;\n  iter->read_record = [](PyRecordReaderObject* self, PyObject* args) {\n    return RecordReaderReadMessage(self, args, nullptr);\n  };\n  Py_INCREF(self);\n  iter->record_reader = self;\n  iter->args = PyTuple_Pack(1, message_type_arg);\n  if (ABSL_PREDICT_FALSE(iter->args == nullptr)) return nullptr;\n  return iter.release();\n}\n\nstatic PyObject* RecordReaderSetFieldProjection(PyRecordReaderObject* self,\n                                                PyObject* args,\n                                                PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"field_projection\", nullptr};\n  PyObject* field_projection_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:set_field_projection\", const_cast<char**>(keywords),\n          &field_projection_arg))) {\n    return nullptr;\n  }\n  std::optional<FieldProjection> field_projection;\n  if (field_projection_arg == Py_None) {\n    field_projection = FieldProjection::All();\n  } else {\n    field_projection = FieldProjectionFromPython(field_projection_arg);\n    if (ABSL_PREDICT_FALSE(field_projection == std::nullopt)) return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  const bool set_field_projection_ok = PythonUnlocked([&] {\n    return self->record_reader->SetFieldProjection(\n        *std::move(field_projection));\n  });\n  if (ABSL_PREDICT_FALSE(!set_field_projection_ok)) {\n    SetExceptionFromRecordReader(self);\n    return nullptr;\n  }\n  Py_RETURN_NONE;\n}\n\nstatic PyObject* RecordReaderLastPos(PyRecordReaderObject* self,\n                                     void* closure) {\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  if (ABSL_PREDICT_FALSE(!kRecordPositionApi.Verify())) return nullptr;\n  if (ABSL_PREDICT_FALSE(!self->record_reader->last_record_is_valid())) {\n    SetRiegeliError(absl::FailedPreconditionError(\"No record was read\"));\n    return nullptr;\n  }\n  return kRecordPositionApi\n      ->RecordPositionToPython(self->record_reader->last_pos())\n      .release();\n}\n\nstatic PyObject* RecordReaderPos(PyRecordReaderObject* self, void* closure) {\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  if (ABSL_PREDICT_FALSE(!kRecordPositionApi.Verify())) return nullptr;\n  return kRecordPositionApi->RecordPositionToPython(self->record_reader->pos())\n      .release();\n}\n\nstatic PyObject* RecordReaderSupportsRandomAccess(PyRecordReaderObject* self,\n                                                  void* closure) {\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  return PyBool_FromLong(self->record_reader->SupportsRandomAccess());\n}\n\nstatic PyObject* RecordReaderSeek(PyRecordReaderObject* self, PyObject* args,\n                                  PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"pos\", nullptr};\n  PyObject* pos_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:seek\", const_cast<char**>(keywords), &pos_arg))) {\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!kRecordPositionApi.Verify())) return nullptr;\n  const std::optional<RecordPosition> pos =\n      kRecordPositionApi->RecordPositionFromPython(pos_arg);\n  if (ABSL_PREDICT_FALSE(pos == std::nullopt)) return nullptr;\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  const bool seek_ok =\n      PythonUnlocked([&] { return self->record_reader->Seek(*pos); });\n  if (ABSL_PREDICT_FALSE(!seek_ok)) {\n    SetExceptionFromRecordReader(self);\n    return nullptr;\n  }\n  Py_RETURN_NONE;\n}\n\nstatic PyObject* RecordReaderSeekNumeric(PyRecordReaderObject* self,\n                                         PyObject* args, PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"pos\", nullptr};\n  PyObject* pos_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:seek_numeric\", const_cast<char**>(keywords),\n          &pos_arg))) {\n    return nullptr;\n  }\n  const std::optional<Position> pos = PositionFromPython(pos_arg);\n  if (ABSL_PREDICT_FALSE(pos == std::nullopt)) return nullptr;\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  const bool seek_ok =\n      PythonUnlocked([&] { return self->record_reader->Seek(*pos); });\n  if (ABSL_PREDICT_FALSE(!seek_ok)) {\n    SetExceptionFromRecordReader(self);\n    return nullptr;\n  }\n  Py_RETURN_NONE;\n}\n\nstatic PyObject* RecordReaderSeekBack(PyRecordReaderObject* self,\n                                      PyObject* args) {\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  const bool seek_back_ok =\n      PythonUnlocked([&] { return self->record_reader->SeekBack(); });\n  if (ABSL_PREDICT_FALSE(!seek_back_ok)) {\n    if (ABSL_PREDICT_FALSE(RecordReaderHasException(self))) {\n      SetExceptionFromRecordReader(self);\n      return nullptr;\n    }\n    Py_RETURN_FALSE;\n  }\n  Py_RETURN_TRUE;\n}\n\nstatic PyObject* RecordReaderSize(PyRecordReaderObject* self, PyObject* args) {\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  const std::optional<Position> size =\n      PythonUnlocked([&] { return self->record_reader->Size(); });\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n    SetExceptionFromRecordReader(self);\n    return nullptr;\n  }\n  return PositionToPython(*size).release();\n}\n\nstatic PyObject* RecordReaderSearch(PyRecordReaderObject* self, PyObject* args,\n                                    PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"test\", nullptr};\n  PyObject* test_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:search\", const_cast<char**>(keywords), &test_arg))) {\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  std::optional<Exception> test_exception;\n  const std::optional<PartialOrdering> result = PythonUnlocked([&] {\n    return self->record_reader->Search(\n        [&](RecordReaderBase&) -> std::optional<PartialOrdering> {\n          PythonLock lock;\n          const PythonPtr test_result(\n              PyObject_CallFunctionObjArgs(test_arg, self, nullptr));\n          if (ABSL_PREDICT_FALSE(test_result == nullptr)) {\n            test_exception.emplace(Exception::Fetch());\n            return std::nullopt;\n          }\n          const std::optional<PartialOrdering> ordering =\n              PartialOrderingFromPython(test_result.get());\n          if (ABSL_PREDICT_FALSE(ordering == std::nullopt)) {\n            test_exception.emplace(Exception::Fetch());\n            return std::nullopt;\n          }\n          return *ordering;\n        });\n  });\n  if (ABSL_PREDICT_FALSE(result == std::nullopt)) {\n    if (test_exception != std::nullopt) {\n      test_exception->Restore();\n    } else {\n      SetExceptionFromRecordReader(self);\n    }\n    return nullptr;\n  }\n  return PartialOrderingToPython(*result).release();\n}\n\nstatic PyObject* RecordReaderSearchForRecord(PyRecordReaderObject* self,\n                                             PyObject* args, PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"test\", nullptr};\n  PyObject* test_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:search_for_record\", const_cast<char**>(keywords),\n          &test_arg))) {\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  std::optional<Exception> test_exception;\n  const std::optional<PartialOrdering> result = PythonUnlocked([&] {\n    return self->record_reader->Search<Chain>(\n        [&](const Chain& record) -> std::optional<PartialOrdering> {\n          PythonLock lock;\n          const PythonPtr record_object = ChainToPython(record);\n          if (ABSL_PREDICT_FALSE(record_object == nullptr)) {\n            test_exception.emplace(Exception::Fetch());\n            return std::nullopt;\n          }\n          const PythonPtr test_result(PyObject_CallFunctionObjArgs(\n              test_arg, record_object.get(), nullptr));\n          if (ABSL_PREDICT_FALSE(test_result == nullptr)) {\n            test_exception.emplace(Exception::Fetch());\n            return std::nullopt;\n          }\n          const std::optional<PartialOrdering> ordering =\n              PartialOrderingFromPython(test_result.get());\n          if (ABSL_PREDICT_FALSE(ordering == std::nullopt)) {\n            test_exception.emplace(Exception::Fetch());\n            return std::nullopt;\n          }\n          return *ordering;\n        });\n  });\n  if (ABSL_PREDICT_FALSE(result == std::nullopt)) {\n    if (test_exception != std::nullopt) {\n      test_exception->Restore();\n      if (PyErr_ExceptionMatches(PyExc_StopIteration)) {\n        PyErr_Clear();\n        Py_RETURN_NONE;\n      }\n    } else {\n      SetExceptionFromRecordReader(self);\n    }\n    return nullptr;\n  }\n  return PartialOrderingToPython(*result).release();\n}\n\nstatic PyObject* RecordReaderSearchForMessage(PyRecordReaderObject* self,\n                                              PyObject* args,\n                                              PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"message_type\", \"test\", nullptr};\n  PyObject* message_type_arg;\n  PyObject* test_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"OO:search_for_message\", const_cast<char**>(keywords),\n          &message_type_arg, &test_arg))) {\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!self->record_reader.Verify())) return nullptr;\n  static constexpr ImportedConstant kDecodeError(\"google.protobuf.message\",\n                                                 \"DecodeError\");\n  if (ABSL_PREDICT_FALSE(!kDecodeError.Verify())) return nullptr;\n  // `RecordReader::Search(test)` sets the recovery function to `nullptr` while\n  // calling `test()`. Save it here to call it explicitly in `test()`.\n  std::function<bool(const SkippedRegion&, RecordReaderBase&)> recovery =\n      self->record_reader->recovery();\n  std::optional<Exception> test_exception;\n  const std::optional<PartialOrdering> result = PythonUnlocked([&] {\n    return self->record_reader->Search<absl::string_view>(\n        [&](absl::string_view record) -> std::optional<PartialOrdering> {\n          PythonLock lock;\n          MemoryView memory_view;\n          PyObject* const record_object = memory_view.ToPython(record);\n          if (ABSL_PREDICT_FALSE(record_object == nullptr)) {\n            test_exception.emplace(Exception::Fetch());\n            return std::nullopt;\n          }\n          // message = message_type.FromString(record)\n          static constexpr Identifier id_FromString(\"FromString\");\n          const PythonPtr message(PyObject_CallMethodObjArgs(\n              message_type_arg, id_FromString.get(), record_object, nullptr));\n          if (ABSL_PREDICT_FALSE(message == nullptr)) {\n            if (recovery != nullptr &&\n                PyErr_ExceptionMatches(kDecodeError.get())) {\n              const Exception exception = Exception::Fetch();\n              if (ABSL_PREDICT_FALSE(!memory_view.Release())) {\n                test_exception.emplace(Exception::Fetch());\n                return std::nullopt;\n              }\n              if (recovery(\n                      SkippedRegion(self->record_reader->last_pos().numeric(),\n                                    self->record_reader->pos().numeric(),\n                                    exception.message()),\n                      *self->record_reader)) {\n                // Declare the skipped record unordered.\n                return PartialOrdering::unordered;\n              }\n              if (ABSL_PREDICT_FALSE(self->recovery_exception.has_value())) {\n                return std::nullopt;\n              }\n              // Cancel the search.\n              PyErr_SetNone(PyExc_StopIteration);\n            }\n            test_exception.emplace(Exception::Fetch());\n            return std::nullopt;\n          }\n          if (ABSL_PREDICT_FALSE(!memory_view.Release())) {\n            test_exception.emplace(Exception::Fetch());\n            return std::nullopt;\n          }\n          const PythonPtr test_result(\n              PyObject_CallFunctionObjArgs(test_arg, message.get(), nullptr));\n          if (ABSL_PREDICT_FALSE(test_result == nullptr)) {\n            test_exception.emplace(Exception::Fetch());\n            return std::nullopt;\n          }\n          const std::optional<PartialOrdering> ordering =\n              PartialOrderingFromPython(test_result.get());\n          if (ABSL_PREDICT_FALSE(ordering == std::nullopt)) {\n            test_exception.emplace(Exception::Fetch());\n            return std::nullopt;\n          }\n          return *ordering;\n        });\n  });\n  if (ABSL_PREDICT_FALSE(result == std::nullopt)) {\n    if (test_exception != std::nullopt) {\n      test_exception->Restore();\n      if (PyErr_ExceptionMatches(PyExc_StopIteration)) {\n        PyErr_Clear();\n        Py_RETURN_NONE;\n      }\n    } else {\n      SetExceptionFromRecordReader(self);\n    }\n    return nullptr;\n  }\n  return PartialOrderingToPython(*result).release();\n}\n\n}  // extern \"C\"\n\nconst PyMethodDef RecordReaderMethods[] = {\n    {\"__enter__\", RecordReaderEnter, METH_NOARGS,\n     R\"doc(\n__enter__(self) -> RecordReader\n\nReturns self.\n)doc\"},\n    {\"__exit__\", reinterpret_cast<PyCFunction>(RecordReaderExit), METH_VARARGS,\n     R\"doc(\n__exit__(self, exc_type, exc_value, traceback) -> bool\n\nCalls close().\n\nSuppresses exceptions from close() if an exception is already in flight.\n\nArgs:\n  exc_type: None or exception in flight (type).\n  exc_value: None or exception in flight (value).\n  traceback: None or exception in flight (traceback).\n)doc\"},\n    {\"close\", reinterpret_cast<PyCFunction>(RecordReaderClose), METH_NOARGS,\n     R\"doc(\nclose(self) -> None\n\nIndicates that reading is done.\n\nVerifies that the file is not truncated at the current position, i.e. that it\neither has more data or ends cleanly. Marks the RecordReader as closed,\ndisallowing further reading.\n\nIf the RecordReader was failed, raises the same exception again.\n\nIf the RecordReader was not failed but already closed, does nothing.\n)doc\"},\n    {\"check_file_format\",\n     reinterpret_cast<PyCFunction>(RecordReaderCheckFileFormat), METH_NOARGS,\n     R\"doc(\ncheck_file_format(self) -> bool\n\nEnsures that the file looks like a valid Riegeli/Records file.\n\nReading functions already check the file format. check_file_format() can verify\nthe file format before (or instead of) performing other operations.\n\nThis ignores the recovery function. If invalid file contents are skipped, then\nchecking the file format is meaningless: any file can be read.\n\nReturns:\n  True if this looks like a Riegeli/records file. False if the file ends before\n  this could be determined.\n\nRaises:\n  RiegeliError: If this is not a Riegeli/records file.\n)doc\"},\n    {\"read_metadata\", reinterpret_cast<PyCFunction>(RecordReaderReadMetadata),\n     METH_NOARGS, R\"doc(\nread_metadata(self) -> RecordsMetadata | None\n\nReturns file metadata.\n\nRecord type in metadata can be conveniently interpreted by get_record_type().\n\nread_metadata() must be called while the RecordReader is at the beginning of the\nfile (calling check_file_format() before is allowed).\n\nReturns:\n  File metadata as parsed RecordsMetadata message, or None at end of file.\n)doc\"},\n    {\"read_serialized_metadata\",\n     reinterpret_cast<PyCFunction>(RecordReaderReadSerializedMetadata),\n     METH_NOARGS, R\"doc(\nread_serialized_metadata(self) -> bytes | None\n\nReturns file metadata.\n\nThis is like read_metadata(), but metadata is returned in the serialized form.\nThis is faster if the caller needs metadata already serialized.\n\nReturns:\n  File metadata as serialized RecordsMetadata message, or None at end of file.\n)doc\"},\n    {\"read_record\", reinterpret_cast<PyCFunction>(RecordReaderReadRecord),\n     METH_NOARGS, R\"doc(\nread_record(self) -> bytes | None\n\nReads the next record.\n\nReturns:\n  The record read as bytes, or None at end of file.\n)doc\"},\n    {\"read_message\", reinterpret_cast<PyCFunction>(RecordReaderReadMessage),\n     METH_VARARGS | METH_KEYWORDS, R\"doc(\nread_message(self, message_type: type[Message]) -> Message | None\n\nReads the next record.\n\nArgs:\n  message_type: Type of the message to parse the record as.\n\nReturns:\n  The record read as a parsed message, or None at end of file.\n)doc\"},\n    {\"read_records\", reinterpret_cast<PyCFunction>(RecordReaderReadRecords),\n     METH_NOARGS, R\"doc(\nread_records(self) -> Iterator[bytes]\n\nReturns an iterator which reads all remaining records.\n\nYields:\n  The next record read as bytes.\n)doc\"},\n    {\"read_messages\", reinterpret_cast<PyCFunction>(RecordReaderReadMessages),\n     METH_VARARGS | METH_KEYWORDS, R\"doc(\nread_messages(self, message_type: type[Message]) -> Iterator[Message]\n\nReturns an iterator which reads all remaining records.\n\nYields:\n  The next record read as parsed message.\n)doc\"},\n    {\"set_field_projection\",\n     reinterpret_cast<PyCFunction>(RecordReaderSetFieldProjection),\n     METH_VARARGS | METH_KEYWORDS, R\"doc(\nset_field_projection(\n    self, field_projection: Iterable[Iterable[int]] | None\n) -> None\n\nLike field_projection constructor argument, but can be done at any time.\n\nArgs:\n  field_projection: If not None, the set of fields to be included in returned\n    records, allowing to exclude the remaining fields (but does not guarantee\n    that they will be excluded). Excluding data makes reading faster. Projection\n    is effective if the file has been written with \"transpose\" in RecordWriter\n    options. Additionally, \"bucket_fraction\" in RecordWriter options with a\n    lower value can make reading with projection faster. A field projection is\n    specified as an iterable of field paths. A field path is specified as an\n    iterable of proto field numbers descending from the root message. A special\n    field EXISTENCE_ONLY can be added to the end of the path; it preserves\n    field existence but ignores its value; warning: for a repeated field this\n    preserves the field count only if the field is not packed.\n)doc\"},\n    {\"seek\", reinterpret_cast<PyCFunction>(RecordReaderSeek),\n     METH_VARARGS | METH_KEYWORDS, R\"doc(\nseek(self, pos: RecordPosition) -> None\n\nSeeks to a position.\n\nThe position should have been obtained by pos for the same file.\n\nArgs:\n  pos: Seek target.\n)doc\"},\n    {\"seek_numeric\", reinterpret_cast<PyCFunction>(RecordReaderSeekNumeric),\n     METH_VARARGS | METH_KEYWORDS, R\"doc(\nseek_numeric(self, pos: int) -> None\n\nSeeks to a position.\n\nThe position can be any integer between 0 and file size. If it points between\nrecords, it is interpreted as the next record.\n\nArgs:\n  pos: Seek target.\n)doc\"},\n    {\"seek_back\", reinterpret_cast<PyCFunction>(RecordReaderSeekBack),\n     METH_NOARGS, R\"doc(\nseek_back(self) -> bool\n\nSeeks back by one record.\n\nReturns:\n  If successful, True. Returns False at the beginning of the file.\n)doc\"},\n    {\"size\", reinterpret_cast<PyCFunction>(RecordReaderSize), METH_NOARGS,\n     R\"doc(\nsize(self) -> int\n\nReturns the size of the file in bytes.\n\nThis is the position corresponding to its end.\n)doc\"},\n    {\"search\", reinterpret_cast<PyCFunction>(RecordReaderSearch),\n     METH_VARARGS | METH_KEYWORDS,\n     R\"doc(\nsearch(self, test: Callable[[RecordReader], int | None]) -> None\n\nSearches the file for a desired record, or for a desired position between\nrecords, given that it is possible to determine whether a given record is before\nor after the desired position.\n\nThe current position before calling search() does not matter.\n\nArgs:\n  test: A function which takes the RecordReader as a parameter, seeked to some\n    record, and returns an int or None:\n     * < 0:  The current record is before the desired position.\n     * == 0: The current record is desired, searching can stop.\n     * > 0:  The current record is after the desired position.\n     * None: It could not be determined which is the case. The current record\n             will be skipped.\n    It can also raise StopIteration to cancel the search.\n\nPreconditions:\n * All < 0 records precede all == 0 records.\n * All == 0 records precede all > 0 records.\n * All < 0 records precede all > 0 records, even if there are no == 0 records.\n\nReturn values:\n * 0: There is some == 0 record, and search() points to some such record.\n * 1: There are no == 0 records but there is some > 0 record, and search()\n   points to the earliest such record.\n * -1: There are no == 0 nor > 0 records, but there is some < 0 record, and\n   search() points to the end of file.\n * None: All records are None, and search() points to the end of file,\n   or search() was cancelled.\n\nTo find the earliest == 0 record instead of an arbitrary one, test() can be\nchanged to return > 0 in place of == 0.\n\nFurther guarantees:\n * If a test() returns == 0, search() points back to the record before test()\n   and returns.\n * If a test() returns < 0, test() will not be called again at earlier\n   positions.\n * If a test() returns > 0, test() will not be called again at later positions.\n * test() will not be called again at the same position.\n\nIt follows that if a test() returns == 0 or > 0, search() points to the record\nbefore the last test() call with one of these results. This allows to\ncommunicate additional context of a == 0 or > 0 result by a side effect of\ntest().\n)doc\"},\n    {\"search_for_record\",\n     reinterpret_cast<PyCFunction>(RecordReaderSearchForRecord),\n     METH_VARARGS | METH_KEYWORDS,\n     R\"doc(\nsearch_for_record(self, test: Callable[[bytes], int | None]) -> None\n\nA variant of search() which reads a record before calling test(), instead of\nletting test() read the record.\n\nArgs:\n  test: A function which takes the record read as bytes as a parameter, and\n    returns an int or None, like in search().\n)doc\"},\n    {\"search_for_message\",\n     reinterpret_cast<PyCFunction>(RecordReaderSearchForMessage),\n     METH_VARARGS | METH_KEYWORDS,\n     R\"doc(\nsearch_for_message(\n    self, message_type: type[Message],\n    test: Callable[[Message], int | None]\n) -> None\n\nA variant of search() which reads a record before calling test(), instead of\nletting test() read the record.\n\nArgs:\n  message_type: Type of the message to parse the record as.\n  test: A function which takes the record read as a parsed message as a\n    parameter, and returns an int or None, like in search().\n)doc\"},\n    {nullptr, nullptr, 0, nullptr},\n};\n\nconst PyGetSetDef RecordReaderGetSet[] = {\n    {const_cast<char*>(\"src\"), reinterpret_cast<getter>(RecordReaderSrc),\n     nullptr, const_cast<char*>(R\"doc(\nsrc: BinaryIO\n\nBinary IO stream being read from.\n)doc\"),\n     nullptr},\n    {const_cast<char*>(\"last_pos\"),\n     reinterpret_cast<getter>(RecordReaderLastPos), nullptr,\n     const_cast<char*>(R\"doc(\nlast_pos: RecordPosition\n\nThe canonical position of the last record read.\n\nThe canonical position is the largest among all equivalent positions.\nSeeking to any equivalent position leads to reading the same record.\n\nlast_pos.numeric returns the position as an int.\n\nPrecondition:\n  a record was successfully read and there was no intervening call to\n  close(), seek(), seek_numeric(), seek_back(), search(), search_for_record(),\n  or search_for_message().\n)doc\"),\n     nullptr},\n    {const_cast<char*>(\"pos\"), reinterpret_cast<getter>(RecordReaderPos),\n     nullptr, const_cast<char*>(R\"doc(\npos: RecordPosition\n\nA position of the next record.\n\nA position of the next record (or the end of file if there is no next record).\n\nA position which is not canonical can be smaller than the equivalent canonical\nposition. Seeking to any equivalent position leads to reading the same record.\n\npos.numeric returns the position as an int.\n\npos is unchanged by close().\n)doc\"),\n     nullptr},\n    {const_cast<char*>(\"supports_random_access\"),\n     reinterpret_cast<getter>(RecordReaderSupportsRandomAccess), nullptr,\n     const_cast<char*>(R\"doc(\nsupports_random_access: bool\n\nTrue if this RecordReader supports random access.\n\nThis includes seek(), seek_numeric(), and size().\n)doc\"),\n     nullptr},\n    {nullptr, nullptr, nullptr, nullptr, nullptr}};\n\nPyTypeObject PyRecordReader_Type = {\n    // clang-format off\n    PyVarObject_HEAD_INIT(&PyType_Type, 0)\n    // clang-format on\n    \"riegeli.records.record_reader.RecordReader\",          // tp_name\n    sizeof(PyRecordReaderObject),                          // tp_basicsize\n    0,                                                     // tp_itemsize\n    reinterpret_cast<destructor>(RecordReaderDestructor),  // tp_dealloc\n#if PY_VERSION_HEX >= 0x03080000\n    0,  // tp_vectorcall_offset\n#else\n    nullptr,  // tp_print\n#endif\n    nullptr,                                       // tp_getattr\n    nullptr,                                       // tp_setattr\n    nullptr,                                       // tp_as_async\n    reinterpret_cast<reprfunc>(RecordReaderRepr),  // tp_repr\n    nullptr,                                       // tp_as_number\n    nullptr,                                       // tp_as_sequence\n    nullptr,                                       // tp_as_mapping\n    nullptr,                                       // tp_hash\n    nullptr,                                       // tp_call\n    nullptr,                                       // tp_str\n    nullptr,                                       // tp_getattro\n    nullptr,                                       // tp_setattro\n    nullptr,                                       // tp_as_buffer\n    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC,  // tp_flags\n    R\"doc(\nRecordReader(\n    src: BinaryIO,\n    *,\n    owns_src: bool = True,\n    assumed_pos: int | None = None,\n    min_buffer_size: int = 4 << 10,\n    max_buffer_size: int = 64 << 10,\n    buffer_size: int | None,\n    field_projection: Iterable[Iterable[int]] | None = None,\n    recovery: Callable[[SkippedRegion], Any] | None = None) -> RecordReader\n\nWill read from the given file.\n\nArgs:\n  src: Binary IO stream to read from.\n  owns_src: If True, src is owned, and close() or __exit__() calls src.close().\n  assumed_pos: If None, src must support random access, RecordReader will\n    support random access, and RecordReader will set the position of src on\n    close(). If an int, it is enough that src supports sequential access, and\n    this position will be assumed initially.\n  min_buffer_size: Tunes the minimal buffer size, which determines how much data\n    at a time is typically read from src. The actual buffer size changes between\n    min_buffer_size and max_buffer_size depending on the access pattern.\n  max_buffer_size: Tunes the maximal buffer size, which determines how much data\n    at a time is typically read from src. The actual buffer size changes between\n    min_buffer_size and max_buffer_size depending on the access pattern.\n  buffer_size: If not None, a shortcut for setting min_buffer_size and\n    max_buffer_size to the same value.\n  field_projection: If not None, the set of fields to be included in returned\n    records, allowing to exclude the remaining fields (but does not guarantee\n    that they will be excluded). Excluding data makes reading faster. Projection\n    is effective if the file has been written with \"transpose\" in RecordWriter\n    options. Additionally, \"bucket_fraction\" in RecordWriter options with a\n    lower value can make reading with projection faster. A field projection is\n    specified as an iterable of field paths. A field path is specified as an\n    iterable of proto field numbers descending from the root message. A special\n    field EXISTENCE_ONLY can be added to the end of the path; it preserves\n    field existence but ignores its value; warning: for a repeated field this\n    preserves the field count only if the field is not packed.\n  recovery: If None, then invalid file contents cause RecordReader to raise\n    RiegeliError. If not None, then invalid file contents cause RecordReader to\n    skip over the invalid region and call this recovery function with a\n    SkippedRegion as an argument. If the recovery function returns normally,\n    reading continues. If the recovery function raises StopIteration, reading\n    ends. If close() is called and file contents were truncated, the recovery\n    function is called if set; the RecordReader remains closed.\n\nThe src argument should be a binary IO stream which supports:\n * close()          - for close() or __exit__() if owns_src\n * readinto1(memoryview) or readinto(memoryview) or read1(int) or read(int)\n * seek(int[, int]) - if assumed_pos is None,\n                      or for seek(), seek_numeric(), or size()\n * tell()           - if assumed_pos is None,\n                      or for seek(), seek_numeric(), or size()\n\nExample values for src:\n * io.FileIO(filename, 'rb')\n * io.open(filename, 'rb') - better with buffering=0, or use io.FileIO() instead\n * open(filename, 'rb')    - better with buffering=0, or use io.FileIO() instead\n * io.BytesIO(contents)\n * tf.io.gfile.GFile(filename, 'rb')\n\nWarning: if owns_src is False and assumed_pos is not None, src will have an\nunpredictable amount of extra data consumed because of buffering.\n)doc\",                                                              // tp_doc\n    reinterpret_cast<traverseproc>(RecordReaderTraverse),  // tp_traverse\n    reinterpret_cast<inquiry>(RecordReaderClear),          // tp_clear\n    nullptr,                                               // tp_richcompare\n    0,                                                     // tp_weaklistoffset\n    nullptr,                                               // tp_iter\n    nullptr,                                               // tp_iternext\n    const_cast<PyMethodDef*>(RecordReaderMethods),         // tp_methods\n    nullptr,                                               // tp_members\n    const_cast<PyGetSetDef*>(RecordReaderGetSet),          // tp_getset\n    nullptr,                                               // tp_base\n    nullptr,                                               // tp_dict\n    nullptr,                                               // tp_descr_get\n    nullptr,                                               // tp_descr_set\n    0,                                                     // tp_dictoffset\n    reinterpret_cast<initproc>(RecordReaderInit),          // tp_init\n    nullptr,                                               // tp_alloc\n    PyType_GenericNew,                                     // tp_new\n    nullptr,                                               // tp_free\n    nullptr,                                               // tp_is_gc\n    nullptr,                                               // tp_bases\n    nullptr,                                               // tp_mro\n    nullptr,                                               // tp_cache\n    nullptr,                                               // tp_subclasses\n    nullptr,                                               // tp_weaklist\n    nullptr,                                               // tp_del\n    0,                                                     // tp_version_tag\n    nullptr,                                               // tp_finalize\n};\n\n// `extern \"C\"` sets the C calling convention for compatibility with the Python\n// API. `static` avoids making symbols public, as `extern \"C\"` trumps anonymous\n// namespace.\nextern \"C\" {\n\nstatic void RecordIterDestructor(PyRecordIterObject* self) {\n  PyObject_GC_UnTrack(self);\n#if PY_VERSION_HEX < 0x030D0000  // < 3.13\n  Py_TRASHCAN_BEGIN(self, RecordIterDestructor);\n#endif\n  Py_XDECREF(self->record_reader);\n  Py_XDECREF(self->args);\n  Py_TYPE(self)->tp_free(self);\n#if PY_VERSION_HEX < 0x030D0000  // < 3.13\n  Py_TRASHCAN_END;\n#endif\n}\n\nstatic int RecordIterTraverse(PyRecordIterObject* self, visitproc visit,\n                              void* arg) {\n  Py_VISIT(self->record_reader);\n  Py_VISIT(self->args);\n  return 0;\n}\n\nstatic int RecordIterClear(PyRecordIterObject* self) {\n  Py_CLEAR(self->record_reader);\n  Py_CLEAR(self->args);\n  return 0;\n}\n\nstatic PyObject* RecordIterNext(PyRecordIterObject* self) {\n  PythonPtr read_record_result(\n      self->read_record(self->record_reader, self->args));\n  if (ABSL_PREDICT_FALSE(read_record_result.get() == Py_None)) return nullptr;\n  return read_record_result.release();\n}\n\n}  // extern \"C\"\n\nPyTypeObject PyRecordIter_Type = {\n    // clang-format off\n    PyVarObject_HEAD_INIT(&PyType_Type, 0)\n    // clang-format on\n    \"RecordIter\",                                        // tp_name\n    sizeof(PyRecordIterObject),                          // tp_basicsize\n    0,                                                   // tp_itemsize\n    reinterpret_cast<destructor>(RecordIterDestructor),  // tp_dealloc\n#if PY_VERSION_HEX >= 0x03080000\n    0,  // tp_vectorcall_offset\n#else\n    nullptr,  // tp_print\n#endif\n    nullptr,                                             // tp_getattr\n    nullptr,                                             // tp_setattr\n    nullptr,                                             // tp_as_async\n    nullptr,                                             // tp_repr\n    nullptr,                                             // tp_as_number\n    nullptr,                                             // tp_as_sequence\n    nullptr,                                             // tp_as_mapping\n    nullptr,                                             // tp_hash\n    nullptr,                                             // tp_call\n    nullptr,                                             // tp_str\n    nullptr,                                             // tp_getattro\n    nullptr,                                             // tp_setattro\n    nullptr,                                             // tp_as_buffer\n    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,             // tp_flags\n    nullptr,                                             // tp_doc\n    reinterpret_cast<traverseproc>(RecordIterTraverse),  // tp_traverse\n    reinterpret_cast<inquiry>(RecordIterClear),          // tp_clear\n    nullptr,                                             // tp_richcompare\n    0,                                                   // tp_weaklistoffset\n    PyObject_SelfIter,                                   // tp_iter\n    reinterpret_cast<iternextfunc>(RecordIterNext),      // tp_iternext\n    nullptr,                                             // tp_methods\n    nullptr,                                             // tp_members\n    nullptr,                                             // tp_getset\n    nullptr,                                             // tp_base\n    nullptr,                                             // tp_dict\n    nullptr,                                             // tp_descr_get\n    nullptr,                                             // tp_descr_set\n    0,                                                   // tp_dictoffset\n    nullptr,                                             // tp_init\n    nullptr,                                             // tp_alloc\n    nullptr,                                             // tp_new\n    nullptr,                                             // tp_free\n    nullptr,                                             // tp_is_gc\n    nullptr,                                             // tp_bases\n    nullptr,                                             // tp_mro\n    nullptr,                                             // tp_cache\n    nullptr,                                             // tp_subclasses\n    nullptr,                                             // tp_weaklist\n    nullptr,                                             // tp_del\n    0,                                                   // tp_version_tag\n    nullptr,                                             // tp_finalize\n};\n\nconst char* const kModuleName = \"riegeli.records.record_reader\";\nconst char kModuleDoc[] = R\"doc(Reads records from a Riegeli/records file.)doc\";\n\nconst PyMethodDef kModuleMethods[] = {\n    {\"get_record_type\", reinterpret_cast<PyCFunction>(GetRecordType),\n     METH_VARARGS | METH_KEYWORDS,\n     R\"doc(\nget_record_type(metadata: RecordsMetadata) -> type[Message] | None\n\nInterprets record_type_name and file_descriptor from metadata.\n\nArgs:\n  metadata: Riegeli/records file metadata, typically returned by\n    RecordReader.read_metadata().\n\nReturns:\n  A generated message type corresponding to the type of records, or None if that\n  information is not available in metadata.\n)doc\"},\n    {nullptr, nullptr, 0, nullptr},\n};\n\nPyModuleDef kModuleDef = {\n    PyModuleDef_HEAD_INIT,\n    kModuleName,                               // m_name\n    kModuleDoc,                                // m_doc\n    -1,                                        // m_size\n    const_cast<PyMethodDef*>(kModuleMethods),  // m_methods\n    nullptr,                                   // m_slots\n    nullptr,                                   // m_traverse\n    nullptr,                                   // m_clear\n    nullptr,                                   // m_free\n};\n\nPyObject* InitModule() {\n  if (ABSL_PREDICT_FALSE(PyType_Ready(&PyRecordReader_Type) < 0)) {\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(PyType_Ready(&PyRecordIter_Type) < 0)) {\n    return nullptr;\n  }\n  PythonPtr module(PyModule_Create(&kModuleDef));\n  if (ABSL_PREDICT_FALSE(module == nullptr)) return nullptr;\n  PythonPtr existence_only = IntToPython(Field::kExistenceOnly);\n  if (ABSL_PREDICT_FALSE(existence_only == nullptr)) return nullptr;\n  if (ABSL_PREDICT_FALSE(PyModule_AddObject(module.get(), \"EXISTENCE_ONLY\",\n                                            existence_only.release()) < 0)) {\n    return nullptr;\n  }\n  Py_INCREF(&PyRecordReader_Type);\n  if (ABSL_PREDICT_FALSE(PyModule_AddObject(module.get(), \"RecordReader\",\n                                            reinterpret_cast<PyObject*>(\n                                                &PyRecordReader_Type)) < 0)) {\n    return nullptr;\n  }\n  return module.release();\n}\n\n}  // namespace\n\nPyMODINIT_FUNC PyInit_record_reader() { return InitModule(); }\n\n}  // namespace riegeli::python\n"
  },
  {
    "path": "python/riegeli/records/record_writer.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// From https://docs.python.org/3/c-api/intro.html:\n// Since Python may define some pre-processor definitions which affect the\n// standard headers on some systems, you must include Python.h before any\n// standard headers are included.\n#define PY_SSIZE_T_CLEAN\n#include <Python.h>\n// clang-format: do not reorder the above include.\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"python/riegeli/base/utils.h\"\n#include \"python/riegeli/bytes/python_writer.h\"\n#include \"python/riegeli/records/record_position.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/records/record_writer.h\"\n\nnamespace riegeli::python {\n\nnamespace {\n\nconstexpr ImportedCapsule<RecordPositionApi> kRecordPositionApi(\n    kRecordPositionCapsuleName);\n\nPyObject* PyFlushType_Type;\n\nPythonPtr DefineFlushType() {\n  static constexpr ImportedConstant kEnum(\"enum\", \"Enum\");\n  if (ABSL_PREDICT_FALSE(!kEnum.Verify())) return nullptr;\n  static constexpr Identifier id_FlushType(\"FlushType\");\n  const PythonPtr values(Py_BuildValue(\n      \"((si)(si)(si))\", \"FROM_OBJECT\", static_cast<int>(FlushType::kFromObject),\n      \"FROM_PROCESS\", static_cast<int>(FlushType::kFromProcess), \"FROM_MACHINE\",\n      static_cast<int>(FlushType::kFromMachine)));\n  if (ABSL_PREDICT_FALSE(values == nullptr)) return nullptr;\n  return PythonPtr(PyObject_CallFunctionObjArgs(kEnum.get(), id_FlushType.get(),\n                                                values.get(), nullptr));\n}\n\nbool FlushTypeFromPython(PyObject* object, FlushType* value) {\n  RIEGELI_ASSERT_NE(PyFlushType_Type, nullptr)\n      << \"Python FlushType not defined yet\";\n  if (ABSL_PREDICT_FALSE(!PyObject_IsInstance(object, PyFlushType_Type))) {\n    PyErr_Format(PyExc_TypeError, \"Expected FlushType, not %s\",\n                 Py_TYPE(object)->tp_name);\n    return false;\n  }\n  static constexpr Identifier id_value(\"value\");\n  const PythonPtr enum_value(PyObject_GetAttr(object, id_value.get()));\n  if (ABSL_PREDICT_FALSE(enum_value == nullptr)) return false;\n  const long long_value = PyLong_AsLong(enum_value.get());\n  if (ABSL_PREDICT_FALSE(long_value == -1) && PyErr_Occurred()) return false;\n  *value = static_cast<FlushType>(long_value);\n  return true;\n}\n\nclass FileDescriptorCollector {\n public:\n  bool Init(PyObject* file_descriptors) {\n    file_descriptors_ = file_descriptors;\n    files_seen_.reset(PySet_New(nullptr));\n    return files_seen_ != nullptr;\n  }\n\n  bool AddFile(PyObject* file_descriptor) {\n    // name = file_descriptor.name\n    static constexpr Identifier id_name(\"name\");\n    const PythonPtr name(PyObject_GetAttr(file_descriptor, id_name.get()));\n    if (ABSL_PREDICT_FALSE(name == nullptr)) return false;\n    // if name in self.files_seen: return\n    const int contains = PySet_Contains(files_seen_.get(), name.get());\n    if (ABSL_PREDICT_FALSE(contains < 0)) return false;\n    if (contains != 0) return true;\n    // self.files_seen.add(name)\n    if (ABSL_PREDICT_FALSE(PySet_Add(files_seen_.get(), name.get()) < 0)) {\n      return false;\n    }\n    // for dependency in file_descriptor.dependencies:\n    //   self.add_file(dependency)\n    static constexpr Identifier id_dependencies(\"dependencies\");\n    const PythonPtr dependencies(\n        PyObject_GetAttr(file_descriptor, id_dependencies.get()));\n    if (ABSL_PREDICT_FALSE(dependencies == nullptr)) return false;\n    const PythonPtr iter(PyObject_GetIter(dependencies.get()));\n    if (ABSL_PREDICT_FALSE(iter == nullptr)) return false;\n    while (const PythonPtr dependency{PyIter_Next(iter.get())}) {\n      if (ABSL_PREDICT_FALSE(!AddFile(dependency.get()))) return false;\n    }\n    if (ABSL_PREDICT_FALSE(PyErr_Occurred() != nullptr)) return false;\n    // file_descriptor_proto = self.file_descriptors.add()\n    static constexpr Identifier id_add(\"add\");\n    const PythonPtr file_descriptor_proto(\n        PyObject_CallMethodObjArgs(file_descriptors_, id_add.get(), nullptr));\n    if (ABSL_PREDICT_FALSE(file_descriptor_proto == nullptr)) return false;\n    // file_descriptor.CopyToProto(file_descriptor_proto)\n    static constexpr Identifier id_CopyToProto(\"CopyToProto\");\n    return PythonPtr(PyObject_CallMethodObjArgs(\n               file_descriptor, id_CopyToProto.get(),\n               file_descriptor_proto.get(), nullptr)) != nullptr;\n  }\n\n private:\n  PyObject* file_descriptors_;\n  PythonPtr files_seen_;\n};\n\n// `extern \"C\"` sets the C calling convention for compatibility with the Python\n// API. `static` avoids making symbols public, as `extern \"C\"` trumps anonymous\n// namespace.\nextern \"C\" {\n\nstatic PyObject* SetRecordType(PyObject* self, PyObject* args,\n                               PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"metadata\", \"message_type\",\n                                             nullptr};\n  PyObject* metadata_arg;\n  PyObject* message_type_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"OO:set_record_type\", const_cast<char**>(keywords),\n          &metadata_arg, &message_type_arg))) {\n    return nullptr;\n  }\n  // message_descriptor = message_type.DESCRIPTOR\n  static constexpr Identifier id_DESCRIPTOR(\"DESCRIPTOR\");\n  const PythonPtr message_descriptor(\n      PyObject_GetAttr(message_type_arg, id_DESCRIPTOR.get()));\n  if (ABSL_PREDICT_FALSE(message_descriptor == nullptr)) return nullptr;\n  // metadata.record_type_name = message_descriptor.full_name\n  static constexpr Identifier id_full_name(\"full_name\");\n  const PythonPtr full_name(\n      PyObject_GetAttr(message_descriptor.get(), id_full_name.get()));\n  if (ABSL_PREDICT_FALSE(full_name == nullptr)) return nullptr;\n  static constexpr Identifier id_record_type_name(\"record_type_name\");\n  if (ABSL_PREDICT_FALSE(PyObject_SetAttr(metadata_arg,\n                                          id_record_type_name.get(),\n                                          full_name.get()) < 0)) {\n    return nullptr;\n  }\n  // file_descriptors = metadata.file_descriptor\n  static constexpr Identifier id_file_descriptor(\"file_descriptor\");\n  const PythonPtr file_descriptors(\n      PyObject_GetAttr(metadata_arg, id_file_descriptor.get()));\n  if (ABSL_PREDICT_FALSE(file_descriptors == nullptr)) return nullptr;\n  // del file_descriptors[:]\n  const PythonPtr slice(PySlice_New(nullptr, nullptr, nullptr));\n  if (ABSL_PREDICT_FALSE(slice == nullptr)) return nullptr;\n  if (ABSL_PREDICT_FALSE(PyObject_DelItem(file_descriptors.get(), slice.get()) <\n                         0)) {\n    return nullptr;\n  }\n  // file_descriptor = message_descriptor.file\n  static constexpr Identifier id_file(\"file\");\n  const PythonPtr file_descriptor(\n      PyObject_GetAttr(message_descriptor.get(), id_file.get()));\n  if (ABSL_PREDICT_FALSE(file_descriptor == nullptr)) return nullptr;\n  // FileDescriptorCollector(file_descriptors).add_file(file_descriptor)\n  FileDescriptorCollector collector;\n  if (ABSL_PREDICT_FALSE(!collector.Init(file_descriptors.get()))) {\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!collector.AddFile(file_descriptor.get()))) {\n    return nullptr;\n  }\n  Py_RETURN_NONE;\n}\n\n}  // extern \"C\"\n\nstruct PyRecordWriterObject {\n  // clang-format off\n  PyObject_HEAD\n  static_assert(true, \"\");  // clang-format workaround.\n  // clang-format on\n\n  PythonWrapped<RecordWriter<PythonWriter>> record_writer;\n};\n\nextern PyTypeObject PyRecordWriter_Type;\n\nvoid SetExceptionFromRecordWriter(PyRecordWriterObject* self) {\n  RIEGELI_ASSERT(!self->record_writer->ok())\n      << \"Failed precondition of SetExceptionFromRecordWriter(): \"\n         \"RecordWriter OK\";\n  if (!self->record_writer->dest().exception().ok()) {\n    self->record_writer->dest().exception().Restore();\n    return;\n  }\n  SetRiegeliError(self->record_writer->status());\n}\n\n// `extern \"C\"` sets the C calling convention for compatibility with the Python\n// API. `static` avoids making symbols public, as `extern \"C\"` trumps anonymous\n// namespace.\nextern \"C\" {\n\nstatic void RecordWriterDestructor(PyRecordWriterObject* self) {\n  PyObject_GC_UnTrack(self);\n  Py_TRASHCAN_BEGIN(self, RecordWriterDestructor);\n  PythonUnlocked([&] { self->record_writer.reset(); });\n  Py_TYPE(self)->tp_free(self);\n  Py_TRASHCAN_END;\n}\n\nstatic int RecordWriterTraverse(PyRecordWriterObject* self, visitproc visit,\n                                void* arg) {\n  if (self->record_writer.has_value()) {\n    return self->record_writer->dest().Traverse(visit, arg);\n  }\n  return 0;\n}\n\nstatic int RecordWriterClear(PyRecordWriterObject* self) {\n  PythonUnlocked([&] { self->record_writer.reset(); });\n  return 0;\n}\n\nstatic int RecordWriterInit(PyRecordWriterObject* self, PyObject* args,\n                            PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"dest\",\n                                             \"owns_dest\",\n                                             \"assumed_pos\",\n                                             \"min_buffer_size\",\n                                             \"max_buffer_size\",\n                                             \"buffer_size\",\n                                             \"options\",\n                                             \"metadata\",\n                                             \"serialized_metadata\",\n                                             nullptr};\n  PyObject* dest_arg;\n  PyObject* owns_dest_arg = nullptr;\n  PyObject* assumed_pos_arg = nullptr;\n  PyObject* min_buffer_size_arg = nullptr;\n  PyObject* max_buffer_size_arg = nullptr;\n  PyObject* buffer_size_arg = nullptr;\n  PyObject* options_arg = nullptr;\n  PyObject* metadata_arg = nullptr;\n  PyObject* serialized_metadata_arg = nullptr;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O|$OOOOOOOO:RecordWriter\",\n          const_cast<char**>(keywords), &dest_arg, &owns_dest_arg,\n          &assumed_pos_arg, &min_buffer_size_arg, &max_buffer_size_arg,\n          &buffer_size_arg, &options_arg, &metadata_arg,\n          &serialized_metadata_arg))) {\n    return -1;\n  }\n\n  PythonWriter::Options python_writer_options;\n  python_writer_options.set_owns_dest(true);\n  if (owns_dest_arg != nullptr) {\n    const int owns_dest_is_true = PyObject_IsTrue(owns_dest_arg);\n    if (ABSL_PREDICT_FALSE(owns_dest_is_true < 0)) return -1;\n    python_writer_options.set_owns_dest(owns_dest_is_true != 0);\n  }\n  if (assumed_pos_arg != nullptr && assumed_pos_arg != Py_None) {\n    const std::optional<Position> assumed_pos =\n        PositionFromPython(assumed_pos_arg);\n    if (ABSL_PREDICT_FALSE(assumed_pos == std::nullopt)) return -1;\n    python_writer_options.set_assumed_pos(*assumed_pos);\n  }\n  if (buffer_size_arg != nullptr && buffer_size_arg != Py_None) {\n    min_buffer_size_arg = buffer_size_arg;\n    max_buffer_size_arg = buffer_size_arg;\n  }\n  if (min_buffer_size_arg != nullptr) {\n    const std::optional<size_t> min_buffer_size =\n        SizeFromPython(min_buffer_size_arg);\n    if (ABSL_PREDICT_FALSE(min_buffer_size == std::nullopt)) return -1;\n    python_writer_options.set_min_buffer_size(*min_buffer_size);\n  }\n  if (max_buffer_size_arg != nullptr) {\n    const std::optional<size_t> max_buffer_size =\n        SizeFromPython(max_buffer_size_arg);\n    if (ABSL_PREDICT_FALSE(max_buffer_size == std::nullopt)) return -1;\n    python_writer_options.set_max_buffer_size(*max_buffer_size);\n  }\n\n  RecordWriterBase::Options record_writer_options;\n  if (options_arg != nullptr) {\n    StrOrBytes options;\n    if (ABSL_PREDICT_FALSE(!options.FromPython(options_arg))) return -1;\n    if (const absl::Status status = record_writer_options.FromString(options);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      SetRiegeliError(status);\n      return -1;\n    }\n  }\n  if (metadata_arg != nullptr && metadata_arg != Py_None) {\n    static constexpr Identifier id_SerializeToString(\"SerializeToString\");\n    const PythonPtr serialized_metadata_str(PyObject_CallMethodObjArgs(\n        metadata_arg, id_SerializeToString.get(), nullptr));\n    if (ABSL_PREDICT_FALSE(serialized_metadata_str == nullptr)) return -1;\n    std::optional<Chain> serialized_metadata =\n        ChainFromPython(serialized_metadata_str.get());\n    if (ABSL_PREDICT_FALSE(serialized_metadata == std::nullopt)) return -1;\n    record_writer_options.set_serialized_metadata(\n        *std::move(serialized_metadata));\n  }\n  if (serialized_metadata_arg != nullptr &&\n      serialized_metadata_arg != Py_None) {\n    std::optional<Chain> serialized_metadata =\n        ChainFromPython(serialized_metadata_arg);\n    if (ABSL_PREDICT_FALSE(serialized_metadata == std::nullopt)) return -1;\n    if (record_writer_options.serialized_metadata() != std::nullopt) {\n      PyErr_SetString(PyExc_TypeError,\n                      \"RecordWriter() got conflicting keyword arguments \"\n                      \"'metadata' and 'serialized_metadata'\");\n      return -1;\n    }\n    record_writer_options.set_serialized_metadata(\n        *std::move(serialized_metadata));\n  }\n\n  PythonWriter python_writer(dest_arg, std::move(python_writer_options));\n  PythonUnlocked([&] {\n    self->record_writer.emplace(std::move(python_writer),\n                                std::move(record_writer_options));\n  });\n  if (ABSL_PREDICT_FALSE(!self->record_writer->ok())) {\n    self->record_writer->dest().Close();\n    SetExceptionFromRecordWriter(self);\n    return -1;\n  }\n  return 0;\n}\n\nstatic PyObject* RecordWriterDest(PyRecordWriterObject* self, void* closure) {\n  PyObject* const dest = ABSL_PREDICT_FALSE(!self->record_writer.has_value())\n                             ? Py_None\n                             : self->record_writer->dest().dest();\n  Py_INCREF(dest);\n  return dest;\n}\n\nstatic PyObject* RecordWriterRepr(PyRecordWriterObject* self) {\n  const PythonPtr format = StringToPython(\"<RecordWriter dest={!r}>\");\n  if (ABSL_PREDICT_FALSE(format == nullptr)) return nullptr;\n  // return format.format(self.dest)\n  PyObject* const dest = ABSL_PREDICT_FALSE(!self->record_writer.has_value())\n                             ? Py_None\n                             : self->record_writer->dest().dest();\n  static constexpr Identifier id_format(\"format\");\n  return PyObject_CallMethodObjArgs(format.get(), id_format.get(), dest,\n                                    nullptr);\n}\n\nstatic PyObject* RecordWriterEnter(PyObject* self, PyObject* args) {\n  // return self\n  Py_INCREF(self);\n  return self;\n}\n\nstatic PyObject* RecordWriterExit(PyRecordWriterObject* self, PyObject* args) {\n  PyObject* exc_type;\n  PyObject* exc_value;\n  PyObject* traceback;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTuple(args, \"OOO:__exit__\", &exc_type,\n                                           &exc_value, &traceback))) {\n    return nullptr;\n  }\n  // self.close(), suppressing exceptions if exc_type != None.\n  if (ABSL_PREDICT_TRUE(self->record_writer.has_value())) {\n    const bool close_ok =\n        PythonUnlocked([&] { return self->record_writer->Close(); });\n    if (ABSL_PREDICT_FALSE(!close_ok) && exc_type == Py_None) {\n      SetExceptionFromRecordWriter(self);\n      return nullptr;\n    }\n  }\n  Py_RETURN_FALSE;\n}\n\nstatic PyObject* RecordWriterClose(PyRecordWriterObject* self, PyObject* args) {\n  if (ABSL_PREDICT_TRUE(self->record_writer.has_value())) {\n    const bool close_ok =\n        PythonUnlocked([&] { return self->record_writer->Close(); });\n    if (ABSL_PREDICT_FALSE(!close_ok)) {\n      SetExceptionFromRecordWriter(self);\n      return nullptr;\n    }\n  }\n  Py_RETURN_NONE;\n}\n\nstatic PyObject* RecordWriterWriteRecord(PyRecordWriterObject* self,\n                                         PyObject* args, PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"record\", nullptr};\n  PyObject* record_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:write_record\", const_cast<char**>(keywords),\n          &record_arg))) {\n    return nullptr;\n  }\n  BytesLike record;\n  if (ABSL_PREDICT_FALSE(!record.FromPython(record_arg))) return nullptr;\n  if (ABSL_PREDICT_FALSE(!self->record_writer.Verify())) return nullptr;\n  const bool write_record_ok =\n      PythonUnlocked([&] { return self->record_writer->WriteRecord(record); });\n  if (ABSL_PREDICT_FALSE(!write_record_ok)) {\n    SetExceptionFromRecordWriter(self);\n    return nullptr;\n  }\n  Py_RETURN_NONE;\n}\n\nstatic PyObject* RecordWriterWriteMessage(PyRecordWriterObject* self,\n                                          PyObject* args, PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"record\", nullptr};\n  PyObject* record_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:write_message\", const_cast<char**>(keywords),\n          &record_arg))) {\n    return nullptr;\n  }\n  // self.write_record(record.SerializeToString())\n  static constexpr Identifier id_SerializeToString(\"SerializeToString\");\n  const PythonPtr serialized_object(PyObject_CallMethodObjArgs(\n      record_arg, id_SerializeToString.get(), nullptr));\n  if (ABSL_PREDICT_FALSE(serialized_object == nullptr)) return nullptr;\n  BytesLike serialized;\n  if (ABSL_PREDICT_FALSE(!serialized.FromPython(serialized_object.get()))) {\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!self->record_writer.Verify())) return nullptr;\n  const bool write_record_ok = PythonUnlocked(\n      [&] { return self->record_writer->WriteRecord(serialized); });\n  if (ABSL_PREDICT_FALSE(!write_record_ok)) {\n    SetExceptionFromRecordWriter(self);\n    return nullptr;\n  }\n  Py_RETURN_NONE;\n}\n\nstatic PyObject* RecordWriterWriteRecords(PyRecordWriterObject* self,\n                                          PyObject* args, PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"records\", nullptr};\n  PyObject* records_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:write_records\", const_cast<char**>(keywords),\n          &records_arg))) {\n    return nullptr;\n  }\n  // for record in records:\n  //   self.write_record(record)\n  const PythonPtr iter(PyObject_GetIter(records_arg));\n  if (ABSL_PREDICT_FALSE(iter == nullptr)) return nullptr;\n  while (const PythonPtr record_object{PyIter_Next(iter.get())}) {\n    BytesLike record;\n    if (ABSL_PREDICT_FALSE(!record.FromPython(record_object.get()))) {\n      return nullptr;\n    }\n    if (ABSL_PREDICT_FALSE(!self->record_writer.Verify())) return nullptr;\n    const bool write_record_ok = PythonUnlocked(\n        [&] { return self->record_writer->WriteRecord(record); });\n    if (ABSL_PREDICT_FALSE(!write_record_ok)) {\n      SetExceptionFromRecordWriter(self);\n      return nullptr;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(PyErr_Occurred() != nullptr)) return nullptr;\n  Py_RETURN_NONE;\n}\n\nstatic PyObject* RecordWriterWriteMessages(PyRecordWriterObject* self,\n                                           PyObject* args, PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"records\", nullptr};\n  PyObject* records_arg;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"O:write_messages\", const_cast<char**>(keywords),\n          &records_arg))) {\n    return nullptr;\n  }\n  // for record in records:\n  //   self.write_record(record.SerializeToString())\n  const PythonPtr iter(PyObject_GetIter(records_arg));\n  if (ABSL_PREDICT_FALSE(iter == nullptr)) return nullptr;\n  while (const PythonPtr record_object{PyIter_Next(iter.get())}) {\n    static constexpr Identifier id_SerializeToString(\"SerializeToString\");\n    const PythonPtr serialized_object(PyObject_CallMethodObjArgs(\n        record_object.get(), id_SerializeToString.get(), nullptr));\n    if (ABSL_PREDICT_FALSE(serialized_object == nullptr)) return nullptr;\n    BytesLike serialized;\n    if (ABSL_PREDICT_FALSE(!serialized.FromPython(serialized_object.get()))) {\n      return nullptr;\n    }\n    if (ABSL_PREDICT_FALSE(!self->record_writer.Verify())) return nullptr;\n    const bool write_record_ok = PythonUnlocked(\n        [&] { return self->record_writer->WriteRecord(serialized); });\n    if (ABSL_PREDICT_FALSE(!write_record_ok)) {\n      SetExceptionFromRecordWriter(self);\n      return nullptr;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(PyErr_Occurred() != nullptr)) return nullptr;\n  Py_RETURN_NONE;\n}\n\nstatic PyObject* RecordWriterFlush(PyRecordWriterObject* self, PyObject* args,\n                                   PyObject* kwargs) {\n  static constexpr const char* keywords[] = {\"flush_type\", nullptr};\n  PyObject* flush_type_arg = nullptr;\n  if (ABSL_PREDICT_FALSE(!PyArg_ParseTupleAndKeywords(\n          args, kwargs, \"|O:flush\", const_cast<char**>(keywords),\n          &flush_type_arg))) {\n    return nullptr;\n  }\n  FlushType flush_type = FlushType::kFromProcess;\n  if (flush_type_arg != nullptr) {\n    if (ABSL_PREDICT_FALSE(!FlushTypeFromPython(flush_type_arg, &flush_type))) {\n      return nullptr;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!self->record_writer.Verify())) return nullptr;\n  const bool flush_ok =\n      PythonUnlocked([&] { return self->record_writer->Flush(flush_type); });\n  if (ABSL_PREDICT_FALSE(!flush_ok)) {\n    SetExceptionFromRecordWriter(self);\n    return nullptr;\n  }\n  Py_RETURN_NONE;\n}\n\nstatic PyObject* RecordWriterLastPos(PyRecordWriterObject* self,\n                                     void* closure) {\n  if (ABSL_PREDICT_FALSE(!self->record_writer.Verify())) return nullptr;\n  if (ABSL_PREDICT_FALSE(!kRecordPositionApi.Verify())) return nullptr;\n  if (ABSL_PREDICT_FALSE(!self->record_writer->last_record_is_valid())) {\n    SetRiegeliError(absl::FailedPreconditionError(\"No record was written\"));\n    return nullptr;\n  }\n  return kRecordPositionApi\n      ->RecordPositionToPython(self->record_writer->LastPos())\n      .release();\n}\n\nstatic PyObject* RecordWriterPos(PyRecordWriterObject* self, void* closure) {\n  if (ABSL_PREDICT_FALSE(!self->record_writer.Verify())) return nullptr;\n  if (ABSL_PREDICT_FALSE(!kRecordPositionApi.Verify())) return nullptr;\n  return kRecordPositionApi->RecordPositionToPython(self->record_writer->Pos())\n      .release();\n}\n\nstatic PyObject* RecordWriterEstimatedSize(PyRecordWriterObject* self,\n                                           PyObject* args) {\n  if (ABSL_PREDICT_FALSE(!self->record_writer.Verify())) return nullptr;\n  return PositionToPython(self->record_writer->EstimatedSize()).release();\n}\n\n}  // extern \"C\"\n\nconst PyMethodDef RecordWriterMethods[] = {\n    {\"__enter__\", RecordWriterEnter, METH_NOARGS,\n     R\"doc(\n__enter__(self) -> RecordWriter\n\nReturns self.\n)doc\"},\n    {\"__exit__\", reinterpret_cast<PyCFunction>(RecordWriterExit), METH_VARARGS,\n     R\"doc(\n__exit__(self, exc_type, exc_value, traceback) -> bool\n\nCalls close().\n\nSuppresses exceptions from close() if an exception is already in flight.\n\nArgs:\n  exc_type: None or exception in flight (type).\n  exc_value: None or exception in flight (value).\n  traceback: None or exception in flight (traceback).\n)doc\"},\n    {\"close\", reinterpret_cast<PyCFunction>(RecordWriterClose), METH_NOARGS,\n     R\"doc(\nclose(self) -> None\n\nIndicates that writing is done.\n\nWrites buffered data to the file. Marks the RecordWriter as closed,\ndisallowing further writing.\n\nIf the RecordWriter was failed, raises the same exception again.\n\nIf the RecordWriter was not failed but already closed, does nothing.\n)doc\"},\n    {\"write_record\", reinterpret_cast<PyCFunction>(RecordWriterWriteRecord),\n     METH_VARARGS | METH_KEYWORDS, R\"doc(\nwrite_record(self, record: bytes | bytearray | memoryview) -> None\n\nWrites the next record.\n\nArgs:\n  record: Record to write as a bytes-like object.\n)doc\"},\n    {\"write_message\", reinterpret_cast<PyCFunction>(RecordWriterWriteMessage),\n     METH_VARARGS | METH_KEYWORDS, R\"doc(\nwrite_message(self, record: Message) -> None\n\nWrites the next record.\n\nArgs:\n  record: Record to write as a proto message.\n)doc\"},\n    {\"write_records\", reinterpret_cast<PyCFunction>(RecordWriterWriteRecords),\n     METH_VARARGS | METH_KEYWORDS, R\"doc(\nwrite_records(\n    self, records: Iterable[bytes | bytearray | memoryview]) -> None\n\nWrites a number of records.\n\nArgs:\n  records: Records to write as an iterable of bytes-like objects.\n)doc\"},\n    {\"write_messages\", reinterpret_cast<PyCFunction>(RecordWriterWriteMessages),\n     METH_VARARGS | METH_KEYWORDS, R\"doc(\nwrite_messages(self, records: Iterable[Message]) -> None\n\nWrites a number of records.\n\nArgs:\n  records: Records to write as an iterable of proto messages.\n)doc\"},\n    {\"flush\", reinterpret_cast<PyCFunction>(RecordWriterFlush),\n     METH_VARARGS | METH_KEYWORDS, R\"doc(\nflush(self, flush_type: FlushType = FlushType.FROM_PROCESS) -> None\n\nFinalizes any open chunk and pushes buffered data to the destination.\nIf parallelism was used in options, waits for any background writing to\ncomplete.\n\nThis makes data written so far visible, but in contrast to close(),\nkeeps the possibility to write more data later. What exactly does it mean\nfor data to be visible depends on the destination.\n\nThis degrades compression density if used too often.\n\nArgs:\n  flush_type: The scope of objects to flush and the intended data durability\n  (without a guarantee).\n   * FlushType.FROM_OBJECT:  Makes data written so far visible in other\n                             objects, propagating flushing through owned\n                             dependencies of the given writer.\n   * FlushType.FROM_PROCESS: Makes data written so far visible outside\n                             the process, propagating flushing through\n                             dependencies of the given writer.\n                             This is the default.\n   * FlushType.FROM_MACHINE: Makes data written so far visible outside\n                             the process and durable in case of operating\n                             system crash, propagating flushing through\n                             dependencies of the given writer.\n)doc\"},\n    {\"estimated_size\", reinterpret_cast<PyCFunction>(RecordWriterEstimatedSize),\n     METH_NOARGS,\n     R\"doc(\nestimated_size(self) -> int\n\nReturns an estimation of the file size if no more data is written, without\naffecting data representation (i.e. without closing the current chunk) and\nwithout blocking.\n\nThis is an underestimation because pending work is not taken into account:\n * The currently open chunk.\n * If parallelism was used in options, chunks being encoded in background.\n\nThe exact file size can be found by flush(FlushType.FROM_OBJECT) which closes\nthe currently open chunk, and pos.chunk_begin (record_index == 0 after flushing)\nwhich might need to wait for some background work to complete.\n)doc\"},\n    {nullptr, nullptr, 0, nullptr},\n};\n\nconst PyGetSetDef RecordWriterGetSet[] = {\n    {const_cast<char*>(\"dest\"), reinterpret_cast<getter>(RecordWriterDest),\n     nullptr, const_cast<char*>(R\"doc(\ndest: BinaryIO\n\nBinary IO stream being written to.\n)doc\"),\n     nullptr},\n    {const_cast<char*>(\"last_pos\"),\n     reinterpret_cast<getter>(RecordWriterLastPos), nullptr,\n     const_cast<char*>(R\"doc(\nlast_pos: RecordPosition\n\nThe canonical position of the last record written.\n\nThe canonical position is the largest among all equivalent positions.\nSeeking to any equivalent position leads to reading the same record.\n\nlast_pos.numeric returns the position as an int.\n\nPrecondition:\n  a record was successfully written\n)doc\"),\n     nullptr},\n    {const_cast<char*>(\"pos\"), reinterpret_cast<getter>(RecordWriterPos),\n     nullptr, const_cast<char*>(R\"doc(\npos: RecordPosition\n\nA position of the next record (or the end of file if there is no next record).\n\nA position which is not canonical can be smaller than the equivalent canonical\nposition. Seeking to any equivalent position leads to reading the same record.\n\npos.numeric returns the position as an int.\n\nAfter opening the file, close(), or flush(), pos is the canonical position of\nthe next record, and pos.record_index == 0.\n)doc\"),\n     nullptr},\n    {nullptr, nullptr, nullptr, nullptr, nullptr}};\n\nPyTypeObject PyRecordWriter_Type = {\n    // clang-format off\n    PyVarObject_HEAD_INIT(&PyType_Type, 0)\n    // clang-format on\n    \"riegeli.records.record_writer.RecordWriter\",          // tp_name\n    sizeof(PyRecordWriterObject),                          // tp_basicsize\n    0,                                                     // tp_itemsize\n    reinterpret_cast<destructor>(RecordWriterDestructor),  // tp_dealloc\n#if PY_VERSION_HEX >= 0x03080000\n    0,  // tp_vectorcall_offset\n#else\n    nullptr,  // tp_print\n#endif\n    nullptr,                                       // tp_getattr\n    nullptr,                                       // tp_setattr\n    nullptr,                                       // tp_as_async\n    reinterpret_cast<reprfunc>(RecordWriterRepr),  // tp_repr\n    nullptr,                                       // tp_as_number\n    nullptr,                                       // tp_as_sequence\n    nullptr,                                       // tp_as_mapping\n    nullptr,                                       // tp_hash\n    nullptr,                                       // tp_call\n    nullptr,                                       // tp_str\n    nullptr,                                       // tp_getattro\n    nullptr,                                       // tp_setattro\n    nullptr,                                       // tp_as_buffer\n    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC,  // tp_flags\n    R\"doc(\nRecordWriter(\n    dest: BinaryIO,\n    *,\n    owns_dest: bool = True,\n    assumed_pos: int | None = None,\n    min_buffer_size: int = 4 << 10,\n    max_buffer_size: int = 64 << 10,\n    buffer_size: int | None,\n    options: str | bytes = '',\n    metadata: RecordsMetadata | None = None,\n    serialized_metadata: bytes | bytearray | memoryview = b''\n) -> RecordWriter\n\nWill write to the given file.\n\nArgs:\n  dest: Binary IO stream to write to.\n  owns_dest: If True, dest is owned, close() or __exit__() calls dest.close(),\n    and flush(flush_type) calls dest.flush() even if flush_type is\n    FlushType.FROM_OBJECT.\n  assumed_pos: If None, dest must support random access. If an int, it is enough\n    that dest supports sequential access, and this position will be assumed\n    initially.\n  min_buffer_size: Tunes the minimal buffer size, which determines how much data\n    at a time is typically written to dest. The actual buffer size changes\n    between min_buffer_size and max_buffer_size depending on the access pattern.\n  max_buffer_size: Tunes the maximal buffer size, which determines how much data\n    at a time is typically written to dest. The actual buffer size changes\n    between min_buffer_size and max_buffer_size depending on the access pattern.\n  buffer_size: If not None, a shortcut for setting min_buffer_size and\n    max_buffer_size to the same value.\n  options: Compression and other writing options. See below.\n  metadata: If not None, file metadata to be written at the beginning (if\n    metadata has any fields set). Metadata are written only when the file is\n    written from the beginning, not when it is appended to. Record type in\n    metadata can be conveniently set by set_record_type().\n  serialized_metadata: If not empty, like metadata, but metadata are passed\n    serialized as a bytes-like object. This is faster if the caller has metadata\n    already serialized. This conflicts with metadata.\n\nThe dest argument should be a binary IO stream which supports:\n * close()          - for close() or __exit__() if owns_dest\n * write(bytes)\n * flush()          - for flush()\n * seek(int[, int]) - if assumed_pos is None\n * tell()           - if assumed_pos is None\n\nExample values for dest (possibly with 'ab' instead of 'wb' for appending):\n * io.FileIO(filename, 'wb')\n * io.open(filename, 'wb') - better with buffering=0, or use io.FileIO() instead\n * open(filename, 'wb')    - better with buffering=0, or use io.FileIO() instead\n * io.BytesIO()            - use owns_dest=False to access dest after closing\n                             the RecordWriter\n * tf.io.gfile.GFile(filename, 'wb')\n\nOptions are documented at\nhttps://github.com/google/riegeli/blob/master/doc/record_writer_options.md\n)doc\",                                                              // tp_doc\n    reinterpret_cast<traverseproc>(RecordWriterTraverse),  // tp_traverse\n    reinterpret_cast<inquiry>(RecordWriterClear),          // tp_clear\n    nullptr,                                               // tp_richcompare\n    0,                                                     // tp_weaklistoffset\n    nullptr,                                               // tp_iter\n    nullptr,                                               // tp_iternext\n    const_cast<PyMethodDef*>(RecordWriterMethods),         // tp_methods\n    nullptr,                                               // tp_members\n    const_cast<PyGetSetDef*>(RecordWriterGetSet),          // tp_getset\n    nullptr,                                               // tp_base\n    nullptr,                                               // tp_dict\n    nullptr,                                               // tp_descr_get\n    nullptr,                                               // tp_descr_set\n    0,                                                     // tp_dictoffset\n    reinterpret_cast<initproc>(RecordWriterInit),          // tp_init\n    nullptr,                                               // tp_alloc\n    PyType_GenericNew,                                     // tp_new\n    nullptr,                                               // tp_free\n    nullptr,                                               // tp_is_gc\n    nullptr,                                               // tp_bases\n    nullptr,                                               // tp_mro\n    nullptr,                                               // tp_cache\n    nullptr,                                               // tp_subclasses\n    nullptr,                                               // tp_weaklist\n    nullptr,                                               // tp_del\n    0,                                                     // tp_version_tag\n    nullptr,                                               // tp_finalize\n};\n\nconst char* const kModuleName = \"riegeli.records.record_writer\";\nconst char kModuleDoc[] = R\"doc(Writes records to a Riegeli/records file.)doc\";\n\nconst PyMethodDef kModuleMethods[] = {\n    {\"set_record_type\", reinterpret_cast<PyCFunction>(SetRecordType),\n     METH_VARARGS | METH_KEYWORDS,\n     R\"doc(\nset_record_type(metadata: RecordsMetadata, message_type: type[Message]) -> None\n\nSets record_type_name and file_descriptor in metadata.\n\nArgs:\n  metadata: Riegeli/records file metadata being filled, typically will become\n    the metadata argument of RecordWriter().\n  message_type: Promised type of records, typically the argument type of\n    RecordWriter.write_message().\n)doc\"},\n    {nullptr, nullptr, 0, nullptr},\n};\n\nPyModuleDef kModuleDef = {\n    PyModuleDef_HEAD_INIT,\n    kModuleName,                               // m_name\n    kModuleDoc,                                // m_doc\n    -1,                                        // m_size\n    const_cast<PyMethodDef*>(kModuleMethods),  // m_methods\n    nullptr,                                   // m_slots\n    nullptr,                                   // m_traverse\n    nullptr,                                   // m_clear\n    nullptr,                                   // m_free\n};\n\nPyObject* InitModule() {\n  if (ABSL_PREDICT_FALSE(PyType_Ready(&PyRecordWriter_Type) < 0)) {\n    return nullptr;\n  }\n  PythonPtr module(PyModule_Create(&kModuleDef));\n  if (ABSL_PREDICT_FALSE(module == nullptr)) return nullptr;\n  PyFlushType_Type = DefineFlushType().release();\n  if (ABSL_PREDICT_FALSE(PyFlushType_Type == nullptr)) return nullptr;\n  if (ABSL_PREDICT_FALSE(PyModule_AddObject(module.get(), \"FlushType\",\n                                            PyFlushType_Type) < 0)) {\n    return nullptr;\n  }\n  Py_INCREF(&PyRecordWriter_Type);\n  if (ABSL_PREDICT_FALSE(PyModule_AddObject(module.get(), \"RecordWriter\",\n                                            reinterpret_cast<PyObject*>(\n                                                &PyRecordWriter_Type)) < 0)) {\n    return nullptr;\n  }\n  return module.release();\n}\n\n}  // namespace\n\nPyMODINIT_FUNC PyInit_record_writer() { return InitModule(); }\n\n}  // namespace riegeli::python\n"
  },
  {
    "path": "python/riegeli/records/records_metadata.proto",
    "content": "edition = \"2024\";\n\npackage riegeli;\n\nimport \"google/protobuf/descriptor.proto\";\n\n// Information about a Riegeli/records file, which may be helpful to interpret\n// file contents.\nmessage RecordsMetadata {\n  // Human-readable explanation of what the file contains.\n  string file_comment = 1;\n\n  // If records are proto messages of a fixed type, the full name of their type.\n  string record_type_name = 2;\n\n  // If `record_type_name` is set, proto file descriptors which should contain\n  // the definition of that type and their dependencies (each file comes after\n  // all its dependencies).\n  //\n  // If `file_descriptor` is empty but `record_type_name` is set (not\n  // recommended), `record_type_name` can be interpreted in the context of an\n  // unspecified proto descriptor database.\n  repeated google.protobuf.FileDescriptorProto file_descriptor = 3;\n\n  // Options originally used to encode the file:\n  // https://github.com/google/riegeli/blob/master/doc/record_writer_options.md\n  //\n  // They are informative here, they are never necessary to decode the file.\n  string record_writer_options = 4;\n\n  // Number of records in the file, so that the reader can tune for it.\n  //\n  // This is informative, the actual number of records may differ.\n  int64 num_records = 5;\n\n  // Clients can define custom metadata in extensions of this message.\n  extensions 1000 to max;\n}\n"
  },
  {
    "path": "python/riegeli/records/skipped_region.py",
    "content": "# Copyright 2018 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\"\"\"Details about a skipped region of invalid file contents.\"\"\"\n\n__all__ = ('SkippedRegion',)\n\n\nclass SkippedRegion:\n  \"\"\"Details about a skipped region of invalid file contents.\n\n  Attributes:\n      begin: File position of the beginning of the skipped region, inclusive.\n      end: File position of the end of the skipped region, exclusive.\n      length: Length of the skipped region, in bytes.\n      message: Message explaining why the region is invalid.\n  \"\"\"\n\n  __slots__ = ('begin', 'end', 'message')\n\n  def __init__(self, begin, end, message):\n    if begin > end:\n      raise ValueError(f'Positions in the wrong order: {begin} > {end}')\n    self.begin = begin\n    self.end = end\n    self.message = message\n\n  @property\n  def length(self):\n    return self.end - self.begin\n\n  def __str__(self):\n    return f'[{self.begin}..{self.end}): {self.message}'\n\n  def __repr__(self):\n    return f'SkippedRegion({self.begin}, {self.end}, {self.message!r})'\n"
  },
  {
    "path": "python/riegeli/records/tests/BUILD",
    "content": "load(\"@com_google_protobuf//bazel:proto_library.bzl\", \"proto_library\")\nload(\"@com_google_protobuf//bazel:py_proto_library.bzl\", \"py_proto_library\")\nload(\"@rules_python//python:defs.bzl\", \"py_test\")\n\npackage(\n    default_visibility = [\n        \"//python/riegeli:__subpackages__\",\n    ],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\npy_test(\n    name = \"records_test\",\n    srcs = [\"records_test.py\"],\n    deps = [\n        \":records_test_py_pb2\",\n        \"//python/riegeli\",\n        \"@absl_py//absl/logging\",\n        \"@absl_py//absl/testing:absltest\",\n        \"@absl_py//absl/testing:parameterized\",\n        \"@com_google_protobuf//:protobuf_python\",\n    ],\n)\n\nproto_library(\n    name = \"records_test_proto\",\n    srcs = [\"records_test.proto\"],\n)\n\npy_proto_library(\n    name = \"records_test_py_pb2\",\n    deps = [\"records_test_proto\"],\n)\n"
  },
  {
    "path": "python/riegeli/records/tests/__init__.py",
    "content": ""
  },
  {
    "path": "python/riegeli/records/tests/records_test.proto",
    "content": "edition = \"2024\";\n\npackage riegeli.tests;\n\nmessage SimpleMessage {\n  int32 id = 1;\n  bytes payload = 2;\n}\n"
  },
  {
    "path": "python/riegeli/records/tests/records_test.py",
    "content": "# Copyright 2018 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport abc\nimport contextlib\nfrom enum import Enum\nimport io\nimport itertools\n\nfrom absl import logging\nfrom absl.testing import absltest\nfrom absl.testing import parameterized\nfrom google.protobuf import message\nimport riegeli\nfrom riegeli.records.tests import records_test_pb2\n\n\ndef combine_named_parameters(*testcase_sets):\n  \"\"\"Allows a parameterized test with multiple independent parameters.\n\n  Example:\n    combine_named_parameters([('BytesIO', BytesIOSpec), ('FileIO', FileIOSpec)],\n                             [('serial', 0), ('parallel', 10)])\n      yields the same elements as\n    [('BytesIO_serial', BytesIOSpec, 0), ('BytesIO_parallel', BytesIOSpec, 10),\n     ('FileIO_serial', FileIOSpec, 0), ('FileIO_parallel', FileIOSpec, 10)]\n  \"\"\"\n  for combination in itertools.product(*testcase_sets):\n    key = '_'.join(name for name, _ in combination)\n    values = [value for _, value in combination]\n    yield tuple([key] + values)\n\n\nclass RandomAccess(Enum):\n  RANDOM_ACCESS = 1\n  SEQUENTIAL_ACCESS_DETECTED = 2\n  SEQUENTIAL_ACCESS_EXPLICIT = 3\n\n\nclass FakeFile:\n  __slots__ = ('_random_access',)\n\n  def __init__(self, random_access):\n    self._random_access = random_access\n\n  def seekable(self):\n    return self._random_access\n\n  def tell(self):\n    if self._random_access:\n      return 0\n    raise NotImplementedError('tell()')\n\n  def __getattr__(self, name):\n    raise NotImplementedError(f'{name}()')\n\n\nclass UnseekableWrapper:\n  __slots__ = ('_wrapped',)\n\n  def __init__(self, wrapped):\n    self._wrapped = wrapped\n\n  def seekable(self):\n    return False\n\n  def tell(self, *args):\n    raise NotImplementedError('tell()')\n\n  def seek(self, *args):\n    raise NotImplementedError('seek()')\n\n  def __getattr__(self, name):\n    return getattr(self._wrapped, name)\n\n\nclass FileSpecBase(metaclass=abc.ABCMeta):\n  __slots__ = ('_random_access', '_file')\n\n  def __init__(self, create_tempfile, random_access):\n    self._random_access = random_access\n    self._file = None\n\n  @abc.abstractmethod\n  def _open_for_writing(self):\n    raise NotImplementedError('_open_for_writing()')\n\n  def writing_open(self):\n    self._open_for_writing()\n    logging.debug('Opened %r for writing', self._file)\n    if self._random_access is RandomAccess.RANDOM_ACCESS:\n      return self._file\n    else:\n      return UnseekableWrapper(self._file)\n\n  @property\n  def writing_should_close(self):\n    return True\n\n  @property\n  def writing_assumed_pos(self):\n    if self._random_access is RandomAccess.SEQUENTIAL_ACCESS_EXPLICIT:\n      return 0\n    return None\n\n  @abc.abstractmethod\n  def _open_for_reading(self):\n    raise NotImplementedError('_open_for_reading()')\n\n  def reading_open(self):\n    self._open_for_reading()\n    logging.debug('Opened %r for reading', self._file)\n    if self._random_access is RandomAccess.RANDOM_ACCESS:\n      return self._file\n    else:\n      return UnseekableWrapper(self._file)\n\n  @property\n  def reading_should_close(self):\n    return True\n\n  @property\n  def reading_assumed_pos(self):\n    if self._random_access is RandomAccess.SEQUENTIAL_ACCESS_EXPLICIT:\n      return 0\n    return None\n\n  def close(self):\n    pass\n\n\nclass BytesIOSpec(FileSpecBase):\n  __slots__ = ()\n\n  def _open_for_writing(self):\n    self._file = io.BytesIO()\n\n  @property\n  def writing_should_close(self):\n    # If BytesIO is closed, it loses bytes written.\n    return False\n\n  def _open_for_reading(self):\n    if self._file is None:\n      raise ValueError('file was not set')\n    self._file.seek(0)\n\n\nclass LocalFileSpecBase(FileSpecBase):\n  __slots__ = ('_filename',)\n\n  def __init__(self, create_tempfile, random_access):\n    super().__init__(create_tempfile, random_access)\n    self._filename = create_tempfile().full_path\n\n\nclass FileIOSpec(LocalFileSpecBase):\n  __slots__ = ()\n\n  def _open_for_writing(self):\n    self._file = io.FileIO(self._filename, mode='wb')\n\n  def _open_for_reading(self):\n    self._file = io.FileIO(self._filename, mode='rb')\n\n\nclass BufferedIOSpec(LocalFileSpecBase):\n  __slots__ = ()\n\n  def _open_for_writing(self):\n    self._file = io.open(self._filename, mode='wb')\n\n  def _open_for_reading(self):\n    self._file = io.open(self._filename, mode='rb')\n\n\nclass BuiltinFileSpec(LocalFileSpecBase):\n  __slots__ = ()\n\n  def _open_for_writing(self):\n    self._file = open(self._filename, mode='wb')\n\n  def _open_for_reading(self):\n    self._file = open(self._filename, mode='rb')\n\n\ndef sample_string(i, size):\n  piece = f'{i} '.encode()\n  result = piece * -(-size // len(piece))  # len(result) >= size\n  return result[:size]\n\n\ndef sample_message(i, size):\n  return records_test_pb2.SimpleMessage(id=i, payload=sample_string(i, size))\n\n\ndef sample_message_id_only(i):\n  return records_test_pb2.SimpleMessage(id=i)\n\n\ndef sample_invalid_message(size):\n  return b'\\xff' * size  # An unfinished varint.\n\n\ndef record_writer_options(parallelism, transpose=False, chunk_size=35000):\n  return (\n      f'{\"transpose,\" if transpose else \"\"}uncompressed,'\n      f'chunk_size:{chunk_size},parallelism:{parallelism}'\n  )\n\n\n_FILE_SPEC_VALUES = (\n    ('BytesIO', BytesIOSpec),\n    ('FileIO', FileIOSpec),\n    ('BufferedIO', BufferedIOSpec),\n    ('BuiltinFile', BuiltinFileSpec),\n)\n\n_RANDOM_ACCESS_VALUES = (\n    ('randomAccess', RandomAccess.RANDOM_ACCESS),\n    ('sequentialAccessDetected', RandomAccess.SEQUENTIAL_ACCESS_DETECTED),\n    ('sequentialAccessExplicit', RandomAccess.SEQUENTIAL_ACCESS_EXPLICIT),\n)\n\n_PARALLELISM_VALUES = (('serial', 0), ('parallel', 10))\n\n_PARAMETERIZE_BY_FILE_SPEC = parameterized.named_parameters(*_FILE_SPEC_VALUES)\n\n_PARAMETERIZE_BY_RANDOM_ACCESS = parameterized.named_parameters(\n    *_RANDOM_ACCESS_VALUES\n)\n\n_PARAMETERIZE_BY_RANDOM_ACCESS_AND_PARALLELISM = parameterized.named_parameters(\n    combine_named_parameters(_RANDOM_ACCESS_VALUES, _PARALLELISM_VALUES)\n)\n\n_PARAMETERIZE_BY_FILE_SPEC_AND_PARALLELISM = parameterized.named_parameters(\n    combine_named_parameters(_FILE_SPEC_VALUES, _PARALLELISM_VALUES)\n)\n\n_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM = (\n    parameterized.named_parameters(\n        combine_named_parameters(\n            _FILE_SPEC_VALUES, _RANDOM_ACCESS_VALUES, _PARALLELISM_VALUES\n        )\n    )\n)\n\n\nclass RecordsTest(parameterized.TestCase):\n\n  def corrupt_at(self, files, index):\n    byte_reader = files.reading_open()\n    contents1 = byte_reader.read(index)\n    contents2 = byte_reader.read(1)\n    contents2 = bytes([(contents2[0] + 1) % 256])\n    contents3 = byte_reader.read()\n    if files.reading_should_close:\n      byte_reader.close()\n    byte_writer = files.writing_open()\n    byte_writer.write(contents1)\n    byte_writer.write(contents2)\n    byte_writer.write(contents3)\n    if files.writing_should_close:\n      byte_writer.close()\n\n  @_PARAMETERIZE_BY_RANDOM_ACCESS_AND_PARALLELISM\n  def test_record_writer_exception_from_file(self, random_access, parallelism):\n    byte_writer = FakeFile(random_access is RandomAccess.RANDOM_ACCESS)\n    with self.assertRaises(NotImplementedError):\n      with riegeli.RecordWriter(\n          byte_writer,\n          assumed_pos=(\n              0\n              if random_access is RandomAccess.SEQUENTIAL_ACCESS_EXPLICIT\n              else None\n          ),\n          options=record_writer_options(parallelism),\n      ) as writer:\n        writer.write_record(sample_string(0, 10000))\n\n  @_PARAMETERIZE_BY_RANDOM_ACCESS\n  def test_record_reader_exception_from_file(self, random_access):\n    byte_reader = FakeFile(random_access is RandomAccess.RANDOM_ACCESS)\n    with self.assertRaises(NotImplementedError):\n      with riegeli.RecordReader(\n          byte_reader,\n          owns_src=False,\n          assumed_pos=(\n              0\n              if random_access is RandomAccess.SEQUENTIAL_ACCESS_EXPLICIT\n              else None\n          ),\n      ) as reader:\n        reader.read_record()\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_write_read_record(self, file_spec, random_access, parallelism):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      positions = []\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n      ) as writer:\n        for i in range(23):\n          pos = writer.pos\n          writer.write_record(sample_string(i, 10000))\n          canonical_pos = writer.last_pos\n          if positions:\n            self.assertGreater(pos, positions[-1])\n          self.assertLessEqual(pos, canonical_pos)\n          positions.append(canonical_pos)\n        writer.close()\n        end_pos = writer.pos\n        self.assertEqual(writer.last_pos, positions[-1])\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n        for i in range(23):\n          pos = reader.pos\n          self.assertEqual(reader.read_record(), sample_string(i, 10000))\n          canonical_pos = reader.last_pos\n          self.assertEqual(canonical_pos, positions[i])\n          self.assertLessEqual(pos, canonical_pos)\n        self.assertIsNone(reader.read_record())\n        self.assertEqual(reader.pos, end_pos)\n        reader.close()\n        self.assertEqual(reader.pos, end_pos)\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_write_read_message(self, file_spec, random_access, parallelism):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      positions = []\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n      ) as writer:\n        for i in range(23):\n          pos = writer.pos\n          writer.write_message(sample_message(i, 10000))\n          canonical_pos = writer.last_pos\n          if positions:\n            self.assertGreater(pos, positions[-1])\n          self.assertLessEqual(pos, canonical_pos)\n          positions.append(canonical_pos)\n        writer.close()\n        end_pos = writer.pos\n        self.assertEqual(writer.last_pos, positions[-1])\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n        for i in range(23):\n          pos = reader.pos\n          self.assertEqual(\n              reader.read_message(records_test_pb2.SimpleMessage),\n              sample_message(i, 10000),\n          )\n          canonical_pos = reader.last_pos\n          self.assertEqual(canonical_pos, positions[i])\n          self.assertLessEqual(pos, canonical_pos)\n        self.assertIsNone(reader.read_message(records_test_pb2.SimpleMessage))\n        self.assertEqual(reader.pos, end_pos)\n        reader.close()\n        self.assertEqual(reader.pos, end_pos)\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_write_read_records(self, file_spec, random_access, parallelism):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n      ) as writer:\n        writer.write_records(sample_string(i, 10000) for i in range(23))\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n        self.assertEqual(\n            list(reader.read_records()),\n            [sample_string(i, 10000) for i in range(23)],\n        )\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_write_read_messages(self, file_spec, random_access, parallelism):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n      ) as writer:\n        writer.write_messages(sample_message(i, 10000) for i in range(23))\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n        self.assertEqual(\n            list(reader.read_messages(records_test_pb2.SimpleMessage)),\n            [sample_message(i, 10000) for i in range(23)],\n        )\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_write_read_messages_with_field_projection(\n      self, file_spec, random_access, parallelism\n  ):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism, transpose=True),\n      ) as writer:\n        writer.write_messages(sample_message(i, 10000) for i in range(23))\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n          field_projection=[[\n              records_test_pb2.SimpleMessage.DESCRIPTOR.fields_by_name[\n                  'id'\n              ].number\n          ]],\n      ) as reader:\n        self.assertEqual(\n            list(reader.read_messages(records_test_pb2.SimpleMessage)),\n            [sample_message_id_only(i) for i in range(23)],\n        )\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_PARALLELISM\n  def test_write_read_messages_with_field_projection_later(\n      self, file_spec, parallelism\n  ):\n    with contextlib.closing(\n        file_spec(\n            self.create_tempfile, random_access=RandomAccess.RANDOM_ACCESS\n        )\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism, transpose=True),\n      ) as writer:\n        writer.write_messages(sample_message(i, 10000) for i in range(23))\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n        for i in range(4):\n          self.assertEqual(\n              reader.read_message(records_test_pb2.SimpleMessage),\n              sample_message(i, 10000),\n          )\n        reader.set_field_projection([[\n            records_test_pb2.SimpleMessage.DESCRIPTOR.fields_by_name[\n                'id'\n            ].number\n        ]])\n        for i in range(4, 14):\n          self.assertEqual(\n              reader.read_message(records_test_pb2.SimpleMessage),\n              sample_message_id_only(i),\n          )\n        reader.set_field_projection(None)\n        for i in range(14, 23):\n          self.assertEqual(\n              reader.read_message(records_test_pb2.SimpleMessage),\n              sample_message(i, 10000),\n          )\n        self.assertIsNone(reader.read_message(records_test_pb2.SimpleMessage))\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_write_read_metadata(self, file_spec, random_access, parallelism):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      metadata_written = riegeli.RecordsMetadata()\n      metadata_written.file_comment = 'Comment'\n      riegeli.set_record_type(metadata_written, records_test_pb2.SimpleMessage)\n      message_written = sample_message(7, 10)\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n          metadata=metadata_written,\n      ) as writer:\n        writer.write_message(message_written)\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n        metadata_read = reader.read_metadata()\n        self.assertEqual(metadata_read, metadata_written)\n        record_type = riegeli.get_record_type(metadata_read)\n        assert record_type is not None\n        self.assertEqual(\n            record_type.DESCRIPTOR.full_name, 'riegeli.tests.SimpleMessage'\n        )\n        message_read = reader.read_message(record_type)\n        assert message_read is not None\n        # Serialize and deserialize because messages have descriptors of\n        # different origins.\n        self.assertEqual(\n            records_test_pb2.SimpleMessage.FromString(\n                message_read.SerializeToString()\n            ),\n            message_written,\n        )\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_invalid_metadata_exception(\n      self, file_spec, random_access, parallelism\n  ):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n          serialized_metadata=sample_invalid_message(100),\n      ):\n        pass\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n        with self.assertRaises(message.DecodeError):\n          reader.read_metadata()\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_invalid_metadata_recovery(\n      self, file_spec, random_access, parallelism\n  ):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n          serialized_metadata=sample_invalid_message(100),\n      ):\n        pass\n\n      def recovery(skipped_region):\n        pass\n\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n          recovery=recovery,\n      ) as reader:\n        self.assertEqual(reader.read_metadata(), riegeli.RecordsMetadata())\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_invalid_metadata_recovery_stop_iteration(\n      self, file_spec, random_access, parallelism\n  ):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n          serialized_metadata=sample_invalid_message(100),\n      ):\n        pass\n\n      def recovery(skipped_region):\n        raise StopIteration\n\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n          recovery=recovery,\n      ) as reader:\n        self.assertIsNone(reader.read_metadata())\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_field_projection(self, file_spec, random_access, parallelism):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=f'{record_writer_options(parallelism)},transpose',\n      ) as writer:\n        for i in range(23):\n          writer.write_message(sample_message(i, 10000))\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n          field_projection=[[\n              records_test_pb2.SimpleMessage.DESCRIPTOR.fields_by_name[\n                  'id'\n              ].number\n          ]],\n      ) as reader:\n        for i in range(23):\n          self.assertEqual(\n              reader.read_message(records_test_pb2.SimpleMessage),\n              records_test_pb2.SimpleMessage(id=i),\n          )\n        self.assertIsNone(reader.read_message(records_test_pb2.SimpleMessage))\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_field_projection_existence_only(\n      self, file_spec, random_access, parallelism\n  ):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=f'{record_writer_options(parallelism)},transpose',\n      ) as writer:\n        for i in range(23):\n          writer.write_message(sample_message(i, 10000))\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n          field_projection=[\n              [\n                  records_test_pb2.SimpleMessage.DESCRIPTOR.fields_by_name[\n                      'id'\n                  ].number\n              ],\n              [\n                  records_test_pb2.SimpleMessage.DESCRIPTOR.fields_by_name[\n                      'payload'\n                  ].number,\n                  riegeli.EXISTENCE_ONLY,\n              ],\n          ],\n      ) as reader:\n        for i in range(23):\n          self.assertEqual(\n              reader.read_message(records_test_pb2.SimpleMessage),\n              records_test_pb2.SimpleMessage(id=i, payload=b''),\n          )\n        self.assertIsNone(reader.read_message(records_test_pb2.SimpleMessage))\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_PARALLELISM\n  def test_seek(self, file_spec, parallelism):\n    with contextlib.closing(\n        file_spec(\n            self.create_tempfile, random_access=RandomAccess.RANDOM_ACCESS\n        )\n    ) as files:\n      positions = []\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n      ) as writer:\n        for i in range(23):\n          pos = writer.pos\n          writer.write_record(sample_string(i, 10000))\n          canonical_pos = writer.last_pos\n          if positions:\n            self.assertGreater(pos, positions[-1])\n          self.assertLessEqual(pos, canonical_pos)\n          positions.append(canonical_pos)\n        writer.close()\n        end_pos = writer.pos\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n        reader.seek(positions[9])\n        self.assertGreater(reader.pos, positions[8])\n        self.assertLessEqual(reader.pos, positions[9])\n        reader.seek(positions[9])\n        self.assertGreater(reader.pos, positions[8])\n        self.assertLessEqual(reader.pos, positions[9])\n        reader.seek(positions[11])\n        self.assertGreater(reader.pos, positions[10])\n        self.assertLessEqual(reader.pos, positions[11])\n        self.assertEqual(reader.read_record(), sample_string(11, 10000))\n        reader.seek(positions[9])\n        self.assertGreater(reader.pos, positions[8])\n        self.assertLessEqual(reader.pos, positions[9])\n        self.assertEqual(reader.read_record(), sample_string(9, 10000))\n        reader.seek(positions[11])\n        self.assertGreater(reader.pos, positions[10])\n        self.assertLessEqual(reader.pos, positions[11])\n        self.assertEqual(reader.read_record(), sample_string(11, 10000))\n        reader.seek(positions[13])\n        self.assertGreater(reader.pos, positions[12])\n        self.assertLessEqual(reader.pos, positions[13])\n        self.assertEqual(reader.read_record(), sample_string(13, 10000))\n        reader.seek(riegeli.RecordPosition(0, 0))\n        self.assertLessEqual(reader.pos, positions[0])\n        self.assertEqual(reader.read_record(), sample_string(0, 10000))\n        reader.seek(end_pos)\n        self.assertLessEqual(reader.pos, end_pos)\n        self.assertIsNone(reader.read_record())\n        reader.seek(positions[11])\n        self.assertGreater(reader.pos, positions[10])\n        self.assertLessEqual(reader.pos, positions[11])\n        reader.close()\n        self.assertGreater(reader.pos, positions[10])\n        self.assertLessEqual(reader.pos, positions[11])\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_PARALLELISM\n  def test_seek_numeric(self, file_spec, parallelism):\n    with contextlib.closing(\n        file_spec(\n            self.create_tempfile, random_access=RandomAccess.RANDOM_ACCESS\n        )\n    ) as files:\n      positions = []\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n      ) as writer:\n        for i in range(23):\n          pos = writer.pos\n          writer.write_record(sample_string(i, 10000))\n          canonical_pos = writer.last_pos\n          if positions:\n            self.assertGreater(pos, positions[-1])\n          self.assertLessEqual(pos, canonical_pos)\n          positions.append(canonical_pos)\n        writer.close()\n        end_pos = writer.pos\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n        reader.seek_numeric(positions[9].numeric)\n        self.assertGreater(reader.pos, positions[8])\n        self.assertLessEqual(reader.pos, positions[9])\n        reader.seek_numeric(positions[9].numeric)\n        self.assertGreater(reader.pos, positions[8])\n        self.assertLessEqual(reader.pos, positions[9])\n        reader.seek_numeric(positions[11].numeric)\n        self.assertGreater(reader.pos, positions[10])\n        self.assertLessEqual(reader.pos, positions[11])\n        self.assertEqual(reader.read_record(), sample_string(11, 10000))\n        reader.seek_numeric(positions[9].numeric)\n        self.assertGreater(reader.pos, positions[8])\n        self.assertLessEqual(reader.pos, positions[9])\n        self.assertEqual(reader.read_record(), sample_string(9, 10000))\n        reader.seek_numeric(positions[11].numeric)\n        self.assertGreater(reader.pos, positions[10])\n        self.assertLessEqual(reader.pos, positions[11])\n        self.assertEqual(reader.read_record(), sample_string(11, 10000))\n        reader.seek_numeric(positions[13].numeric)\n        self.assertGreater(reader.pos, positions[12])\n        self.assertLessEqual(reader.pos, positions[13])\n        self.assertEqual(reader.read_record(), sample_string(13, 10000))\n        reader.seek_numeric(0)\n        self.assertLessEqual(reader.pos, positions[0])\n        self.assertEqual(reader.read_record(), sample_string(0, 10000))\n        reader.seek_numeric(end_pos.numeric)\n        self.assertLessEqual(reader.pos, end_pos)\n        self.assertIsNone(reader.read_record())\n        reader.seek_numeric(positions[11].numeric)\n        self.assertGreater(reader.pos, positions[10])\n        self.assertLessEqual(reader.pos, positions[11])\n        reader.close()\n        self.assertGreater(reader.pos, positions[10])\n        self.assertLessEqual(reader.pos, positions[11])\n\n  @_PARAMETERIZE_BY_FILE_SPEC\n  def test_seek_back(self, file_spec):\n    with contextlib.closing(\n        file_spec(\n            self.create_tempfile, random_access=RandomAccess.RANDOM_ACCESS\n        )\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism=0),\n      ) as writer:\n        for i in range(23):\n          writer.write_record(sample_string(i, 10000))\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n        reader.seek_numeric(reader.size())\n        for i in reversed(range(23)):\n          self.assertTrue(reader.seek_back())\n          self.assertEqual(reader.read_record(), sample_string(i, 10000))\n          self.assertTrue(reader.seek_back())\n        self.assertFalse(reader.seek_back())\n\n  @_PARAMETERIZE_BY_FILE_SPEC\n  def test_search(self, file_spec):\n    with contextlib.closing(\n        file_spec(\n            self.create_tempfile, random_access=RandomAccess.RANDOM_ACCESS\n        )\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism=0),\n      ) as writer:\n        positions = []\n        for i in range(23):\n          writer.write_message(sample_message(i, 10000))\n          positions.append(writer.last_pos)\n        writer.close()\n        end_pos = writer.pos\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n\n        def test_function(search_target):\n          def test(record_reader):\n            msg = record_reader.read_message(records_test_pb2.SimpleMessage)\n            return (msg.id > search_target) - (msg.id < search_target)\n\n          return test\n\n        self.assertEqual(reader.search(test_function(7)), 0)\n        self.assertEqual(reader.pos, positions[7])\n        self.assertEqual(reader.search(test_function(0)), 0)\n        self.assertEqual(reader.pos, positions[0])\n        self.assertEqual(reader.search(test_function(22)), 0)\n        self.assertEqual(reader.pos, positions[22])\n        self.assertEqual(reader.search(test_function(23)), -1)\n        self.assertEqual(reader.pos, end_pos)\n\n  @_PARAMETERIZE_BY_FILE_SPEC\n  def test_search_for_record(self, file_spec):\n    with contextlib.closing(\n        file_spec(\n            self.create_tempfile, random_access=RandomAccess.RANDOM_ACCESS\n        )\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism=0),\n      ) as writer:\n        positions = []\n        for i in range(23):\n          writer.write_message(sample_message(i, 10000))\n          positions.append(writer.last_pos)\n        writer.close()\n        end_pos = writer.pos\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n\n        def test_function(search_target):\n          def test(record):\n            msg = records_test_pb2.SimpleMessage.FromString(record)\n            return (msg.id > search_target) - (msg.id < search_target)\n\n          return test\n\n        self.assertEqual(reader.search_for_record(test_function(7)), 0)\n        self.assertEqual(reader.pos, positions[7])\n        self.assertEqual(reader.search_for_record(test_function(0)), 0)\n        self.assertEqual(reader.pos, positions[0])\n        self.assertEqual(reader.search_for_record(test_function(22)), 0)\n        self.assertEqual(reader.pos, positions[22])\n        self.assertEqual(reader.search_for_record(test_function(23)), -1)\n        self.assertEqual(reader.pos, end_pos)\n\n  @_PARAMETERIZE_BY_FILE_SPEC\n  def test_search_for_record_stop_iteration(self, file_spec):\n    with contextlib.closing(\n        file_spec(\n            self.create_tempfile, random_access=RandomAccess.RANDOM_ACCESS\n        )\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism=0),\n      ) as writer:\n        for i in range(23):\n          writer.write_message(sample_message(i, 10000))\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n\n        def test(record):\n          raise StopIteration\n\n        self.assertIsNone(reader.search_for_record(test))\n\n  @_PARAMETERIZE_BY_FILE_SPEC\n  def test_search_for_message(self, file_spec):\n    with contextlib.closing(\n        file_spec(\n            self.create_tempfile, random_access=RandomAccess.RANDOM_ACCESS\n        )\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism=0),\n      ) as writer:\n        positions = []\n        for i in range(23):\n          writer.write_message(sample_message(i, 10000))\n          positions.append(writer.last_pos)\n        writer.close()\n        end_pos = writer.pos\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n\n        def test_function(search_target):\n\n          def test(msg):\n            return (msg.id > search_target) - (msg.id < search_target)\n\n          return test\n\n        self.assertEqual(\n            reader.search_for_message(\n                records_test_pb2.SimpleMessage, test_function(7)\n            ),\n            0,\n        )\n        self.assertEqual(reader.pos, positions[7])\n        self.assertEqual(\n            reader.search_for_message(\n                records_test_pb2.SimpleMessage, test_function(0)\n            ),\n            0,\n        )\n        self.assertEqual(reader.pos, positions[0])\n        self.assertEqual(\n            reader.search_for_message(\n                records_test_pb2.SimpleMessage, test_function(22)\n            ),\n            0,\n        )\n        self.assertEqual(reader.pos, positions[22])\n        self.assertEqual(\n            reader.search_for_message(\n                records_test_pb2.SimpleMessage, test_function(23)\n            ),\n            -1,\n        )\n        self.assertEqual(reader.pos, end_pos)\n\n  @_PARAMETERIZE_BY_FILE_SPEC\n  def test_search_for_message_stop_iteration(self, file_spec):\n    with contextlib.closing(\n        file_spec(\n            self.create_tempfile, random_access=RandomAccess.RANDOM_ACCESS\n        )\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism=0),\n      ) as writer:\n        for i in range(23):\n          writer.write_message(sample_message(i, 10000))\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n\n        def test(record):\n          raise StopIteration\n\n        self.assertIsNone(\n            reader.search_for_message(records_test_pb2.SimpleMessage, test)\n        )\n\n  @_PARAMETERIZE_BY_FILE_SPEC\n  def test_search_for_invalid_message_exception(self, file_spec):\n    with contextlib.closing(\n        file_spec(\n            self.create_tempfile, random_access=RandomAccess.RANDOM_ACCESS\n        )\n    ) as files:\n      # Write 1 valid message, 1 invalid message, and 1 valid message, each in a\n      # separate chunk.\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism=0, chunk_size=15000),\n      ) as writer:\n        writer.write_message(sample_message(0, 10000))\n        writer.write_record(sample_invalid_message(10000))\n        writer.write_message(sample_message(2, 10000))\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n\n        def test_function(search_target):\n          def test(msg):\n            return (msg.id > search_target) - (msg.id < search_target)\n\n          return test\n\n        with self.assertRaises(message.DecodeError):\n          reader.search_for_message(\n              records_test_pb2.SimpleMessage, test_function(1)\n          )\n\n  @_PARAMETERIZE_BY_FILE_SPEC\n  def test_search_for_invalid_message_recovery(self, file_spec):\n    with contextlib.closing(\n        file_spec(\n            self.create_tempfile, random_access=RandomAccess.RANDOM_ACCESS\n        )\n    ) as files:\n      # Write 1 valid message, 1 invalid message, and 1 valid message, each in a\n      # separate chunk.\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism=0, chunk_size=15000),\n      ) as writer:\n        positions = []\n        writer.write_message(sample_message(0, 10000))\n        positions.append(writer.last_pos)\n        writer.write_record(sample_invalid_message(10000))\n        positions.append(writer.last_pos)\n        writer.write_message(sample_message(2, 10000))\n        positions.append(writer.last_pos)\n        writer.close()\n\n      def recovery(skipped_region):\n        pass\n\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n          recovery=recovery,\n      ) as reader:\n\n        def test_function(search_target):\n          def test(msg):\n            return (msg.id > search_target) - (msg.id < search_target)\n\n          return test\n\n        self.assertEqual(\n            reader.search_for_message(\n                records_test_pb2.SimpleMessage, test_function(1)\n            ),\n            1,\n        )\n        self.assertEqual(reader.pos, positions[2])\n\n  @_PARAMETERIZE_BY_FILE_SPEC\n  def test_search_for_invalid_message_recovery_stop_iteration(self, file_spec):\n    with contextlib.closing(\n        file_spec(\n            self.create_tempfile, random_access=RandomAccess.RANDOM_ACCESS\n        )\n    ) as files:\n      # Write 1 valid message, 1 invalid message, and 1 valid message, each in a\n      # separate chunk.\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism=0, chunk_size=15000),\n      ) as writer:\n        positions = []\n        writer.write_message(sample_message(0, 10000))\n        positions.append(writer.last_pos)\n        writer.write_record(sample_invalid_message(10000))\n        positions.append(writer.last_pos)\n        writer.write_message(sample_message(2, 10000))\n        positions.append(writer.last_pos)\n        writer.close()\n\n      def recovery(skipped_region):\n        raise StopIteration\n\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n          recovery=recovery,\n      ) as reader:\n\n        def test_function(search_target):\n          def test(msg):\n            return (msg.id > search_target) - (msg.id < search_target)\n\n          return test\n\n        self.assertIsNone(\n            reader.search_for_message(\n                records_test_pb2.SimpleMessage, test_function(1)\n            )\n        )\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_corruption_exception(self, file_spec, random_access, parallelism):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      positions = []\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n      ) as writer:\n        for i in range(23):\n          writer.write_record(sample_string(i, 10000))\n          positions.append(writer.last_pos)\n      # Corrupt the header of the chunk containing records [9..12).\n      self.corrupt_at(files, positions[9].chunk_begin + 20)\n      # Read records [0..9) successfully (all before the corrupted chunk).\n      reader = riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      )\n      for i in range(9):\n        self.assertEqual(reader.read_record(), sample_string(i, 10000))\n      with self.assertRaises(riegeli.RiegeliError):\n        reader.read_record()\n      with self.assertRaises(riegeli.RiegeliError):\n        reader.close()\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_corruption_recovery(self, file_spec, random_access, parallelism):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      positions = []\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n      ) as writer:\n        for i in range(23):\n          writer.write_record(sample_string(i, 10000))\n          positions.append(writer.last_pos)\n      # Corrupt the header of the chunk containing records [9..12).\n      self.corrupt_at(files, positions[9].chunk_begin + 20)\n      # Read records [0..9) and [15..23) successfully (all except the corrupted\n      # chunk and the next chunk which intersects the same block).\n      skipped_regions = []\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n          recovery=skipped_regions.append,\n      ) as reader:\n        for i in range(9):\n          self.assertEqual(reader.read_record(), sample_string(i, 10000))\n        for i in range(15, 23):\n          self.assertEqual(reader.read_record(), sample_string(i, 10000))\n        self.assertIsNone(reader.read_record())\n      self.assertLen(skipped_regions, 1)\n      skipped_region = skipped_regions[0]\n      self.assertEqual(skipped_region.begin, positions[9].numeric)\n      self.assertEqual(skipped_region.end, positions[15].numeric)\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_corruption_recovery_stop_iteration(\n      self, file_spec, random_access, parallelism\n  ):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      positions = []\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n      ) as writer:\n        for i in range(23):\n          writer.write_record(sample_string(i, 10000))\n          positions.append(writer.last_pos)\n      # Corrupt the header of the chunk containing records [9..12).\n      self.corrupt_at(files, positions[9].chunk_begin + 20)\n      # Read records [0..9) successfully (all before the corrupted chunk).\n      skipped_regions = []\n\n      def recovery(skipped_region):\n        skipped_regions.append(skipped_region)\n        raise StopIteration\n\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n          recovery=recovery,\n      ) as reader:\n        for i in range(9):\n          self.assertEqual(reader.read_record(), sample_string(i, 10000))\n        self.assertIsNone(reader.read_record())\n      self.assertLen(skipped_regions, 1)\n      skipped_region = skipped_regions[0]\n      self.assertEqual(skipped_region.begin, positions[9].numeric)\n      self.assertEqual(skipped_region.end, positions[15].numeric)\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_corruption_recovery_exception(\n      self, file_spec, random_access, parallelism\n  ):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      positions = []\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n      ) as writer:\n        for i in range(23):\n          writer.write_record(sample_string(i, 10000))\n          positions.append(writer.last_pos)\n      # Corrupt the header of the chunk containing records [9..12).\n      self.corrupt_at(files, positions[9].chunk_begin + 20)\n\n      # Propagate exception from the recovery function\n      def recovery(skipped_region):\n        raise KeyboardInterrupt\n\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n          recovery=recovery,\n      ) as reader:\n        for i in range(9):\n          self.assertEqual(reader.read_record(), sample_string(i, 10000))\n        with self.assertRaises(KeyboardInterrupt):\n          reader.read_record()\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_invalid_message_exception(\n      self, file_spec, random_access, parallelism\n  ):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n      ) as writer:\n        for i in range(9):\n          writer.write_message(sample_message(i, 10000))\n        for i in range(9, 10):\n          writer.write_record(sample_invalid_message(10000))\n        for i in range(10, 14):\n          writer.write_message(sample_message(i, 10000))\n        for i in range(14, 15):\n          writer.write_record(sample_invalid_message(10000))\n        for i in range(15, 23):\n          writer.write_message(sample_message(i, 10000))\n      # Read messages [0..9), [10..14), and [15, 23) successfully (all except\n      # invalid messages), raising exceptions for invalid messages\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n      ) as reader:\n        for i in range(9):\n          self.assertEqual(\n              reader.read_message(records_test_pb2.SimpleMessage),\n              sample_message(i, 10000),\n          )\n        with self.assertRaises(message.DecodeError):\n          reader.read_message(records_test_pb2.SimpleMessage)\n        for i in range(10, 14):\n          self.assertEqual(\n              reader.read_message(records_test_pb2.SimpleMessage),\n              sample_message(i, 10000),\n          )\n        with self.assertRaises(message.DecodeError):\n          reader.read_message(records_test_pb2.SimpleMessage)\n        for i in range(15, 23):\n          self.assertEqual(\n              reader.read_message(records_test_pb2.SimpleMessage),\n              sample_message(i, 10000),\n          )\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_invalid_message_recovery(\n      self, file_spec, random_access, parallelism\n  ):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      positions = []\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n      ) as writer:\n        for i in range(9):\n          writer.write_message(sample_message(i, 10000))\n          positions.append(writer.last_pos)\n        for i in range(9, 10):\n          writer.write_record(sample_invalid_message(10000))\n          positions.append(writer.last_pos)\n        for i in range(10, 14):\n          writer.write_message(sample_message(i, 10000))\n          positions.append(writer.last_pos)\n        for i in range(14, 15):\n          writer.write_record(sample_invalid_message(10000))\n          positions.append(writer.last_pos)\n        for i in range(15, 23):\n          writer.write_message(sample_message(i, 10000))\n          positions.append(writer.last_pos)\n      # Read messages [0..9), [10..14), and [15, 23) successfully (all except\n      # invalid messages).\n      skipped_regions = []\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n          recovery=skipped_regions.append,\n      ) as reader:\n        for i in range(9):\n          self.assertEqual(\n              reader.read_message(records_test_pb2.SimpleMessage),\n              sample_message(i, 10000),\n          )\n        for i in range(10, 14):\n          self.assertEqual(\n              reader.read_message(records_test_pb2.SimpleMessage),\n              sample_message(i, 10000),\n          )\n        for i in range(15, 23):\n          self.assertEqual(\n              reader.read_message(records_test_pb2.SimpleMessage),\n              sample_message(i, 10000),\n          )\n        self.assertIsNone(reader.read_message(records_test_pb2.SimpleMessage))\n      self.assertLen(skipped_regions, 2)\n      skipped_region = skipped_regions[0]\n      self.assertEqual(skipped_region.begin, positions[9].numeric)\n      self.assertEqual(skipped_region.end, positions[10].numeric)\n      skipped_region = skipped_regions[1]\n      self.assertEqual(skipped_region.begin, positions[14].numeric)\n      self.assertEqual(skipped_region.end, positions[15].numeric)\n\n  @_PARAMETERIZE_BY_FILE_SPEC_AND_RANDOM_ACCESS_AND_PARALLELISM\n  def test_invalid_message_recovery_stop_iteration(\n      self, file_spec, random_access, parallelism\n  ):\n    with contextlib.closing(\n        file_spec(self.create_tempfile, random_access)\n    ) as files:\n      positions = []\n      with riegeli.RecordWriter(\n          files.writing_open(),\n          owns_dest=files.writing_should_close,\n          assumed_pos=files.writing_assumed_pos,\n          options=record_writer_options(parallelism),\n      ) as writer:\n        for i in range(9):\n          writer.write_message(sample_message(i, 10000))\n          positions.append(writer.last_pos)\n        for i in range(9, 10):\n          writer.write_record(sample_invalid_message(10000))\n          positions.append(writer.last_pos)\n        for i in range(10, 14):\n          writer.write_message(sample_message(i, 10000))\n          positions.append(writer.last_pos)\n        for i in range(14, 15):\n          writer.write_record(sample_invalid_message(10000))\n          positions.append(writer.last_pos)\n        for i in range(15, 23):\n          writer.write_message(sample_message(i, 10000))\n          positions.append(writer.last_pos)\n      # Read messages [0..9) successfully (all before the first invalid\n      # message).\n      skipped_regions = []\n\n      def recovery(skipped_region):\n        skipped_regions.append(skipped_region)\n        raise StopIteration\n\n      with riegeli.RecordReader(\n          files.reading_open(),\n          owns_src=files.reading_should_close,\n          assumed_pos=files.reading_assumed_pos,\n          recovery=recovery,\n      ) as reader:\n        for i in range(9):\n          self.assertEqual(\n              reader.read_message(records_test_pb2.SimpleMessage),\n              sample_message(i, 10000),\n          )\n        self.assertIsNone(reader.read_message(records_test_pb2.SimpleMessage))\n      self.assertLen(skipped_regions, 1)\n      skipped_region = skipped_regions[0]\n      self.assertEqual(skipped_region.begin, positions[9].numeric)\n      self.assertEqual(skipped_region.end, positions[10].numeric)\n\n\nif __name__ == '__main__':\n  absltest.main()\n"
  },
  {
    "path": "python/riegeli/tensorflow/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_binary\")\nload(\"@rules_python//python:defs.bzl\", \"py_library\", \"py_test\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\npy_library(\n    name = \"riegeli_dataset_ops\",\n    srcs = [\"ops/riegeli_dataset_ops.py\"],\n    data = [\":ops/_riegeli_dataset_ops.so\"],\n)\n\ncc_binary(\n    name = \"ops/_riegeli_dataset_ops.so\",\n    srcs = [\n        \"//riegeli/tensorflow:kernels/riegeli_dataset_ops.cc\",\n        \"//riegeli/tensorflow:ops/riegeli_dataset_ops.cc\",\n    ],\n    # tensorflow/core/lib/core/refcount.h needs NDEBUG consistency between\n    # translation units.\n    copts = [\"-DNDEBUG\"],\n    linkshared = True,\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/records:record_position\",\n        \"//riegeli/records:record_reader\",\n        \"//riegeli/records:skipped_region\",\n        \"//riegeli/tensorflow/io:file_reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/synchronization\",\n        \"@local_config_tf//:libtensorflow_framework\",\n        \"@local_config_tf//:tf_header_lib\",\n    ],\n)\n\npy_test(\n    name = \"riegeli_dataset_test\",\n    srcs = [\"kernel_tests/riegeli_dataset_test.py\"],\n    deps = [\n        \":riegeli_dataset_ops\",\n        \"//python/riegeli\",\n    ],\n)\n"
  },
  {
    "path": "python/riegeli/tensorflow/__init__.py",
    "content": ""
  },
  {
    "path": "python/riegeli/tensorflow/kernel_tests/__init__.py",
    "content": ""
  },
  {
    "path": "python/riegeli/tensorflow/kernel_tests/riegeli_dataset_test.py",
    "content": "# Copyright 2019 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\"\"\"Tests for RiegeliDataset.\"\"\"\n\nimport os\n\nimport riegeli\nfrom riegeli.tensorflow.ops import riegeli_dataset_ops\nimport tensorflow as tf\nfrom tensorflow.python.data.ops import dataset_ops\nfrom tensorflow.python.data.util import nest\nfrom tensorflow.python.eager import context\nfrom tensorflow.python.framework import errors\nfrom tensorflow.python.framework import sparse_tensor\nfrom tensorflow.python.framework import test_util\nfrom tensorflow.python.ops import tensor_array_ops\nfrom tensorflow.python.platform import test\n\n\n# Adapted from tensorflow/python/data/kernel_tests/test_base.py\n# which has restricted visibility.\nclass DatasetTestBase(test.TestCase):\n  \"\"\"Base class for dataset tests.\"\"\"\n\n  def assertValuesEqual(self, expected, actual):\n    \"\"\"Asserts that two values are equal.\"\"\"\n    if sparse_tensor.is_sparse(expected):\n      self.assertAllEqual(expected.indices, actual.indices)\n      self.assertAllEqual(expected.values, actual.values)\n      self.assertAllEqual(expected.dense_shape, actual.dense_shape)\n    else:\n      self.assertAllEqual(expected, actual)\n\n  def getNext(self, dataset, requires_initialization=False, shared_name=None):\n    \"\"\"Returns a callable that returns the next element of the dataset.\n\n    Example use:\n    ```python\n    # In both graph and eager modes\n    dataset = ...\n    get_next = self.getNext(dataset)\n    result = self.evaluate(get_next())\n    ```\n\n    Args:\n      dataset: A dataset whose elements will be returned.\n      requires_initialization: Indicates that when the test is executed in graph\n        mode, it should use an initializable iterator to iterate through the\n        dataset (e.g. when it contains stateful nodes). Defaults to False.\n      shared_name: (Optional.) If non-empty, the returned iterator will be\n        shared under the given name across multiple sessions that share the same\n        devices (e.g. when using a remote server).\n\n    Returns:\n      A callable that returns the next element of `dataset`. Any `TensorArray`\n      objects `dataset` outputs are stacked.\n    \"\"\"\n\n    def ta_wrapper(gn):\n      def _wrapper():\n        r = gn()\n        if isinstance(r, tensor_array_ops.TensorArray):\n          return r.stack()\n        else:\n          return r\n\n      return _wrapper\n\n    if context.executing_eagerly():\n      iterator = iter(dataset)\n      return ta_wrapper(iterator._next_internal)  # pylint: disable=protected-access\n    else:\n      if requires_initialization:\n        iterator = dataset_ops.make_initializable_iterator(dataset, shared_name)\n        self.evaluate(iterator.initializer)\n      else:\n        iterator = dataset_ops.make_one_shot_iterator(dataset)\n      get_next = iterator.get_next()\n      return ta_wrapper(lambda: get_next)\n\n  def _compareOutputToExpected(\n      self, result_values, expected_values, assert_items_equal\n  ):\n    if assert_items_equal:\n      self.assertItemsEqual(result_values, expected_values)\n      return\n    for i in range(len(result_values)):\n      nest.assert_same_structure(result_values[i], expected_values[i])\n      for result_value, expected_value in zip(\n          nest.flatten(result_values[i]), nest.flatten(expected_values[i])\n      ):\n        self.assertValuesEqual(expected_value, result_value)\n\n  def assertDatasetProduces(\n      self,\n      dataset,\n      expected_output=None,\n      expected_shapes=None,\n      expected_error=None,\n      requires_initialization=False,\n      num_test_iterations=1,\n      assert_items_equal=False,\n      expected_error_iter=1,\n  ):\n    \"\"\"Asserts that a dataset produces the expected output / error.\n\n    Args:\n      dataset: A dataset to check for the expected output / error.\n      expected_output: A list of elements that the dataset is expected to\n        produce.\n      expected_shapes: A list of TensorShapes which is expected to match\n        output_shapes of dataset.\n      expected_error: A tuple `(type, predicate)` identifying the expected error\n        `dataset` should raise. The `type` should match the expected exception\n        type, while `predicate` should either be 1) a unary function that inputs\n        the raised exception and returns a boolean indicator of success or 2) a\n        regular expression that is expected to match the error message\n        partially.\n      requires_initialization: Indicates that when the test is executed in graph\n        mode, it should use an initializable iterator to iterate through the\n        dataset (e.g. when it contains stateful nodes). Defaults to False.\n      num_test_iterations: Number of times `dataset` will be iterated. Defaults\n        to 2.\n      assert_items_equal: Tests expected_output has (only) the same elements\n        regardless of order.\n      expected_error_iter: How many times to iterate before expecting an error,\n        if an error is expected.\n    \"\"\"\n    self.assertTrue(\n        expected_error is not None or expected_output is not None,\n        'Exactly one of expected_output or expected error should be provided.',\n    )\n    if expected_error:\n      self.assertTrue(\n          expected_output is None,\n          (\n              'Exactly one of expected_output or expected error should be'\n              ' provided.'\n          ),\n      )\n      with self.assertRaisesWithPredicateMatch(\n          expected_error[0], expected_error[1]\n      ):\n        get_next = self.getNext(\n            dataset, requires_initialization=requires_initialization\n        )\n        for _ in range(expected_error_iter):\n          self.evaluate(get_next())\n      return\n    if expected_shapes:\n      self.assertEqual(\n          expected_shapes, dataset_ops.get_legacy_output_shapes(dataset)\n      )\n    self.assertGreater(num_test_iterations, 0)\n    for _ in range(num_test_iterations):\n      get_next = self.getNext(\n          dataset, requires_initialization=requires_initialization\n      )\n      result = []\n      for _ in range(len(expected_output)):\n        result.append(self.evaluate(get_next()))\n      self._compareOutputToExpected(result, expected_output, assert_items_equal)\n      with self.assertRaises(errors.OutOfRangeError):\n        self.evaluate(get_next())\n      with self.assertRaises(errors.OutOfRangeError):\n        self.evaluate(get_next())\n\n\n@test_util.run_all_in_graph_and_eager_modes\nclass RiegeliDatasetTest(DatasetTestBase):\n\n  def setUp(self):\n    super().setUp()\n    self._num_files = 2\n    self._num_records = 7\n\n    self.test_filenames = self._create_files()\n\n  def dataset_fn(self, filenames, num_epochs=1, batch_size=None):\n    repeat_dataset = riegeli_dataset_ops.RiegeliDataset(filenames).repeat(\n        num_epochs\n    )\n    if batch_size:\n      return repeat_dataset.batch(batch_size)\n    return repeat_dataset\n\n  def _record(self, f, r):\n    return f'Record {r} of file {f}'.encode()\n\n  def _create_files(self):\n    filenames = []\n    for i in range(self._num_files):\n      filename = os.path.join(self.get_temp_dir(), f'riegeli.{i}')\n      filenames.append(filename)\n\n      # Note: if records were serialized proto messages, passing\n      # options='transpose' to RecordWriter would make compression better.\n      with riegeli.RecordWriter(tf.io.gfile.GFile(filename, 'wb')) as writer:\n        for j in range(self._num_records):\n          writer.write_record(self._record(i, j))\n    return filenames\n\n  def test_read_one_epoch(self):\n    # Basic test: read from file 0.\n    dataset = self.dataset_fn(self.test_filenames[0])\n    self.assertDatasetProduces(\n        dataset,\n        expected_output=[self._record(0, i) for i in range(self._num_records)],\n    )\n\n    # Basic test: read from file 1.\n    dataset = self.dataset_fn(self.test_filenames[1])\n    self.assertDatasetProduces(\n        dataset,\n        expected_output=[self._record(1, i) for i in range(self._num_records)],\n    )\n\n    # Basic test: read from both files.\n    dataset = self.dataset_fn(self.test_filenames)\n    expected_output = []\n    for j in range(self._num_files):\n      expected_output.extend(\n          [self._record(j, i) for i in range(self._num_records)]\n      )\n    self.assertDatasetProduces(dataset, expected_output=expected_output)\n\n  def test_read_ten_epochs(self):\n    dataset = self.dataset_fn(self.test_filenames, num_epochs=10)\n    expected_output = []\n    for j in range(self._num_files):\n      expected_output.extend(\n          [self._record(j, i) for i in range(self._num_records)]\n      )\n    self.assertDatasetProduces(dataset, expected_output=expected_output * 10)\n\n  def test_read_ten_epochs_of_batches(self):\n    dataset = self.dataset_fn(\n        self.test_filenames, num_epochs=10, batch_size=self._num_records\n    )\n    expected_output = []\n    for j in range(self._num_files):\n      expected_output.append(\n          [self._record(j, i) for i in range(self._num_records)]\n      )\n    self.assertDatasetProduces(dataset, expected_output=expected_output * 10)\n\n\nif __name__ == '__main__':\n  tf.test.main()\n"
  },
  {
    "path": "python/riegeli/tensorflow/ops/__init__.py",
    "content": ""
  },
  {
    "path": "python/riegeli/tensorflow/ops/riegeli_dataset_ops.py",
    "content": "# Copyright 2019 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\"\"\"TensorFlow dataset for Riegeli/records files.\"\"\"\n\nimport tensorflow as tf\nfrom tensorflow.python.data.ops import dataset_ops\nfrom tensorflow.python.data.util import convert\nfrom tensorflow.python.framework import load_library\n\ngen_riegeli_dataset_ops = load_library.load_op_library(\n    tf.compat.v1.resource_loader.get_path_to_datafile('_riegeli_dataset_ops.so')\n)\n\n__all__ = ('RiegeliDataset',)\n\n_DEFAULT_MIN_BUFFER_SIZE = 4 << 10\n_DEFAULT_MAX_BUFFER_SIZE = 64 << 10\n\n\nclass RiegeliDataset(dataset_ops.DatasetSource):\n  \"\"\"A `Dataset` comprising records from one or more Riegeli/records files.\"\"\"\n\n  __slots__ = ('_filenames', '_min_buffer_size', '_max_buffer_size')\n\n  def __init__(\n      self,\n      filenames,\n      min_buffer_size=None,\n      max_buffer_size=None,\n      buffer_size=None,\n  ):\n    \"\"\"Creates a `RiegeliDataset`.\n\n    Args:\n      filenames: A `tf.string` tensor containing one or more filenames.\n      min_buffer_size: A `tf.int64` scalar which tunes the minimal buffer size,\n        which determines how much data at a time is typically read from the\n        file. The actual buffer size changes between min_buffer_size and\n        max_buffer_size depending on the access pattern. Default: 4K.\n      max_buffer_size: A `tf.int64` scalar which tunes the maximal buffer size,\n        which determines how much data at a time is typically read from the\n        file. The actual buffer size changes between min_buffer_size and\n        max_buffer_size depending on the access pattern. Default: 64K.\n      buffer_size: If not None, a shortcut for setting min_buffer_size and\n        max_buffer_size to the same value.\n    \"\"\"\n    if buffer_size is not None:\n      min_buffer_size = buffer_size\n      max_buffer_size = buffer_size\n    self._filenames = tf.convert_to_tensor(filenames, name='filenames')\n    self._min_buffer_size = convert.optional_param_to_tensor(\n        'min_buffer_size',\n        min_buffer_size,\n        argument_default=_DEFAULT_MIN_BUFFER_SIZE,\n    )\n    self._max_buffer_size = convert.optional_param_to_tensor(\n        'max_buffer_size',\n        max_buffer_size,\n        argument_default=_DEFAULT_MAX_BUFFER_SIZE,\n    )\n    variant_tensor = gen_riegeli_dataset_ops.riegeli_dataset(\n        self._filenames, self._min_buffer_size, self._max_buffer_size\n    )\n    super().__init__(variant_tensor)\n\n  @property\n  def element_spec(self):\n    return tf.TensorSpec([], tf.dtypes.string)\n"
  },
  {
    "path": "python/setup.py",
    "content": "# Copyright 2019 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\"\"\"PIP package setup for Riegeli.\"\"\"\n\nimport setuptools\nfrom setuptools import dist\n\nwith open('README.md', 'r') as fh:\n  long_description = fh.read()\n\n\nclass BinaryDistribution(dist.Distribution):\n  \"\"\"This class is needed in order to create OS specific wheels.\"\"\"\n\n  def has_ext_modules(self):\n    return True\n\n\nsetuptools.setup(\n    name='riegeli',\n    version='0.0.1',\n    description='File format for storing a sequence of records',\n    long_description=long_description,\n    long_description_content_type='text/markdown',\n    url='https://github.com/google/riegeli',\n    author='Google LLC',\n    author_email='compression-dev@google.com',\n    license='Apache License, Version 2.0',\n    python_requires='>=3.5,<4',\n    install_requires=[\n        'protobuf>=3.8.0,<4',\n    ],\n    extras_require={\n        'tensorflow': ['tensorflow>=1.15,<3'],\n    },\n    packages=setuptools.find_packages(),\n    include_package_data=True,\n    package_data={'': ['**/*.so']},\n    distclass=BinaryDistribution,\n    classifiers=[\n        'Programming Language :: Python',\n        'Intended Audience :: Developers',\n        'Programming Language :: Python :: 3',\n        'Programming Language :: Python :: 3.5',\n        'Programming Language :: Python :: 3.6',\n        'Programming Language :: Python :: 3.7',\n        'Programming Language :: Python :: 3.8',\n        'License :: OSI Approved :: Apache Software License',\n        'Operating System :: OS Independent',\n        'Topic :: Software Development :: Libraries',\n        'Topic :: Software Development :: Libraries :: Python Modules',\n    ],\n)\n"
  },
  {
    "path": "riegeli/.gitignore",
    "content": "bazel-*\n"
  },
  {
    "path": "riegeli/BUILD",
    "content": "# Riegeli, file format for storing a sequence of records.\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\nexports_files([\"LICENSE\"])\n"
  },
  {
    "path": "riegeli/base/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"type_traits\",\n    hdrs = [\"type_traits.h\"],\n    deps = [\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/utility\",\n    ],\n)\n\ncc_library(\n    name = \"constexpr\",\n    srcs = [\"port.h\"],\n    hdrs = [\"constexpr.h\"],\n    deps = [\"@com_google_absl//absl/base:nullability\"],\n)\n\ncc_library(\n    name = \"null_safe_memcpy\",\n    hdrs = [\"null_safe_memcpy.h\"],\n    deps = [\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n    ],\n)\n\ncc_library(\n    name = \"compare\",\n    hdrs = [\"compare.h\"],\n    deps = [\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:compare\",\n    ],\n)\n\ncc_library(\n    name = \"iterable\",\n    hdrs = [\"iterable.h\"],\n    deps = [\n        \":type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n    ],\n)\n\ncc_library(\n    name = \"stream_utils\",\n    srcs = [\"stream_utils.cc\"],\n    hdrs = [\"stream_utils.h\"],\n    deps = [\n        \":types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"debug\",\n    srcs = [\"debug.cc\"],\n    hdrs = [\"debug.h\"],\n    deps = [\n        \":stream_utils\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"assert\",\n    srcs = [\n        \"assert.cc\",\n        \"port.h\",\n    ],\n    hdrs = [\"assert.h\"],\n    deps = [\n        \":debug\",\n        \":stream_utils\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/log:absl_log\",\n    ],\n)\n\ncc_library(\n    name = \"types\",\n    hdrs = [\"types.h\"],\n    deps = [\"@com_google_absl//absl/base:nullability\"],\n)\n\ncc_library(\n    name = \"arithmetic\",\n    hdrs = [\"arithmetic.h\"],\n    deps = [\n        \":assert\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/numeric:bits\",\n        \"@com_google_absl//absl/numeric:int128\",\n    ],\n)\n\ncc_library(\n    name = \"buffering\",\n    hdrs = [\"buffering.h\"],\n    deps = [\n        \":arithmetic\",\n        \":types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n    ],\n)\n\ncc_library(\n    name = \"estimated_allocated_size\",\n    hdrs = [\"estimated_allocated_size.h\"],\n    deps = [\n        \":arithmetic\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n    ],\n)\n\ncc_library(\n    name = \"new_aligned\",\n    hdrs = [\"new_aligned.h\"],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":estimated_allocated_size\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/numeric:bits\",\n    ],\n)\n\ncc_library(\n    name = \"string_utils\",\n    srcs = [\"string_utils.cc\"],\n    hdrs = [\"string_utils.h\"],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/strings:resize_and_overwrite\",\n    ],\n)\n\ncc_library(\n    name = \"cord_utils\",\n    srcs = [\"cord_utils.cc\"],\n    hdrs = [\"cord_utils.h\"],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":buffering\",\n        \":string_utils\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/numeric:bits\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"unicode\",\n    srcs = [\"unicode.cc\"],\n    hdrs = [\"unicode.h\"],\n    features = select({\n        # unicode.cc has #define before #include to influence what the included\n        # files provide.\n        \"@platforms//os:windows\": [\"-use_header_modules\"],\n        \"//conditions:default\": [],\n    }),\n    deps = select({\n        \"@platforms//os:windows\": [\n            \":arithmetic\",\n            \"@com_google_absl//absl/base:core_headers\",\n            \"@com_google_absl//absl/base:nullability\",\n            \"@com_google_absl//absl/strings:resize_and_overwrite\",\n            \"@com_google_absl//absl/strings:string_view\",\n            \"@com_google_absl//absl/types:span\",\n        ],\n        \"//conditions:default\": [],\n    }),\n)\n\ncc_library(\n    name = \"type_id\",\n    hdrs = [\"type_id.h\"],\n    deps = [\n        \":compare\",\n        \"@com_google_absl//absl/base:nullability\",\n    ],\n)\n\ncc_library(\n    name = \"reset\",\n    hdrs = [\"reset.h\"],\n    deps = [\n        \":assert\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"type_erased_ref\",\n    hdrs = [\"type_erased_ref.h\"],\n    deps = [\n        \":type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/meta:type_traits\",\n    ],\n)\n\ncc_library(\n    name = \"initializer\",\n    srcs = [\"initializer_internal.h\"],\n    hdrs = [\n        \"initializer.h\",\n        \"invoker.h\",\n        \"maker.h\",\n        \"temporary_storage.h\",\n    ],\n    deps = [\n        \":assert\",\n        \":reset\",\n        \":type_erased_ref\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n    ],\n)\n\ncc_library(\n    name = \"global\",\n    hdrs = [\"global.h\"],\n    deps = [\"@com_google_absl//absl/base:nullability\"],\n)\n\ncc_library(\n    name = \"string_ref\",\n    hdrs = [\"string_ref.h\"],\n    deps = [\n        \":assert\",\n        \":compare\",\n        \":initializer\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base:config\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"bytes_ref\",\n    hdrs = [\"bytes_ref.h\"],\n    deps = [\n        \":compare\",\n        \":initializer\",\n        \":string_ref\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"c_string_ref\",\n    hdrs = [\"c_string_ref.h\"],\n    deps = [\n        \":compare\",\n        \":initializer\",\n        \":string_ref\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"memory_estimator\",\n    srcs = [\"memory_estimator.cc\"],\n    hdrs = [\"memory_estimator.h\"],\n    deps = [\n        \":arithmetic\",\n        \":estimated_allocated_size\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/container:flat_hash_map\",\n        \"@com_google_absl//absl/container:flat_hash_set\",\n        \"@com_google_absl//absl/container:inlined_vector\",\n        \"@com_google_absl//absl/container:node_hash_map\",\n        \"@com_google_absl//absl/container:node_hash_set\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"closing_ptr\",\n    hdrs = [\"closing_ptr.h\"],\n    deps = [\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n    ],\n)\n\ncc_library(\n    name = \"dependency\",\n    hdrs = [\n        \"dependency.h\",\n        \"dependency_base.h\",\n        \"dependency_manager.h\",\n    ],\n    deps = [\n        \":assert\",\n        \":bytes_ref\",\n        \":compare\",\n        \":initializer\",\n        \":reset\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/meta:type_traits\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"moving_dependency\",\n    hdrs = [\"moving_dependency.h\"],\n    deps = [\n        \":dependency\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n    ],\n)\n\ncc_library(\n    name = \"stable_dependency\",\n    hdrs = [\"stable_dependency.h\"],\n    deps = [\n        \":assert\",\n        \":dependency\",\n        \":initializer\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n    ],\n)\n\ncc_library(\n    name = \"any\",\n    srcs = [\"any_internal.h\"],\n    hdrs = [\n        \"any.h\",\n        \"any_initializer.h\",\n    ],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":closing_ptr\",\n        \":compare\",\n        \":dependency\",\n        \":initializer\",\n        \":memory_estimator\",\n        \":type_erased_ref\",\n        \":type_id\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/meta:type_traits\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"cord_iterator_span\",\n    srcs = [\"cord_iterator_span.cc\"],\n    hdrs = [\"cord_iterator_span.h\"],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":dependency\",\n        \":string_utils\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:resize_and_overwrite\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"status\",\n    srcs = [\n        \"errno_mapping.cc\",\n        \"status.cc\",\n    ],\n    hdrs = [\n        \"errno_mapping.h\",\n        \"status.h\",\n    ],\n    features = select({\n        # errno_mapping.cc has #define before #include to influence what the\n        # included files provide.\n        \"@platforms//os:windows\": [\"-use_header_modules\"],\n        \"//conditions:default\": [],\n    }),\n    deps = [\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/status:statusor\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ] + select({\n        \"@platforms//os:windows\": [\n            \":arithmetic\",\n            \":unicode\",\n            \"@com_google_absl//absl/types:span\",\n        ],\n        \"//conditions:default\": [],\n    }),\n)\n\ncc_library(\n    name = \"object\",\n    srcs = [\"object.cc\"],\n    hdrs = [\"object.h\"],\n    deps = [\n        \":assert\",\n        \":initializer\",\n        \":type_id\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n    ],\n)\n\ncc_library(\n    name = \"external_data\",\n    srcs = [\"external_data.cc\"],\n    hdrs = [\"external_data.h\"],\n    deps = [\"@com_google_absl//absl/strings:string_view\"],\n)\n\ncc_library(\n    name = \"shared_ptr\",\n    hdrs = [\n        \"intrusive_shared_ptr.h\",\n        \"ownership.h\",\n        \"ref_count.h\",\n        \"shared_ptr.h\",\n    ],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":compare\",\n        \":external_data\",\n        \":initializer\",\n        \":new_aligned\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n    ],\n)\n\ncc_library(\n    name = \"buffer\",\n    srcs = [\"buffer.cc\"],\n    hdrs = [\"buffer.h\"],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":buffering\",\n        \":estimated_allocated_size\",\n        \":external_data\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"shared_buffer\",\n    srcs = [\"shared_buffer.cc\"],\n    hdrs = [\"shared_buffer.h\"],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":buffer\",\n        \":external_data\",\n        \":initializer\",\n        \":shared_ptr\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"sized_shared_buffer\",\n    srcs = [\"sized_shared_buffer.cc\"],\n    hdrs = [\"sized_shared_buffer.h\"],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":buffering\",\n        \":shared_buffer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"chain_and_external_ref\",\n    srcs = [\"chain.cc\"],\n    hdrs = [\n        \"chain_base.h\",\n        \"chain_details.h\",\n        \"external_ref_base.h\",\n        \"external_ref_support.h\",\n    ],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":buffering\",\n        \":bytes_ref\",\n        \":compare\",\n        \":cord_utils\",\n        \":external_data\",\n        \":initializer\",\n        \":iterable\",\n        \":memory_estimator\",\n        \":new_aligned\",\n        \":null_safe_memcpy\",\n        \":shared_ptr\",\n        \":stream_utils\",\n        \":string_utils\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/container:inlined_vector\",\n        \"@com_google_absl//absl/meta:type_traits\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:resize_and_overwrite\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"chain\",\n    hdrs = [\"chain.h\"],\n    deps = [\":chain_and_external_ref\"],\n)\n\ncc_library(\n    name = \"external_ref\",\n    hdrs = [\"external_ref.h\"],\n    deps = [\":chain_and_external_ref\"],\n)\n\ncc_library(\n    name = \"byte_fill\",\n    srcs = [\"byte_fill.cc\"],\n    hdrs = [\"byte_fill.h\"],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":chain\",\n        \":compare\",\n        \":cord_utils\",\n        \":external_data\",\n        \":external_ref\",\n        \":global\",\n        \":iterable\",\n        \":shared_buffer\",\n        \":types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:bits\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"uninitialized_vector\",\n    hdrs = [\"uninitialized_vector.h\"],\n    deps = [\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/container:inlined_vector\",\n    ],\n)\n\ncc_library(\n    name = \"compact_string\",\n    srcs = [\"compact_string.cc\"],\n    hdrs = [\"compact_string.h\"],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":bytes_ref\",\n        \":compare\",\n        \":estimated_allocated_size\",\n        \":external_data\",\n        \":new_aligned\",\n        \":null_safe_memcpy\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base:config\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/hash\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"optional_compact_string\",\n    hdrs = [\"optional_compact_string.h\"],\n    deps = [\n        \":assert\",\n        \":bytes_ref\",\n        \":compact_string\",\n        \":compare\",\n        \":iterable\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"hybrid_direct_map\",\n    srcs = [\n        \"hybrid_direct_common.h\",\n        \"hybrid_direct_internal.h\",\n    ],\n    hdrs = [\"hybrid_direct_map.h\"],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":compare\",\n        \":debug\",\n        \":initializer\",\n        \":iterable\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/container:flat_hash_map\",\n    ],\n)\n\ncc_library(\n    name = \"hybrid_direct_set\",\n    srcs = [\n        \"hybrid_direct_common.h\",\n        \"hybrid_direct_internal.h\",\n    ],\n    hdrs = [\"hybrid_direct_set.h\"],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":compare\",\n        \":iterable\",\n        \":type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/container:flat_hash_set\",\n    ],\n)\n\ncc_library(\n    name = \"binary_search\",\n    hdrs = [\"binary_search.h\"],\n    deps = [\n        \":compare\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/status:statusor\",\n    ],\n)\n\ncc_library(\n    name = \"parallelism\",\n    srcs = [\"parallelism.cc\"],\n    hdrs = [\"parallelism.h\"],\n    visibility = [\"//riegeli:__subpackages__\"],\n    deps = [\n        \":assert\",\n        \":global\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/functional:any_invocable\",\n        \"@com_google_absl//absl/synchronization\",\n        \"@com_google_absl//absl/time\",\n    ],\n)\n\ncc_library(\n    name = \"background_cleaning\",\n    srcs = [\"background_cleaning.cc\"],\n    hdrs = [\"background_cleaning.h\"],\n    deps = [\n        \":assert\",\n        \":global\",\n        \":parallelism\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/synchronization\",\n        \"@com_google_absl//absl/time\",\n    ],\n)\n\ncc_library(\n    name = \"recycling_pool\",\n    hdrs = [\"recycling_pool.h\"],\n    deps = [\n        \":arithmetic\",\n        \":assert\",\n        \":background_cleaning\",\n        \":compare\",\n        \":global\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/container:flat_hash_map\",\n        \"@com_google_absl//absl/container:inlined_vector\",\n        \"@com_google_absl//absl/container:node_hash_map\",\n        \"@com_google_absl//absl/synchronization\",\n        \"@com_google_absl//absl/time\",\n    ],\n)\n\ncc_library(\n    name = \"options_parser\",\n    srcs = [\"options_parser.cc\"],\n    hdrs = [\"options_parser.h\"],\n    deps = [\n        \":assert\",\n        \":initializer\",\n        \":object\",\n        \":string_ref\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/functional:any_invocable\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/base/any.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_ANY_H_\n#define RIEGELI_BASE_ANY_H_\n\n#include <stddef.h>\n\n#include <algorithm>\n#include <cstddef>\n#include <cstring>\n#include <memory>\n#include <new>  // IWYU pragma: keep\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/any_initializer.h\"\n#include \"riegeli/base/any_internal.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/closing_ptr.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/dependency_base.h\"\n#include \"riegeli/base/dependency_manager.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/memory_estimator.h\"\n#include \"riegeli/base/temporary_storage.h\"\n#include \"riegeli/base/type_id.h\"\n#include \"riegeli/base/type_traits.h\"\n\nnamespace riegeli {\n\nnamespace any_internal {\n\n// Common base class of `Any` and `AnyRef`.\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\nclass AnyBase : public WithEqual<AnyBase<Handle, inline_size, inline_align>> {\n public:\n  // Returns a `Handle` to the `Manager`, or a default `Handle` for an empty\n  // `AnyBase`.\n  Handle get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return methods_and_handle_.handle;\n  }\n\n  // If `Handle` supports `operator*`, `AnyBase<Handle>` can be used as a smart\n  // pointer to the result of `operator*`, for convenience.\n  template <typename DependentHandle = Handle,\n            std::enable_if_t<HasDereference<DependentHandle>::value, int> = 0>\n  decltype(*std::declval<DependentHandle>()) operator*() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    AssertNotNull(\"Failed precondition of AnyBase::operator*: null handle\");\n    return *get();\n  }\n\n  template <typename DependentHandle = Handle,\n            std::enable_if_t<HasArrow<DependentHandle>::value, int> = 0>\n  Handle operator->() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    AssertNotNull(\"Failed precondition of AnyBase::operator->: null handle\");\n    return get();\n  }\n\n  // `AnyBase<Handle>` can be compared against `nullptr`. If `Handle` supports\n  // `operator==` with `nullptr`, then delegates the comparison to `Handle`,\n  // otherwise returns `true` for a non-empty `AnyBase`.\n  friend bool operator==(const AnyBase& a, std::nullptr_t) {\n    return a.EqualNullptr();\n  }\n\n  // If `true`, the `AnyBase` owns the dependent object, i.e. closing the host\n  // object should close the dependent object.\n  bool IsOwning() const {\n    return methods_and_handle_.methods->is_owning(repr_.storage);\n  }\n\n  // If `true`, `get()` stays unchanged when an `AnyBase` is moved.\n  static constexpr bool kIsStable = inline_size == 0;\n\n  // If the stored `Manager` has exactly this type or a reference to it, returns\n  // a pointer to the `Manager`. Otherwise returns `nullptr`.\n  template <\n      typename Manager,\n      std::enable_if_t<SupportsDependency<Handle, Manager&&>::value, int> = 0>\n  Manager* GetIf() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  template <\n      typename Manager,\n      std::enable_if_t<SupportsDependency<Handle, Manager&&>::value, int> = 0>\n  const Manager* GetIf() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns the `TypeId` corresponding to the stored `Manager` type, stripping\n  // any toplevel reference.\n  TypeId type_id() const { return methods_and_handle_.methods->type_id; }\n\n  // Returns `true` when the stored `Manager` has exactly this type or a\n  // reference to it.\n  //\n  // Same as `type_id() == TypeId::For<Manager>()`.\n  //\n  // Same as `GetIf<Manager>() != nullptr` but more efficient if the type\n  // matches.\n  template <\n      typename Manager,\n      std::enable_if_t<SupportsDependency<Handle, Manager&&>::value, int> = 0>\n  bool Holds() const {\n    return type_id() == TypeId::For<Manager>();\n  }\n\n  // Supports `MemoryEstimator`.\n  friend void RiegeliRegisterSubobjects(const AnyBase* self,\n                                        MemoryEstimator& memory_estimator) {\n    self->methods_and_handle_.methods->register_subobjects(self->repr_.storage,\n                                                           memory_estimator);\n  }\n\n protected:\n  // The state is left uninitialized.\n  AnyBase() noexcept {}\n\n  AnyBase(AnyBase&& that) noexcept;\n  AnyBase& operator=(AnyBase&& that) noexcept;\n\n  ~AnyBase() { Destroy(); }\n\n  void Reset(std::nullptr_t = nullptr);\n\n  // Initializes the state.\n  //\n  // If `Manager` is already a compatible `Any` or `AnyRef`, possibly wrapped in\n  // `ClosingPtrType`, or an rvalue reference to it, adopts its storage instead\n  // of keeping an indirection. This causes `GetIf()` to see through it.\n  void Initialize();\n  template <\n      typename Manager,\n      std::enable_if_t<\n          std::conjunction_v<std::negation<IsAny<Handle, Manager>>,\n                             std::negation<IsAnyClosingPtr<Handle, Manager>>>,\n          int> = 0>\n  void Initialize(Manager&& manager);\n  template <typename Manager,\n            std::enable_if_t<IsAny<Handle, Manager>::value, int> = 0>\n  void Initialize(Manager&& manager);\n  template <typename Manager,\n            std::enable_if_t<IsAnyClosingPtr<Handle, Manager>::value, int> = 0>\n  void Initialize(Manager&& manager);\n  template <\n      typename Manager,\n      std::enable_if_t<\n          std::conjunction_v<std::negation<IsAny<Handle, Manager>>,\n                             std::negation<IsAnyClosingPtr<Handle, Manager>>>,\n          int> = 0>\n  void Initialize(Initializer<Manager> manager);\n  template <typename Manager,\n            std::enable_if_t<IsAny<Handle, Manager>::value, int> = 0>\n  void Initialize(Initializer<Manager> manager);\n  template <typename Manager,\n            std::enable_if_t<IsAnyClosingPtr<Handle, Manager>::value, int> = 0>\n  void Initialize(Initializer<Manager> manager);\n  void InitializeFromAnyInitializer(AnyInitializer<Handle> manager);\n\n  template <typename Manager,\n            std::enable_if_t<!std::is_reference_v<Manager>, int> = 0>\n  void Adopt(Manager&& manager);\n  template <typename Manager,\n            std::enable_if_t<std::is_rvalue_reference_v<Manager>, int> = 0>\n  void Adopt(Manager&& manager);\n\n  // Destroys the state, leaving it uninitialized.\n  void Destroy();\n\n private:\n  // For adopting the state from an instantiation with a different `inline_size`\n  // and `inline_align`.\n  template <typename OtherHandle, size_t other_inline_size,\n            size_t other_inline_align>\n  friend class AnyBase;\n  // For adopting the state from an instantiation held in an `AnyInitializer`.\n  friend class AnyInitializer<Handle>;\n\n  using Repr = any_internal::Repr<Handle, inline_size, inline_align>;\n  using MethodsAndHandle = any_internal::MethodsAndHandle<Handle>;\n  using NullMethods = any_internal::NullMethods<Handle>;\n  template <typename Manager>\n  using MethodsFor = any_internal::MethodsFor<\n      Handle, Manager, IsInline<Handle, Manager, inline_size, inline_align>()>;\n\n  static constexpr size_t kAvailableSize =\n      AvailableSize<Handle, inline_size, inline_align>();\n  static constexpr size_t kAvailableAlign =\n      AvailableAlign<Handle, inline_size, inline_align>();\n\n  template <typename DependentHandle = Handle,\n            std::enable_if_t<IsComparableAgainstNullptr<DependentHandle>::value,\n                             int> = 0>\n  void AssertNotNull(absl::string_view message) const {\n    RIEGELI_ASSERT(get() != nullptr) << message;\n  }\n  template <typename DependentHandle = Handle,\n            std::enable_if_t<\n                !IsComparableAgainstNullptr<DependentHandle>::value, int> = 0>\n  void AssertNotNull(ABSL_ATTRIBUTE_UNUSED absl::string_view message) const {}\n\n  template <typename DependentHandle = Handle,\n            std::enable_if_t<IsComparableAgainstNullptr<DependentHandle>::value,\n                             int> = 0>\n  bool EqualNullptr() const {\n    return get() == nullptr;\n  }\n  template <typename DependentHandle = Handle,\n            std::enable_if_t<\n                !IsComparableAgainstNullptr<DependentHandle>::value, int> = 0>\n  bool EqualNullptr() const {\n    return methods_and_handle_.methods == &NullMethods::kMethods;\n  }\n\n  MethodsAndHandle methods_and_handle_;\n  Repr repr_;\n};\n\n}  // namespace any_internal\n\n// `Any<Handle>` refers to an optionally owned object which is accessed as\n// `Handle` and stored as some `Manager` type decided when the `Any` is\n// initialized.\n//\n// Often `Handle` is some pointer `Base*`, and then `Manager` can be e.g.\n// `T*` (not owned), `T` (owned), or `std::unique_ptr<T>` (owned), with some `T`\n// derived from `Base`.\n//\n// `Any<Handle>` holds a `Dependency<Handle, Manager>` for some `Manager` type,\n// erasing the `Manager` parameter from the type of the `Any`, or is empty.\ntemplate <typename Handle, size_t inline_size = 0, size_t inline_align = 0>\nclass ABSL_NULLABILITY_COMPATIBLE Any\n    : public any_internal::AnyBase<Handle, inline_size, inline_align> {\n private:\n  // Indirection through `InliningImpl` is needed for MSVC for some reason.\n  template <typename... InlineManagers>\n  struct InliningImpl {\n    using type =\n        Any<Handle,\n            UnsignedMax(inline_size,\n                        sizeof(Dependency<Handle, InlineManagers>)...),\n            UnsignedMax(inline_align,\n                        alignof(Dependency<Handle, InlineManagers>)...)>;\n  };\n\n public:\n  // `Any<Handle>::Inlining<InlineManagers...>` enlarges inline storage of\n  // `Any<Handle>`.\n  //\n  // `InlineManagers` specify the size of inline storage, which allows to avoid\n  // heap allocation if `Manager` is among `InlineManagers`, or if\n  // `Dependency<Handle, Manager>` fits there regarding size and alignment.\n  // By default inline storage is enough for a pointer.\n  template <typename... InlineManagers>\n  using Inlining = typename InliningImpl<InlineManagers...>::type;\n\n  // Creates an empty `Any`.\n  Any() noexcept { this->Initialize(); }\n  /*implicit*/ Any(std::nullptr_t) { this->Initialize(); }\n\n  // Holds a `Dependency<Handle, TargetT<Manager>>`.\n  //\n  // If `TargetT<Manager>` is already a compatible `Any` or `AnyRef`, possibly\n  // wrapped in `ClosingPtrType`, or an rvalue reference to it, adopts its\n  // storage instead of keeping an indirection. This causes `GetIf()` to see\n  // through it.\n  template <typename Manager,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<Any, TargetT<Manager>>,\n                                   TargetSupportsDependency<Handle, Manager>>,\n                int> = 0>\n  /*implicit*/ Any(Manager&& manager);\n  template <typename Manager,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<Any, TargetT<Manager>>,\n                                   TargetSupportsDependency<Handle, Manager>>,\n                int> = 0>\n  Any& operator=(Manager&& manager);\n\n  // Holds the `Dependency` specified when the `AnyInitializer` was constructed.\n  //\n  // `AnyInitializer` is accepted as a template parameter to avoid this\n  // constructor triggering implicit conversions of other parameter types to\n  // `AnyInitializer`, which causes template instantiation cycles.\n  template <typename Manager,\n            std::enable_if_t<std::is_same_v<Manager, AnyInitializer<Handle>>,\n                             int> = 0>\n  /*implicit*/ Any(Manager manager);\n  template <typename Manager,\n            std::enable_if_t<std::is_same_v<Manager, AnyInitializer<Handle>>,\n                             int> = 0>\n  Any& operator=(Manager manager);\n\n  // Assignment operator which materializes `Any` from its `Initializer`\n  // except from the `Any` itself, which is handled below.\n  template <typename Manager,\n            std::enable_if_t<std::conjunction_v<SameRef<Any, TargetT<Manager>>,\n                                                NotSameRef<Any, Manager>>,\n                             int> = 0>\n  Any& operator=(Manager&& manager) {\n    riegeli::Reset(*this, std::forward<Manager>(manager));\n    return *this;\n  }\n\n  Any(Any&& that) = default;\n  Any& operator=(Any&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `Any`. This avoids\n  // constructing a temporary `Any` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(std::nullptr_t = nullptr) {\n    Any::AnyBase::Reset();\n  }\n\n private:\n  // For `ABSL_NULLABILITY_COMPATIBLE`.\n  using pointer = std::conditional_t<std::is_pointer_v<Handle>, Handle, void*>;\n};\n\n// Specialization of `DependencyManagerImpl<Any<Handle>>`:\n// a dependency with ownership determined at runtime.\ntemplate <typename Handle, size_t inline_size, size_t inline_align,\n          typename ManagerStorage>\nclass DependencyManagerImpl<Any<Handle, inline_size, inline_align>,\n                            ManagerStorage>\n    : public DependencyBase<ManagerStorage> {\n public:\n  using DependencyManagerImpl::DependencyBase::DependencyBase;\n\n  bool IsOwning() const { return this->manager().IsOwning(); }\n\n  static constexpr bool kIsStable =\n      DependencyManagerImpl::DependencyBase::kIsStable ||\n      Any<Handle, inline_size, inline_align>::kIsStable;\n\n protected:\n  DependencyManagerImpl(DependencyManagerImpl&& that) = default;\n  DependencyManagerImpl& operator=(DependencyManagerImpl&& that) = default;\n\n  ~DependencyManagerImpl() = default;\n\n  Handle ptr() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return this->manager().get();\n  }\n};\n\n// Specialization of\n// `DependencyManagerImpl<ClosingPtrType<Any<Handle>>>`:\n// a dependency with ownership determined at runtime.\ntemplate <typename Handle, size_t inline_size, size_t inline_align,\n          typename ManagerStorage>\nclass DependencyManagerImpl<\n    std::unique_ptr<Any<Handle, inline_size, inline_align>, NullDeleter>,\n    ManagerStorage>\n    : public DependencyBase<std::unique_ptr<\n          Any<Handle, inline_size, inline_align>, NullDeleter>> {\n public:\n  using DependencyManagerImpl::DependencyBase::DependencyBase;\n\n  bool IsOwning() const {\n    return this->manager() != nullptr && this->manager()->IsOwning();\n  }\n\n  static constexpr bool kIsStable = true;\n\n protected:\n  DependencyManagerImpl(DependencyManagerImpl&& that) = default;\n  DependencyManagerImpl& operator=(DependencyManagerImpl&& that) = default;\n\n  ~DependencyManagerImpl() = default;\n\n  Handle ptr() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return this->manager()->get();\n  }\n};\n\n// `AnyRef<Handle>` refers to an optionally owned object which is accessed as\n// `Handle` and was passed as some `Manager` type decided when the `AnyRef` was\n// initialized.\n//\n// Often `Handle` is some pointer `Base*`, and then `Manager` can be e.g.\n// `T&` (not owned), `T&&` (owned), or `std::unique_ptr<T>` (owned), with some\n// `T` derived from `Base`.\n//\n// `AnyRef<Handle>` holds a `Dependency<Handle, Manager&&>` (which collapses to\n// `Dependency<Handle, Manager&>` if `Manager` is itself an lvalue reference)\n// for some `Manager` type, erasing the `Manager` parameter from the type of the\n// `AnyRef`, or is empty.\n//\n// `AnyRef<Handle>(manager)` does not own `manager`, even if it involves\n// temporaries, hence it should be used only as a parameter of a function or\n// constructor, so that the temporaries outlive its usage. Instead of storing an\n// `AnyRef<Handle>` in a variable or returning it from a function, consider\n// `riegeli::OwningMaker<Manager>()`, `MakerTypeFor<Manager, ManagerArgs...>`,\n// or `Any<Handle>`.\n//\n// This allows to pass an unowned dependency by lvalue reference instead of by\n// pointer, which allows for a more idiomatic API for passing an object which\n// does not need to be valid after the function returns. And this allows to pass\n// an owned dependency by rvalue reference instead of by value, which avoids\n// moving it.\ntemplate <typename Handle>\nclass ABSL_NULLABILITY_COMPATIBLE AnyRef\n    : public any_internal::AnyBase<Handle, 0, 0> {\n public:\n  // Creates an empty `AnyRef`.\n  AnyRef() noexcept { this->Initialize(); }\n  /*implicit*/ AnyRef(std::nullptr_t) { this->Initialize(); }\n\n  // Holds a `Dependency<Handle, TargetRefT<Manager>&&>` when\n  // `TargetRefT<Manager>` is not a reference.\n  //\n  // If `TargetT<Manager>` is already a compatible `Any` or `AnyRef`, possibly\n  // wrapped in `ClosingPtrType`, points to its storage instead of keeping an\n  // indirection. This causes `GetIf()` to see through it.\n  template <typename Manager,\n            std::enable_if_t<\n                std::conjunction_v<\n                    NotSameRef<AnyRef, TargetT<Manager>>,\n                    NotSameRef<Any<Handle>, TargetT<Manager>>,\n                    std::negation<std::is_reference<TargetRefT<Manager>>>,\n                    SupportsDependency<Handle, TargetRefT<Manager>&&>>,\n                int> = 0>\n  /*implicit*/ AnyRef(Manager&& manager ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                      TemporaryStorage<TargetRefT<Manager>>&& storage\n                          ABSL_ATTRIBUTE_LIFETIME_BOUND = {});\n\n  // Holds a `DependencyRef<Handle, Manager>` when `TargetRefT<Manager>` is a\n  // reference.\n  //\n  // If `TargetT<Manager>` is an rvalue reference to an already a compatible\n  // `Any` or `AnyRef`, possibly wrapped in `ClosingPtrType`, points to its\n  // storage instead of keeping an indirection. This causes `GetIf()` to see\n  // through it.\n  //\n  // This constructor is separate so that it does not need `storage`.\n  template <typename Manager,\n            std::enable_if_t<std::conjunction_v<\n                                 NotSameRef<AnyRef, TargetT<Manager>>,\n                                 NotSameRef<Any<Handle>, TargetT<Manager>>,\n                                 std::is_reference<TargetRefT<Manager>>,\n                                 TargetRefSupportsDependency<Handle, Manager>>,\n                             int> = 0>\n  /*implicit*/ AnyRef(Manager&& manager ABSL_ATTRIBUTE_LIFETIME_BOUND);\n\n  // Adopts the `Dependency` from `Any<Handle>` with no inline storage.\n  //\n  // This constructor is separate so that it does not need temporary storage nor\n  // `ABSL_ATTRIBUTE_LIFETIME_BOUND`.\n  template <\n      typename Manager,\n      std::enable_if_t<std::is_same_v<TargetT<Manager>, Any<Handle>>, int> = 0>\n  /*implicit*/ AnyRef(Manager&& manager);\n\n  // Holds the `Dependency` specified when the `AnyInitializer` was constructed.\n  //\n  // Prefer taking parameters as `AnyRef<Handle>` instead of\n  // `AnyInitializer<Handle>` if they are ultimately always converted to\n  // `AnyRef<Handle>`, because this constructor may involve heap allocation.\n  //\n  // `AnyInitializer` is accepted as a template parameter to avoid this\n  // constructor triggering implicit conversions of other parameter types to\n  // `AnyInitializer`, which causes template instantiation cycles.\n  template <typename Manager,\n            std::enable_if_t<std::is_same_v<Manager, AnyInitializer<Handle>>,\n                             int> = 0>\n  /*implicit*/ AnyRef(Manager manager);\n\n  AnyRef(AnyRef&& that) = default;\n  AnyRef& operator=(AnyRef&&) = delete;\n\n private:\n  // For `ABSL_NULLABILITY_COMPATIBLE`.\n  using pointer = std::conditional_t<std::is_pointer_v<Handle>, Handle, void*>;\n};\n\n// Specialization of `DependencyManagerImpl<AnyRef<Handle>>`:\n// a dependency with ownership determined at runtime.\ntemplate <typename Handle, typename ManagerStorage>\nclass DependencyManagerImpl<AnyRef<Handle>, ManagerStorage>\n    : public DependencyBase<ManagerStorage> {\n public:\n  using DependencyManagerImpl::DependencyBase::DependencyBase;\n\n  bool IsOwning() const { return this->manager().IsOwning(); }\n\n  static constexpr bool kIsStable =\n      DependencyManagerImpl::DependencyBase::kIsStable ||\n      AnyRef<Handle>::kIsStable;\n\n protected:\n  DependencyManagerImpl(DependencyManagerImpl&& that) = default;\n  DependencyManagerImpl& operator=(DependencyManagerImpl&& that) = default;\n\n  ~DependencyManagerImpl() = default;\n\n  Handle ptr() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return this->manager().get();\n  }\n};\n\n// Specialization of\n// `DependencyManagerImpl<ClosingPtrType<AnyRef<Handle>, Deleter>>`:\n// a dependency with ownership determined at runtime.\ntemplate <typename Handle, typename ManagerStorage>\nclass DependencyManagerImpl<std::unique_ptr<AnyRef<Handle>, NullDeleter>,\n                            ManagerStorage>\n    : public DependencyBase<std::unique_ptr<AnyRef<Handle>, NullDeleter>> {\n public:\n  using DependencyManagerImpl::DependencyBase::DependencyBase;\n\n  bool IsOwning() const {\n    return this->manager() != nullptr && this->manager()->IsOwning();\n  }\n\n  static constexpr bool kIsStable = true;\n\n protected:\n  DependencyManagerImpl(DependencyManagerImpl&& that) = default;\n  DependencyManagerImpl& operator=(DependencyManagerImpl&& that) = default;\n\n  ~DependencyManagerImpl() = default;\n\n  Handle ptr() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return this->manager()->get();\n  }\n};\n\n// Type-erased object like `absl::string_view`, `std::string` or `const char*`\n// which stores and possibly owns a string.\n//\n// Do not mutate a stored `std::string` in-place through `GetIf()` if its length\n// changes. Assign a new one.\nusing AnyString =\n    Any<absl::string_view>::Inlining<absl::string_view, std::string>;\n\n// Implementation details follow.\n\nnamespace any_internal {\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ninline AnyBase<Handle, inline_size, inline_align>::AnyBase(\n    AnyBase&& that) noexcept {\n  that.methods_and_handle_.methods->move(that.repr_.storage, repr_.storage,\n                                         &methods_and_handle_);\n  that.methods_and_handle_.methods = &NullMethods::kMethods;\n  that.methods_and_handle_.handle = SentinelHandle<Handle>();\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ninline AnyBase<Handle, inline_size, inline_align>&\nAnyBase<Handle, inline_size, inline_align>::operator=(AnyBase&& that) noexcept {\n  if (ABSL_PREDICT_TRUE(&that != this)) {\n    Destroy();\n    that.methods_and_handle_.methods->move(that.repr_.storage, repr_.storage,\n                                           &methods_and_handle_);\n    that.methods_and_handle_.methods = &NullMethods::kMethods;\n    that.methods_and_handle_.handle = SentinelHandle<Handle>();\n  }\n  return *this;\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ninline void AnyBase<Handle, inline_size, inline_align>::Initialize() {\n  methods_and_handle_.methods = &NullMethods::kMethods;\n  new (&methods_and_handle_.handle)\n      Handle(any_internal::SentinelHandle<Handle>());\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <typename Manager,\n          std::enable_if_t<std::conjunction_v<\n                               std::negation<IsAny<Handle, Manager>>,\n                               std::negation<IsAnyClosingPtr<Handle, Manager>>>,\n                           int>>\ninline void AnyBase<Handle, inline_size, inline_align>::Initialize(\n    Manager&& manager) {\n  Initialize<Manager>(Initializer<Manager>(std::forward<Manager>(manager)));\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <typename Manager,\n          std::enable_if_t<IsAny<Handle, Manager>::value, int>>\ninline void AnyBase<Handle, inline_size, inline_align>::Initialize(\n    Manager&& manager) {\n  using ManagerValue = std::remove_reference_t<Manager>;\n  // `manager.methods_and_handle_.methods->used_size <=\n  //      ManagerValue::kAvailableSize`, hence if\n  // `ManagerValue::kAvailableSize <= kAvailableSize` then\n  // `manager.methods_and_handle_.methods->used_size <= kAvailableSize`.\n  // No need to check possibly at runtime.\n  if ((ManagerValue::kAvailableSize <= kAvailableSize ||\n       manager.methods_and_handle_.methods->used_size <= kAvailableSize) &&\n      // Same for alignment.\n      (ManagerValue::kAvailableAlign <= kAvailableAlign ||\n       manager.methods_and_handle_.methods->used_align <= kAvailableAlign)) {\n    // Adopt `manager` by moving its representation as is.\n    manager.methods_and_handle_.methods->move(\n        manager.repr_.storage, repr_.storage, &methods_and_handle_);\n    manager.methods_and_handle_.methods = &NullMethods::kMethods;\n    manager.methods_and_handle_.handle = SentinelHandle<Handle>();\n    return;\n  }\n  // Adopt `manager` by moving its representation to the heap if `Manager` is\n  // a value, or referring to it if `Manager` is a reference.\n  Adopt<Manager>(std::forward<Manager>(manager));\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <typename Manager,\n          std::enable_if_t<IsAnyClosingPtr<Handle, Manager>::value, int>>\ninline void AnyBase<Handle, inline_size, inline_align>::Initialize(\n    Manager&& manager) {\n  if (manager == nullptr) {\n    Initialize();\n    return;\n  }\n  // Adopt `*manager` by referring to its representation.\n  manager->methods_and_handle_.methods->make_reference(\n      manager->repr_.storage, repr_.storage, &methods_and_handle_);\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <typename Manager,\n          std::enable_if_t<std::conjunction_v<\n                               std::negation<IsAny<Handle, Manager>>,\n                               std::negation<IsAnyClosingPtr<Handle, Manager>>>,\n                           int>>\ninline void AnyBase<Handle, inline_size, inline_align>::Initialize(\n    Initializer<Manager> manager) {\n  methods_and_handle_.methods = &MethodsFor<Manager>::kMethods;\n  MethodsFor<Manager>::Construct(repr_.storage, &methods_and_handle_.handle,\n                                 std::move(manager));\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <typename Manager,\n          std::enable_if_t<IsAny<Handle, Manager>::value, int>>\ninline void AnyBase<Handle, inline_size, inline_align>::Initialize(\n    Initializer<Manager> manager) {\n  // Materialize `Manager` to adopt its storage.\n  Initialize<Manager>(std::move(manager).Reference());\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <typename Manager,\n          std::enable_if_t<IsAnyClosingPtr<Handle, Manager>::value, int>>\ninline void AnyBase<Handle, inline_size, inline_align>::Initialize(\n    Initializer<Manager> manager) {\n  // Materialize `Manager` to adopt its storage.\n  Initialize<Manager>(std::move(manager).Construct());\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ninline void\nAnyBase<Handle, inline_size, inline_align>::InitializeFromAnyInitializer(\n    AnyInitializer<Handle> manager) {\n  std::move(manager).Construct(repr_.storage, &methods_and_handle_,\n                               kAvailableSize, kAvailableAlign);\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <typename Manager,\n          std::enable_if_t<!std::is_reference_v<Manager>, int>>\ninline void AnyBase<Handle, inline_size, inline_align>::Adopt(\n    Manager&& manager) {\n  manager.methods_and_handle_.methods->move_to_heap(\n      manager.repr_.storage, repr_.storage, &methods_and_handle_);\n  manager.methods_and_handle_.methods = &NullMethods::kMethods;\n  manager.methods_and_handle_.handle = SentinelHandle<Handle>();\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <typename Manager,\n          std::enable_if_t<std::is_rvalue_reference_v<Manager>, int>>\ninline void AnyBase<Handle, inline_size, inline_align>::Adopt(\n    Manager&& manager) {\n  manager.methods_and_handle_.methods->make_reference(\n      manager.repr_.storage, repr_.storage, &methods_and_handle_);\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ninline void AnyBase<Handle, inline_size, inline_align>::Destroy() {\n  methods_and_handle_.methods->destroy(repr_.storage);\n  methods_and_handle_.handle.~Handle();\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ninline void AnyBase<Handle, inline_size, inline_align>::Reset(std::nullptr_t) {\n  methods_and_handle_.methods->destroy(repr_.storage);\n  methods_and_handle_.methods = &NullMethods::kMethods;\n  methods_and_handle_.handle = SentinelHandle<Handle>();\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <typename Manager,\n          std::enable_if_t<SupportsDependency<Handle, Manager&&>::value, int>>\ninline Manager* AnyBase<Handle, inline_size, inline_align>::GetIf()\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  if (!Holds<Manager>()) return nullptr;\n  return &methods_and_handle_.methods->get_raw_manager(repr_.storage)\n              .template Cast<Manager&>();\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <typename Manager,\n          std::enable_if_t<SupportsDependency<Handle, Manager&&>::value, int>>\ninline const Manager* AnyBase<Handle, inline_size, inline_align>::GetIf() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  if (!Holds<Manager>()) return nullptr;\n  return &methods_and_handle_.methods->get_raw_manager(repr_.storage)\n              .template Cast<const Manager&>();\n}\n\n}  // namespace any_internal\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <\n    typename Manager,\n    std::enable_if_t<\n        std::conjunction_v<NotSameRef<Any<Handle, inline_size, inline_align>,\n                                      TargetT<Manager>>,\n                           TargetSupportsDependency<Handle, Manager>>,\n        int>>\ninline Any<Handle, inline_size, inline_align>::Any(Manager&& manager) {\n  this->template Initialize<TargetT<Manager>>(std::forward<Manager>(manager));\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <\n    typename Manager,\n    std::enable_if_t<\n        std::conjunction_v<NotSameRef<Any<Handle, inline_size, inline_align>,\n                                      TargetT<Manager>>,\n                           TargetSupportsDependency<Handle, Manager>>,\n        int>>\ninline Any<Handle, inline_size, inline_align>&\nAny<Handle, inline_size, inline_align>::operator=(Manager&& manager) {\n  this->Destroy();\n  this->template Initialize<TargetT<Manager>>(std::forward<Manager>(manager));\n  return *this;\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <\n    typename Manager,\n    std::enable_if_t<std::is_same_v<Manager, AnyInitializer<Handle>>, int>>\ninline Any<Handle, inline_size, inline_align>::Any(Manager manager) {\n  this->InitializeFromAnyInitializer(std::move(manager));\n}\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\ntemplate <\n    typename Manager,\n    std::enable_if_t<std::is_same_v<Manager, AnyInitializer<Handle>>, int>>\ninline Any<Handle, inline_size, inline_align>&\nAny<Handle, inline_size, inline_align>::operator=(Manager manager) {\n  this->Destroy();\n  this->InitializeFromAnyInitializer(std::move(manager));\n  return *this;\n}\n\ntemplate <typename Handle>\ntemplate <\n    typename Manager,\n    std::enable_if_t<std::conjunction_v<\n                         NotSameRef<AnyRef<Handle>, TargetT<Manager>>,\n                         NotSameRef<Any<Handle>, TargetT<Manager>>,\n                         std::negation<std::is_reference<TargetRefT<Manager>>>,\n                         SupportsDependency<Handle, TargetRefT<Manager>&&>>,\n                     int>>\ninline AnyRef<Handle>::AnyRef(Manager&& manager ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                              TemporaryStorage<TargetRefT<Manager>>&& storage\n                                  ABSL_ATTRIBUTE_LIFETIME_BOUND) {\n  this->template Initialize<TargetRefT<Manager>&&>(\n      Initializer<TargetRefT<Manager>>(std::forward<Manager>(manager))\n          .Reference(std::move(storage)));\n}\n\ntemplate <typename Handle>\ntemplate <typename Manager,\n          std::enable_if_t<\n              std::conjunction_v<NotSameRef<AnyRef<Handle>, TargetT<Manager>>,\n                                 NotSameRef<Any<Handle>, TargetT<Manager>>,\n                                 std::is_reference<TargetRefT<Manager>>,\n                                 TargetRefSupportsDependency<Handle, Manager>>,\n              int>>\ninline AnyRef<Handle>::AnyRef(Manager&& manager ABSL_ATTRIBUTE_LIFETIME_BOUND) {\n  this->template Initialize<TargetRefT<Manager>>(\n      std::forward<Manager>(manager));\n}\n\ntemplate <typename Handle>\ntemplate <typename Manager,\n          std::enable_if_t<std::is_same_v<TargetT<Manager>, Any<Handle>>, int>>\ninline AnyRef<Handle>::AnyRef(Manager&& manager) {\n  this->template Initialize<TargetT<Manager>>(std::forward<Manager>(manager));\n}\n\ntemplate <typename Handle>\ntemplate <\n    typename Manager,\n    std::enable_if_t<std::is_same_v<Manager, AnyInitializer<Handle>>, int>>\ninline AnyRef<Handle>::AnyRef(Manager manager) {\n  this->InitializeFromAnyInitializer(std::move(manager));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_ANY_H_\n"
  },
  {
    "path": "riegeli/base/any_initializer.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_ANY_INITIALIZER_H_\n#define RIEGELI_BASE_ANY_INITIALIZER_H_\n\n#include <stddef.h>\n\n#include <cstddef>\n#include <cstring>\n#include <new>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"riegeli/base/any_internal.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/type_erased_ref.h\"\n#include \"riegeli/base/type_traits.h\"\n\nnamespace riegeli {\n\nnamespace any_internal {\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\nclass AnyBase;\n}  // namespace any_internal\n\n// A parameter of type `AnyInitializer<Handle>` allows the caller to specify an\n// `Any<Handle>` by passing a value convertible to `Any<Handle>`.\n//\n// In contrast to accepting `Any<Handle>` directly, this allows to construct the\n// object in-place, avoiding constructing a temporary and moving from it. This\n// also avoids specifying `::Inlining<...>` in the interface while benefiting\n// from that in the implementation.\n//\n// `AnyInitializer<Handle>` also allows to initialize an `Any<Handle>` for a\n// `Handle` type which is neither default-constructible nor supporting\n// `RiegeliDependencySentinel()`, which makes `Any<Handle>` immovable.\n//\n// `AnyInitializer<Handle>` is similar to `Initializer<Any<Handle>>`, except\n// that it efficiently handles `Any<Handle>` specializations with any inline\n// storage constraints.\n//\n// `AnyInitializer<Handle>(manager)` does not own `manager`, even if it involves\n// temporaries, hence it should be used only as a parameter of a function or\n// constructor, so that the temporaries outlive its usage. Instead of storing\n// an `AnyInitializer<Handle>` in a variable or returning it from a function,\n// consider `riegeli::OwningMaker<Manager>(manager_args...)`,\n// `MakerTypeFor<Manager, ManagerArgs...>`, or `Any<Handle>`.\ntemplate <typename Handle>\nclass ABSL_NULLABILITY_COMPATIBLE AnyInitializer {\n public:\n  // An `Any` will be empty.\n  AnyInitializer() noexcept : construct_(ConstructMethodEmpty) {}\n  /*implicit*/ AnyInitializer(std::nullptr_t)\n      : construct_(ConstructMethodEmpty) {}\n\n  // An `Any` will hold a `Dependency<Handle, TargetT<Manager>>`.\n  //\n  // If `TargetT<Manager>` is already a compatible `Any` or `AnyRef`, possibly\n  // wrapped in `ClosingPtrType`, or an rvalue reference to it, adopts its\n  // storage instead of keeping an indirection. This causes `GetIf()` to see\n  // through it.\n  template <typename Manager,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<AnyInitializer, TargetT<Manager>>,\n                                   TargetSupportsDependency<Handle, Manager>>,\n                int> = 0>\n  /*implicit*/ AnyInitializer(Manager&& manager ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : construct_(ConstructMethod<Manager>),\n        context_(std::forward<Manager>(manager)) {}\n\n  AnyInitializer(AnyInitializer&& that) = default;\n  AnyInitializer& operator=(AnyInitializer&&) = delete;\n\n private:\n  // For `Construct()`.\n  template <typename OtherHandle, size_t inline_size, size_t inline_align>\n  friend class any_internal::AnyBase;\n\n  // For `ABSL_NULLABILITY_COMPATIBLE`.\n  using pointer = std::conditional_t<std::is_pointer_v<Handle>, Handle, void*>;\n\n  using Storage = any_internal::Storage;\n  using MethodsAndHandle = any_internal::MethodsAndHandle<Handle>;\n  using NullMethods = any_internal::NullMethods<Handle>;\n  template <typename Manager, bool is_inline>\n  using MethodsFor = any_internal::MethodsFor<Handle, Manager, is_inline>;\n\n  static void ConstructMethodEmpty(TypeErasedRef context, Storage dest,\n                                   MethodsAndHandle* dest_methods_and_handle,\n                                   size_t available_size,\n                                   size_t available_align);\n\n  template <\n      typename Manager,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::negation<any_internal::IsAny<Handle, TargetT<Manager>>>,\n              std::negation<\n                  any_internal::IsAnyClosingPtr<Handle, TargetT<Manager>>>>,\n          int> = 0>\n  static void ConstructMethod(TypeErasedRef context, Storage dest,\n                              MethodsAndHandle* dest_methods_and_handle,\n                              size_t available_size, size_t available_align);\n  template <typename Manager,\n            std::enable_if_t<\n                any_internal::IsAny<Handle, TargetT<Manager>>::value, int> = 0>\n  static void ConstructMethod(TypeErasedRef context, Storage dest,\n                              MethodsAndHandle* dest_methods_and_handle,\n                              size_t available_size, size_t available_align);\n  template <typename Manager,\n            std::enable_if_t<\n                any_internal::IsAnyClosingPtr<Handle, TargetT<Manager>>::value,\n                int> = 0>\n  static void ConstructMethod(TypeErasedRef context, Storage dest,\n                              MethodsAndHandle* dest_methods_and_handle,\n                              size_t available_size, size_t available_align);\n\n  template <typename Target,\n            std::enable_if_t<!std::is_reference_v<Target>, int> = 0>\n  static void Adopt(Target&& target, Storage dest,\n                    MethodsAndHandle* dest_methods_and_handle);\n  template <typename Target,\n            std::enable_if_t<std::is_rvalue_reference_v<Target>, int> = 0>\n  static void Adopt(Target&& target, Storage dest,\n                    MethodsAndHandle* dest_methods_and_handle);\n\n  // Constructs `dest` with `*dest_methods_and_handle` by moving from `*this`.\n  void Construct(Storage dest, MethodsAndHandle* dest_methods_and_handle,\n                 size_t available_size, size_t available_align) && {\n    construct_(context_, dest, dest_methods_and_handle, available_size,\n               available_align);\n  }\n\n  void (*construct_)(TypeErasedRef context, Storage dest,\n                     MethodsAndHandle* dest_methods_and_handle,\n                     size_t available_size, size_t available_align);\n  TypeErasedRef context_;\n};\n\n// Implementation details follow.\n\ntemplate <typename Handle>\nvoid AnyInitializer<Handle>::ConstructMethodEmpty(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef context,\n    ABSL_ATTRIBUTE_UNUSED Storage dest,\n    MethodsAndHandle* dest_methods_and_handle,\n    ABSL_ATTRIBUTE_UNUSED size_t available_size,\n    ABSL_ATTRIBUTE_UNUSED size_t available_align) {\n  dest_methods_and_handle->methods = &NullMethods::kMethods;\n  new (&dest_methods_and_handle->handle)\n      Handle(any_internal::SentinelHandle<Handle>());\n}\n\ntemplate <typename Handle>\ntemplate <typename Manager,\n          std::enable_if_t<\n              std::conjunction_v<\n                  std::negation<any_internal::IsAny<Handle, TargetT<Manager>>>,\n                  std::negation<\n                      any_internal::IsAnyClosingPtr<Handle, TargetT<Manager>>>>,\n              int>>\nvoid AnyInitializer<Handle>::ConstructMethod(\n    TypeErasedRef context, Storage dest,\n    MethodsAndHandle* dest_methods_and_handle, size_t available_size,\n    size_t available_align) {\n  using Target = TargetT<Manager>;\n  // This is equivalent to calling `MethodsFor<Target, true>::Construct()`\n  // or `MethodsFor<Target, false>::Construct()`. Separate allocation of\n  // `Dependency<Handle, Target>` from its construction, so that the code for\n  // construction can be shared between the two cases, reducing the code size.\n  Dependency<Handle, Target>* dep_ptr;\n  const any_internal::Methods<Handle>* methods_ptr;\n  if (any_internal::ReprIsInline<Handle, Target>(available_size,\n                                                 available_align)) {\n    dep_ptr = reinterpret_cast<Dependency<Handle, Target>*>(dest);\n    methods_ptr = &MethodsFor<Target, true>::kMethods;\n  } else {\n    if constexpr (alignof(Dependency<Handle, Target>) >\n                  __STDCPP_DEFAULT_NEW_ALIGNMENT__) {\n      dep_ptr = static_cast<Dependency<Handle, Target>*>(operator new(\n          sizeof(Dependency<Handle, Target>),\n          std::align_val_t(alignof(Dependency<Handle, Target>))));\n    } else {\n      dep_ptr = static_cast<Dependency<Handle, Target>*>(operator new(\n          sizeof(Dependency<Handle, Target>)));\n    }\n    new (dest) Dependency<Handle, Target>*(dep_ptr);\n    methods_ptr = &MethodsFor<Target, false>::kMethods;\n  }\n  new (dep_ptr) Dependency<Handle, Target>(context.Cast<Manager>());\n  dest_methods_and_handle->methods = methods_ptr;\n  new (&dest_methods_and_handle->handle) Handle(dep_ptr->get());\n}\n\ntemplate <typename Handle>\ntemplate <\n    typename Manager,\n    std::enable_if_t<any_internal::IsAny<Handle, TargetT<Manager>>::value, int>>\nvoid AnyInitializer<Handle>::ConstructMethod(\n    TypeErasedRef context, Storage dest,\n    MethodsAndHandle* dest_methods_and_handle, size_t available_size,\n    size_t available_align) {\n  using Target = TargetT<Manager>;\n  using TargetValue = std::remove_reference_t<Target>;\n  // Materialize `Target` to adopt its storage.\n  [&](Target&& target) {\n    // `target.methods_and_handle_.methods->used_size <=\n    //      TargetValue::kAvailableSize`, hence if\n    // `TargetValue::kAvailableSize == 0` then\n    // `target.methods_and_handle_.methods->used_size <= available_size`.\n    // No need to check possibly at runtime.\n    if ((TargetValue::kAvailableSize == 0 ||\n         target.methods_and_handle_.methods->used_size <= available_size) &&\n        // Same for alignment.\n        (TargetValue::kAvailableAlign == 0 ||\n         target.methods_and_handle_.methods->used_align <= available_align)) {\n      // Adopt `target` instead of wrapping it.\n      target.methods_and_handle_.methods->move(target.repr_.storage, dest,\n                                               dest_methods_and_handle);\n      target.methods_and_handle_.methods = &NullMethods::kMethods;\n      target.methods_and_handle_.handle =\n          any_internal::SentinelHandle<Handle>();\n      return;\n    }\n    Adopt<Target>(std::forward<Target>(target), dest, dest_methods_and_handle);\n  }(Initializer<Target>(context.Cast<Manager>()).Reference());\n}\n\ntemplate <typename Handle>\ntemplate <\n    typename Manager,\n    std::enable_if_t<\n        any_internal::IsAnyClosingPtr<Handle, TargetT<Manager>>::value, int>>\nvoid AnyInitializer<Handle>::ConstructMethod(\n    TypeErasedRef context, Storage dest,\n    MethodsAndHandle* dest_methods_and_handle,\n    ABSL_ATTRIBUTE_UNUSED size_t available_size,\n    ABSL_ATTRIBUTE_UNUSED size_t available_align) {\n  using Target = TargetT<Manager>;\n  // Materialize `Target` to adopt its storage.\n  const Target target =\n      Initializer<Target>(context.Cast<Manager>()).Construct();\n  if (target == nullptr) {\n    dest_methods_and_handle->methods = &NullMethods::kMethods;\n    new (&dest_methods_and_handle->handle)\n        Handle(any_internal::SentinelHandle<Handle>());\n    return;\n  }\n  // Adopt `*manager` by referring to its representation.\n  target->methods_and_handle_.methods->make_reference(\n      target->repr_.storage, dest, dest_methods_and_handle);\n}\n\ntemplate <typename Handle>\ntemplate <typename Target, std::enable_if_t<!std::is_reference_v<Target>, int>>\ninline void AnyInitializer<Handle>::Adopt(\n    Target&& target, Storage dest, MethodsAndHandle* dest_methods_and_handle) {\n  target.methods_and_handle_.methods->move_to_heap(target.repr_.storage, dest,\n                                                   dest_methods_and_handle);\n  target.methods_and_handle_.methods = &NullMethods::kMethods;\n  target.methods_and_handle_.handle = any_internal::SentinelHandle<Handle>();\n}\n\ntemplate <typename Handle>\ntemplate <typename Target,\n          std::enable_if_t<std::is_rvalue_reference_v<Target>, int>>\ninline void AnyInitializer<Handle>::Adopt(\n    Target&& target, Storage dest, MethodsAndHandle* dest_methods_and_handle) {\n  target.methods_and_handle_.methods->make_reference(target.repr_.storage, dest,\n                                                     dest_methods_and_handle);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_ANY_INITIALIZER_H_\n"
  },
  {
    "path": "riegeli/base/any_internal.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_ANY_INTERNAL_H_\n#define RIEGELI_BASE_ANY_INTERNAL_H_\n\n#include <stddef.h>\n\n#include <algorithm>\n#include <memory>\n#include <new>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/meta/type_traits.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/closing_ptr.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/dependency_base.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/memory_estimator.h\"\n#include \"riegeli/base/type_erased_ref.h\"\n#include \"riegeli/base/type_id.h\"\n\nnamespace riegeli {\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\nclass Any;\ntemplate <typename Handle>\nclass AnyRef;\n\nnamespace any_internal {\n\n// Variants of `Repr`:\n//  * Empty `Any`: `Repr` is not used\n//  * Stored inline: `storage` holds `Dependency<Handle, Manager>`\n//  * Held by pointer: `storage` holds `Dependency<Handle, Manager>*`\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\nstruct Repr {\n  // clang-format off\n  alignas(UnsignedMax(alignof(void*), inline_align))\n      char storage[UnsignedMax(sizeof(void*), inline_size)];\n  // clang-format on\n};\n\n// By convention, a parameter of type `Storage` points to\n// `Repr<Handle, inline_size, inline_align>::storage`.\nusing Storage = char[];\n\n// A `Dependency<Handle, Manager>` is stored inline in\n// `Repr<Handle, inline_size, inline_align>` if it is movable and it fits there.\n//\n// If `inline_size == 0`, the dependency is also required to be stable (because\n// then `Any` declares itself stable) and trivially relocatable (because then\n// `Any` declares itself with trivial ABI and optimizes moving to a plain memory\n// copy of the representation).\n\n// Properties of inline storage in an `Any` instance are expressed as two\n// numbers: `available_size` and `available_align`, while constraints of a\n// movable `Dependency` instance on its storage are expressed as two numbers:\n// `used_size` and `used_align`, such that\n// `used_size <= available_size && used_align <= available_align` implies that\n// the movable `Dependency` can be stored inline in the `Any`.\n//\n// This formulation allows reevaluating the condition with different values of\n// `available_size` and `available_align` when considering adopting the storage\n// for a different `Any` instance, at either compile time or runtime.\n\n// Returns `available_size`: `sizeof` the storage, except that 0 indicates\n// `inline_size == 0`, which means the minimal size of any inline storage with\n// the given alignment, while also putting additional constraints on the\n// `Dependency` (stability and trivial relocatability).\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\nconstexpr size_t AvailableSize() {\n  if (inline_size == 0) return 0;\n  return sizeof(Repr<Handle, inline_size, inline_align>);\n}\n\n// Returns `available_align`: `alignof` the storage, except that 0 means the\n// minimal alignment of any inline storage.\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\nconstexpr size_t AvailableAlign() {\n  if (alignof(Repr<Handle, inline_size, inline_align>) ==\n      alignof(Repr<Handle, 0, 0>)) {\n    return 0;\n  }\n  return alignof(Repr<Handle, inline_size, inline_align>);\n}\n\n// Returns `used_size`: `sizeof` the `Dependency`, except that 0 indicates\n// compatibility with `inline_size == 0`, which means fitting under the minimal\n// size of any inline storage with the given alignment, and being stable.\ntemplate <typename Handle, typename Manager>\nconstexpr size_t UsedSize() {\n  if (sizeof(Dependency<Handle, Manager>) <=\n          sizeof(Repr<Handle, 0, alignof(Dependency<Handle, Manager>)>) &&\n      Dependency<Handle, Manager>::kIsStable) {\n    return 0;\n  }\n  return sizeof(Dependency<Handle, Manager>);\n}\n\n// Returns `used_align`: `alignof` the storage, except that 0 means fitting\n// under the minimal alignment of any inline storage. Making this a special\n// case allows to optimize out comparisons of a compile time `used_align`\n// against a runtime `available_align`.\ntemplate <typename Handle, typename Manager>\nconstexpr size_t UsedAlign() {\n  if (alignof(Dependency<Handle, Manager>) <= alignof(Repr<Handle, 0, 0>)) {\n    return 0;\n  }\n  return alignof(Dependency<Handle, Manager>);\n}\n\ntemplate <typename Handle, typename Manager>\nconstexpr bool ReprIsInline(size_t available_size, size_t available_align) {\n  return std::is_move_constructible_v<Dependency<Handle, Manager>> &&\n         UsedSize<Handle, Manager>() <= available_size &&\n         UsedAlign<Handle, Manager>() <= available_align;\n}\n\ntemplate <typename Handle, typename Manager, size_t inline_size,\n          size_t inline_align>\nconstexpr bool IsInline() {\n  return ReprIsInline<Handle, Manager>(\n      AvailableSize<Handle, inline_size, inline_align>(),\n      AvailableAlign<Handle, inline_size, inline_align>());\n}\n\ntemplate <typename Handle>\nstruct MethodsAndHandle;\n\n// Method pointers.\ntemplate <typename Handle>\nstruct Methods {\n  // Destroys `self`.\n  void (*destroy)(Storage self);\n  // Constructs `dest` with `*dest_handle` by moving from `src`. Destroys `src`.\n  void (*move)(Storage src, Storage dest,\n               MethodsAndHandle<Handle>* dest_methods_and_handle);\n  // Constructs a differently represented `dest` with `*dest_methods_and_handle`\n  // by moving from `src` to the heap and pointing `dest` to that. Destroys\n  // `src`. Used only if `used_size > 0`.\n  void (*move_to_heap)(Storage src, Storage dest,\n                       MethodsAndHandle<Handle>* dest_methods_and_handle);\n  // Constructs a differently represented `dest` with `*dest_methods_and_handle`\n  // by pointing `dest` to `src`.\n  void (*make_reference)(Storage src, Storage dest,\n                         MethodsAndHandle<Handle>* dest_methods_and_handle);\n  size_t used_size;\n  size_t used_align;\n  TypeId type_id;\n  bool (*is_owning)(const Storage self);\n  // Returns the `Manager&` stored in `self`, with the `Manager` type\n  // corresponding to `type_id`. Used only if `type_id != nullptr`.\n  // If `self` is const then `Manager` should be const, otherwise `Manager`\n  // can be non-const.\n  TypeErasedRef (*get_raw_manager)(const Storage self);\n  void (*register_subobjects)(const Storage self,\n                              MemoryEstimator& memory_estimator);\n};\n\n// Grouped members so that their address can be passed together.\ntemplate <typename Handle>\nstruct MethodsAndHandle {\n  MethodsAndHandle() noexcept {}\n\n  const Methods<Handle>* methods;\n  union {\n    Handle handle;\n  };\n};\n\ntemplate <typename Handle>\nstruct NullMethods;\ntemplate <typename Handle, typename Manager>\nstruct MethodsForReference;\ntemplate <typename Handle, typename Manager, bool is_inline>\nstruct MethodsFor;\n\n// `IsAny` detects `Any` or `AnyRef` type with the given `Handle`, or an rvalue\n// reference to it.\n\ntemplate <typename Handle, typename T>\nstruct IsAny : std::false_type {};\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\nstruct IsAny<Handle, Any<Handle, inline_size, inline_align>> : std::true_type {\n};\n\ntemplate <typename Handle>\nstruct IsAny<Handle, AnyRef<Handle>> : std::true_type {};\n\ntemplate <typename Handle, typename T>\nstruct IsAny<Handle, T&&> : IsAny<Handle, T> {};\n\n// `IsAnyClosingPtr` detects `Any` or `AnyRef` type with the given `Handle`,\n// wrapped in `ClosingPtrType` or in an rvalue reference to it.\n\ntemplate <typename Handle, typename T>\nstruct IsAnyClosingPtr : std::false_type {};\n\ntemplate <typename Handle, size_t inline_size, size_t inline_align>\nstruct IsAnyClosingPtr<\n    Handle,\n    std::unique_ptr<Any<Handle, inline_size, inline_align>, NullDeleter>>\n    : std::true_type {};\n\ntemplate <typename Handle>\nstruct IsAnyClosingPtr<Handle, std::unique_ptr<AnyRef<Handle>, NullDeleter>>\n    : std::true_type {};\n\ntemplate <typename Handle, typename T>\nstruct IsAnyClosingPtr<Handle, T&&> : IsAnyClosingPtr<Handle, T> {};\n\ntemplate <typename Handle>\ninline Handle SentinelHandle() {\n  return Initializer<Handle>(\n      RiegeliDependencySentinel(static_cast<Handle*>(nullptr)));\n}\n\n// Implementation details follow.\n\ntemplate <typename Handle>\nstruct NullMethods {\n private:\n  static void Destroy(ABSL_ATTRIBUTE_UNUSED Storage self) {}\n  static void Move(ABSL_ATTRIBUTE_UNUSED Storage src,\n                   ABSL_ATTRIBUTE_UNUSED Storage dest,\n                   MethodsAndHandle<Handle>* dest_methods_and_handle) {\n    dest_methods_and_handle->methods = &kMethods;\n    new (&dest_methods_and_handle->handle) Handle(SentinelHandle<Handle>());\n  }\n  static bool IsOwning(ABSL_ATTRIBUTE_UNUSED const Storage self) {\n    return false;\n  }\n  static void RegisterSubobjects(\n      ABSL_ATTRIBUTE_UNUSED const Storage self,\n      ABSL_ATTRIBUTE_UNUSED MemoryEstimator& memory_estimator) {}\n\n public:\n  static constexpr Methods<Handle> kMethods = {\n      Destroy, Move,    nullptr,  Move,    0,\n      0,       nullptr, IsOwning, nullptr, RegisterSubobjects};\n};\n\ntemplate <typename Handle, typename Manager>\nstruct MethodsForReference {\n private:\n  static Dependency<Handle, Manager>* dep_ptr(const Storage self) {\n    return *std::launder(\n        reinterpret_cast<Dependency<Handle, Manager>* const*>(self));\n  }\n\n  static void Destroy(ABSL_ATTRIBUTE_UNUSED Storage self) {}\n  static void Move(Storage src, Storage dest,\n                   MethodsAndHandle<Handle>* dest_methods_and_handle) {\n    new (dest) Dependency<Handle, Manager>*(dep_ptr(src));\n    dest_methods_and_handle->methods = &kMethods;\n    new (&dest_methods_and_handle->handle) Handle(dep_ptr(dest)->get());\n  }\n  static bool IsOwning(const Storage self) { return dep_ptr(self)->IsOwning(); }\n  static TypeErasedRef GetRawManager(const Storage self) {\n    return TypeErasedRef(dep_ptr(self)->manager());\n  }\n  static void RegisterSubobjects(\n      ABSL_ATTRIBUTE_UNUSED const Storage self,\n      ABSL_ATTRIBUTE_UNUSED MemoryEstimator& memory_estimator) {}\n\n public:\n  static constexpr Methods<Handle> kMethods = {\n      Destroy,\n      Move,\n      nullptr,\n      Move,\n      0,\n      0,\n      TypeId::For<absl::remove_cvref_t<Manager>>(),\n      IsOwning,\n      GetRawManager,\n      RegisterSubobjects};\n};\n\ntemplate <typename Handle, typename Manager>\nstruct MethodsFor<Handle, Manager, false> {\n  static void Construct(Storage self, Handle* self_handle,\n                        Initializer<Manager> manager) {\n    new (self) Dependency<Handle, Manager>*(\n        new Dependency<Handle, Manager>(std::move(manager)));\n    new (self_handle) Handle(dep_ptr(self)->get());\n  }\n\n private:\n  static Dependency<Handle, Manager>* dep_ptr(const Storage self) {\n    return *std::launder(\n        reinterpret_cast<Dependency<Handle, Manager>* const*>(self));\n  }\n\n  static void Destroy(Storage self) { delete dep_ptr(self); }\n  static void Move(Storage src, Storage dest,\n                   MethodsAndHandle<Handle>* dest_methods_and_handle) {\n    new (dest) Dependency<Handle, Manager>*(dep_ptr(src));\n    dest_methods_and_handle->methods = &kMethods;\n    new (&dest_methods_and_handle->handle) Handle(dep_ptr(dest)->get());\n  }\n  static void MakeReference(Storage src, Storage dest,\n                            MethodsAndHandle<Handle>* dest_methods_and_handle) {\n    new (dest) Dependency<Handle, Manager>*(dep_ptr(src));\n    dest_methods_and_handle->methods =\n        &MethodsForReference<Handle, Manager>::kMethods;\n    new (&dest_methods_and_handle->handle) Handle(dep_ptr(dest)->get());\n  }\n  static bool IsOwning(const Storage self) { return dep_ptr(self)->IsOwning(); }\n  static TypeErasedRef GetRawManager(const Storage self) {\n    return TypeErasedRef(dep_ptr(self)->manager());\n  }\n  static void RegisterSubobjects(const Storage self,\n                                 MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterDynamicObject(dep_ptr(self));\n  }\n\n public:\n  static constexpr Methods<Handle> kMethods = {\n      Destroy,\n      Move,\n      nullptr,\n      MakeReference,\n      0,\n      0,\n      TypeId::For<absl::remove_cvref_t<Manager>>(),\n      IsOwning,\n      GetRawManager,\n      RegisterSubobjects};\n};\n\ntemplate <typename Handle, typename Manager>\nstruct MethodsFor<Handle, Manager, true> {\n  static void Construct(Storage self, Handle* self_handle,\n                        Initializer<Manager> manager) {\n    new (self) Dependency<Handle, Manager>(std::move(manager));\n    new (self_handle) Handle(dep(self).get());\n  }\n\n private:\n  static Dependency<Handle, Manager>& dep(Storage self) {\n    return *std::launder(reinterpret_cast<Dependency<Handle, Manager>*>(self));\n  }\n  static const Dependency<Handle, Manager>& dep(const Storage self) {\n    return *std::launder(\n        reinterpret_cast<const Dependency<Handle, Manager>*>(self));\n  }\n  static Dependency<Handle, Manager>* dep_ptr(const Storage self) {\n    return *std::launder(\n        reinterpret_cast<Dependency<Handle, Manager>* const*>(self));\n  }\n\n  static void Destroy(Storage self) {\n    dep(self).~Dependency<Handle, Manager>();\n  }\n  static void Move(Storage src, Storage dest,\n                   MethodsAndHandle<Handle>* dest_methods_and_handle) {\n    new (dest) Dependency<Handle, Manager>(std::move(dep(src)));\n    dep(src).~Dependency<Handle, Manager>();\n    dest_methods_and_handle->methods = &kMethods;\n    new (&dest_methods_and_handle->handle) Handle(dep(dest).get());\n  }\n  static void MoveToHeap(Storage src, Storage dest,\n                         MethodsAndHandle<Handle>* dest_methods_and_handle) {\n    new (dest) Dependency<Handle, Manager>*(\n        new Dependency<Handle, Manager>(std::move(dep(src))));\n    dep(src).~Dependency<Handle, Manager>();\n    dest_methods_and_handle->methods =\n        &MethodsFor<Handle, Manager, false>::kMethods;\n    new (&dest_methods_and_handle->handle) Handle(dep_ptr(dest)->get());\n  }\n  static void MakeReference(Storage src, Storage dest,\n                            MethodsAndHandle<Handle>* dest_methods_and_handle) {\n    new (dest) Dependency<Handle, Manager>*(&dep(src));\n    dest_methods_and_handle->methods =\n        &MethodsForReference<Handle, Manager>::kMethods;\n    new (&dest_methods_and_handle->handle) Handle(dep_ptr(dest)->get());\n  }\n  static bool IsOwning(const Storage self) { return dep(self).IsOwning(); }\n  static TypeErasedRef GetRawManager(const Storage self) {\n    return TypeErasedRef(dep(self).manager());\n  }\n  static void RegisterSubobjects(const Storage self,\n                                 MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&dep(self));\n  }\n\n public:\n  static constexpr Methods<Handle> kMethods = {\n      Destroy,\n      Move,\n      MoveToHeap,\n      MakeReference,\n      UsedSize<Handle, Manager>(),\n      UsedAlign<Handle, Manager>(),\n      TypeId::For<absl::remove_cvref_t<Manager>>(),\n      IsOwning,\n      GetRawManager,\n      RegisterSubobjects};\n};\n\n}  // namespace any_internal\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_ANY_INTERNAL_H_\n"
  },
  {
    "path": "riegeli/base/arithmetic.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_ARITHMETIC_H_\n#define RIEGELI_BASE_ARITHMETIC_H_\n\n#include <stddef.h>\n\n#include <limits>\n#include <type_traits>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/bits.h\"\n#include \"absl/numeric/int128.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/type_traits.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `IsUnsignedInt<T>::value` is `true` for unsigned integral types, including\n// `absl::uint128`.\ntemplate <typename T>\nstruct IsUnsignedInt\n    : std::conjunction<std::is_integral<T>, std::is_unsigned<T>> {};\ntemplate <>\nstruct IsUnsignedInt<absl::uint128> : std::true_type {};\n\n// `IsSignedInt<T>::value` is `true` for signed integral types, including\n// `absl::int128`.\ntemplate <typename T>\nstruct IsSignedInt : std::conjunction<std::is_integral<T>, std::is_signed<T>> {\n};\ntemplate <>\nstruct IsSignedInt<absl::int128> : std::true_type {};\n\n// `IsInt<T>::value` is `true` for integral types, including `absl::uint128` and\n// `absl::int128`.\ntemplate <typename T>\nstruct IsInt : std::is_integral<T> {};\ntemplate <>\nstruct IsInt<absl::uint128> : std::true_type {};\ntemplate <>\nstruct IsInt<absl::int128> : std::true_type {};\n\n// `MakeUnsigned<T>::type` and `MakeUnsignedT<T>` transform a signed integral\n// type to the corresponding unsigned type, including `absl::int128`, and leave\n// unsigned integral types unchanged, including `absl::uint128`.\ntemplate <typename T>\nstruct MakeUnsigned : std::make_unsigned<T> {};\n\ntemplate <>\nstruct MakeUnsigned<absl::int128> {\n  using type = absl::uint128;\n};\n\ntemplate <>\nstruct MakeUnsigned<absl::uint128> {\n  using type = absl::uint128;\n};\n\ntemplate <typename T>\nusing MakeUnsignedT = typename MakeUnsigned<T>::type;\n\n// `IntCast<A>(value)` converts between integral types, asserting that `value`\n// fits in the target type.\n\ntemplate <typename A, typename B,\n          std::enable_if_t<\n              std::conjunction_v<IsUnsignedInt<A>, IsUnsignedInt<B>>, int> = 0>\nconstexpr A IntCast(B value) {\n  RIEGELI_ASSERT_LE(value, std::numeric_limits<A>::max())\n      << \"Failed precondition of IntCast(): value out of range\";\n  return static_cast<A>(value);\n}\n\ntemplate <typename A, typename B,\n          std::enable_if_t<std::conjunction_v<IsUnsignedInt<A>, IsSignedInt<B>>,\n                           int> = 0>\nconstexpr A IntCast(B value) {\n  RIEGELI_ASSERT_GE(value, 0)\n      << \"Failed precondition of IntCast(): value out of range\";\n  RIEGELI_ASSERT_LE(static_cast<MakeUnsignedT<B>>(value),\n                    std::numeric_limits<A>::max())\n      << \"Failed precondition of IntCast(): value out of range\";\n  return static_cast<A>(value);\n}\n\ntemplate <typename A, typename B,\n          std::enable_if_t<std::conjunction_v<IsSignedInt<A>, IsUnsignedInt<B>>,\n                           int> = 0>\nconstexpr A IntCast(B value) {\n  RIEGELI_ASSERT_LE(value, MakeUnsignedT<A>{std::numeric_limits<A>::max()})\n      << \"Failed precondition of IntCast(): value out of range\";\n  return static_cast<A>(value);\n}\n\ntemplate <typename A, typename B,\n          std::enable_if_t<std::conjunction_v<IsSignedInt<A>, IsSignedInt<B>>,\n                           int> = 0>\nconstexpr A IntCast(B value) {\n  RIEGELI_ASSERT_GE(value, std::numeric_limits<A>::min())\n      << \"Failed precondition of IntCast(): value out of range\";\n  RIEGELI_ASSERT_LE(value, std::numeric_limits<A>::max())\n      << \"Failed precondition of IntCast(): value out of range\";\n  return static_cast<A>(value);\n}\n\n// `UnsignedCast(value)` converts `value` to the corresponding unsigned type,\n// asserting that `value` was non-negative.\ntemplate <typename T,\n          std::enable_if_t<std::disjunction_v<IsSignedInt<T>, IsUnsignedInt<T>>,\n                           int> = 0>\nconstexpr MakeUnsignedT<T> UnsignedCast(T value) {\n  return IntCast<MakeUnsignedT<T>>(value);\n}\n\n// `NegatingUnsignedCast(value)` converts `-value` to the corresponding unsigned\n// type, asserting that `value` was non-positive, and correctly handling\n// `std::numeric_limits<T>::min()`.\ntemplate <typename T, std::enable_if_t<IsSignedInt<T>::value, int> = 0>\nconstexpr MakeUnsignedT<T> NegatingUnsignedCast(T value) {\n  RIEGELI_ASSERT_LE(value, 0)\n      << \"Failed precondition of NegatingUnsignedCast(): positive value\";\n  // Negate in the unsigned space to correctly handle\n  // `std::numeric_limits<T>::min()`.\n  return static_cast<MakeUnsignedT<T>>(0 -\n                                       static_cast<MakeUnsignedT<T>>(value));\n}\n\n// `SignedMin()` returns the minimum of its arguments, which must be signed\n// integers, as their widest type.\n\ntemplate <typename A, std::enable_if_t<IsSignedInt<A>::value, int> = 0>\nconstexpr A SignedMin(A a) {\n  return a;\n}\n\ntemplate <typename A, typename B,\n          std::enable_if_t<std::conjunction_v<IsSignedInt<A>, IsSignedInt<B>>,\n                           int> = 0>\nconstexpr std::common_type_t<A, B> SignedMin(A a, B b) {\n  return a <= b ? a : b;\n}\n\ntemplate <\n    typename A, typename B, typename... Rest,\n    std::enable_if_t<std::conjunction_v<\n                         std::bool_constant<(sizeof...(Rest) > 0)>,\n                         IsSignedInt<A>, IsSignedInt<B>, IsSignedInt<Rest>...>,\n                     int> = 0>\nconstexpr std::common_type_t<A, B, Rest...> SignedMin(A a, B b, Rest... rest) {\n  return SignedMin(SignedMin(a, b), rest...);\n}\n\n// `SignedMax()` returns the maximum of its arguments, which must be signed\n// integers, as their widest type.\n\ntemplate <typename A, std::enable_if_t<IsSignedInt<A>::value, int> = 0>\nconstexpr A SignedMax(A a) {\n  return a;\n}\n\ntemplate <typename A, typename B,\n          std::enable_if_t<std::conjunction_v<IsSignedInt<A>, IsSignedInt<B>>,\n                           int> = 0>\nconstexpr std::common_type_t<A, B> SignedMax(A a, B b) {\n  return a >= b ? a : b;\n}\n\ntemplate <\n    typename A, typename B, typename... Rest,\n    std::enable_if_t<std::conjunction_v<\n                         std::bool_constant<(sizeof...(Rest) > 0)>,\n                         IsSignedInt<A>, IsSignedInt<B>, IsSignedInt<Rest>...>,\n                     int> = 0>\nconstexpr std::common_type_t<A, B, Rest...> SignedMax(A a, B b, Rest... rest) {\n  return SignedMax(SignedMax(a, b), rest...);\n}\n\n// `UnsignedMin()` returns the minimum of its arguments, which must be unsigned\n// integers, as their narrowest type.\n\ntemplate <typename A, std::enable_if_t<IsUnsignedInt<A>::value, int> = 0>\nconstexpr A UnsignedMin(A a) {\n  return a;\n}\n\ntemplate <typename A, typename B,\n          std::enable_if_t<\n              std::conjunction_v<IsUnsignedInt<A>, IsUnsignedInt<B>>, int> = 0>\nconstexpr IntersectionTypeT<A, B> UnsignedMin(A a, B b) {\n  return static_cast<IntersectionTypeT<A, B>>(a <= b ? a : b);\n}\n\ntemplate <typename A, typename B, typename... Rest,\n          std::enable_if_t<\n              std::conjunction_v<std::bool_constant<(sizeof...(Rest) > 0)>,\n                                 IsUnsignedInt<A>, IsUnsignedInt<B>,\n                                 IsUnsignedInt<Rest>...>,\n              int> = 0>\nconstexpr IntersectionTypeT<A, B, Rest...> UnsignedMin(A a, B b, Rest... rest) {\n  return UnsignedMin(UnsignedMin(a, b), rest...);\n}\n\n// `UnsignedMax()` returns the maximum of its arguments, which must be unsigned\n// integers, as their widest type.\n\ntemplate <typename A, std::enable_if_t<IsUnsignedInt<A>::value, int> = 0>\nconstexpr A UnsignedMax(A a) {\n  return a;\n}\n\ntemplate <typename A, typename B,\n          std::enable_if_t<\n              std::conjunction_v<IsUnsignedInt<A>, IsUnsignedInt<B>>, int> = 0>\nconstexpr std::common_type_t<A, B> UnsignedMax(A a, B b) {\n  return a >= b ? a : b;\n}\n\ntemplate <typename A, typename B, typename... Rest,\n          std::enable_if_t<\n              std::conjunction_v<std::bool_constant<(sizeof...(Rest) > 0)>,\n                                 IsUnsignedInt<A>, IsUnsignedInt<B>,\n                                 IsUnsignedInt<Rest>...>,\n              int> = 0>\nconstexpr std::common_type_t<A, B, Rest...> UnsignedMax(A a, B b,\n                                                        Rest... rest) {\n  return UnsignedMax(UnsignedMax(a, b), rest...);\n}\n\n// `UnsignedClamp(value, min_value, max_value)` is at least `min_value`,\n// at most `max(max_value, min_value)`, preferably `value`.\n//\n// If `min_value <= max_value`, then it is equivalent to `std::clamp()`,\n// otherwise `min_value` wins.\ntemplate <\n    typename Value, typename Min, typename Max,\n    std::enable_if_t<std::conjunction_v<IsUnsignedInt<Value>,\n                                        IsUnsignedInt<Min>, IsUnsignedInt<Max>>,\n                     int> = 0>\nconstexpr std::common_type_t<IntersectionTypeT<Value, Max>, Min> UnsignedClamp(\n    Value value, Min min, Max max) {\n  return UnsignedMax(UnsignedMin(value, max), min);\n}\n\n// `SaturatingIntCast()` converts an integer value to another integer type, or\n// returns the appropriate bound of the type if conversion would overflow.\n\ntemplate <typename A, typename B,\n          std::enable_if_t<\n              std::conjunction_v<IsUnsignedInt<A>, IsUnsignedInt<B>>, int> = 0>\nconstexpr A SaturatingIntCast(B value) {\n  if (ABSL_PREDICT_FALSE(value > std::numeric_limits<A>::max())) {\n    return std::numeric_limits<A>::max();\n  }\n  return static_cast<A>(value);\n}\n\ntemplate <typename A, typename B,\n          std::enable_if_t<std::conjunction_v<IsUnsignedInt<A>, IsSignedInt<B>>,\n                           int> = 0>\nconstexpr A SaturatingIntCast(B value) {\n  if (ABSL_PREDICT_FALSE(value < 0)) return 0;\n  if (ABSL_PREDICT_FALSE(static_cast<MakeUnsignedT<B>>(value) >\n                         std::numeric_limits<A>::max())) {\n    return std::numeric_limits<A>::max();\n  }\n  return static_cast<A>(value);\n}\n\ntemplate <typename A, typename B,\n          std::enable_if_t<std::conjunction_v<IsSignedInt<A>, IsUnsignedInt<B>>,\n                           int> = 0>\nconstexpr A SaturatingIntCast(B value) {\n  if (ABSL_PREDICT_FALSE(value >\n                         MakeUnsignedT<A>{std::numeric_limits<A>::max()})) {\n    return std::numeric_limits<A>::max();\n  }\n  return static_cast<A>(value);\n}\n\ntemplate <typename A, typename B,\n          std::enable_if_t<std::conjunction_v<IsSignedInt<A>, IsSignedInt<B>>,\n                           int> = 0>\nconstexpr A SaturatingIntCast(B value) {\n  if (ABSL_PREDICT_FALSE(value < std::numeric_limits<A>::min())) {\n    return std::numeric_limits<A>::min();\n  }\n  if (ABSL_PREDICT_FALSE(value > std::numeric_limits<A>::max())) {\n    return std::numeric_limits<A>::max();\n  }\n  return static_cast<A>(value);\n}\n\n// `SaturatingAdd()` adds unsigned values, or returns max possible value of the\n// type if addition would overflow.\n\ntemplate <typename T, std::enable_if_t<IsUnsignedInt<T>::value, int> = 0>\nconstexpr T SaturatingAdd() {\n  return 0;\n}\n\ntemplate <typename T, std::enable_if_t<IsUnsignedInt<T>::value, int> = 0>\nconstexpr T SaturatingAdd(T a) {\n  return a;\n}\n\ntemplate <typename T, std::enable_if_t<IsUnsignedInt<T>::value, int> = 0>\nconstexpr T SaturatingAdd(T a, T b) {\n  if (ABSL_PREDICT_FALSE(b > std::numeric_limits<T>::max() - a)) {\n    return std::numeric_limits<T>::max();\n  }\n  return a + b;\n}\n\ntemplate <typename T, typename... Rest,\n          std::enable_if_t<\n              std::conjunction_v<std::bool_constant<(sizeof...(Rest) > 0)>,\n                                 IsUnsignedInt<T>, IsUnsignedInt<Rest>...>,\n              int> = 0>\nconstexpr T SaturatingAdd(T a, T b, Rest... rest) {\n  return SaturatingAdd(SaturatingAdd(a, b), rest...);\n}\n\n// `SaturatingSub()` subtracts unsigned values, or returns 0 if subtraction\n// would underflow.\ntemplate <typename T, typename U,\n          std::enable_if_t<\n              std::conjunction_v<IsUnsignedInt<T>, IsUnsignedInt<U>>, int> = 0>\nconstexpr T SaturatingSub(T a, U b) {\n  if (ABSL_PREDICT_FALSE(b > a)) return 0;\n  return a - IntCast<T>(b);\n}\n\n// `RoundDown()` rounds an unsigned value downwards to the nearest multiple of\n// the given power of 2.\ntemplate <size_t alignment, typename T,\n          std::enable_if_t<\n              std::conjunction_v<IsUnsignedInt<T>,\n                                 std::integral_constant<\n                                     bool, absl::has_single_bit(alignment)>>,\n              int> = 0>\nconstexpr T RoundDown(T value) {\n  return value & ~T{alignment - 1};\n}\n\n// `RoundUp()` rounds an unsigned value upwards to the nearest multiple of the\n// given power of 2.\ntemplate <size_t alignment, typename T,\n          std::enable_if_t<\n              std::conjunction_v<IsUnsignedInt<T>,\n                                 std::integral_constant<\n                                     bool, absl::has_single_bit(alignment)>>,\n              int> = 0>\nconstexpr T RoundUp(T value) {\n  return ((value - 1) | T{alignment - 1}) + 1;\n}\n\n// `PtrDistance(first, last)` returns `last - first` as `size_t`, asserting that\n// `first <= last`.\ntemplate <typename A>\nconstexpr size_t PtrDistance(const A* absl_nullable first,\n                             const A* absl_nullable last) {\n  RIEGELI_ASSERT_EQ(first == nullptr, last == nullptr)\n      << \"Failed precondition of PtrDistance(): \"\n         \"nullptr compared with non-nullptr\";\n  RIEGELI_ASSERT_LE(first, last)\n      << \"Failed precondition of PtrDistance(): pointers in the wrong order\";\n  return IntCast<size_t>(last - first);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_ARITHMETIC_H_\n"
  },
  {
    "path": "riegeli/base/assert.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/assert.h\"\n\n#include <string>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/log/absl_log.h\"\n#include \"riegeli/base/stream_utils.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli::assert_internal {\n\nCheckResult::CheckResult(const char* function, const char* prefix)\n    : header_(new StringOStream(new std::string())) {\n  header() << \"Check failed in \" << function << \": \" << prefix;\n}\n\nCheckFailed::CheckFailed(const char* file, int line, CheckResult check_result)\n    : file_(file),\n      line_(line),\n      check_result_(check_result),\n      details_(new StringOStream(new std::string())) {}\n\nCheckFailed::~CheckFailed() {\n  if (!details_->dest()->empty()) {\n    check_result_.header() << \"; \" << *details_->dest();\n  }\n  ABSL_LOG(FATAL).AtLocation(file_, line_) << *check_result_.header().dest();\n}\n\nvoid CheckNotNullFailed(const char* file, int line, const char* function,\n                        const char* expression) {\n  CheckResult check_result(function, expression);\n  check_result.header() << \" != nullptr\";\n  CheckFailed check_failed(file, line, check_result);\n}\n\nCheckResult CheckImpossibleResult(const char* function) {\n  return CheckResult(function, \"Impossible\");\n}\n\n}  // namespace riegeli::assert_internal\n"
  },
  {
    "path": "riegeli/base/assert.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_ASSERT_H_\n#define RIEGELI_BASE_ASSERT_H_\n\n#include <stddef.h>\n\n#include <cassert>\n#include <ostream>  // IWYU pragma: export\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/debug.h\"\n#include \"riegeli/base/port.h\"\n#include \"riegeli/base/stream_utils.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `RIEGELI_DEBUG` determines whether assertions are verified or just assumed.\n// By default it follows `NDEBUG`.\n\n#ifndef RIEGELI_DEBUG\n#ifdef NDEBUG\n#define RIEGELI_DEBUG 0\n#else\n#define RIEGELI_DEBUG 1\n#endif\n#endif\n\nnamespace assert_internal {\n\n#if __cpp_lib_unreachable\n#define RIEGELI_INTERNAL_UNREACHABLE() ::std::unreachable()\n#elif RIEGELI_INTERNAL_HAS_BUILTIN(__builtin_unreachable) || \\\n    RIEGELI_INTERNAL_IS_GCC_VERSION(4, 5)\n#define RIEGELI_INTERNAL_UNREACHABLE() __builtin_unreachable()\n#elif defined(_WIN32)\n#define RIEGELI_INTERNAL_UNREACHABLE() __assume(false)\n#else\n#define RIEGELI_INTERNAL_UNREACHABLE() for (;;)\n#endif\n\n// Indicates that a check succeeded or failed.\n//\n// If it failed, stores a stream for writing the header.\nclass CheckResult {\n public:\n  // A check succeeded.\n  CheckResult() = default;\n\n  // A check failed. The header will begin with\n  // \"Check failed in function: prefix\".\n  explicit CheckResult(const char* function, const char* prefix);\n\n  CheckResult(const CheckResult& that) = default;\n  CheckResult& operator=(const CheckResult& that) = default;\n\n  // Returns `true` if the check succeeded.\n  explicit operator bool() const { return header_ == nullptr; }\n\n  // Returns the header stream.\n  //\n  // Precondition: the check failed, i.e. `*this` is `false`.\n  StringOStream& header() {\n    assert(header_ != nullptr);\n    return *header_;\n  }\n\n private:\n  StringOStream* absl_nullable header_ = nullptr;\n};\n\n// Stores a `CheckResult` and a stream for adding details to the message.\n// The message is \"header; details\", or just \"header\" if details are empty.\n// In the destructor, outputs the message and terminates the program.\nclass CheckFailed {\n public:\n  explicit CheckFailed(const char* file, int line, CheckResult check_result);\n\n  // Allows to add details to the message by writing to the stream.\n  std::ostream& details() { return *details_; }\n\n  // Prints the check failure message and terminates the program.\n  ABSL_ATTRIBUTE_NORETURN ~CheckFailed();\n\n private:\n  const char* file_;\n  int line_;\n  CheckResult check_result_;\n  StringOStream* details_;\n};\n\n// Indicates that a check failed with the message header\n// \"Check failed in function: assertion (a vs. b)\".\ntemplate <typename A, typename B>\nABSL_ATTRIBUTE_COLD CheckResult CheckOpResult(const char* function,\n                                              const char* assertion, const A& a,\n                                              const B& b) {\n  CheckResult check_result(function, assertion);\n  check_result.header() << \" (\" << riegeli::Debug(a) << \" vs. \"\n                        << riegeli::Debug(b) << \")\";\n  return check_result;\n}\n\n// Indicates that a check failed with the message header\n// \"Check failed in function: expression is OK (status)\".\n\nnamespace assert_internal {\n\ntemplate <typename T, typename Enable = void>\nstruct HasStatus : std::false_type {};\n\ntemplate <typename T>\nstruct HasStatus<T, std::void_t<decltype(std::declval<T>().status())>>\n    : std::true_type {};\n\n}  // namespace assert_internal\n\ntemplate <typename StatusType>\nABSL_ATTRIBUTE_COLD CheckResult CheckOkResult(const char* function,\n                                              const char* expression,\n                                              const StatusType& status) {\n  if constexpr (!assert_internal::HasStatus<StatusType>::value) {\n    // `absl::Status`.\n    CheckResult check_result(function, expression);\n    check_result.header() << \" is OK (\" << status << \")\";\n    return check_result;\n  } else {\n    // `absl::StatusOr<T>`.\n    return CheckOkResult(function, expression, status.status());\n  }\n}\n\n// Writes \"Check failed in function: expression != nullptr\" and terminates\n// the program.\nABSL_ATTRIBUTE_NORETURN void CheckNotNullFailed(const char* file, int line,\n                                                const char* function,\n                                                const char* expression);\n\n// Indicates that a check failed with the message header\n// \"Check failed in function: Impossible\".\nABSL_ATTRIBUTE_COLD CheckResult CheckImpossibleResult(const char* function);\n\n// These functions allow using `a` and `b` multiple times without reevaluation.\n// They are small enough to be inlined, with the slow path delegated to\n// `CheckOpResult()`.\n#define RIEGELI_INTERNAL_DEFINE_CHECK_OP(name, op)                            \\\n  template <typename A, typename B>                                           \\\n  inline CheckResult Check##name(const char* function, const char* assertion, \\\n                                 const A& a, const B& b) {                    \\\n    if (ABSL_PREDICT_TRUE(a op b)) return CheckResult();                      \\\n    CheckResult check_result = CheckOpResult(function, assertion, a, b);      \\\n    if (check_result) RIEGELI_INTERNAL_UNREACHABLE();                         \\\n    return check_result;                                                      \\\n  }                                                                           \\\n  static_assert(true, \"\")  // Eat a semicolon.\n\nRIEGELI_INTERNAL_DEFINE_CHECK_OP(Eq, ==);\nRIEGELI_INTERNAL_DEFINE_CHECK_OP(Ne, !=);\nRIEGELI_INTERNAL_DEFINE_CHECK_OP(Lt, <);\nRIEGELI_INTERNAL_DEFINE_CHECK_OP(Gt, >);\nRIEGELI_INTERNAL_DEFINE_CHECK_OP(Le, <=);\nRIEGELI_INTERNAL_DEFINE_CHECK_OP(Ge, >=);\n\n#undef RIEGELI_INTERNAL_DEFINE_CHECK_OP\n\ntemplate <typename StatusType>\ninline CheckResult CheckOk(const char* function, const char* expression,\n                           const StatusType& status) {\n  if (ABSL_PREDICT_TRUE(status.ok())) return CheckResult();\n  CheckResult check_result = CheckOkResult(function, expression, status);\n  if (check_result) RIEGELI_INTERNAL_UNREACHABLE();\n  return check_result;\n}\n\ntemplate <typename T>\ninline T&& CheckNotNull(const char* file, int line, const char* function,\n                        const char* expression, T&& value) {\n  if (ABSL_PREDICT_FALSE(value == nullptr)) {\n    CheckNotNullFailed(file, line, function, expression);\n  }\n  return std::forward<T>(value);\n}\n\n#if !RIEGELI_DEBUG\n\n// These functions allow using `a` and `b` multiple times without reevaluation.\n// They are small enough to be inlined.\n#define RIEGELI_INTERNAL_DEFINE_EVAL_ASSERT_OP(name, op)   \\\n  template <typename A, typename B>                        \\\n  inline bool EvalAssert##name(const A& a, const B& b) {   \\\n    return true || a op b; /* Check that this compiles. */ \\\n  }                                                        \\\n  static_assert(true, \"\")  // Eat a semicolon.\n\nRIEGELI_INTERNAL_DEFINE_EVAL_ASSERT_OP(Eq, ==);\nRIEGELI_INTERNAL_DEFINE_EVAL_ASSERT_OP(Ne, !=);\nRIEGELI_INTERNAL_DEFINE_EVAL_ASSERT_OP(Lt, <);\nRIEGELI_INTERNAL_DEFINE_EVAL_ASSERT_OP(Gt, >);\nRIEGELI_INTERNAL_DEFINE_EVAL_ASSERT_OP(Le, <=);\nRIEGELI_INTERNAL_DEFINE_EVAL_ASSERT_OP(Ge, >=);\n\n#undef RIEGELI_INTERNAL_DEFINE_EVAL_ASSERT_OP\n\ntemplate <typename T>\ninline T&& EvalAssertNotNull(T&& value) {\n  ABSL_ATTRIBUTE_UNUSED const bool condition =\n      true || value == nullptr;  // Check that this compiles.\n  return std::forward<T>(value);\n}\n\ntemplate <typename StatusType>\ninline bool EvalAssertOk(const StatusType& status) {\n  return true || status.ok();  // Check that this compiles.\n}\n\ntemplate <typename T>\ninline T&& AssumeNotNull(T&& value) {\n  if (value == nullptr) RIEGELI_INTERNAL_UNREACHABLE();\n  return std::forward<T>(value);\n}\n\n#ifdef _MSC_VER\n// Silence MSVC warning for destructor that does not return.\n#pragma warning(push)\n#pragma warning(disable : 4722)\n#endif\n\nclass UnreachableStream {\n public:\n  UnreachableStream() { RIEGELI_INTERNAL_UNREACHABLE(); }\n\n  ABSL_ATTRIBUTE_NORETURN ~UnreachableStream() {\n    RIEGELI_INTERNAL_UNREACHABLE();\n  }\n\n  template <typename T>\n  UnreachableStream& operator<<(ABSL_ATTRIBUTE_UNUSED T&& src) {\n    return *this;\n  }\n};\n\n#ifdef _MSC_VER\n#pragma warning(pop)\n#endif\n\n#endif  // !RIEGELI_DEBUG\n\n// Allow `MACRO()` expanding to `if (condition) ...; else ...`, to be usable as\n// `if (condition) MACRO();` without a warning about ambiguous `else`.\n// The definition of `MACRO()` must begin with `RIEGELI_INTERNAL_BLOCK_ELSE`.\n#define RIEGELI_INTERNAL_BLOCK_ELSE \\\n  switch (0)                        \\\n  case 0:                           \\\n  default:\n\n}  // namespace assert_internal\n\n// `RIEGELI_CHECK(expr)` checks that `expr` is `true`, terminating the program\n// if not.\n//\n// `RIEGELI_CHECK_{EQ,NE,LT,GT,LE,GE}(a, b)` check the relationship between `a`\n// and `b`, and include values of `a` and `b` in the failure message. The values\n// must be printable using `riegeli::Debug()`.\n//\n// `RIEGELI_CHECK_NOTNULL(expr)` checks that `expr` is not `nullptr` and returns\n// `expr`.\n//\n// `RIEGELI_CHECK_OK(expr)` checks that `expr.ok()`, and includes either\n// `expr.status()` or `expr` in the failure message. Supported types include\n// `absl::Status`, `absl::StatusOr<T>`, and classes deriving from `Object`.\n//\n// `RIEGELI_CHECK_NOTNULL(expr)` is an expression which evaluates to `expr`.\n// The remaining `RIEGELI_CHECK*` macros can be followed by streaming `<<`\n// operators in order to append more details to the failure message\n// (streamed expressions are evaluated only on assertion failure).\n//\n// `RIEGELI_CHECK_UNREACHABLE()` checks that this point is not reached.\n\n#if defined(__clang__) || RIEGELI_INTERNAL_IS_GCC_VERSION(2, 6)\n#define RIEGELI_INTERNAL_FUNCTION __PRETTY_FUNCTION__\n#elif defined(_WIN32)\n#define RIEGELI_INTERNAL_FUNCTION __FUNCSIG__\n#else\n#define RIEGELI_INTERNAL_FUNCTION __func__\n#endif\n\n#define RIEGELI_INTERNAL_CHECK_OP(name, op, a, b)                            \\\n  RIEGELI_INTERNAL_BLOCK_ELSE                                                \\\n  if (const ::riegeli::assert_internal::CheckResult riegeli_internal_check = \\\n          ::riegeli::assert_internal::Check##name(RIEGELI_INTERNAL_FUNCTION, \\\n                                                  #a \" \" #op \" \" #b, a, b))  \\\n    ;                                                                        \\\n  else                                                                       \\\n    ::riegeli::assert_internal::CheckFailed(__FILE__, __LINE__,              \\\n                                            riegeli_internal_check)          \\\n        .details()\n\n#define RIEGELI_CHECK(expr)                                                \\\n  RIEGELI_INTERNAL_BLOCK_ELSE                                              \\\n  if (ABSL_PREDICT_TRUE(expr))                                             \\\n    ;                                                                      \\\n  else                                                                     \\\n    ::riegeli::assert_internal::CheckFailed(                               \\\n        __FILE__, __LINE__,                                                \\\n        ::riegeli::assert_internal::CheckResult(RIEGELI_INTERNAL_FUNCTION, \\\n                                                #expr))                    \\\n        .details()\n#define RIEGELI_CHECK_EQ(a, b) RIEGELI_INTERNAL_CHECK_OP(Eq, ==, a, b)\n#define RIEGELI_CHECK_NE(a, b) RIEGELI_INTERNAL_CHECK_OP(Ne, !=, a, b)\n#define RIEGELI_CHECK_LT(a, b) RIEGELI_INTERNAL_CHECK_OP(Lt, <, a, b)\n#define RIEGELI_CHECK_GT(a, b) RIEGELI_INTERNAL_CHECK_OP(Gt, >, a, b)\n#define RIEGELI_CHECK_LE(a, b) RIEGELI_INTERNAL_CHECK_OP(Le, <=, a, b)\n#define RIEGELI_CHECK_GE(a, b) RIEGELI_INTERNAL_CHECK_OP(Ge, >=, a, b)\n#define RIEGELI_CHECK_OK(status)                                             \\\n  RIEGELI_INTERNAL_BLOCK_ELSE                                                \\\n  if (const ::riegeli::assert_internal::CheckResult riegeli_internal_check = \\\n          ::riegeli::assert_internal::CheckOk(RIEGELI_INTERNAL_FUNCTION,     \\\n                                              #status, status))              \\\n    ;                                                                        \\\n  else                                                                       \\\n    ::riegeli::assert_internal::CheckFailed(__FILE__, __LINE__,              \\\n                                            riegeli_internal_check)          \\\n        .details()\n#define RIEGELI_CHECK_NOTNULL(expr)         \\\n  ::riegeli::assert_internal::CheckNotNull( \\\n      __FILE__, __LINE__, RIEGELI_INTERNAL_FUNCTION, #expr, expr)\n#define RIEGELI_CHECK_UNREACHABLE()                      \\\n  ::riegeli::assert_internal::CheckFailed(               \\\n      __FILE__, __LINE__,                                \\\n      ::riegeli::assert_internal::CheckImpossibleResult( \\\n          RIEGELI_INTERNAL_FUNCTION))                    \\\n      .details()\n\n// If `RIEGELI_DEBUG` is `true`, `RIEGELI_ASSERT*` macros are equivalent to the\n// corresponding `RIEGELI_CHECK*` macros.\n//\n// If `RIEGELI_DEBUG` is `false`, they do nothing except for ensuring that the\n// assertion compiles, and that any code appending to the stream compiles.\n//\n// There is no `RIEGELI_ASSERT_NOTNULL` because the argument is returned, and\n// thus it is necessarily always evaluated also if `RIEGELI_DEBUG` is `false`\n// (the semantics of `RIEGELI_ASSERT*` of doing nothing if `RIEGELI_DEBUG` is\n// `false` cannot be followed). Use `RIEGELI_EVAL_ASSERT_NOTNULL` instead.\n//\n// There is no `RIEGELI_ASSERT_UNREACHABLE` because no following code is\n// expected, and thus this point is necessarily never reached also if\n// `RIEGELI_DEBUG` is `false` (the semantics of `RIEGELI_ASSERT*` of doing\n// nothing if `RIEGELI_DEBUG` is `false` cannot be followed). Use\n// `RIEGELI_ASSUME_UNREACHABLE` instead.\n\n#if RIEGELI_DEBUG\n\n#define RIEGELI_ASSERT RIEGELI_CHECK\n#define RIEGELI_ASSERT_EQ RIEGELI_CHECK_EQ\n#define RIEGELI_ASSERT_NE RIEGELI_CHECK_NE\n#define RIEGELI_ASSERT_LT RIEGELI_CHECK_LT\n#define RIEGELI_ASSERT_GT RIEGELI_CHECK_GT\n#define RIEGELI_ASSERT_LE RIEGELI_CHECK_LE\n#define RIEGELI_ASSERT_GE RIEGELI_CHECK_GE\n#define RIEGELI_ASSERT_OK RIEGELI_CHECK_OK\n\n#else  // !RIEGELI_DEBUG\n\n#define RIEGELI_ASSERT(expr)  \\\n  RIEGELI_INTERNAL_BLOCK_ELSE \\\n  if (true || (expr))         \\\n    ;                         \\\n  else                        \\\n    ::riegeli::assert_internal::UnreachableStream()\n\n#define RIEGELI_ASSERT_EQ(a, b) RIEGELI_ASSERT((a) == (b))\n#define RIEGELI_ASSERT_NE(a, b) RIEGELI_ASSERT((a) != (b))\n#define RIEGELI_ASSERT_LT(a, b) RIEGELI_ASSERT((a) < (b))\n#define RIEGELI_ASSERT_GT(a, b) RIEGELI_ASSERT((a) > (b))\n#define RIEGELI_ASSERT_LE(a, b) RIEGELI_ASSERT((a) <= (b))\n#define RIEGELI_ASSERT_GE(a, b) RIEGELI_ASSERT((a) >= (b))\n#define RIEGELI_ASSERT_OK(status) RIEGELI_ASSERT((status).ok())\n\n#endif  // !RIEGELI_DEBUG\n\n// If `RIEGELI_DEBUG` is `true`, `RIEGELI_EVAL_ASSERT*` macros are equivalent to\n// the corresponding `RIEGELI_CHECK*` macros.\n//\n// If `RIEGELI_DEBUG` is `false`, they evaluate the arguments, but do not check\n// the assertion, although they verify that evaluating the assertion and any\n// code appending to the stream compiles.\n//\n// There is no `RIEGELI_EVAL_ASSERT_UNREACHABLE` because there is no argument\n// to evaluate, and because no following code is expected, and thus this point\n// is necessarily never reached also if `RIEGELI_DEBUG` is `false` (the\n// semantics of `RIEGELI_EVAL_ASSERT*` of doing nothing besides evaluating the\n// arguments cannot be followed). Use `RIEGELI_ASSUME_UNREACHABLE` instead.\n\n#if RIEGELI_DEBUG\n\n#define RIEGELI_EVAL_ASSERT RIEGELI_CHECK\n#define RIEGELI_EVAL_ASSERT_EQ RIEGELI_CHECK_EQ\n#define RIEGELI_EVAL_ASSERT_NE RIEGELI_CHECK_NE\n#define RIEGELI_EVAL_ASSERT_LT RIEGELI_CHECK_LT\n#define RIEGELI_EVAL_ASSERT_GT RIEGELI_CHECK_GT\n#define RIEGELI_EVAL_ASSERT_LE RIEGELI_CHECK_LE\n#define RIEGELI_EVAL_ASSERT_GE RIEGELI_CHECK_GE\n#define RIEGELI_EVAL_ASSERT_OK RIEGELI_CHECK_OK\n#define RIEGELI_EVAL_ASSERT_NOTNULL RIEGELI_CHECK_NOTNULL\n\n#else  // !RIEGELI_DEBUG\n\n#define RIEGELI_INTERNAL_EVAL_ASSERT_OP(name, a, b)       \\\n  RIEGELI_INTERNAL_BLOCK_ELSE                             \\\n  if (::riegeli::assert_internal::EvalAssert##name(a, b)) \\\n    ;                                                     \\\n  else                                                    \\\n    ::riegeli::assert_internal::UnreachableStream()\n\n#define RIEGELI_EVAL_ASSERT(expr) \\\n  RIEGELI_INTERNAL_BLOCK_ELSE     \\\n  if ((expr) || true)             \\\n    ;                             \\\n  else                            \\\n    ::riegeli::assert_internal::UnreachableStream()\n#define RIEGELI_EVAL_ASSERT_EQ(a, b) RIEGELI_INTERNAL_EVAL_ASSERT_OP(Eq, a, b)\n#define RIEGELI_EVAL_ASSERT_NE(a, b) RIEGELI_INTERNAL_EVAL_ASSERT_OP(Ne, a, b)\n#define RIEGELI_EVAL_ASSERT_LT(a, b) RIEGELI_INTERNAL_EVAL_ASSERT_OP(Lt, a, b)\n#define RIEGELI_EVAL_ASSERT_GT(a, b) RIEGELI_INTERNAL_EVAL_ASSERT_OP(Gt, a, b)\n#define RIEGELI_EVAL_ASSERT_LE(a, b) RIEGELI_INTERNAL_EVAL_ASSERT_OP(Le, a, b)\n#define RIEGELI_EVAL_ASSERT_GE(a, b) RIEGELI_INTERNAL_EVAL_ASSERT_OP(Ge, a, b)\n#define RIEGELI_EVAL_ASSERT_NOTNULL(expr) \\\n  ::riegeli::assert_internal::EvalAssertNotNull(expr)\n#define RIEGELI_EVAL_ASSERT_OK(status)                  \\\n  RIEGELI_INTERNAL_BLOCK_ELSE                           \\\n  if (::riegeli::assert_internal::EvalAssertOk(status)) \\\n    ;                                                   \\\n  else                                                  \\\n    ::riegeli::assert_internal::UnreachableStream()\n\n#endif\n\n// If `RIEGELI_DEBUG` is `true`, `RIEGELI_ASSUME*` macros are equivalent to the\n// corresponding `RIEGELI_CHECK*` macros.\n//\n// If `RIEGELI_DEBUG` is `false`, the behavior is undefined if the assertion\n// fails, which allows the compiler to perform optimizations based on that.\n//\n// The condition is evaluated unconditionally, but this should not be relied\n// upon, as a future implementation might not ensure this. To make it optimized\n// out when `RIEGELI_DEBUG` is `false`, it should use only operations which are\n// expected to be optimized out when the result of the condition is not needed,\n// in particular it should not call non-inline functions.\n\n#if RIEGELI_DEBUG\n\n#define RIEGELI_ASSUME RIEGELI_CHECK\n#define RIEGELI_ASSUME_EQ RIEGELI_CHECK_EQ\n#define RIEGELI_ASSUME_NE RIEGELI_CHECK_NE\n#define RIEGELI_ASSUME_LT RIEGELI_CHECK_LT\n#define RIEGELI_ASSUME_GT RIEGELI_CHECK_GT\n#define RIEGELI_ASSUME_LE RIEGELI_CHECK_LE\n#define RIEGELI_ASSUME_GE RIEGELI_CHECK_GE\n#define RIEGELI_ASSUME_OK RIEGELI_CHECK_OK\n#define RIEGELI_ASSUME_NOTNULL RIEGELI_CHECK_NOTNULL\n#define RIEGELI_ASSUME_UNREACHABLE RIEGELI_CHECK_UNREACHABLE\n\n#else  // !RIEGELI_DEBUG\n\n#define RIEGELI_ASSUME(expr)  \\\n  RIEGELI_INTERNAL_BLOCK_ELSE \\\n  if (expr)                   \\\n    ;                         \\\n  else                        \\\n    RIEGELI_ASSUME_UNREACHABLE()\n\n#define RIEGELI_ASSUME_EQ(a, b) RIEGELI_ASSUME((a) == (b))\n#define RIEGELI_ASSUME_NE(a, b) RIEGELI_ASSUME((a) != (b))\n#define RIEGELI_ASSUME_LT(a, b) RIEGELI_ASSUME((a) < (b))\n#define RIEGELI_ASSUME_GT(a, b) RIEGELI_ASSUME((a) > (b))\n#define RIEGELI_ASSUME_LE(a, b) RIEGELI_ASSUME((a) <= (b))\n#define RIEGELI_ASSUME_GE(a, b) RIEGELI_ASSUME((a) >= (b))\n#define RIEGELI_ASSUME_OK(status) RIEGELI_ASSUME((status).ok())\n#define RIEGELI_ASSUME_NOTNULL(expr) \\\n  ::riegeli::assert_internal::AssumeNotNull(expr)\n#define RIEGELI_ASSUME_UNREACHABLE() \\\n  ::riegeli::assert_internal::UnreachableStream()\n\n#endif  // !RIEGELI_DEBUG\n\n// Asserts that a region of memory is initialized, which is checked when running\n// under memory sanitizer.\ninline void AssertInitialized(ABSL_ATTRIBUTE_UNUSED const char* data,\n                              ABSL_ATTRIBUTE_UNUSED size_t size) {\n#ifdef MEMORY_SANITIZER\n  __msan_check_mem_is_initialized(data, size);\n#endif\n}\n\n// Marks that a region of memory should be treated as uninitialized, which is\n// checked when running under memory sanitizer.\ninline void MarkPoisoned(ABSL_ATTRIBUTE_UNUSED const char* data,\n                         ABSL_ATTRIBUTE_UNUSED size_t size) {\n#ifdef MEMORY_SANITIZER\n  __msan_poison(data, size);\n#endif\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_ASSERT_H_\n"
  },
  {
    "path": "riegeli/base/background_cleaning.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/background_cleaning.h\"\n\n#include <list>\n\n#include \"absl/base/thread_annotations.h\"\n#include \"absl/synchronization/mutex.h\"\n#include \"absl/time/time.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/parallelism.h\"\n\nnamespace riegeli {\n\nBackgroundCleanee::~BackgroundCleanee() = default;  // Key method.\n\nBackgroundCleaner::Token BackgroundCleaner::Register(\n    BackgroundCleanee* cleanee) {\n  absl::MutexLock lock(mutex_);\n  entries_.emplace_front(cleanee, absl::InfiniteFuture());\n  return Token(entries_.begin());\n}\n\nvoid BackgroundCleaner::Unregister(Token token) {\n  absl::MutexLock lock(mutex_);\n  CancelCleaningInternal(token);\n  if (next_ == token.iter()) ++next_;\n  entries_.erase(token.iter());\n}\n\nvoid BackgroundCleaner::CancelCleaning(Token token) {\n  absl::MutexLock lock(mutex_);\n  CancelCleaningInternal(token);\n  if (token.iter()->deadline == absl::InfiniteFuture()) return;\n  // Move `token.iter()` before `next_`.\n  if (next_ == token.iter()) {\n    ++next_;\n  } else {\n    entries_.splice(next_, entries_, token.iter());\n  }\n  token.iter()->deadline = absl::InfiniteFuture();\n}\n\n// Waits until this cleanee is not being cleaned.\ninline void BackgroundCleaner::CancelCleaningInternal(Token token) {\n  struct Args {\n    BackgroundCleanee** current_cleanee;\n    BackgroundCleanee* cleanee_to_unregister;\n  };\n  Args args{&current_cleanee_, token.iter()->cleanee};\n  mutex_.Await(absl::Condition(\n      +[](Args* args) {\n        return *args->current_cleanee != args->cleanee_to_unregister;\n      },\n      &args));\n}\n\nvoid BackgroundCleaner::ScheduleCleaningSlow(Token token, absl::Time deadline) {\n  absl::MutexLock lock(mutex_);\n  if (token.iter()->deadline <= deadline) {\n    // Cleaning is already scheduled with the same or earlier deadline.\n    return;\n  }\n\n  // Move `token.iter()` to the right place after `next_`.\n  Entries::iterator iter =\n      token.iter()->deadline == absl::InfiniteFuture()\n          ? entries_.end()  // Schedule new cleaning: move from before `next_`.\n          : token.iter();   // Reduce deadline: move backwards.\n  for (;;) {\n    if (iter == next_) {\n      // Insert `token.iter()` before `iter` which is `next_`.\n      next_ = token.iter();\n      deadline_reduced_ = true;\n      break;\n    }\n    const Entries::iterator last_iter = iter;\n    --iter;\n    if (iter->deadline <= deadline) {\n      // Insert `token.iter()` after `iter`, i.e. before `last_iter`.\n      // This might be its old place, then `splice()` does nothing.\n      iter = last_iter;\n      break;\n    }\n  }\n  entries_.splice(iter, entries_, token.iter());\n  RIEGELI_ASSERT(next_ != entries_.end())\n      << \"next_ must cover at least token.iter()\";\n  token.iter()->deadline = deadline;\n\n  // Start a background thread if needed.\n  if (!no_background_thread_) return;\n  no_background_thread_ = false;\n  internal::ThreadPool::global().Schedule([this] {\n    absl::MutexLock lock(mutex_);\n    BackgroundThread();\n    no_background_thread_ = true;\n  });\n}\n\ninline void BackgroundCleaner::BackgroundThread()\n    ABSL_EXCLUSIVE_LOCKS_REQUIRED(mutex_) {\n  if (next_ == entries_.end()) return;\n  for (;;) {\n    // Wait until the next deadline.\n    do {\n      deadline_reduced_ = false;\n      mutex_.AwaitWithDeadline(absl::Condition(&deadline_reduced_),\n                               next_->deadline);\n      if (next_ == entries_.end()) return;\n    } while (deadline_reduced_);\n    // Schedule cleaning.\n    for (;;) {\n      const absl::Time now = TimeNow();\n      if (next_->deadline > now) break;\n      BackgroundCleanee* const cleanee = next_->cleanee;\n      next_->deadline = absl::InfiniteFuture();\n      ++next_;\n      current_cleanee_ = cleanee;\n      mutex_.unlock();\n      cleanee->Clean(now);\n      mutex_.lock();\n      current_cleanee_ = nullptr;\n      if (next_ == entries_.end()) return;\n    }\n  }\n}\n\nBackgroundCleaner::~BackgroundCleaner() {\n  RIEGELI_CHECK(entries_.empty())\n      << \"Failed precondition of BackgroundCleaner::~BackgroundCleaner(): \"\n         \"some cleanees remain registered\";\n  absl::MutexLock lock(mutex_);\n  // Request the background thread to exit.\n  deadline_reduced_ = true;\n  mutex_.Await(absl::Condition(&no_background_thread_));\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/background_cleaning.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_BACKGROUND_CLEANING_H_\n#define RIEGELI_BASE_BACKGROUND_CLEANING_H_\n\n#include <list>\n\n#include \"absl/base/thread_annotations.h\"\n#include \"absl/synchronization/mutex.h\"\n#include \"absl/time/clock.h\"\n#include \"absl/time/time.h\"\n#include \"riegeli/base/global.h\"\n\nnamespace riegeli {\n\n// An interface of objects which need background cleaning.\nclass BackgroundCleanee {\n public:\n  virtual ~BackgroundCleanee();\n\n protected:\n  friend class BackgroundCleaner;  // For `Clean()`.\n\n  // Called from a background thread when a scheduled cleaning time arrived.\n  //\n  // `now` is the current time, passed as a parameter so that there is no need\n  // to call `BackgroundCleaner::TimeNow()` again if that time is needed to\n  // decide what to clean.\n  virtual void Clean(absl::Time now) = 0;\n};\n\n// Manages objects which need background cleaning, scheduling cleaning calls\n// from a background thread.\nclass BackgroundCleaner {\n private:\n  struct Entry {\n    explicit Entry(BackgroundCleanee* cleanee, absl::Time deadline)\n        : cleanee(cleanee), deadline(deadline) {}\n\n    BackgroundCleanee* cleanee;\n    absl::Time deadline;\n  };\n  using Entries = std::list<Entry>;\n\n public:\n  // Registration token of an object which needs background cleaning.\n  class Token {\n   public:\n    Token() = default;\n\n    Token(const Token& that) = default;\n    Token& operator=(const Token& that) = default;\n\n   private:\n    friend class BackgroundCleaner;  // For `Token()` and `iter()`.\n\n    explicit Token(Entries::iterator iter) : iter_(iter) {}\n\n    Entries::iterator iter() const { return iter_; }\n\n    Entries::iterator iter_{};\n  };\n\n  BackgroundCleaner() = default;\n\n  BackgroundCleaner(const BackgroundCleaner&) = delete;\n  BackgroundCleaner& operator=(const BackgroundCleaner&) = delete;\n\n  // Precondition: all registered cleanees have been unregistered.\n  ~BackgroundCleaner();\n\n  // Returns a default global `BackgroundCleaner`.\n  static BackgroundCleaner& global() {\n    return Global([] { return BackgroundCleaner(); });\n  }\n\n  // Registers the cleanee, allowing `ScheduleCleaning()` calls.\n  //\n  // Thread safe.\n  Token Register(BackgroundCleanee* cleanee);\n\n  // Unregisters the cleanee corresponding to `token`, invalidating `token` and\n  // cancelling any pending cleaning.\n  //\n  // This might block if the cleanee is being cleaned or will be cleaned soon,\n  // so this must not be called under a mutex needed for cleaning.\n  //\n  // Thread safe.\n  void Unregister(Token token);\n\n  // Cancels any pending cleaning corresponding to `token`. Does not unregister\n  // the cleanee.\n  //\n  // This might block if the cleanee is being cleaned or will be cleaned soon,\n  // so this must not be called under a mutex needed for cleaning.\n  //\n  // Thread safe.\n  void CancelCleaning(Token token);\n\n  // Schedules cleaning the cleanee corresponding to `token` at `deadline`.\n  //\n  // If `deadline` is `absl::InfiniteFuture()`, cleaning will never happen.\n  // If `deadline` is in the past, cleaning will be scheduled immediately.\n  //\n  // If `ScheduleCleaning()` is called again for the same cleanee with a pending\n  // cleaning, its deadline can be reduced, but extending the deadline has no\n  // effect.\n  //\n  // Thread safe.\n  void ScheduleCleaning(Token token, absl::Time deadline);\n\n  // Returns the current time according to the same clock that\n  // `BackgroundCleaner` is using.\n  //\n  // Thread safe.\n  absl::Time TimeNow();\n\n private:\n  void CancelCleaningInternal(Token token)\n      ABSL_EXCLUSIVE_LOCKS_REQUIRED(mutex_);\n\n  void ScheduleCleaningSlow(Token token, absl::Time deadline);\n\n  void BackgroundThread();\n\n  absl::Mutex mutex_;\n  // Registered cleanees, partitioned so that `entries_` before `next_` do not\n  // have pending cleaning and have `deadline == absl::InfiniteFuture()`, while\n  // `entries_` at and after `next_` have pending cleaning and are sorted by\n  // their `deadline` which is never `absl::InfiniteFuture()`.\n  Entries entries_ ABSL_GUARDED_BY(mutex_);\n  Entries::iterator next_ ABSL_GUARDED_BY(mutex_) = entries_.begin();\n  // If not `nullptr`, this cleanee is currently being cleaned. This is used to\n  // avoid a race between `Unregister()` and cleaning.\n  BackgroundCleanee* current_cleanee_ ABSL_GUARDED_BY(mutex_) = nullptr;\n  // If `true`, the next deadline might have been reduced since the background\n  // thread started waiting for it. This wakes up the thread and lets it recheck\n  // the next deadline.\n  //\n  // This is also used to request the thread to exit when `next_ == next_end()`.\n  bool deadline_reduced_ ABSL_GUARDED_BY(mutex_) = false;\n  // If `false`, the background thread is active. This is negated for easier\n  // `absl::Condition()`.\n  bool no_background_thread_ ABSL_GUARDED_BY(mutex_) = true;\n};\n\n// Implementation details follow.\n\ninline void BackgroundCleaner::ScheduleCleaning(Token token,\n                                                absl::Time deadline) {\n  if (deadline == absl::InfiniteFuture()) return;\n  ScheduleCleaningSlow(token, deadline);\n}\n\ninline absl::Time BackgroundCleaner::TimeNow() { return absl::Now(); }\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_BACKGROUND_CLEANING_H_\n"
  },
  {
    "path": "riegeli/base/binary_search.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_BINARY_SEARCH_H_\n#define RIEGELI_BASE_BINARY_SEARCH_H_\n\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/status/statusor.h\"\n#include \"riegeli/base/compare.h\"\n\nnamespace riegeli {\n\n// Explains the result of a binary search.\n//\n// Assumptions:\n//  * All `less` positions precede all `equivalent` positions.\n//  * All `equivalent` positions precede all `greater` positions.\n//  * All `less` positions precede all `greater` positions,\n//    even if there are no `equivalent` positions.\n//\n// Interpretation of the result of a binary search, depending on `ordering`:\n//  * `equivalent` - There is some `equivalent` position,\n//                   and `found` is some such position.\n//  * `greater`    - There are no `equivalent` positions\n//                   but there is some `greater` position,\n//                   and `found` is the earliest such position.\n//  * `less`       - There are no `equivalent` nor `greater` positions\n//                   but there is some `less` position,\n//                   and `found` is the end of the range to search.\n//  * `unordered`  - All positions are `unordered`,\n//                   and `found` is the end of the range to search.\ntemplate <typename Pos>\nstruct SearchResult {\n  PartialOrdering ordering;\n  Pos found;\n};\n\n// The `test()` parameter of `BinarySearch()` is a function which returns\n// an ordering (a value comparable with literal 0, such as\n// `{Partial,Strong}Ordering`, `{std,absl}::{partial,weak,strong}_ordering`,\n// or `int`) or `SearchGuide<Traits::Pos>`.\n//\n// If the earliest interesting position after `current` can be found\n// independently from `test(current)`, `test(current)` can return an ordering.\n// The next position will be `traits.Next(current)`.\n//\n// If the earliest interesting position after `current` can be more easily found\n// as a side effect of `test(current)`, `test(current)` can return\n// `SearchGuide<Pos>`. If `ordering >= 0` (i.e. `ordering` is `equivalent` or\n// `greater`), the associated `next` should be `current` (or another position\n// to replace `current` with). Otherwise (i.e. `ordering` is `less` or\n// `unordered`), the associated `next` should be the earliest interesting\n// position after `current`.\ntemplate <typename Pos>\nstruct SearchGuide {\n  template <typename Ordering,\n            std::enable_if_t<IsOrdering<Ordering>::value, int> = 0>\n  explicit SearchGuide(Ordering ordering, const Pos& next)\n      : ordering(AsPartialOrdering(ordering)), next(next) {}\n  template <typename Ordering,\n            std::enable_if_t<IsOrdering<Ordering>::value, int> = 0>\n  explicit SearchGuide(Ordering ordering, Pos&& next)\n      : ordering(AsPartialOrdering(ordering)), next(std::move(next)) {}\n\n  SearchGuide(const SearchGuide& that) = default;\n  SearchGuide& operator=(const SearchGuide& that) = default;\n\n  SearchGuide(SearchGuide&& that) = default;\n  SearchGuide& operator=(SearchGuide&& that) = default;\n\n  template <typename OtherPos,\n            std::enable_if_t<std::is_convertible_v<OtherPos, Pos>, int> = 0>\n  /*implicit*/ SearchGuide(const SearchGuide<OtherPos>& that)\n      : ordering(that.ordering), next(that.next) {}\n  template <typename OtherPos,\n            std::enable_if_t<std::is_convertible_v<OtherPos, Pos>, int> = 0>\n  SearchGuide& operator=(const SearchGuide<OtherPos>& that) {\n    ordering = that.ordering;\n    next = that.next;\n    return *this;\n  }\n\n  template <typename OtherPos,\n            std::enable_if_t<std::is_convertible_v<OtherPos, Pos>, int> = 0>\n  /*implicit*/ SearchGuide(SearchGuide<OtherPos>&& that)\n      : ordering(that.ordering), next(std::move(that.next)) {}\n  template <typename OtherPos,\n            std::enable_if_t<std::is_convertible_v<OtherPos, Pos>, int> = 0>\n  SearchGuide& operator=(SearchGuide<OtherPos>&& that) {\n    ordering = that.ordering;\n    next = std::move(that.next);\n    return *this;\n  }\n\n  PartialOrdering ordering;\n  Pos next;\n};\n\ntemplate <typename Pos, typename Ordering>\nexplicit SearchGuide(Ordering ordering, Pos next)\n    -> SearchGuide<std::decay_t<Pos>>;\n\nnamespace binary_search_internal {\n\ntemplate <typename T, typename Pos, typename Enable = void>\nstruct IsSearchGuide : std::false_type {};\n\ntemplate <typename Pos, typename OtherPos>\nstruct IsSearchGuide<SearchGuide<OtherPos>, Pos>\n    : std::is_convertible<OtherPos, Pos> {};\n\ntemplate <typename T, typename Pos>\nstruct IsOrderingOrSearchGuide\n    : std::disjunction<IsOrdering<T>, IsSearchGuide<T, Pos>> {};\n\ntemplate <typename T, typename Pos>\nstruct IsOptionalOrderingOrSearchGuide : std::false_type {};\n\ntemplate <typename T, typename Pos>\nstruct IsOptionalOrderingOrSearchGuide<std::optional<T>, Pos>\n    : IsOrderingOrSearchGuide<T, Pos> {};\n\ntemplate <typename T, typename Pos>\nstruct IsStatusOrOrderingOrSearchGuide : std::false_type {};\n\ntemplate <typename T, typename Pos>\nstruct IsStatusOrOrderingOrSearchGuide<absl::StatusOr<T>, Pos>\n    : IsOrderingOrSearchGuide<T, Pos> {};\n\ntemplate <typename Test, typename Pos, typename Enable = void>\nstruct TestReturnsOrderingOrSearchGuide : std::false_type {};\n\ntemplate <typename Test, typename Pos>\nstruct TestReturnsOrderingOrSearchGuide<\n    Test, Pos,\n    std::enable_if_t<IsOrderingOrSearchGuide<\n        decltype(std::declval<Test>()(std::declval<Pos>())), Pos>::value>>\n    : std::true_type {};\n\ntemplate <typename Test, typename Pos, typename Enable = void>\nstruct TestReturnsOptionalOrderingOrSearchGuide : std::false_type {};\n\ntemplate <typename Test, typename Pos>\nstruct TestReturnsOptionalOrderingOrSearchGuide<\n    Test, Pos,\n    std::enable_if_t<IsOptionalOrderingOrSearchGuide<\n        decltype(std::declval<Test>()(std::declval<Pos>())), Pos>::value>>\n    : std::true_type {};\n\ntemplate <typename Test, typename Pos, typename Enable = void>\nstruct TestReturnsStatusOrOrderingOrSearchGuide : std::false_type {};\n\ntemplate <typename Test, typename Pos>\nstruct TestReturnsStatusOrOrderingOrSearchGuide<\n    Test, Pos,\n    std::enable_if_t<IsStatusOrOrderingOrSearchGuide<\n        decltype(std::declval<Test>()(std::declval<Pos>())), Pos>::value>>\n    : std::true_type {};\n\n}  // namespace binary_search_internal\n\n// Searches a sequence of elements for a desired element, or for a desired\n// position between elements, given that it is possible to determine whether a\n// given position is before or after the desired position.\n//\n// The `traits` parameter specifies the space of possible positions.\n// See `DefaultSearchTraits` documentation for details. The default `traits` are\n// `DefaultSearchTraits<Pos>()`.\n//\n// The `low` (inclusive) and `high` (exclusive) parameters specify the range to\n// search.\n//\n// The `test()` function takes `current` of type `Traits::Pos` as a parameter\n// and returns an ordering:\n//  * `less`       - `current` is before the desired position.\n//  * `equivalent` - `current` is desired, searching can stop.\n//  * `greater`    - `current` is after the desired position.\n//  * `unordered`  - It could not be determined which is the case. `current`\n//                   will be skipped.\n//\n// Alternatively, `test()` can return `SearchGuide<Traits::Pos>`. See\n// `SearchGuide` documentation for details.\n//\n// Preconditions:\n//  * All `less` positions precede all `equivalent` positions.\n//  * All `equivalent` positions precede all `greater` positions.\n//  * All `less` positions precede all `greater` positions,\n//    even if there are no `equivalent` positions.\n//\n// For interpretation of the result, see `SearchResult` documentation.\n//\n// To find the earliest `equivalent` position instead of an arbitrary one,\n// `test()` can be changed to return `greater` in place of `equivalent`.\n//\n// Further guarantees:\n//  * Each `traits.Next(current)` immediately follows a `test(current)` which\n//    returned `less` or `unordered`.\n//  * Each `test(current)` immediately follows a `traits.Next()` which returned\n//    `current`, or a `test()` which returned a `SearchGuide` containing `less`\n//    or `unordered` together with `current`, or a `traits.Middle()` which\n//    returned `current`.\n//  * If `test(current)` returns `equivalent`, `BinarySearch()` immediately\n//    returns `current`.\n//  * If `test(current)` returns `less`, `test()` will not be called again\n//    with arguments before `current`.\n//  * If `test(current)` returns `greater`, `test()` will not be called again\n//    with arguments after `current`.\n//  * `test()` will not be called again with the same argument.\n//\n// It follows that if a `test()` returns `equivalent` or `greater`,\n// `BinarySearch()` returns the argument of the last `test()` call with one of\n// these results. This allows to communicate additional context of an\n// `equivalent` or `greater` result by a side effect of `test()`.\ntemplate <\n    typename Pos, typename Test,\n    std::enable_if_t<binary_search_internal::TestReturnsOrderingOrSearchGuide<\n                         Test, Pos>::value,\n                     int> = 0>\nSearchResult<Pos> BinarySearch(Pos low, Pos high, Test&& test);\ntemplate <\n    typename Traits, typename Test,\n    std::enable_if_t<binary_search_internal::TestReturnsOrderingOrSearchGuide<\n                         Test, typename Traits::Pos>::value,\n                     int> = 0>\nSearchResult<typename Traits::Pos> BinarySearch(typename Traits::Pos low,\n                                                typename Traits::Pos high,\n                                                Test&& test,\n                                                const Traits& traits);\n\n// A variant of `BinarySearch()` which supports cancellation.\n//\n// If a `test()` returns `std::nullopt`, `BinarySearch()` returns\n// `std::nullopt`.\ntemplate <typename Pos, typename Test,\n          std::enable_if_t<\n              binary_search_internal::TestReturnsOptionalOrderingOrSearchGuide<\n                  Test, Pos>::value,\n              int> = 0>\nstd::optional<SearchResult<Pos>> BinarySearch(Pos low, Pos high, Test&& test);\ntemplate <typename Traits, typename Test,\n          std::enable_if_t<\n              binary_search_internal::TestReturnsOptionalOrderingOrSearchGuide<\n                  Test, typename Traits::Pos>::value,\n              int> = 0>\nstd::optional<SearchResult<typename Traits::Pos>> BinarySearch(\n    typename Traits::Pos low, typename Traits::Pos high, Test&& test,\n    const Traits& traits);\n\n// A variant of `BinarySearch()` which supports cancellation with a `Status`.\n//\n// If a `test()` returns a failed `absl::StatusOr`, `BinarySearch()` returns\n// the corresponding failed `absl::StatusOr`.\ntemplate <typename Pos, typename Test,\n          std::enable_if_t<\n              binary_search_internal::TestReturnsStatusOrOrderingOrSearchGuide<\n                  Test, Pos>::value,\n              int> = 0>\nabsl::StatusOr<SearchResult<Pos>> BinarySearch(Pos low, Pos high, Test&& test);\ntemplate <typename Traits, typename Test,\n          std::enable_if_t<\n              binary_search_internal::TestReturnsStatusOrOrderingOrSearchGuide<\n                  Test, typename Traits::Pos>::value,\n              int> = 0>\nabsl::StatusOr<SearchResult<typename Traits::Pos>> BinarySearch(\n    typename Traits::Pos low, typename Traits::Pos high, Test&& test,\n    const Traits& traits);\n\n// The `traits` parameter of `BinarySearch()` specifies the space of positions\n// to search.\n//\n// Some positions might be determined to be uninteresting, which means that for\n// the purposes of the search they are equivalent to a nearby interesting\n// position. They are skipped during the search.\n//\n// `DefaultSearchTraits<Pos>` might be appropriate for positions of an\n// arithmetic type. If custom traits are needed instead, these comments specify\n// generalized requirements of the traits.\ntemplate <typename T>\nclass DefaultSearchTraits {\n public:\n  // Identifies a position between elements being searched. This type must be\n  // copyable.\n  using Pos = T;\n\n  // Returns the earliest interesting position after `current`.\n  //\n  // `Next()` is used only if the `test()` parameter of `BinarySearch()` returns\n  // an ordering. If `test()` returns `SearchGuide<Pos>`, the result of `test()`\n  // provides the next position instead.\n  //\n  // Precondition: `test(current)` returned `less` or `unordered`.\n  T Next(T current) const { return current + 1; }\n\n  // Returns `true` if the range between `low` and `high` contains no positions.\n  bool Empty(T low, T high) const { return low >= high; }\n\n  // Returns a position in the range from `low` (inclusive) to `high`\n  // (exclusive) which is approximately halfway between `low` and `high`.\n  // Returns `std::nullopt` if the range contains no interesting positions.\n  std::optional<T> Middle(T low, T high) const {\n    if (low >= high) return std::nullopt;\n    return low + (high - low) / 2;\n  }\n};\n\n// Implementation details follow.\n\nnamespace binary_search_internal {\n\ntemplate <typename Traits, typename Ordering,\n          std::enable_if_t<IsOrdering<Ordering>::value, int> = 0>\ninline SearchGuide<typename Traits::Pos> GetSearchGuide(\n    Ordering ordering, typename Traits::Pos&& pos, const Traits& traits) {\n  return SearchGuide<typename Traits::Pos>(\n      AsPartialOrdering(ordering),\n      ordering >= 0 ? std::move(pos) : traits.Next(std::move(pos)));\n}\n\ntemplate <typename Traits, typename OtherPos>\ninline SearchGuide<typename Traits::Pos> GetSearchGuide(\n    SearchGuide<OtherPos>&& guide,\n    ABSL_ATTRIBUTE_UNUSED typename Traits::Pos&& pos,\n    ABSL_ATTRIBUTE_UNUSED const Traits& traits) {\n  return std::move(guide);\n}\n\ntemplate <typename Pos, typename TestResult, typename Enable = void>\nstruct CancelSearch;\n\ntemplate <typename Pos, typename Ordering>\nstruct CancelSearch<Pos, Ordering,\n                    std::enable_if_t<IsOrdering<Ordering>::value>> {\n  static PartialOrdering DoCancel(ABSL_ATTRIBUTE_UNUSED const Pos& pos) {\n    return PartialOrdering::equivalent;\n  }\n  static PartialOrdering DoNotCancel(Ordering ordering) {\n    return AsPartialOrdering(ordering);\n  }\n};\n\ntemplate <typename Pos, typename OtherPos>\nstruct CancelSearch<Pos, SearchGuide<OtherPos>> {\n  static SearchGuide<Pos> DoCancel(const Pos& pos) {\n    return SearchGuide<Pos>(PartialOrdering::equivalent, pos);\n  }\n  static SearchGuide<Pos> DoNotCancel(SearchGuide<OtherPos>&& guide) {\n    return std::move(guide);\n  }\n};\n\n}  // namespace binary_search_internal\n\ntemplate <\n    typename Pos, typename Test,\n    std::enable_if_t<binary_search_internal::TestReturnsOrderingOrSearchGuide<\n                         Test, Pos>::value,\n                     int>>\ninline SearchResult<Pos> BinarySearch(Pos low, Pos high, Test&& test) {\n  return BinarySearch(std::move(low), std::move(high), std::forward<Test>(test),\n                      DefaultSearchTraits<Pos>());\n}\n\ntemplate <\n    typename Traits, typename Test,\n    std::enable_if_t<binary_search_internal::TestReturnsOrderingOrSearchGuide<\n                         Test, typename Traits::Pos>::value,\n                     int>>\ninline SearchResult<typename Traits::Pos> BinarySearch(\n    typename Traits::Pos low, typename Traits::Pos high, Test&& test,\n    const Traits& traits) {\n  // Invariants:\n  //  * All positions between the original `low` and the current `low` are\n  //    `less` or `unordered`.\n  //  * All positions between the current `high` and the original `high` are\n  //    `greater` or `unordered`.\n  //\n  // Invariants depending on `greater_result.ordering`:\n  //  * `greater`   - `greater_result.found` is the first `greater` position\n  //                  between the current `high` and the original `high`.\n  //  * `less`      - There are no such positions but there are `less` positions\n  //                  between the original `low` and the current `low`,\n  //                  and `greater_result.found` is `high`.\n  //  * `unordered` - There are no such positions either,\n  //                  and `greater_result.found` is `high`.\n  using Pos = typename Traits::Pos;\n  SearchResult<Pos> greater_result = {PartialOrdering::unordered, high};\n\nagain:\n  std::optional<Pos> middle_before_unordered = traits.Middle(low, high);\n  if (middle_before_unordered == std::nullopt) return greater_result;\n  Pos middle = *middle_before_unordered;\n  // Invariant: all positions between `*middle_before_unordered` and `middle`\n  // are `unordered`.\n  bool unordered_found = false;\n  for (;;) {\n    auto test_result = test(middle);\n    SearchGuide<Pos> guide = binary_search_internal::GetSearchGuide(\n        std::move(test_result), std::move(middle), traits);\n    if (guide.ordering < 0) {\n      if (!(greater_result.ordering >= 0)) {\n        greater_result.ordering = PartialOrdering::less;\n      }\n      low = std::move(guide.next);\n      goto again;\n    }\n    if (guide.ordering == 0) {\n      // Assign instead of returning for NRVO.\n      greater_result.ordering = PartialOrdering::equivalent;\n      greater_result.found = std::move(guide.next);\n      return greater_result;\n    }\n    if (guide.ordering > 0) {\n      greater_result.ordering = PartialOrdering::greater;\n      greater_result.found = std::move(guide.next);\n      if (unordered_found) break;\n      // Use the position from `guide` instead of `*middle_before_unordered`\n      // in case the guide provides an earlier upper bound.\n      high = greater_result.found;\n      goto again;\n    }\n    unordered_found = true;\n    if (traits.Empty(guide.next, high)) break;\n    middle = std::move(guide.next);\n  }\n  // Either a `greater` position was found after some `unordered` positions,\n  // or all positions between `*middle_before_unordered` and `high` are\n  // `unordered`.\n  high = *std::move(middle_before_unordered);\n  goto again;\n}\n\ntemplate <typename Pos, typename Test,\n          std::enable_if_t<\n              binary_search_internal::TestReturnsOptionalOrderingOrSearchGuide<\n                  Test, Pos>::value,\n              int>>\ninline std::optional<SearchResult<Pos>> BinarySearch(Pos low, Pos high,\n                                                     Test&& test) {\n  return BinarySearch(std::move(low), std::move(high), std::forward<Test>(test),\n                      DefaultSearchTraits<Pos>());\n}\n\ntemplate <typename Traits, typename Test,\n          std::enable_if_t<\n              binary_search_internal::TestReturnsOptionalOrderingOrSearchGuide<\n                  Test, typename Traits::Pos>::value,\n              int>>\ninline std::optional<SearchResult<typename Traits::Pos>> BinarySearch(\n    typename Traits::Pos low, typename Traits::Pos high, Test&& test,\n    const Traits& traits) {\n  bool cancelled = false;\n  SearchResult<typename Traits::Pos> result = BinarySearch(\n      std::move(low), std::move(high),\n      [&](const typename Traits::Pos& pos) {\n        auto test_result = test(pos);\n        using Cancel = binary_search_internal::CancelSearch<\n            typename Traits::Pos, std::decay_t<decltype(*test_result)>>;\n        if (ABSL_PREDICT_FALSE(test_result == std::nullopt)) {\n          cancelled = true;\n          return Cancel::DoCancel(pos);\n        }\n        return Cancel::DoNotCancel(*std::move(test_result));\n      },\n      traits);\n  if (ABSL_PREDICT_FALSE(cancelled)) return std::nullopt;\n  return result;\n}\n\ntemplate <typename Pos, typename Test,\n          std::enable_if_t<\n              binary_search_internal::TestReturnsStatusOrOrderingOrSearchGuide<\n                  Test, Pos>::value,\n              int>>\ninline absl::StatusOr<SearchResult<Pos>> BinarySearch(Pos low, Pos high,\n                                                      Test&& test) {\n  return BinarySearch(std::move(low), std::move(high), std::forward<Test>(test),\n                      DefaultSearchTraits<Pos>());\n}\n\ntemplate <typename Traits, typename Test,\n          std::enable_if_t<\n              binary_search_internal::TestReturnsStatusOrOrderingOrSearchGuide<\n                  Test, typename Traits::Pos>::value,\n              int>>\ninline absl::StatusOr<SearchResult<typename Traits::Pos>> BinarySearch(\n    typename Traits::Pos low, typename Traits::Pos high, Test&& test,\n    const Traits& traits) {\n  absl::Status status;\n  SearchResult<typename Traits::Pos> result = BinarySearch(\n      std::move(low), std::move(high),\n      [&](const typename Traits::Pos& pos) {\n        auto test_result = test(pos);\n        using Cancel = binary_search_internal::CancelSearch<\n            typename Traits::Pos, std::decay_t<decltype(*test_result)>>;\n        if (ABSL_PREDICT_FALSE(!test_result.ok())) {\n          status = test_result.status();\n          return Cancel::DoCancel(pos);\n        }\n        return Cancel::DoNotCancel(*std::move(test_result));\n      },\n      traits);\n  if (ABSL_PREDICT_FALSE(!status.ok())) return status;\n  return result;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_BINARY_SEARCH_H_\n"
  },
  {
    "path": "riegeli/base/buffer.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/buffer.h\"\n\n#include <ostream>\n\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n\nnamespace riegeli {\n\nvoid Buffer::DumpStructure(absl::string_view substr, std::ostream& dest) const {\n  dest << \"[buffer] {\";\n  if (!substr.empty()) {\n    if (substr.data() != data()) {\n      dest << \" space_before: \" << PtrDistance(data(), substr.data());\n    }\n    dest << \" space_after: \"\n         << PtrDistance(substr.data() + substr.size(), data() + capacity());\n  }\n  dest << \" }\";\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/buffer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_BUFFER_H_\n#define RIEGELI_BASE_BUFFER_H_\n\n#include <stddef.h>\n\n#include <iosfwd>\n#include <new>  // IWYU pragma: keep\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/estimated_allocated_size.h\"\n#include \"riegeli/base/external_data.h\"\n\nnamespace riegeli {\n\n// Dynamically allocated byte buffer.\nclass ABSL_ATTRIBUTE_TRIVIAL_ABI Buffer {\n public:\n  Buffer() = default;\n\n  // Ensures at least `min_capacity` of space.\n  explicit Buffer(size_t min_capacity);\n\n  // The source `Buffer` is left deallocated.\n  Buffer(Buffer&& that) noexcept;\n  Buffer& operator=(Buffer&& that) noexcept;\n\n  ~Buffer() { DeleteInternal(); }\n\n  // Ensures at least `min_capacity` of space. Existing contents are lost.\n  //\n  // Drops the allocation if the resulting capacity would be wasteful for\n  // `min_capacity`.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(size_t min_capacity = 0);\n\n  // Returns the data pointer.\n  char* data() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return data_; }\n\n  // Returns the usable data size. It can be greater than the requested size.\n  size_t capacity() const { return capacity_; }\n\n  // Indicates support for `ExternalRef(Buffer&&, substr)`.\n  friend void RiegeliSupportsExternalRef(Buffer*) {}\n\n  // Supports `ExternalRef`.\n  friend ExternalStorage RiegeliToExternalStorage(Buffer* self) {\n    self->capacity_ = 0;\n    return ExternalStorage(std::exchange(self->data_, nullptr),\n                           [](void* ptr) { operator delete(ptr); });\n  }\n\n  // Supports `ExternalRef` and `Chain::Block`.\n  friend void RiegeliDumpStructure(const Buffer* self, absl::string_view substr,\n                                   std::ostream& dest) {\n    self->DumpStructure(substr, dest);\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const Buffer* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterDynamicMemory(self->data_, self->capacity_);\n  }\n\n private:\n  void AllocateInternal(size_t min_capacity);\n  void DeleteInternal() { operator delete(data_, capacity_); }\n  void DumpStructure(absl::string_view substr, std::ostream& dest) const;\n\n  char* data_ = nullptr;\n  size_t capacity_ = 0;\n  // Invariant: if `data_ == nullptr` then `capacity_ == 0`\n};\n\n// Implementation details follow.\n\ninline Buffer::Buffer(size_t min_capacity) { AllocateInternal(min_capacity); }\n\ninline Buffer::Buffer(Buffer&& that) noexcept\n    : data_(std::exchange(that.data_, nullptr)),\n      capacity_(std::exchange(that.capacity_, 0)) {}\n\ninline Buffer& Buffer::operator=(Buffer&& that) noexcept {\n  // Exchange `that.data_` early to support self-assignment.\n  char* const data = std::exchange(that.data_, nullptr);\n  DeleteInternal();\n  data_ = data;\n  capacity_ = std::exchange(that.capacity_, 0);\n  return *this;\n}\n\ninline void Buffer::Reset(size_t min_capacity) {\n  if (data_ != nullptr) {\n    if (capacity_ >= min_capacity && !Wasteful(capacity_, min_capacity)) return;\n    DeleteInternal();\n    data_ = nullptr;\n    capacity_ = 0;\n  }\n  AllocateInternal(min_capacity);\n}\n\ninline void Buffer::AllocateInternal(size_t min_capacity) {\n  if (min_capacity > 0) {\n    const size_t capacity = EstimatedAllocatedSize(min_capacity);\n    data_ = static_cast<char*>(operator new(capacity));\n    capacity_ = capacity;\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_BUFFER_H_\n"
  },
  {
    "path": "riegeli/base/buffering.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_BUFFERING_H_\n#define RIEGELI_BASE_BUFFERING_H_\n\n#include <stddef.h>\n\n#include <optional>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/types.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// Typical bounds of sizes of memory blocks holding pieces of data in objects.\ninline constexpr size_t kDefaultMinBlockSize = 512;\ninline constexpr size_t kDefaultMaxBlockSize = size_t{64} << 10;\n\n// When deciding whether to copy an array of bytes or share memory, prefer\n// copying up to this length.\n//\n// Copying can often be done in an inlined fast path. Sharing has more overhead,\n// especially in a virtual slow path, so copying sufficiently short lengths\n// performs better.\ninline constexpr size_t kMaxBytesToCopy = 511;\n\n// Recommends the length of a buffer by modifying the base recommendation.\n//\n// If `pos` did not pass `size_hint` yet, returns the remaining length instead\n// of `base_length`.\ninline Position ApplySizeHint(Position base_length,\n                              std::optional<Position> size_hint, Position pos) {\n  if (size_hint != std::nullopt && pos <= *size_hint) return *size_hint - pos;\n  return base_length;\n}\n\n// Recommends the length of a buffer by modifying the base recommendation.\n//\n// The following constraints are applied, in the order of weakest to strongest:\n//  * At least `recommended_length`.\n//  * At most `max_length`.\n//  * At least `min_length`.\ninline size_t ApplyBufferConstraints(Position base_length, size_t min_length,\n                                     size_t recommended_length,\n                                     size_t max_length) {\n  return UnsignedClamp(UnsignedMax(base_length, recommended_length), min_length,\n                       max_length);\n}\n\n// Heuristics for whether a data structure with `allocated` bytes utilizing\n// `used` bytes for actual data is considered wasteful: `allocated` is larger\n// than `2 * used + kDefaultMinBlockSize` (512).\ninline bool Wasteful(size_t allocated, size_t used) {\n  if (ABSL_PREDICT_FALSE(used > allocated)) return false;\n  const size_t unused = allocated - used;\n  if (ABSL_PREDICT_TRUE(unused <= kDefaultMinBlockSize)) return false;\n  return unused - riegeli::kDefaultMinBlockSize > used;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_BUFFERING_H_\n"
  },
  {
    "path": "riegeli/base/byte_fill.cc",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/byte_fill.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <cstring>\n#include <ios>\n#include <limits>\n#include <ostream>\n#include <utility>\n#include <variant>\n\n#include \"absl/numeric/bits.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/cord_buffer.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_utils.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/global.h\"\n#include \"riegeli/base/shared_buffer.h\"\n#include \"riegeli/base/types.h\"\n\nnamespace riegeli {\n\ninline const char* ByteFill::ZeroBlock::Data() {\n  return Global([] { return new char[kSize](); });\n}\n\nChain::Block ByteFill::ZeroBlock::ToChainBlock(absl::string_view substr) {\n  if (substr.size() == kSize) {\n    return Global([] {\n      return Chain::Block(ZeroBlock(), absl::string_view(Data(), kSize));\n    });\n  }\n  return Chain::Block(ZeroBlock(), substr);\n}\n\nabsl::Cord ByteFill::ZeroBlock::ToCord(absl::string_view substr) {\n  static constexpr auto kNullReleaser = [] {};\n  if (substr.size() == kSize) {\n    return Global([] {\n      return absl::MakeCordFromExternal(absl::string_view(Data(), kSize),\n                                        kNullReleaser);\n    });\n  }\n  return absl::MakeCordFromExternal(substr, kNullReleaser);\n}\n\nvoid ByteFill::ZeroBlock::DumpStructure(std::ostream& dest) {\n  dest << \"[zero_fill] { }\";\n}\n\nvoid ByteFill::LargeBlock::DumpStructure(absl::string_view substr,\n                                         std::ostream& dest) const {\n  dest << \"[large_fill] {\";\n  const size_t ref_count = buffer_.GetRefCount();\n  if (ref_count != 1) dest << \" ref_count: \" << ref_count;\n  if (buffer_.capacity() != substr.size()) {\n    dest << \" capacity: \" << buffer_.capacity();\n  }\n  dest << \" }\";\n}\n\nByteFill::Blocks::Blocks(Position size, char fill) {\n  if (size == 0) return;\n  if (fill == '\\0') {\n    RIEGELI_ASSERT(std::holds_alternative<ZeroBlock>(block_));\n    num_blocks_ = (size - 1) / ZeroBlock::kSize + 1;\n    non_last_block_size_ = uint32_t{ZeroBlock::kSize};\n    last_block_size_ =\n        static_cast<uint32_t>(size - 1) % uint32_t{ZeroBlock::kSize} + 1;\n    data_ = ZeroBlock::Data();\n    return;\n  }\n  if (size <= SmallBlock::kSize) {\n    num_blocks_ = 1;\n    last_block_size_ = IntCast<uint32_t>(size);\n    data_ = block_.emplace<SmallBlock>(fill).data();\n    return;\n  }\n  if (size <= kMaxSizeForSingleBlock) {\n    num_blocks_ = 1;\n    non_last_block_size_ = IntCast<uint32_t>(size);\n    last_block_size_ = non_last_block_size_;\n  } else {\n    const int block_size_bits = SignedMin(\n        (kBlockSizeBitsBias + IntCast<int>(absl::bit_width(size))) / 2, 16);\n    num_blocks_ = ((size - 1) >> block_size_bits) + 1;\n    non_last_block_size_ = uint32_t{1} << block_size_bits;\n    last_block_size_ =\n        (static_cast<uint32_t>(size - 1) & (non_last_block_size_ - 1)) + 1;\n  }\n  data_ = block_.emplace<LargeBlock>(non_last_block_size_, fill).data();\n}\n\nByteFill::operator Chain() const {\n  Chain dest;\n  if (size_ <= (fill_ == '\\0' ? Chain::kMaxBytesToCopyToEmpty\n                              : Blocks::kMaxSizeForSingleBlock)) {\n    if (size_ > 0) {\n      const absl::Span<char> buffer = dest.AppendFixedBuffer(\n          IntCast<size_t>(size_),\n          Chain::Options().set_size_hint(IntCast<size_t>(size_)));\n      std::memset(buffer.data(), fill_, buffer.size());\n    }\n  } else {\n    RIEGELI_CHECK_LE(size_, std::numeric_limits<size_t>::max())\n        << \"Chain size overflow\";\n    Chain::Options options;\n    options.set_size_hint(IntCast<size_t>(size_));\n    const Blocks blocks = this->blocks();\n    BlockIterator iter = blocks.cbegin();\n    RIEGELI_ASSERT(iter != blocks.cend()) << \"Empty ByteFill handled above\";\n    do {\n      dest.Append(*iter, options);\n    } while (++iter != blocks.cend());\n  }\n  return dest;\n}\n\nByteFill::operator absl::Cord() const {\n  absl::Cord dest;\n  if (size_ <= UnsignedMin(fill_ == '\\0'\n                               ? cord_internal::kMaxBytesToCopyToEmptyCord\n                               : Blocks::kMaxSizeForSingleBlock,\n                           absl::CordBuffer::kDefaultLimit)) {\n    if (size_ > 0) {\n      absl::CordBuffer buffer =\n          absl::CordBuffer::CreateWithDefaultLimit(IntCast<size_t>(size_));\n      buffer.SetLength(IntCast<size_t>(size_));\n      std::memset(buffer.data(), fill_, IntCast<size_t>(size_));\n      dest.Append(std::move(buffer));\n    }\n  } else {\n    RIEGELI_CHECK_LE(size_, std::numeric_limits<size_t>::max())\n        << \"Cord size overflow\";\n    const Blocks blocks = this->blocks();\n    BlockIterator iter = blocks.cbegin();\n    RIEGELI_ASSERT(iter != blocks.cend()) << \"Empty ByteFill handled above\";\n    do {\n      ExternalRef(*iter).AppendTo(dest);\n    } while (++iter != blocks.cend());\n  }\n  return dest;\n}\n\nvoid ByteFill::AssignTo(Chain& dest) const {\n  dest.Clear();\n  if (size_ <= (fill_ == '\\0' ? Chain::kMaxBytesToCopyToEmpty\n                              : Blocks::kMaxSizeForSingleBlock)) {\n    if (empty()) return;\n    const absl::Span<char> buffer = dest.AppendFixedBuffer(\n        IntCast<size_t>(size_),\n        Chain::Options().set_size_hint(IntCast<size_t>(size_)));\n    std::memset(buffer.data(), fill_, buffer.size());\n  } else {\n    RIEGELI_CHECK_LE(size_, std::numeric_limits<size_t>::max())\n        << \"Chain size overflow\";\n    Chain::Options options;\n    options.set_size_hint(IntCast<size_t>(size_));\n    const Blocks blocks = this->blocks();\n    BlockIterator iter = blocks.cbegin();\n    RIEGELI_ASSERT(iter != blocks.cend()) << \"Empty ByteFill handled above\";\n    do {\n      dest.Append(*iter, options);\n    } while (++iter != blocks.cend());\n  }\n}\n\nvoid ByteFill::AssignTo(absl::Cord& dest) const {\n  if (size_ <= UnsignedMin(fill_ == '\\0'\n                               ? cord_internal::kMaxBytesToCopyToEmptyCord\n                               : Blocks::kMaxSizeForSingleBlock,\n                           absl::CordBuffer::kDefaultLimit)) {\n    if (size_ == 0) {\n      dest.Clear();\n    } else {\n      absl::CordBuffer buffer = dest.GetAppendBuffer(0, 0);\n      dest.Clear();\n      if (buffer.capacity() < IntCast<size_t>(size_)) {\n        buffer =\n            absl::CordBuffer::CreateWithDefaultLimit(IntCast<size_t>(size_));\n      }\n      buffer.SetLength(IntCast<size_t>(size_));\n      std::memset(buffer.data(), fill_, IntCast<size_t>(size_));\n      dest.Append(std::move(buffer));\n    }\n  } else {\n    dest.Clear();\n    RIEGELI_CHECK_LE(size_, std::numeric_limits<size_t>::max())\n        << \"Cord size overflow\";\n    const Blocks blocks = this->blocks();\n    BlockIterator iter = blocks.cbegin();\n    RIEGELI_ASSERT(iter != blocks.cend()) << \"Empty ByteFill handled above\";\n    do {\n      ExternalRef(*iter).AppendTo(dest);\n    } while (++iter != blocks.cend());\n  }\n}\n\nvoid ByteFill::AppendTo(Chain& dest) const {\n  if (size_ <= (fill_ == '\\0' ? dest.MaxBytesToCopy()\n                              : Blocks::kMaxSizeForSingleBlock)) {\n    size_t length = IntCast<size_t>(size_);\n    while (length > 0) {\n      const absl::Span<char> buffer = dest.AppendBuffer(1, length, length);\n      std::memset(buffer.data(), fill_, buffer.size());\n      length -= buffer.size();\n    }\n  } else {\n    const Blocks blocks = this->blocks();\n    BlockIterator iter = blocks.cbegin();\n    RIEGELI_ASSERT(iter != blocks.cend()) << \"Empty ByteFill handled above\";\n    do {\n      dest.Append(*iter);\n    } while (++iter != blocks.cend());\n  }\n}\n\nvoid ByteFill::AppendTo(Chain& dest, Chain::Options options) const {\n  if (size_ <= (fill_ == '\\0' ? dest.MaxBytesToCopy(options)\n                              : Blocks::kMaxSizeForSingleBlock)) {\n    size_t length = IntCast<size_t>(size_);\n    while (length > 0) {\n      const absl::Span<char> buffer =\n          dest.AppendBuffer(1, length, length, options);\n      std::memset(buffer.data(), fill_, buffer.size());\n      length -= buffer.size();\n    }\n  } else {\n    const Blocks blocks = this->blocks();\n    BlockIterator iter = blocks.cbegin();\n    RIEGELI_ASSERT(iter != blocks.cend()) << \"Empty ByteFill handled above\";\n    do {\n      dest.Append(*iter, options);\n    } while (++iter != blocks.cend());\n  }\n}\n\nvoid ByteFill::AppendTo(absl::Cord& dest) const {\n  if (size_ <= UnsignedMin(fill_ == '\\0'\n                               ? cord_internal::MaxBytesToCopyToCord(dest)\n                               : Blocks::kMaxSizeForSingleBlock,\n                           absl::CordBuffer::kDefaultLimit)) {\n    size_t length = IntCast<size_t>(size_);\n    if (length == 0) return;\n    {\n      absl::CordBuffer buffer = dest.GetAppendBuffer(0, 1);\n      const size_t existing_length = buffer.length();\n      if (existing_length > 0) {\n        buffer.SetLength(\n            UnsignedMin(existing_length + length, buffer.capacity()));\n        std::memset(buffer.data() + existing_length, fill_,\n                    buffer.length() - existing_length);\n        length -= buffer.length() - existing_length;\n        dest.Append(std::move(buffer));\n        if (length == 0) return;\n      }\n    }\n    absl::CordBuffer buffer = absl::CordBuffer::CreateWithDefaultLimit(length);\n    buffer.SetLength(length);\n    std::memset(buffer.data(), fill_, length);\n    dest.Append(std::move(buffer));\n  } else {\n    RIEGELI_CHECK_LE(size_, std::numeric_limits<size_t>::max() - dest.size())\n        << \"Cord size overflow\";\n    const Blocks blocks = this->blocks();\n    BlockIterator iter = blocks.cbegin();\n    RIEGELI_ASSERT(iter != blocks.cend()) << \"Empty ByteFill handled above\";\n    do {\n      ExternalRef(*iter).AppendTo(dest);\n    } while (++iter != blocks.cend());\n  }\n}\n\nvoid ByteFill::PrependTo(Chain& dest) const {\n  if (size_ <= (fill_ == '\\0' ? dest.MaxBytesToCopy()\n                              : Blocks::kMaxSizeForSingleBlock)) {\n    size_t length = IntCast<size_t>(size_);\n    while (length > 0) {\n      const absl::Span<char> buffer = dest.PrependBuffer(1, length, length);\n      std::memset(buffer.data(), fill_, buffer.size());\n      length -= buffer.size();\n    }\n  } else {\n    const Blocks blocks = this->blocks();\n    BlockIterator iter = blocks.cend();\n    RIEGELI_ASSERT(iter != blocks.cbegin()) << \"Empty ByteFill handled above\";\n    do {\n      --iter;\n      dest.Prepend(*iter);\n    } while (iter != blocks.cbegin());\n  }\n}\n\nvoid ByteFill::PrependTo(Chain& dest, Chain::Options options) const {\n  if (size_ <= (fill_ == '\\0' ? dest.MaxBytesToCopy(options)\n                              : Blocks::kMaxSizeForSingleBlock)) {\n    size_t length = IntCast<size_t>(size_);\n    while (length > 0) {\n      const absl::Span<char> buffer =\n          dest.PrependBuffer(1, length, length, options);\n      std::memset(buffer.data(), fill_, buffer.size());\n      length -= buffer.size();\n    }\n  } else {\n    const Blocks blocks = this->blocks();\n    BlockIterator iter = blocks.cend();\n    RIEGELI_ASSERT(iter != blocks.cbegin()) << \"Empty ByteFill handled above\";\n    do {\n      --iter;\n      dest.Prepend(*iter, options);\n    } while (iter != blocks.cbegin());\n  }\n}\n\nvoid ByteFill::PrependTo(absl::Cord& dest) const {\n  if (size_ <= UnsignedMin(fill_ == '\\0'\n                               ? cord_internal::MaxBytesToCopyToCord(dest)\n                               : Blocks::kMaxSizeForSingleBlock,\n                           absl::CordBuffer::kDefaultLimit)) {\n    if (empty()) return;\n    absl::CordBuffer buffer =\n        absl::CordBuffer::CreateWithDefaultLimit(IntCast<size_t>(size_));\n    buffer.SetLength(IntCast<size_t>(size_));\n    std::memset(buffer.data(), fill_, IntCast<size_t>(size_));\n    dest.Prepend(std::move(buffer));\n  } else {\n    RIEGELI_CHECK_LE(size_, std::numeric_limits<size_t>::max() - dest.size())\n        << \"Cord size overflow\";\n    const Blocks blocks = this->blocks();\n    BlockIterator iter = blocks.cend();\n    RIEGELI_ASSERT(iter != blocks.cbegin()) << \"Empty ByteFill handled above\";\n    do {\n      --iter;\n      ExternalRef(*iter).PrependTo(dest);\n    } while (iter != blocks.cbegin());\n  }\n}\n\nvoid ByteFill::Output(std::ostream& dest) const {\n  for (const absl::string_view fragment : blocks()) {\n    dest.write(fragment.data(), IntCast<std::streamsize>(fragment.size()));\n  }\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/byte_fill.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_BYTE_FILL_H_\n#define RIEGELI_BASE_BYTE_FILL_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <cstring>\n#include <iosfwd>\n#include <iterator>\n#include <limits>\n#include <utility>\n#include <variant>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/external_data.h\"\n#include \"riegeli/base/iterable.h\"\n#include \"riegeli/base/shared_buffer.h\"\n#include \"riegeli/base/types.h\"\n\nnamespace riegeli {\n\n// Represents a byte sequence of the given size with all bytes equal to the\n// given value.\nclass ByteFill {\n public:\n  class BlockRef;\n  class BlockIterator;\n  class Blocks;\n\n  // Constructs a `ByteFill` with `size` occurrences of `fill`.\n  explicit ByteFill(Position size, char fill = '\\0')\n      : size_(size), fill_(fill) {}\n\n  ByteFill(const ByteFill& that) = default;\n  ByteFill& operator=(const ByteFill& that) = default;\n\n  bool empty() const { return size() == 0; }\n  Position size() const { return size_; }\n  char fill() const { return fill_; }\n\n  // Removes `difference` occurrences, and returns `ByteFill` corresponding\n  // to the removed fragment.\n  //\n  // Precondition: `difference <= size()`\n  ByteFill Extract(Position difference) {\n    RIEGELI_ASSERT_LE(difference, size_)\n        << \"Failed precondition of ByteFill::Extract(): size underflow\";\n    size_ -= difference;\n    return ByteFill(difference, fill_);\n  }\n\n  // A sequence of non-empty `absl::string_view` blocks comprising data of the\n  // `ByteFill`.\n  Blocks blocks() const;\n\n  // Converts the data to `Chain`.\n  explicit operator Chain() const;\n\n  // Converts the data to `absl::Cord`.\n  explicit operator absl::Cord() const;\n\n  // Supports `riegeli::Reset(Chain&, ByteFill)`.\n  friend void RiegeliReset(Chain& dest, ByteFill src) { src.AssignTo(dest); }\n\n  // Supports `riegeli::Reset(absl::Cord&, ByteFill)`.\n  friend void RiegeliReset(absl::Cord& dest, ByteFill src) {\n    src.AssignTo(dest);\n  }\n\n  // Appends the data to `dest`.\n  void AppendTo(Chain& dest) const;\n  void AppendTo(Chain& dest, Chain::Options options) const;\n\n  // Appends the data to `dest`.\n  void AppendTo(absl::Cord& dest) const;\n\n  // Prepends the data to `dest`.\n  void PrependTo(Chain& dest) const;\n  void PrependTo(Chain& dest, Chain::Options options) const;\n\n  // Prepends the data to `dest`.\n  void PrependTo(absl::Cord& dest) const;\n\n  // Default stringification by `absl::StrCat()` etc.\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, ByteFill src) {\n    Position length = src.size_;\n    while (ABSL_PREDICT_FALSE(length > std::numeric_limits<size_t>::max())) {\n      dest.Append(std::numeric_limits<size_t>::max(), src.fill_);\n      length -= std::numeric_limits<size_t>::max();\n    }\n    if (length > 0) dest.Append(IntCast<size_t>(length), src.fill_);\n  }\n\n  // Writes the occurrences to `out` as unformatted bytes.\n  friend std::ostream& operator<<(std::ostream& dest, ByteFill src) {\n    src.Output(dest);\n    return dest;\n  }\n\n private:\n  class ZeroBlock;\n  class SmallBlock;\n  class LargeBlock;\n\n  void AssignTo(Chain& dest) const;\n  void AssignTo(absl::Cord& dest) const;\n  void Output(std::ostream& dest) const;\n\n  Position size_;\n  char fill_;\n};\n\n// Represents a block of zeros backed by a shared array for `ExternalRef`.\nclass ByteFill::ZeroBlock {\n public:\n  static constexpr size_t kSize = size_t{64} << 10;\n\n  static const char* Data();\n\n  ZeroBlock() = default;\n\n  ZeroBlock(const ZeroBlock& that) = default;\n  ZeroBlock& operator=(const ZeroBlock& that) = default;\n\n  // Supports `ExternalRef`.\n  friend Chain::Block RiegeliToChainBlock(\n      ABSL_ATTRIBUTE_UNUSED const ZeroBlock* self, absl::string_view substr) {\n    return ToChainBlock(substr);\n  }\n\n  // Supports `ExternalRef`.\n  friend absl::Cord RiegeliToCord(ABSL_ATTRIBUTE_UNUSED const ZeroBlock* self,\n                                  absl::string_view substr) {\n    return ToCord(substr);\n  }\n\n  // Supports `ExternalRef`.\n  friend ExternalStorage RiegeliToExternalStorage(\n      ABSL_ATTRIBUTE_UNUSED const ZeroBlock* self) {\n    return ExternalStorage(nullptr, [](ABSL_ATTRIBUTE_UNUSED void* ptr) {});\n  }\n\n  // Supports `ExternalRef` and `Chain::Block`.\n  friend void RiegeliDumpStructure(ABSL_ATTRIBUTE_UNUSED const ZeroBlock* self,\n                                   std::ostream& dest) {\n    DumpStructure(dest);\n  }\n\n private:\n  static Chain::Block ToChainBlock(absl::string_view substr);\n  static absl::Cord ToCord(absl::string_view substr);\n  static void DumpStructure(std::ostream& dest);\n};\n\nclass ByteFill::SmallBlock {\n public:\n  static constexpr size_t kSize = 64;\n\n  explicit SmallBlock(char fill) { std::memset(data_, fill, kSize); }\n\n  SmallBlock(const SmallBlock& that) = default;\n  SmallBlock& operator=(const SmallBlock& that) = default;\n\n  const char* data() const { return data_; }\n\n  // Supports `ExternalRef`.\n  friend bool RiegeliExternalCopy(\n      ABSL_ATTRIBUTE_UNUSED const SmallBlock* self) {\n    return true;\n  }\n\n private:\n  char data_[kSize];\n};\n\nclass ByteFill::LargeBlock {\n public:\n  explicit LargeBlock(size_t size, char fill) : buffer_(size) {\n    std::memset(buffer_.mutable_data(), fill, size);\n  }\n\n  LargeBlock(const LargeBlock& that) = default;\n  LargeBlock& operator=(const LargeBlock& that) = default;\n\n  LargeBlock(LargeBlock&& that) = default;\n  LargeBlock& operator=(LargeBlock&& that) = default;\n\n  const char* data() const { return buffer_.data(); }\n\n  // Indicates support for:\n  //  * `ExternalRef(const LargeBlock&, substr)`\n  //  * `ExternalRef(LargeBlock&&, substr)`\n  friend void RiegeliSupportsExternalRef(const LargeBlock*) {}\n\n  // Supports `ExternalRef`.\n  friend ExternalStorage RiegeliToExternalStorage(LargeBlock* self) {\n    return RiegeliToExternalStorage(&self->buffer_);\n  }\n\n  // Supports `ExternalRef` and `Chain::Block`.\n  friend void RiegeliDumpStructure(const LargeBlock* self,\n                                   absl::string_view substr,\n                                   std::ostream& dest) {\n    self->DumpStructure(substr, dest);\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const LargeBlock* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->buffer_);\n  }\n\n private:\n  void DumpStructure(absl::string_view substr, std::ostream& dest) const;\n\n  SharedBuffer buffer_;\n};\n\nclass ByteFill::BlockRef {\n public:\n  BlockRef(const BlockRef& that) = default;\n  BlockRef& operator=(const BlockRef& that) = default;\n\n  /*implicit*/ operator absl::string_view() const {\n    return absl::string_view(data(), size());\n  }\n\n  bool empty() const { return false; }\n  const char* data() const;\n  size_t size() const;\n\n  // Indicates support for:\n  //  * `ExternalRef(BlockRef)`\n  //  * `ExternalRef(BlockRef, substr)`\n  friend void RiegeliSupportsExternalRef(const BlockRef*) {}\n\n  // Supports `ExternalRef`.\n  template <typename Callback>\n  friend void RiegeliExternalDelegate(const BlockRef* self,\n                                      absl::string_view substr,\n                                      Callback&& delegate_to) {\n    self->ExternalDelegate(substr, std::forward<Callback>(delegate_to));\n  }\n\n private:\n  friend class ByteFill;  // For `BlockRef()`.\n\n  explicit BlockRef(const ByteFill::Blocks* blocks,\n                    Position block_index_complement)\n      : blocks_(blocks), block_index_complement_(block_index_complement) {}\n\n  template <typename Callback>\n  void ExternalDelegate(absl::string_view substr, Callback&& delegate_to) const;\n\n  const Blocks* blocks_;\n  // `block_index_complement_` is `blocks_->num_blocks_ - block_index`. Working\n  // with the complement makes it easier to handle special case at 1 (a block\n  // with size `blocks_->last_block_size_`).\n  Position block_index_complement_;\n};\n\nclass ByteFill::BlockIterator : public WithCompare<BlockIterator> {\n public:\n  using iterator_concept = std::random_access_iterator_tag;\n  // `iterator_category` is only `std::input_iterator_tag` because the\n  // `LegacyForwardIterator` requirement and above require `reference` to be\n  // a true reference type.\n  using iterator_category = std::input_iterator_tag;\n  using value_type = BlockRef;\n  using reference = value_type;\n  using pointer = ArrowProxy<reference>;\n  using difference_type = ptrdiff_t;\n\n  BlockIterator() = default;\n\n  BlockIterator(const BlockIterator& that) = default;\n  BlockIterator& operator=(const BlockIterator& that) = default;\n\n  reference operator*() const;\n  pointer operator->() const;\n  BlockIterator& operator++();\n  BlockIterator operator++(int);\n  BlockIterator& operator--();\n  BlockIterator operator--(int);\n  BlockIterator& operator+=(difference_type n);\n  BlockIterator operator+(difference_type n) const;\n  BlockIterator& operator-=(difference_type n);\n  BlockIterator operator-(difference_type n) const;\n  reference operator[](difference_type n) const;\n\n  friend bool operator==(BlockIterator a, BlockIterator b) {\n    RIEGELI_ASSERT_EQ(a.blocks_, b.blocks_)\n        << \"Failed precondition of operator==(ByteFill::BlockIterator): \"\n           \"incomparable iterators\";\n    return b.block_index_complement_ == a.block_index_complement_;\n  }\n  friend StrongOrdering RIEGELI_COMPARE(BlockIterator a, BlockIterator b) {\n    RIEGELI_ASSERT_EQ(a.blocks_, b.blocks_)\n        << \"Failed precondition of operator<=>(ByteFill::BlockIterator): \"\n           \"incomparable iterators\";\n    return riegeli::Compare(b.block_index_complement_,\n                            a.block_index_complement_);\n  }\n  friend difference_type operator-(BlockIterator a, BlockIterator b) {\n    RIEGELI_ASSERT_EQ(a.blocks_, b.blocks_)\n        << \"Failed precondition of operator-(ByteFill::BlockIterator): \"\n           \"incomparable iterators\";\n    return b.block_index_complement_ - a.block_index_complement_;\n  }\n  friend BlockIterator operator+(difference_type n, BlockIterator a) {\n    return a + n;\n  }\n\n private:\n  friend class ByteFill;  // For `BlockIterator()`.\n\n  explicit BlockIterator(const Blocks* blocks, Position block_index_complement)\n      : blocks_(blocks), block_index_complement_(block_index_complement) {}\n\n  const Blocks* blocks_ = nullptr;\n  // `block_index_complement_` is `blocks_->num_blocks_ - block_index`. Working\n  // with the complement makes it easier to handle special cases at 0 (`end()`)\n  // and 1 (a block with size `blocks_->last_block_size_`).\n  Position block_index_complement_ = 0;\n};\n\nclass ByteFill::Blocks {\n public:\n  using value_type = BlockRef;\n  using reference = value_type;\n  using const_reference = reference;\n  using iterator = BlockIterator;\n  using const_iterator = iterator;\n  using reverse_iterator = std::reverse_iterator<iterator>;\n  using const_reverse_iterator = reverse_iterator;\n  using size_type = size_t;\n  using difference_type = ptrdiff_t;\n\n  Blocks() = default;\n\n  Blocks(Blocks&& that) noexcept;\n  Blocks& operator=(Blocks&&) = delete;\n\n  iterator begin() const { return iterator(this, num_blocks_); }\n  iterator cbegin() const { return begin(); }\n  iterator end() const { return iterator(this, 0); }\n  iterator cend() const { return end(); }\n\n  reverse_iterator rbegin() const { return reverse_iterator(end()); }\n  reverse_iterator crbegin() const { return rbegin(); }\n  reverse_iterator rend() const { return reverse_iterator(begin()); }\n  reverse_iterator crend() const { return rend(); }\n\n  bool empty() const { return size() == 0; }\n  size_type size() const { return num_blocks_; }\n\n  reference operator[](size_type n) const {\n    RIEGELI_ASSERT_LT(n, size())\n        << \"Failed precondition of ByteFill::Blocks::operator[]: \"\n           \"block index out of range\";\n    return BlockRef(this, num_blocks_ - n);\n  }\n  reference at(size_type n) const {\n    RIEGELI_CHECK_LT(n, size())\n        << \"Failed precondition of ByteFill::Blocks::at(): \"\n           \"block index out of range\";\n    return BlockRef(this, num_blocks_ - n);\n  }\n  reference front() const {\n    RIEGELI_ASSERT(!empty())\n        << \"Failed precondition of ByteFill::Blocks::front(): no blocks\";\n    return BlockRef(this, num_blocks_);\n  }\n  reference back() const {\n    RIEGELI_ASSERT(!empty())\n        << \"Failed precondition of ByteFill::Blocks::back(): no blocks\";\n    return BlockRef(this, 1);\n  }\n\n private:\n  // For `kMaxSizeForSingleBlock`, `Blocks()`, `data()`, `size()`, and\n  // `ExternalDelegate()`.\n  friend class ByteFill;\n\n  // Find a balance between the number of blocks and the block size.\n  // The following parameters yield:\n  //  *    1K =   1 *  1K\n  //  *    2K =   1 *  2K\n  //  *    4K =   2 *  2K\n  //  *    8K =   2 *  4K\n  //  *   16K =   4 *  4K\n  //  *   32K =   4 *  8K\n  //  *   64K =   8 *  8K\n  //  *  128K =   8 * 16K\n  //  *  256K =  16 * 16K\n  //  *  512K =  16 * 32K\n  //  * 1M    =  32 * 32K\n  //  * 2M    =  32 * 64K\n  //  * 4M    =  64 * 64K\n  //  * 8M    = 128 * 64K\n  static constexpr int kBlockSizeBitsBias = 10;\n  static constexpr Position kMaxSizeForSingleBlock =\n      Position{1} << (kBlockSizeBitsBias + 1);\n\n  explicit Blocks(Position size, char fill);\n\n  const char* data() const { return data_; }\n  size_t size(Position block_index_complement) const {\n    return block_index_complement == 1 ? last_block_size_\n                                       : non_last_block_size_;\n  }\n\n  template <typename Callback>\n  void ExternalDelegate(absl::string_view substr, Callback&& delegate_to) const;\n\n  Position num_blocks_ = 0;\n  uint32_t non_last_block_size_ = 0;\n  uint32_t last_block_size_ = 0;\n  // If `num_blocks_ > 0` then `data_` is:\n  //  * When `block_` is `ZeroBlock`:  `ZeroBlock::Data()`\n  //  * When `block_` is `SmallBlock`: `small_block.data()`\n  //  * When `block_` is `LargeBlock`: `large_block.data()`\n  const char* data_ = nullptr;\n  std::variant<ZeroBlock, SmallBlock, LargeBlock> block_;\n};\n\n// Implementation details follow.\n\ninline const char* ByteFill::BlockRef::data() const { return blocks_->data(); }\n\ninline size_t ByteFill::BlockRef::size() const {\n  return blocks_->size(block_index_complement_);\n}\n\ntemplate <typename Callback>\ninline void ByteFill::BlockRef::ExternalDelegate(absl::string_view substr,\n                                                 Callback&& delegate_to) const {\n  blocks_->ExternalDelegate(substr, std::forward<Callback>(delegate_to));\n}\n\ninline ByteFill::BlockIterator::reference ByteFill::BlockIterator::operator*()\n    const {\n  RIEGELI_ASSERT_GT(block_index_complement_, 0u)\n      << \"Failed precondition of ByteFill::BlockIterator::operator*: \"\n         \"iterator is end()\";\n  return BlockRef(blocks_, block_index_complement_);\n}\n\ninline ByteFill::BlockIterator::pointer ByteFill::BlockIterator::operator->()\n    const {\n  return pointer(**this);\n}\n\ninline ByteFill::BlockIterator& ByteFill::BlockIterator::operator++() {\n  RIEGELI_ASSERT_GT(block_index_complement_, 0u)\n      << \"Failed precondition of ByteFill::BlockIterator::operator++: \"\n         \"iterator is end()\";\n  --block_index_complement_;\n  return *this;\n}\n\ninline ByteFill::BlockIterator ByteFill::BlockIterator::operator++(int) {\n  const BlockIterator tmp = *this;\n  ++*this;\n  return tmp;\n}\n\ninline ByteFill::BlockIterator& ByteFill::BlockIterator::operator--() {\n  RIEGELI_ASSERT_LT(block_index_complement_, blocks_->size())\n      << \"Failed precondition of ByteFill::BlockIterator::operator--: \"\n         \"iterator is begin()\";\n  ++block_index_complement_;\n  return *this;\n}\n\ninline ByteFill::BlockIterator ByteFill::BlockIterator::operator--(int) {\n  const BlockIterator tmp = *this;\n  --*this;\n  return tmp;\n}\n\ninline ByteFill::BlockIterator& ByteFill::BlockIterator::operator+=(\n    difference_type n) {\n  if (n >= 0) {\n    RIEGELI_ASSERT_LE(UnsignedCast(n), block_index_complement_)\n        << \"Failed precondition of ByteFill::BlockIterator::operator+=: \"\n           \"iterator after end()\";\n  } else {\n    RIEGELI_ASSERT_LE(NegatingUnsignedCast(n),\n                      blocks_->size() - block_index_complement_)\n        << \"Failed precondition of ByteFill::BlockIterator::operator+=: \"\n           \"iterator before begin()\";\n  }\n  block_index_complement_ -= static_cast<Position>(n);\n  return *this;\n}\n\ninline ByteFill::BlockIterator ByteFill::BlockIterator::operator+(\n    difference_type n) const {\n  return BlockIterator(*this) += n;\n}\n\ninline ByteFill::BlockIterator& ByteFill::BlockIterator::operator-=(\n    difference_type n) {\n  if (n >= 0) {\n    RIEGELI_ASSERT_LE(UnsignedCast(n),\n                      blocks_->size() - block_index_complement_)\n        << \"Failed precondition of ByteFill::BlockIterator::operator-=: \"\n           \"iterator before begin()\";\n  } else {\n    RIEGELI_ASSERT_LE(NegatingUnsignedCast(n), block_index_complement_)\n        << \"Failed precondition of ByteFill::BlockIterator::operator-=: \"\n           \"iterator after end()\";\n  }\n  block_index_complement_ += static_cast<Position>(n);\n  return *this;\n}\n\ninline ByteFill::BlockIterator ByteFill::BlockIterator::operator-(\n    difference_type n) const {\n  return BlockIterator(*this) -= n;\n}\n\ninline ByteFill::BlockIterator::reference ByteFill::BlockIterator::operator[](\n    difference_type n) const {\n  return *(*this + n);\n}\n\ninline ByteFill::Blocks::Blocks(Blocks&& that) noexcept\n    : num_blocks_(std::exchange(that.num_blocks_, 0)),\n      last_block_size_(that.last_block_size_),\n      data_(that.data_),\n      block_(std::move(that.block_)) {\n  if (SmallBlock* const small_block = std::get_if<SmallBlock>(&block_)) {\n    data_ = small_block->data();\n  }\n}\n\ntemplate <typename Callback>\ninline void ByteFill::Blocks::ExternalDelegate(absl::string_view substr,\n                                               Callback&& delegate_to) const {\n  struct Visitor {\n    void operator()(const ZeroBlock& zero_ref) const {\n      std::forward<Callback>(delegate_to)(zero_ref, substr);\n    }\n    void operator()(const SmallBlock& small_block) const {\n      std::forward<Callback>(delegate_to)(small_block, substr);\n    }\n    void operator()(const LargeBlock& large_block) const {\n      std::forward<Callback>(delegate_to)(large_block, substr);\n    }\n\n    absl::string_view substr;\n    Callback&& delegate_to;\n  };\n  std::visit(Visitor{substr, std::forward<Callback>(delegate_to)}, block_);\n}\n\ninline ByteFill::Blocks ByteFill::blocks() const {\n  return Blocks(size_, fill_);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_BYTE_FILL_H_\n"
  },
  {
    "path": "riegeli/base/bytes_ref.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_BYTES_REF_H_\n#define RIEGELI_BASE_BYTES_REF_H_\n\n#include <stddef.h>\n\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/string_ref.h\"\n#include \"riegeli/base/temporary_storage.h\"\n#include \"riegeli/base/type_traits.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `BytesRef` stores an `absl::string_view` representing text or binary data\n// (see `StringRef` for text data), possibly converted from\n// `absl::Span<const char>` or temporary `std::string`.\n//\n// It is intended for function parameters when the implementation needs\n// an `absl::string_view`, and the caller might have another representation\n// of the string.\n//\n// It is convertible from:\n//  * types convertible to `absl::string_view`\n//  * types convertible to `std::string`, e.g. `BytesInitializer`\n//  * types convertible to `absl::Span<const char>`,\n//    e.g. `std::vector<char>` or `std::array<char, length>`.\n//\n// `BytesRef` does not own string contents and is efficiently copyable.\nclass BytesRef : public StringRef, public WithCompare<BytesRef> {\n public:\n  // Stores an empty `absl::string_view`.\n  BytesRef() = default;\n\n  // Stores `str` converted to `absl::string_view`.\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  /*implicit*/ BytesRef(const char* str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : StringRef(absl::string_view(str)) {}\n\n  // Stores `str` converted to `StringRef` and then to `absl::string_view`.\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<BytesRef, T>,\n                                   std::is_convertible<T&&, absl::string_view>>,\n                int> = 0>\n  /*implicit*/ BytesRef(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : StringRef(std::forward<T>(str)) {}\n\n  // Stores `str` converted to `absl::string_view`.\n  /*implicit*/ BytesRef(\n      absl::Span<const char> str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : StringRef(absl::string_view(str.data(), str.size())) {}\n\n  // Stores `str` materialized, then converted to `StringRef` and then to\n  // `absl::string_view`.\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<\n                    NotSameRef<BytesRef, T>,\n                    std::negation<std::is_convertible<T&&, absl::string_view>>,\n                    std::is_convertible<T&&, std::string>>,\n                int> = 0>\n  /*implicit*/ BytesRef(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                        TemporaryStorage<std::string>&& storage\n                            ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : StringRef(std::forward<T>(str), std::move(storage)) {}\n\n  // Stores `str` converted to `absl::Span<const char>` and then to\n  // `absl::string_view`.\n  template <\n      typename T,\n      std::enable_if_t<\n          std::conjunction_v<NotSameRef<BytesRef, T>,\n                             std::negation<std::is_convertible<T&&, StringRef>>,\n                             NotSameRef<absl::Span<const char>, T>,\n                             std::is_convertible<T&&, absl::Span<const char>>>,\n          int> = 0>\n  /*implicit*/ BytesRef(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : BytesRef(absl::Span<const char>(std::forward<T>(str))) {}\n\n  BytesRef(const BytesRef& that) = default;\n  BytesRef& operator=(const BytesRef&) = delete;\n\n  friend bool operator==(BytesRef a, BytesRef b) {\n    return absl::string_view(a) == absl::string_view(b);\n  }\n  friend riegeli::StrongOrdering RIEGELI_COMPARE(BytesRef a, BytesRef b) {\n    return riegeli::Compare(absl::string_view(a), absl::string_view(b));\n  }\n\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<NotSameRef<BytesRef, T>,\n                                          std::is_convertible<T&&, StringRef>>,\n                       int> = 0>\n  friend bool operator==(BytesRef a, T&& b) {\n    return a == BytesRef(std::forward<T>(b));\n  }\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<NotSameRef<BytesRef, T>,\n                                          std::is_convertible<T&&, StringRef>>,\n                       int> = 0>\n  friend riegeli::StrongOrdering RIEGELI_COMPARE(BytesRef a, T&& b) {\n    return riegeli::Compare(a, BytesRef(std::forward<T>(b)));\n  }\n\n  // `absl::Span<const char>` is already comparable against types convertible to\n  // `absl::Span<const char>`, which includes `BytesRef`.\n};\n\n// `BytesInitializer` is convertible from the same types as `BytesRef`,\n// but efficiently takes ownership of `std::string`.\n//\n// `BytesInitializer` behaves like `Initializer<std::string>`.\nclass BytesInitializer : public Initializer<std::string> {\n public:\n  BytesInitializer() = default;\n\n  // Stores `str` converted to `absl::string_view` and then to `std::string`.\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  /*implicit*/ BytesInitializer(const char* str ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                                TemporaryStorage<MakerType<absl::string_view>>&&\n                                    storage ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : Initializer(std::move(storage).emplace(absl::string_view(str))) {}\n\n  // Stores `str` converted to `std::string`.\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<BytesInitializer, T>,\n                                   std::is_convertible<T&&, std::string>>,\n                int> = 0>\n  /*implicit*/ BytesInitializer(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : Initializer(std::forward<T>(str)) {}\n\n  // Stores `str` converted to `BytesRef`, then to `absl::string_view`, and then\n  // to `std::string`.\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<\n                           NotSameRef<BytesInitializer, T>,\n                           std::negation<std::is_convertible<T&&, std::string>>,\n                           std::is_convertible<T&&, BytesRef>>,\n                       int> = 0>\n  /*implicit*/ BytesInitializer(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                                TemporaryStorage<MakerType<absl::string_view>>&&\n                                    storage ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : Initializer(\n            std::move(storage).emplace(BytesRef(std::forward<T>(str)))) {}\n\n  BytesInitializer(BytesInitializer&& that) = default;\n  BytesInitializer& operator=(BytesInitializer&&) = delete;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_BYTES_REF_H_\n"
  },
  {
    "path": "riegeli/base/c_string_ref.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_C_STRING_REF_H_\n#define RIEGELI_BASE_C_STRING_REF_H_\n\n#include <cstddef>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/string_ref.h\"\n#include \"riegeli/base/temporary_storage.h\"\n#include \"riegeli/base/type_traits.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `CStringRef` stores a pointer to a C-style NUL-terminated string\n// or `nullptr`, possibly converted from another string representation.\n//\n// It is intended for function parameters when the implementation needs\n// a C-style NUL-terminated string, and the caller might have another\n// representation of the string.\n//\n// It is convertible from:\n//  * `std::nullptr_t`\n//  * types convertible to `const char*`\n//  * types supporting `c_str()`, e.g. `std::string` or mutable `CompactString`\n//  * types convertible to `absl::string_view`\n//\n// It copies string contents when this is needed for NUL-termination,\n// e.g. for types convertible to `absl::string_view` excluding `std::string`\n// and mutable `CompactString`. In that case the string is stored in a storage\n// object passed as a default argument to the constructor.\n//\n// `CStringRef` does not own string contents and is efficiently copyable.\nclass ABSL_NULLABILITY_COMPATIBLE CStringRef : public WithEqual<CStringRef> {\n private:\n  template <typename T, typename Enable = void>\n  struct HasCStr : std::false_type {};\n\n  template <typename T>\n  struct HasCStr<T, std::enable_if_t<std::is_convertible_v<\n                        decltype(std::declval<T>().c_str()), const char*>>>\n      : std::true_type {};\n\n public:\n  // Stores `nullptr`.\n  CStringRef() = default;\n  /*implicit*/ CStringRef(std::nullptr_t) {}\n\n  // Stores `str`.\n  /*implicit*/ CStringRef(\n      const char* absl_nullable str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : c_str_(str) {}\n\n  // Stores `str` converted to `const char*`.\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<CStringRef, T>,\n                                   NotSameRef<std::nullptr_t, T>,\n                                   NotSameRef<const char*, T>,\n                                   std::is_convertible<T&&, const char*>>,\n                int> = 0>\n  /*implicit*/ CStringRef(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : c_str_(std::forward<T>(str)) {}\n\n  // Stores `str.c_str()`. This applies e.g. to `std::string` and\n  // mutable `CompactString`.\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<\n                    NotSameRef<CStringRef, T>, NotSameRef<std::nullptr_t, T>,\n                    std::negation<std::is_convertible<T&&, const char*>>,\n                    HasCStr<T&&>>,\n                int> = 0>\n  /*implicit*/ CStringRef(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : c_str_(std::forward<T>(str).c_str()) {}\n\n  // Stores a pointer to the first character of a NUL-terminated copy of `str`\n  // converted to `StringRef` and then to `absl::string_view`.\n  //\n  // The string is stored in a storage object passed as a default argument to\n  // this constructor.\n  template <\n      typename T,\n      std::enable_if_t<\n          std::conjunction_v<\n              NotSameRef<CStringRef, T>, NotSameRef<std::nullptr_t, T>,\n              std::negation<std::is_convertible<T&&, const char*>>,\n              std::negation<HasCStr<T&&>>, std::is_convertible<T&&, StringRef>>,\n          int> = 0>\n  /*implicit*/ CStringRef(T&& str, TemporaryStorage<std::string>&& storage\n                                       ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : CStringRef(std::move(storage).emplace(\n            absl::string_view(StringRef(std::forward<T>(str))))) {}\n\n  CStringRef(const CStringRef& that) = default;\n  CStringRef& operator=(const CStringRef&) = delete;\n\n  // Returns the pointer to the C-style NUL-terminated string, or `nullptr`.\n  const char* absl_nullable c_str() const { return c_str_; }\n\n  friend bool operator==(CStringRef a, std::nullptr_t) {\n    return a.c_str_ == nullptr;\n  }\n\n private:\n  using pointer = const char*;  // For `ABSL_NULLABILITY_COMPATIBLE`.\n\n  const char* absl_nullable c_str_ = nullptr;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_C_STRING_REF_H_\n"
  },
  {
    "path": "riegeli/base/chain.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include <stddef.h>\n\n#include <algorithm>\n#include <cstring>\n#include <ios>\n#include <limits>\n#include <memory>\n#include <optional>\n#include <ostream>\n#include <string>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/container/inlined_vector.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/resize_and_overwrite.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain_base.h\"\n#include \"riegeli/base/chain_details.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/external_ref_base.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/intrusive_shared_ptr.h\"\n#include \"riegeli/base/invoker.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/memory_estimator.h\"\n#include \"riegeli/base/new_aligned.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/ownership.h\"\n#include \"riegeli/base/stream_utils.h\"\n#include \"riegeli/base/string_utils.h\"\n\nnamespace riegeli {\n\nnamespace {\n\n// Stores an `absl::Cord` which must be flat, i.e.\n// `src.TryFlat() != std::nullopt`.\n//\n// This design relies on the fact that moving a flat `absl::Cord` results in a\n// flat `absl::Cord`.\nclass FlatCordBlock {\n public:\n  explicit FlatCordBlock(Initializer<absl::Cord> src);\n\n  FlatCordBlock(FlatCordBlock&& that) = default;\n  FlatCordBlock& operator=(FlatCordBlock&& that) = default;\n\n  const absl::Cord& src() const { return src_; }\n\n  /*implicit*/ operator absl::string_view() const;\n\n  // Supports `ExternalRef` and `Chain::Block`.\n  friend void RiegeliDumpStructure(\n      ABSL_ATTRIBUTE_UNUSED const FlatCordBlock* self, std::ostream& dest) {\n    dest << \"[cord] { }\";\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const FlatCordBlock* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->src_);\n  }\n\n private:\n  // Invariant: `src_.TryFlat() != std::nullopt`\n  absl::Cord src_;\n};\n\ninline FlatCordBlock::FlatCordBlock(Initializer<absl::Cord> src)\n    : src_(std::move(src)) {\n  RIEGELI_ASSERT(src_.TryFlat() != std::nullopt)\n      << \"Failed precondition of FlatCordBlock::FlatCordBlock(): \"\n         \"Cord is not flat\";\n}\n\ninline FlatCordBlock::operator absl::string_view() const {\n  if (const std::optional<absl::string_view> flat = src_.TryFlat();\n      flat != std::nullopt) {\n    return *flat;\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Failed invariant of FlatCordBlock: Cord is not flat\";\n}\n\n}  // namespace\n\nnamespace chain_internal {\n\nvoid DumpStructureDefault(std::ostream& dest) { dest << \"[external] { }\"; }\n\n}  // namespace chain_internal\n\nvoid RiegeliDumpStructure(const std::string* self, std::ostream& dest) {\n  dest << \"[string] { capacity: \" << self->capacity() << \" }\";\n}\n\ninline IntrusiveSharedPtr<Chain::RawBlock> Chain::RawBlock::NewInternal(\n    size_t min_capacity) {\n  RIEGELI_ASSERT_GT(min_capacity, 0u)\n      << \"Failed precondition of Chain::RawBlock::NewInternal(): zero capacity\";\n  size_t raw_capacity;\n  return IntrusiveSharedPtr<RawBlock>(SizeReturningNewAligned<RawBlock>(\n      kInternalAllocatedOffset() + min_capacity, &raw_capacity, &raw_capacity));\n}\n\ninline Chain::RawBlock::RawBlock(const size_t* raw_capacity)\n    : substr_(allocated_begin_, 0),\n      // Redundant cast is needed for `-fsanitize=bounds`.\n      allocated_end_(static_cast<char*>(allocated_begin_) +\n                     (*raw_capacity - kInternalAllocatedOffset())) {\n  RIEGELI_ASSERT(is_internal()) << \"A RawBlock with allocated_end_ != nullptr \"\n                                   \"should be considered internal\";\n  RIEGELI_ASSERT_LE(capacity(), RawBlock::kMaxCapacity)\n      << \"Chain block capacity overflow\";\n}\n\ninline IntrusiveSharedPtr<Chain::RawBlock> Chain::RawBlock::Copy() {\n  IntrusiveSharedPtr<RawBlock> block = NewInternal(size());\n  block->Append(*this);\n  RIEGELI_ASSERT(!block->wasteful())\n      << \"A full block should not be considered wasteful\";\n  return block;\n}\n\ninline size_t Chain::RawBlock::space_before() const {\n  RIEGELI_ASSERT(is_internal())\n      << \"Failed precondition of Chain::RawBlock::space_before(): \"\n         \"block not internal\";\n  return PtrDistance(allocated_begin_, data_begin());\n}\n\ninline size_t Chain::RawBlock::space_after() const {\n  RIEGELI_ASSERT(is_internal())\n      << \"Failed precondition of Chain::RawBlock::space_after(): \"\n         \"block not internal\";\n  return PtrDistance(data_end(), allocated_end_);\n}\n\ninline bool Chain::RawBlock::tiny(size_t extra_size) const {\n  if (is_internal()) {\n    RIEGELI_ASSERT_LE(size(), capacity())\n        << \"Failed invariant of Chain::RawBlock: size greater than capacity\";\n    RIEGELI_ASSERT_LE(extra_size, capacity() - size())\n        << \"Failed precondition of Chain::RawBlock::tiny(): \"\n           \"extra size greater than remaining space\";\n  } else {\n    RIEGELI_ASSERT_EQ(extra_size, 0u)\n        << \"Failed precondition of Chain::RawBlock::tiny(): \"\n           \"non-zero extra size of external block\";\n  }\n  return size() + extra_size < kDefaultMinBlockSize;\n}\n\ninline bool Chain::RawBlock::wasteful(size_t extra_size) const {\n  if (is_internal()) {\n    RIEGELI_ASSERT_LE(size(), capacity())\n        << \"Failed invariant of Chain::RawBlock: size greater than capacity\";\n    RIEGELI_ASSERT_LE(extra_size, capacity() - size())\n        << \"Failed precondition of Chain::RawBlock::wasteful(): \"\n           \"extra size greater than remaining space\";\n  } else {\n    RIEGELI_ASSERT_EQ(extra_size, 0u)\n        << \"Failed precondition of Chain::RawBlock::wasteful(): \"\n           \"non-zero extra size of external block\";\n    return false;\n  }\n  return Wasteful(kInternalAllocatedOffset() + capacity(), size() + extra_size);\n}\n\ninline void Chain::RawBlock::DumpStructure(std::ostream& dest) const {\n  dest << \"block {\";\n  const size_t ref_count = ref_count_.GetCount();\n  if (ref_count != 1) dest << \" ref_count: \" << ref_count;\n  dest << \" size: \" << size();\n  if (is_internal()) {\n    if (space_before() > 0) dest << \" space_before: \" << space_before();\n    dest << \" space_after: \" << space_after();\n  } else {\n    dest << \" \";\n    external_.methods->dump_structure(*this, dest);\n  }\n  dest << \" }\";\n}\n\nsize_t Chain::RawBlock::DynamicSizeOf() const {\n  if (is_internal()) {\n    return kInternalAllocatedOffset() + capacity();\n  } else {\n    return external_.methods->dynamic_sizeof;\n  }\n}\n\nvoid Chain::RawBlock::RegisterSubobjects(\n    MemoryEstimator& memory_estimator) const {\n  if (!is_internal()) {\n    external_.methods->register_subobjects(this, memory_estimator);\n  }\n}\n\ninline bool Chain::RawBlock::can_append(size_t length) const {\n  return is_mutable() && (empty() ? capacity() : space_after()) >= length;\n}\n\ninline bool Chain::RawBlock::can_prepend(size_t length) const {\n  return is_mutable() && (empty() ? capacity() : space_before()) >= length;\n}\n\ninline absl::Span<char> Chain::RawBlock::AppendBuffer(size_t max_length)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT(is_mutable())\n      << \"Failed precondition of Chain::RawBlock::AppendBuffer(): \"\n         \"block is immutable\";\n  if (empty()) substr_ = absl::string_view(allocated_begin_, 0);\n  const size_t length = UnsignedMin(space_after(), max_length);\n  const absl::Span<char> buffer(const_cast<char*>(data_end()), length);\n  substr_ = absl::string_view(data_begin(), size() + length);\n  return buffer;\n}\n\ninline absl::Span<char> Chain::RawBlock::PrependBuffer(size_t max_length)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT(is_mutable())\n      << \"Failed precondition of Chain::RawBlock::PrependBuffer(): \"\n         \"block is immutable\";\n  if (empty()) substr_ = absl::string_view(allocated_end_, 0);\n  const size_t length = UnsignedMin(space_before(), max_length);\n  const absl::Span<char> buffer(const_cast<char*>(data_begin()) - length,\n                                length);\n  substr_ = absl::string_view(data_begin() - length, size() + length);\n  return buffer;\n}\n\ninline void Chain::RawBlock::Append(absl::string_view src,\n                                    size_t space_before) {\n  if (empty()) {\n    // Redundant cast is needed for `-fsanitize=bounds`.\n    substr_ = absl::string_view(\n        static_cast<char*>(allocated_begin_) + space_before, 0);\n  }\n  AppendWithExplicitSizeToCopy(src, src.size());\n}\n\ninline void Chain::RawBlock::AppendWithExplicitSizeToCopy(absl::string_view src,\n                                                          size_t size_to_copy) {\n  RIEGELI_ASSERT_GE(size_to_copy, src.size())\n      << \"Failed precondition of \"\n         \"Chain::RawBlock::AppendWithExplicitSizeToCopy(): \"\n         \"size to copy too small\";\n  RIEGELI_ASSERT(can_append(size_to_copy))\n      << \"Failed precondition of \"\n         \"Chain::RawBlock::AppendWithExplicitSizeToCopy(): \"\n         \"not enough space\";\n  std::memcpy(const_cast<char*>(data_end()), src.data(), size_to_copy);\n  substr_ = absl::string_view(data_begin(), size() + src.size());\n}\n\ninline void Chain::RawBlock::Prepend(absl::string_view src,\n                                     size_t space_after) {\n  RIEGELI_ASSERT(can_prepend(src.size()))\n      << \"Failed precondition of Chain::RawBlock::Prepend(): \"\n         \"not enough space\";\n  if (empty()) substr_ = absl::string_view(allocated_end_ - space_after, 0);\n  std::memcpy(const_cast<char*>(data_begin() - src.size()), src.data(),\n              src.size());\n  substr_ = absl::string_view(data_begin() - src.size(), size() + src.size());\n}\n\nsize_t Chain::BlockIterator::CharIndexInChainInternal() const {\n  if (ptr_ == kBeginShortData) {\n    return 0;\n  } else if (ptr_ == kEndShortData ||\n             ptr_ == BlockPtrPtr::from_ptr(chain_->end_)) {\n    return chain_->size();\n  } else if (chain_->has_here()) {\n    switch (block_index()) {\n      case 0:\n        return 0;\n      case 1:\n        return chain_->begin_[0].block_ptr->size();\n      default:\n        RIEGELI_ASSUME_UNREACHABLE()\n            << \"Failed invariant of Chain: \"\n               \"only two block pointers fit without allocating their array\";\n    }\n  } else {\n    const size_t offset_base =\n        chain_->begin_[chain_->block_offsets()].block_offset;\n    return ptr_.as_ptr()[chain_->block_offsets()].block_offset - offset_base;\n  }\n}\n\nChain::Block Chain::Block::ToChainBlock(absl::string_view substr) && {\n  if (substr.size() == block_->size()) return std::move(*this);\n  return Block(std::move(*this), substr);\n}\n\nabsl::Cord Chain::Block::ToCord(absl::string_view substr) && {\n  if (const FlatCordBlock* const cord_ptr =\n          block_->checked_external_object<FlatCordBlock>()) {\n    if (substr.size() == cord_ptr->src().size()) return cord_ptr->src();\n    return cord_ptr->src().Subcord(\n        PtrDistance(absl::string_view(*cord_ptr).data(), substr.data()),\n        substr.size());\n  }\n  return absl::MakeCordFromExternal(substr, [block = std::move(block_)] {});\n}\n\nabsl::Cord Chain::Block::ToCord(absl::string_view substr) const& {\n  if (const FlatCordBlock* const cord_ptr =\n          block_->checked_external_object<FlatCordBlock>()) {\n    if (substr.size() == cord_ptr->src().size()) return cord_ptr->src();\n    return cord_ptr->src().Subcord(\n        PtrDistance(absl::string_view(*cord_ptr).data(), substr.data()),\n        substr.size());\n  }\n  return absl::MakeCordFromExternal(substr, [block = block_] {});\n}\n\nvoid Chain::Block::DumpStructure(absl::string_view substr,\n                                 std::ostream& dest) const {\n  dest << \"[block] { offset: \"\n       << PtrDistance(block_->data_begin(), substr.data()) << \" \";\n  block_->DumpStructure(dest);\n  dest << \" }\";\n}\n\nChain::Chain(const absl::Cord& src) { Initialize(src); }\n\nChain::Chain(absl::Cord&& src) { Initialize(std::move(src)); }\n\nChain::Chain(const Chain& that) { Initialize(that); }\n\nChain& Chain::operator=(const Chain& that) {\n  if (ABSL_PREDICT_TRUE(&that != this)) {\n    UnrefBlocks();\n    Initialize(that);\n  }\n  return *this;\n}\n\nbool Chain::ClearSlow() {\n  RIEGELI_ASSERT_NE(begin_, end_)\n      << \"Failed precondition of Chain::ClearSlow(): \"\n         \"no blocks, use Clear() instead\";\n  const bool block_remains = front()->TryClear();\n  BlockPtr* const new_end = begin_ + (block_remains ? 1 : 0);\n  UnrefBlocks(new_end, end_);\n  end_ = new_end;\n  return block_remains;\n}\n\nvoid Chain::Reset(BytesRef src) {\n  size_ = 0;\n  if (begin_ != end_ && ClearSlow()) {\n    Append(src, Options().set_size_hint(src.size()));\n    return;\n  }\n  Initialize(src);\n}\n\nvoid Chain::Reset(Block src) {\n  size_ = 0;\n  UnrefBlocks();\n  end_ = begin_;\n  if (src.raw_block() != nullptr) Initialize(std::move(src));\n}\n\nvoid Chain::Reset(const absl::Cord& src) {\n  size_ = 0;\n  if (begin_ != end_ && ClearSlow()) {\n    Append(src, Options().set_size_hint(src.size()));\n    return;\n  }\n  Initialize(src);\n}\n\nvoid Chain::Reset(absl::Cord&& src) {\n  size_ = 0;\n  if (begin_ != end_ && ClearSlow()) {\n    const size_t size = src.size();\n    Append(std::move(src), Options().set_size_hint(size));\n    return;\n  }\n  Initialize(std::move(src));\n}\n\nvoid Chain::InitializeSlow(absl::string_view src) {\n  RIEGELI_ASSERT_GT(src.size(), kMaxShortDataSize)\n      << \"Failed precondition of Chain::InitializeSlow(string_view): \"\n         \"string too short, use Initialize() instead\";\n  IntrusiveSharedPtr<RawBlock> block =\n      RawBlock::NewInternal(UnsignedMin(src.size(), kDefaultMaxBlockSize));\n  const absl::Span<char> buffer = block->AppendBuffer(src.size());\n  std::memcpy(buffer.data(), src.data(), buffer.size());\n  Initialize(Block(std::move(block)));\n  Options options;\n  options.set_size_hint(src.size());\n  src.remove_prefix(buffer.size());\n  Append(src, options);\n}\n\ninline void Chain::Initialize(const absl::Cord& src) {\n  RIEGELI_ASSERT_EQ(size_, 0u)\n      << \"Failed precondition of Chain::Initialize(const Cord&): \"\n         \"size not reset\";\n  InitializeFromCord(src);\n}\n\ninline void Chain::Initialize(absl::Cord&& src) {\n  RIEGELI_ASSERT_EQ(size_, 0u)\n      << \"Failed precondition of Chain::Initialize(absl::Cord&&): \"\n         \"size not reset\";\n  InitializeFromCord(std::move(src));\n}\n\ntemplate <typename CordRef>\ninline void Chain::InitializeFromCord(CordRef&& src) {\n  if (const std::optional<absl::string_view> flat = src.TryFlat();\n      flat != std::nullopt) {\n    if (flat->size() <= kMaxBytesToCopyToEmpty) {\n      Initialize(*flat);\n    } else {\n      Initialize(\n          Block(riegeli::Maker<FlatCordBlock>(std::forward<CordRef>(src))));\n    }\n    return;\n  }\n  AppendCordSlow(std::forward<CordRef>(src),\n                 Options().set_size_hint(src.size()));\n}\n\ninline void Chain::Initialize(const Chain& src) {\n  size_ = src.size_;\n  end_ = begin_;\n  if (src.begin_ == src.end_) {\n    EnsureHasHere();\n    std::memcpy(short_data_begin(), src.short_data_begin(), kMaxShortDataSize);\n  } else {\n    AppendBlocks<ShareOwnership>(src.begin_, src.end_);\n  }\n}\n\ninline std::string Chain::ToString() const {\n  if (begin_ == end_) return std::string(short_data());\n  std::string dest;\n  absl::StringResizeAndOverwrite(dest, size_, [&](char* data, size_t size) {\n    CopyToSlow(data);\n    return size;\n  });\n  return dest;\n}\n\nabsl::string_view Chain::FlattenSlow() {\n  RIEGELI_ASSERT_GT(end_ - begin_, 1)\n      << \"Failed precondition of Chain::FlattenSlow(): \"\n         \"contents already flat, use Flatten() instead\";\n  if (front()->empty()) {\n    PopFront();\n    if (end_ - begin_ == 1) return *front();\n  }\n  if (back()->empty()) {\n    PopBack();\n    if (end_ - begin_ == 1) return *back();\n  }\n  IntrusiveSharedPtr<RawBlock> block =\n      RawBlock::NewInternal(NewBlockCapacity(0, size_, size_, Options()));\n  const BlockPtr* iter = begin_;\n  do {\n    block->Append(*iter->block_ptr);\n    ++iter;\n  } while (iter != end_);\n  UnrefBlocks(begin_, end_);\n  end_ = begin_;\n  PushBack(std::move(block));\n  return *back();\n}\n\ninline Chain::BlockPtr* Chain::NewBlockPtrs(size_t capacity) {\n  return std::allocator<BlockPtr>().allocate(2 * capacity);\n}\n\nvoid Chain::UnrefBlocksSlow(const BlockPtr* begin, const BlockPtr* end) {\n  RIEGELI_ASSERT_LT(begin, end)\n      << \"Failed precondition of Chain::UnrefBlocksSlow(): \"\n         \"no blocks, use UnrefBlocks() instead\";\n  do {\n    (begin++)->block_ptr->Unref();\n  } while (begin != end);\n}\n\ninline void Chain::DropPassedBlocks(PassOwnership) {\n  size_ = 0;\n  end_ = begin_;\n}\n\ninline void Chain::DropPassedBlocks(ShareOwnership) const {}\n\nvoid Chain::CopyTo(char* dest) const {\n  if (begin_ == end_) {\n    riegeli::null_safe_memcpy(dest, short_data_begin(), size_);\n    return;\n  }\n  CopyToSlow(dest);\n}\n\ninline void Chain::CopyToSlow(char* dest) const {\n  RIEGELI_ASSERT_NE(begin_, end_)\n      << \"Failed precondition of Chain::CopyToSlow(): \"\n         \"no blocks, use CopyTo() instead\";\n  const BlockPtr* iter = begin_;\n  do {\n    std::memcpy(dest, iter->block_ptr->data_begin(), iter->block_ptr->size());\n    dest += iter->block_ptr->size();\n    ++iter;\n  } while (iter != end_);\n}\n\nvoid Chain::AppendTo(std::string& dest) const& {\n  const size_t old_size = dest.size();\n  RIEGELI_CHECK_LE(size_, std::numeric_limits<size_t>::max() - old_size)\n      << \"Failed precondition of Chain::AppendTo(string&): \"\n         \"string size overflow\";\n  riegeli::StringResizeAndOverwriteAmortized(dest, old_size + size_,\n                                             [&](char* data, size_t size) {\n                                               CopyTo(data + old_size);\n                                               return size;\n                                             });\n}\n\nvoid Chain::AppendTo(std::string& dest) && {\n  if (dest.empty() && PtrDistance(begin_, end_) == 1) {\n    if (std::string* const string_ptr =\n            back()->checked_external_object_with_unique_owner<std::string>()) {\n      RIEGELI_ASSERT_EQ(back()->size(), string_ptr->size())\n          << \"Failed invariant of Chain::RawBlock: \"\n             \"block size differs from string size\";\n      if (dest.capacity() <= string_ptr->capacity()) {\n        dest = std::move(*string_ptr);\n        size_ = 0;\n        PopBack();\n        return;\n      }\n    }\n  }\n  const size_t old_size = dest.size();\n  RIEGELI_CHECK_LE(size_, std::numeric_limits<size_t>::max() - old_size)\n      << \"Failed precondition of Chain::AppendTo(string&): \"\n         \"string size overflow\";\n  riegeli::StringResizeAndOverwriteAmortized(dest, old_size + size_,\n                                             [&](char* data, size_t size) {\n                                               CopyTo(data + old_size);\n                                               return size;\n                                             });\n}\n\nvoid Chain::AppendTo(absl::Cord& dest) const& {\n  RIEGELI_CHECK_LE(size_, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Chain::AppendTo(Cord&): Cord size overflow\";\n  if (begin_ == end_) {\n    dest.Append(short_data());\n    return;\n  }\n  AppendToSlow(dest);\n}\n\nvoid Chain::AppendTo(absl::Cord& dest) && {\n  RIEGELI_CHECK_LE(size_, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Chain::AppendTo(Cord&): Cord size overflow\";\n  if (begin_ == end_) {\n    dest.Append(short_data());\n    return;\n  }\n  std::move(*this).AppendToSlow(dest);\n}\n\ninline void Chain::AppendToSlow(absl::Cord& dest) const& {\n  RIEGELI_ASSERT_NE(begin_, end_)\n      << \"Failed precondition of Chain::AppendToSlow(Cord&): \"\n         \"no blocks, use AppendTo() instead\";\n  const BlockPtr* iter = begin_;\n  do {\n    ExternalRef(riegeli::Invoker(MakeBlock(), iter->block_ptr),\n                *iter->block_ptr)\n        .AppendTo(dest);\n    ++iter;\n  } while (iter != end_);\n}\n\ninline void Chain::AppendToSlow(absl::Cord& dest) && {\n  RIEGELI_ASSERT_NE(begin_, end_)\n      << \"Failed precondition of Chain::AppendToSlow(Cord&): \"\n         \"no blocks, use AppendTo() instead\";\n  size_ = 0;\n  const BlockPtr* iter = begin_;\n  do {\n    ExternalRef(riegeli::Invoker(MakeBlock(),\n                                 IntrusiveSharedPtr<RawBlock>(iter->block_ptr)),\n                *iter->block_ptr)\n        .AppendTo(dest);\n    ++iter;\n  } while (iter != end_);\n  end_ = begin_;\n}\n\nvoid Chain::PrependTo(absl::Cord& dest) const& {\n  RIEGELI_CHECK_LE(size_, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Chain::PrependTo(Cord&): Cord size overflow\";\n  if (begin_ == end_) {\n    dest.Prepend(short_data());\n    return;\n  }\n  PrependToSlow(dest);\n}\n\nvoid Chain::PrependTo(absl::Cord& dest) && {\n  RIEGELI_CHECK_LE(size_, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Chain::PrependTo(Cord&): Cord size overflow\";\n  if (begin_ == end_) {\n    dest.Prepend(short_data());\n    return;\n  }\n  std::move(*this).PrependToSlow(dest);\n}\n\ninline void Chain::PrependToSlow(absl::Cord& dest) const& {\n  RIEGELI_ASSERT_NE(end_, begin_)\n      << \"Failed precondition of Chain::PrependToSlow(Cord&): \"\n         \"no blocks, use PrependTo() instead\";\n  const BlockPtr* iter = end_;\n  do {\n    --iter;\n    ExternalRef(riegeli::Invoker(MakeBlock(), iter->block_ptr),\n                *iter->block_ptr)\n        .PrependTo(dest);\n  } while (iter != begin_);\n}\n\ninline void Chain::PrependToSlow(absl::Cord& dest) && {\n  RIEGELI_ASSERT_NE(end_, begin_)\n      << \"Failed precondition of Chain::PrependToSlow(Cord&): \"\n         \"no blocks, use PrependTo() instead\";\n  const BlockPtr* iter = end_;\n  size_ = 0;\n  do {\n    --iter;\n    ExternalRef(riegeli::Invoker(MakeBlock(),\n                                 IntrusiveSharedPtr<RawBlock>(iter->block_ptr)),\n                *iter->block_ptr)\n        .PrependTo(dest);\n  } while (iter != begin_);\n  end_ = begin_;\n}\n\nChain::operator std::string() const& { return ToString(); }\n\nChain::operator std::string() && {\n  if (PtrDistance(begin_, end_) == 1) {\n    if (std::string* const string_ptr =\n            back()->checked_external_object_with_unique_owner<std::string>()) {\n      RIEGELI_ASSERT_EQ(back()->size(), string_ptr->size())\n          << \"Failed invariant of Chain::RawBlock: \"\n             \"block size differs from string size\";\n      const std::string dest = std::move(*string_ptr);\n      size_ = 0;\n      PopBack();\n      return dest;\n    }\n  }\n  return ToString();\n}\n\nChain::operator absl::Cord() const& {\n  if (begin_ == end_) return absl::Cord(short_data());\n  absl::Cord dest;\n  AppendToSlow(dest);\n  return dest;\n}\n\nChain::operator absl::Cord() && {\n  if (begin_ == end_) return absl::Cord(short_data());\n  absl::Cord dest;\n  std::move(*this).AppendToSlow(dest);\n  return dest;\n}\n\nChain::BlockAndChar Chain::BlockAndCharIndex(size_t char_index_in_chain) const {\n  RIEGELI_ASSERT_LE(char_index_in_chain, size())\n      << \"Failed precondition of Chain::BlockAndCharIndex(): \"\n         \"position out of range\";\n  if (char_index_in_chain == size()) {\n    return BlockAndChar{blocks().cend(), 0};\n  } else if (begin_ == end_) {\n    return BlockAndChar{blocks().cbegin(), char_index_in_chain};\n  } else if (has_here()) {\n    BlockIterator block_iter = blocks().cbegin();\n    if (char_index_in_chain >= block_iter->size()) {\n      char_index_in_chain -= block_iter->size();\n      ++block_iter;\n      RIEGELI_ASSERT_LT(char_index_in_chain, block_iter->size())\n          << \"Failed invariant of Chain: \"\n             \"only two block pointers fit without allocating their array\";\n    }\n    return BlockAndChar{block_iter, char_index_in_chain};\n  } else {\n    const size_t offset_base = begin_[block_offsets()].block_offset;\n    const BlockPtr* const found =\n        std::upper_bound(begin_ + block_offsets() + 1, end_ + block_offsets(),\n                         char_index_in_chain,\n                         [&](size_t value, BlockPtr element) {\n                           return value < element.block_offset - offset_base;\n                         }) -\n        1;\n    return BlockAndChar{\n        BlockIterator(this, PtrDistance(begin_ + block_offsets(), found)),\n        char_index_in_chain - (found->block_offset - offset_base)};\n  }\n}\n\nvoid Chain::DumpStructure(std::ostream& dest) const {\n  dest << \"chain {\\n  size: \" << size_\n       << \" memory: \" << riegeli::EstimateMemory(*this);\n  for (const BlockPtr* iter = begin_; iter != end_; ++iter) {\n    dest << \"\\n  \";\n    iter->block_ptr->DumpStructure(dest);\n  }\n  dest << \"\\n}\\n\";\n}\n\nvoid Chain::RegisterSubobjects(MemoryEstimator& memory_estimator) const {\n  if (has_allocated()) {\n    memory_estimator.RegisterMemory(\n        2 *\n        PtrDistance(block_ptrs_.allocated.begin, block_ptrs_.allocated.end) *\n        sizeof(BlockPtr));\n  }\n  for (const BlockPtr* iter = begin_; iter != end_; ++iter) {\n    if (memory_estimator.RegisterNode(iter->block_ptr)) {\n      memory_estimator.RegisterDynamicObject(iter->block_ptr);\n    }\n  }\n}\n\ninline IntrusiveSharedPtr<Chain::RawBlock> Chain::SetBack(\n    IntrusiveSharedPtr<RawBlock> block) {\n  return IntrusiveSharedPtr<RawBlock>(\n      std::exchange(end_[-1].block_ptr, block.Release()));\n  // There is no need to adjust block offsets because the size of the last block\n  // is not reflected in block offsets.\n}\n\ninline IntrusiveSharedPtr<Chain::RawBlock> Chain::SetFront(\n    IntrusiveSharedPtr<RawBlock> block) {\n  IntrusiveSharedPtr<RawBlock> old_block = SetFrontSameSize(std::move(block));\n  RefreshFront();\n  return old_block;\n}\n\ninline IntrusiveSharedPtr<Chain::RawBlock> Chain::SetFrontSameSize(\n    IntrusiveSharedPtr<RawBlock> block) {\n  return IntrusiveSharedPtr<RawBlock>(\n      std::exchange(begin_[0].block_ptr, block.Release()));\n}\n\ninline void Chain::RefreshFront() {\n  if (has_allocated()) {\n    begin_[block_offsets()].block_offset =\n        begin_ + 1 == end_ ? size_t{0}\n                           : begin_[block_offsets() + 1].block_offset -\n                                 begin_[0].block_ptr->size();\n  }\n}\n\ninline void Chain::PushBack(IntrusiveSharedPtr<RawBlock> block) {\n  ReserveBack(1);\n  end_[0].block_ptr = block.Release();\n  if (has_allocated()) {\n    end_[block_offsets()].block_offset =\n        begin_ == end_ ? size_t{0}\n                       : end_[block_offsets() - 1].block_offset +\n                             end_[-1].block_ptr->size();\n  }\n  ++end_;\n}\n\ninline void Chain::PushFront(IntrusiveSharedPtr<RawBlock> block) {\n  ReserveFront(1);\n  BlockPtr* const old_begin = begin_;\n  --begin_;\n  begin_[0].block_ptr = block.Release();\n  if (has_allocated()) {\n    begin_[block_offsets()].block_offset =\n        old_begin == end_ ? size_t{0}\n                          : begin_[block_offsets() + 1].block_offset -\n                                begin_[0].block_ptr->size();\n  }\n}\n\ninline IntrusiveSharedPtr<Chain::RawBlock> Chain::PopBack() {\n  RIEGELI_ASSERT_NE(begin_, end_)\n      << \"Failed precondition of Chain::PopBack(): no blocks\";\n  --end_;\n  return IntrusiveSharedPtr<RawBlock>(end_[0].block_ptr);\n}\n\ninline IntrusiveSharedPtr<Chain::RawBlock> Chain::PopFront() {\n  RIEGELI_ASSERT_NE(begin_, end_)\n      << \"Failed precondition of Chain::PopFront(): no blocks\";\n  if (has_here()) {\n    // Shift the remaining 0 or 1 block pointers to the left by 1 because\n    // `begin_` must remain at `block_ptrs_.here`. There might be no pointer to\n    // copy; it is more efficient to copy the array slot unconditionally.\n    IntrusiveSharedPtr<RawBlock> block(\n        std::exchange(block_ptrs_.here[0], block_ptrs_.here[1]).block_ptr);\n    --end_;\n    return block;\n  } else {\n    ++begin_;\n    return IntrusiveSharedPtr<RawBlock>(begin_[-1].block_ptr);\n  }\n}\n\ntemplate <typename Ownership>\ninline void Chain::AppendBlocks(const BlockPtr* begin, const BlockPtr* end) {\n  if (begin == end) return;\n  ReserveBack(PtrDistance(begin, end));\n  BlockPtr* dest_iter = end_;\n  dest_iter->block_ptr = begin->block_ptr->Ref<Ownership>();\n  if (has_allocated()) {\n    const size_t offsets = block_offsets();\n    size_t offset = begin_ == end_ ? size_t{0}\n                                   : dest_iter[offsets - 1].block_offset +\n                                         dest_iter[-1].block_ptr->size();\n    dest_iter[offsets].block_offset = offset;\n    ++begin;\n    ++dest_iter;\n    while (begin != end) {\n      dest_iter->block_ptr = begin->block_ptr->Ref<Ownership>();\n      offset += dest_iter[-1].block_ptr->size();\n      dest_iter[offsets].block_offset = offset;\n      ++begin;\n      ++dest_iter;\n    }\n  } else {\n    ++begin;\n    ++dest_iter;\n    if (begin != end) {\n      dest_iter->block_ptr = begin->block_ptr->Ref<Ownership>();\n      ++begin;\n      ++dest_iter;\n      RIEGELI_ASSERT_EQ(begin, end)\n          << \"Failed invariant of Chain: \"\n             \"only two block pointers fit without allocating their array\";\n    }\n  }\n  end_ = dest_iter;\n}\n\ntemplate <typename Ownership>\ninline void Chain::PrependBlocks(const BlockPtr* begin, const BlockPtr* end) {\n  if (begin == end) return;\n  ReserveFront(PtrDistance(begin, end));\n  BlockPtr* dest_iter = begin_;\n  BlockPtr* const old_begin = begin_;\n  begin_ -= PtrDistance(begin, end);  // For `has_allocated()` to work.\n  --end;\n  --dest_iter;\n  dest_iter->block_ptr = end->block_ptr->Ref<Ownership>();\n  if (has_allocated()) {\n    const size_t offsets = block_offsets();\n    size_t offset = old_begin == end_ ? size_t{0}\n                                      : dest_iter[offsets + 1].block_offset -\n                                            dest_iter->block_ptr->size();\n    dest_iter[offsets].block_offset = offset;\n    while (end != begin) {\n      --end;\n      --dest_iter;\n      dest_iter->block_ptr = end->block_ptr->Ref<Ownership>();\n      offset -= dest_iter->block_ptr->size();\n      dest_iter[offsets].block_offset = offset;\n    }\n  } else {\n    if (end != begin) {\n      --end;\n      --dest_iter;\n      dest_iter->block_ptr = end->block_ptr->Ref<Ownership>();\n      RIEGELI_ASSERT_EQ(begin, end)\n          << \"Failed invariant of Chain: \"\n             \"only two block pointers fit without allocating their array\";\n    }\n  }\n}\n\ninline void Chain::ReserveBack(size_t extra_capacity) {\n  BlockPtr* const allocated_end =\n      has_here() ? block_ptrs_.here + 2 : block_ptrs_.allocated.end;\n  if (ABSL_PREDICT_FALSE(extra_capacity > PtrDistance(end_, allocated_end))) {\n    // The slow path is in a separate function to make easier for the compiler\n    // to make good inlining decisions.\n    ReserveBackSlow(extra_capacity);\n  }\n}\n\ninline void Chain::ReserveFront(size_t extra_capacity) {\n  BlockPtr* const allocated_begin =\n      has_here() ? block_ptrs_.here : block_ptrs_.allocated.begin;\n  if (ABSL_PREDICT_FALSE(extra_capacity >\n                         PtrDistance(allocated_begin, begin_))) {\n    // The slow path is in a separate function to make easier for the compiler\n    // to make good inlining decisions.\n    ReserveFrontSlow(extra_capacity);\n  }\n}\n\ninline void Chain::ReserveBackSlow(size_t extra_capacity) {\n  RIEGELI_ASSERT_GT(extra_capacity, 0u)\n      << \"Failed precondition of Chain::ReserveBackSlow(): \"\n         \"nothing to do, use ReserveBack() instead\";\n  BlockPtr* old_allocated_begin;\n  BlockPtr* old_allocated_end;\n  if (has_here()) {\n    old_allocated_begin = block_ptrs_.here;\n    old_allocated_end = block_ptrs_.here + 2;\n  } else {\n    old_allocated_begin = block_ptrs_.allocated.begin;\n    old_allocated_end = block_ptrs_.allocated.end;\n  }\n  RIEGELI_ASSERT_GT(extra_capacity, PtrDistance(end_, old_allocated_end))\n      << \"Failed precondition of Chain::ReserveBackSlow(): \"\n         \"extra capacity fits in allocated space, use ReserveBack() instead\";\n  RIEGELI_ASSERT_LE(extra_capacity, std::numeric_limits<size_t>::max() /\n                                            (2 * sizeof(BlockPtr)) -\n                                        PtrDistance(old_allocated_begin, end_))\n      << \"Failed invariant of Chain: array of block pointers overflow, \"\n         \"possibly blocks are too small\";\n  const size_t old_capacity =\n      PtrDistance(old_allocated_begin, old_allocated_end);\n  const size_t size = PtrDistance(begin_, end_);\n  if (size + extra_capacity <= old_capacity && 2 * size <= old_capacity) {\n    RIEGELI_ASSERT(has_allocated())\n        << \"The case of has_here() if there is space without reallocation \"\n           \"was handled in ReserveBack()\";\n    // Existing array has enough capacity and is at most half full: move\n    // contents to the beginning of the array. This is enough to make the\n    // amortized cost of adding one element constant as long as prepending\n    // leaves space at both ends.\n    BlockPtr* const new_begin = old_allocated_begin;\n    // Moving left, so block pointers must be moved before block offsets.\n    std::memmove(new_begin, begin_, size * sizeof(BlockPtr));\n    std::memmove(new_begin + old_capacity, begin_ + old_capacity,\n                 size * sizeof(BlockPtr));\n    begin_ = new_begin;\n    end_ = new_begin + size;\n    return;\n  }\n  // Reallocate the array, without keeping space before the contents. This is\n  // enough to make the amortized cost of adding one element constant if\n  // prepending leaves space at both ends.\n  RIEGELI_ASSERT_LE(old_capacity / 2, std::numeric_limits<size_t>::max() /\n                                              (2 * sizeof(BlockPtr)) -\n                                          old_capacity)\n      << \"Failed invariant of Chain: array of block pointers overflow, \"\n         \"possibly blocks are too small\";\n  const size_t new_capacity =\n      UnsignedMax(PtrDistance(begin_, end_) + extra_capacity,\n                  old_capacity + old_capacity / 2, size_t{16});\n  BlockPtr* const new_allocated_begin = NewBlockPtrs(new_capacity);\n  BlockPtr* const new_allocated_end = new_allocated_begin + new_capacity;\n  BlockPtr* const new_begin = new_allocated_begin;\n  BlockPtr* const new_end = new_begin + size;\n  std::memcpy(new_begin, begin_, size * sizeof(BlockPtr));\n  if (has_allocated()) {\n    std::memcpy(new_begin + new_capacity, begin_ + old_capacity,\n                size * sizeof(BlockPtr));\n  } else if (size >= 1) {\n    RIEGELI_ASSERT_LE(size, 2u)\n        << \"Failed invariant of Chain: \"\n           \"only two block pointers fit without allocating their array\";\n    new_begin[new_capacity].block_offset = 0;\n    if (size == 2) {\n      new_begin[new_capacity + 1].block_offset = new_begin[0].block_ptr->size();\n    }\n  }\n  DeleteBlockPtrs();\n  block_ptrs_.allocated.begin = new_allocated_begin;\n  block_ptrs_.allocated.end = new_allocated_end;\n  begin_ = new_begin;\n  end_ = new_end;\n}\n\ninline void Chain::ReserveFrontSlow(size_t extra_capacity) {\n  RIEGELI_ASSERT_GT(extra_capacity, 0u)\n      << \"Failed precondition of Chain::ReserveFrontSlow(): \"\n         \"nothing to do, use ReserveFront() instead\";\n  BlockPtr* old_allocated_begin;\n  BlockPtr* old_allocated_end;\n  if (has_here()) {\n    if (ABSL_PREDICT_TRUE(extra_capacity <=\n                          PtrDistance(end_, block_ptrs_.here + 2))) {\n      // There is space without reallocation. Shift 1 block pointer to the right\n      // by 1, or 0 block pointers by 1 or 2, because `begin_` must remain at\n      // `block_ptrs_.here`. There might be no pointer to copy; it is cheaper to\n      // copy the array slot unconditionally.\n      block_ptrs_.here[1] = block_ptrs_.here[0];\n      begin_ += extra_capacity;\n      end_ += extra_capacity;\n      return;\n    }\n    old_allocated_begin = block_ptrs_.here;\n    old_allocated_end = end_;\n  } else {\n    old_allocated_begin = block_ptrs_.allocated.begin;\n    old_allocated_end = block_ptrs_.allocated.end;\n  }\n  RIEGELI_ASSERT_GT(extra_capacity, PtrDistance(old_allocated_begin, begin_))\n      << \"Failed precondition of Chain::ReserveFrontSlow(): \"\n         \"extra capacity fits in allocated space, use ReserveFront() instead\";\n  RIEGELI_ASSERT_LE(extra_capacity, std::numeric_limits<size_t>::max() /\n                                            (2 * sizeof(BlockPtr)) -\n                                        PtrDistance(begin_, old_allocated_end))\n      << \"Failed invariant of Chain: array of block pointers overflow, \"\n         \"possibly blocks are too small\";\n  const size_t old_capacity =\n      PtrDistance(old_allocated_begin, old_allocated_end);\n  const size_t size = PtrDistance(begin_, end_);\n  if (size + extra_capacity <= old_capacity && 2 * size <= old_capacity) {\n    RIEGELI_ASSERT(has_allocated())\n        << \"The case of has_here() if there is space without reallocation \"\n           \"was handled above\";\n    // Existing array has enough capacity and is at most half full: move\n    // contents to the middle of the array. This makes the amortized cost of\n    // adding one element constant.\n    BlockPtr* const new_begin =\n        old_allocated_begin + (old_capacity - size + extra_capacity) / 2;\n    // Moving right, so block offsets must be moved before block pointers.\n    std::memmove(new_begin + old_capacity, begin_ + old_capacity,\n                 size * sizeof(BlockPtr));\n    std::memmove(new_begin, begin_, size * sizeof(BlockPtr));\n    begin_ = new_begin;\n    end_ = new_begin + size;\n    return;\n  }\n  // Reallocate the array, keeping space after the contents unchanged. This\n  // makes the amortized cost of adding one element constant.\n  RIEGELI_ASSERT_LE(old_capacity / 2, std::numeric_limits<size_t>::max() /\n                                              (2 * sizeof(BlockPtr)) -\n                                          old_capacity)\n      << \"Failed invariant of Chain: array of block pointers overflow, \"\n         \"possibly blocks are too small\";\n  const size_t new_capacity =\n      UnsignedMax(PtrDistance(begin_, old_allocated_end) + extra_capacity,\n                  old_capacity + old_capacity / 2, size_t{16});\n  BlockPtr* const new_allocated_begin = NewBlockPtrs(new_capacity);\n  BlockPtr* const new_allocated_end = new_allocated_begin + new_capacity;\n  BlockPtr* const new_end =\n      new_allocated_end - PtrDistance(end_, old_allocated_end);\n  BlockPtr* const new_begin = new_end - size;\n  std::memcpy(new_begin, begin_, size * sizeof(BlockPtr));\n  if (has_allocated()) {\n    std::memcpy(new_begin + new_capacity, begin_ + old_capacity,\n                size * sizeof(BlockPtr));\n  } else if (size >= 1) {\n    RIEGELI_ASSERT_LE(size, 2u)\n        << \"Failed invariant of Chain: \"\n           \"only two block pointers fit without allocating their array\";\n    new_begin[new_capacity].block_offset = 0;\n    if (size == 2) {\n      new_begin[new_capacity + 1].block_offset = new_begin[0].block_ptr->size();\n    }\n  }\n  DeleteBlockPtrs();\n  block_ptrs_.allocated.begin = new_allocated_begin;\n  block_ptrs_.allocated.end = new_allocated_end;\n  begin_ = new_begin;\n  end_ = new_end;\n}\n\ninline size_t Chain::NewBlockCapacity(size_t replaced_length, size_t min_length,\n                                      size_t recommended_length,\n                                      Options options) const {\n  RIEGELI_ASSERT_LE(replaced_length, size_)\n      << \"Failed precondition of Chain::NewBlockCapacity(): \"\n         \"length to replace greater than current size\";\n  RIEGELI_ASSERT_LE(min_length, RawBlock::kMaxCapacity - replaced_length)\n      << \"Chain block capacity overflow\";\n  return replaced_length +\n         ApplyBufferConstraints(\n             ApplySizeHint(\n                 UnsignedMax(size_, SaturatingSub(options.min_block_size(),\n                                                  replaced_length)),\n                 options.size_hint(), size_),\n             min_length, recommended_length,\n             SaturatingSub(options.max_block_size(), replaced_length));\n}\n\nabsl::Span<char> Chain::AppendBuffer(size_t min_length,\n                                     size_t recommended_length,\n                                     size_t max_length, Options options) {\n  RIEGELI_ASSERT_LE(min_length, max_length)\n      << \"Failed precondition of Chain::AppendBuffer(): \"\n         \"min_length > max_length\";\n  RIEGELI_CHECK_LE(min_length, std::numeric_limits<size_t>::max() - size_)\n      << \"Failed precondition of Chain::AppendBuffer(): \"\n         \"Chain size overflow\";\n  if (begin_ == end_) {\n    RIEGELI_ASSERT_LE(size_, kMaxShortDataSize)\n        << \"Failed invariant of Chain: short data size too large\";\n    if (min_length <= kMaxShortDataSize - size_) {\n      // Do not bother returning short data if `recommended_length` or\n      // `size_hint` is larger, because data will likely need to be copied later\n      // to a real block.\n      if (recommended_length <= kMaxShortDataSize - size_ &&\n          (options.size_hint() == std::nullopt ||\n           *options.size_hint() <= kMaxShortDataSize)) {\n        // Append the new space to short data.\n        EnsureHasHere();\n        const absl::Span<char> buffer(\n            short_data_begin() + size_,\n            UnsignedMin(max_length, kMaxShortDataSize - size_));\n        size_ += buffer.size();\n        return buffer;\n      } else if (min_length == 0) {\n        return absl::Span<char>();\n      }\n    }\n    // Merge short data with the new space to a new block.\n    IntrusiveSharedPtr<RawBlock> block;\n    if (ABSL_PREDICT_FALSE(min_length > RawBlock::kMaxCapacity - size_)) {\n      block = RawBlock::NewInternal(kMaxShortDataSize);\n      block->AppendWithExplicitSizeToCopy(short_data(), kMaxShortDataSize);\n      PushBack(std::move(block));\n      block = RawBlock::NewInternal(\n          NewBlockCapacity(0, min_length, recommended_length, options));\n    } else {\n      block = RawBlock::NewInternal(NewBlockCapacity(\n          size_, UnsignedMax(min_length, kMaxShortDataSize - size_),\n          recommended_length, options));\n      block->AppendWithExplicitSizeToCopy(short_data(), kMaxShortDataSize);\n    }\n    PushBack(std::move(block));\n  } else {\n    if (back()->can_append(min_length)) {\n      // New space can be appended in place.\n    } else if (min_length == 0) {\n      return absl::Span<char>();\n    } else if (back()->tiny() &&\n               ABSL_PREDICT_TRUE(min_length <=\n                                 RawBlock::kMaxCapacity - back()->size())) {\n      // The last block must be rewritten. Merge it with the new space to a\n      // new block.\n      IntrusiveSharedPtr<RawBlock> block =\n          RawBlock::NewInternal(NewBlockCapacity(back()->size(), min_length,\n                                                 recommended_length, options));\n      block->Append(*back());\n      SetBack(std::move(block));\n    } else {\n      IntrusiveSharedPtr<RawBlock> block;\n      if (back()->wasteful()) {\n        // The last block must be rewritten. Rewrite it separately from the new\n        // block to avoid rewriting the same data again if the new block gets\n        // only partially filled.\n        IntrusiveSharedPtr<RawBlock> last = SetBack(back()->Copy());\n        if (last->TryClear() && last->can_append(min_length)) {\n          // Reuse this block.\n          block = std::move(last);\n        }\n      }\n      if (block == nullptr) {\n        // Append a new block.\n        block = RawBlock::NewInternal(\n            NewBlockCapacity(0, min_length, recommended_length, options));\n      }\n      PushBack(std::move(block));\n    }\n  }\n  const absl::Span<char> buffer = back()->AppendBuffer(\n      UnsignedMin(max_length, std::numeric_limits<size_t>::max() - size_));\n  RIEGELI_ASSERT_GE(buffer.size(), min_length)\n      << \"Chain::RawBlock::AppendBuffer() returned less than the free space\";\n  size_ += buffer.size();\n  return buffer;\n}\n\nabsl::Span<char> Chain::PrependBuffer(size_t min_length,\n                                      size_t recommended_length,\n                                      size_t max_length, Options options) {\n  RIEGELI_ASSERT_LE(min_length, max_length)\n      << \"Failed precondition of Chain::PrependBuffer(): \"\n         \"min_length > max_length\";\n  RIEGELI_CHECK_LE(min_length, std::numeric_limits<size_t>::max() - size_)\n      << \"Failed precondition of Chain::PrependBuffer(): \"\n         \"Chain size overflow\";\n  if (begin_ == end_) {\n    RIEGELI_ASSERT_LE(size_, kMaxShortDataSize)\n        << \"Failed invariant of Chain: short data size too large\";\n    if (min_length <= kMaxShortDataSize - size_) {\n      // Do not bother returning short data if `recommended_length` or\n      // `size_hint` is larger, because data will likely need to be copied later\n      // to a real block.\n      if (recommended_length <= kMaxShortDataSize - size_ &&\n          (options.size_hint() == std::nullopt ||\n           *options.size_hint() <= kMaxShortDataSize)) {\n        // Prepend the new space to short data.\n        EnsureHasHere();\n        const absl::Span<char> buffer(\n            short_data_begin(),\n            UnsignedMin(max_length, kMaxShortDataSize - size_));\n        std::memmove(buffer.data() + buffer.size(), short_data_begin(), size_);\n        size_ += buffer.size();\n        return buffer;\n      } else if (min_length == 0) {\n        return absl::Span<char>();\n      }\n    }\n    // Merge short data with the new space to a new block.\n    IntrusiveSharedPtr<RawBlock> block;\n    if (ABSL_PREDICT_FALSE(min_length > RawBlock::kMaxCapacity - size_)) {\n      block = RawBlock::NewInternal(kMaxShortDataSize);\n      block->AppendWithExplicitSizeToCopy(short_data(), kMaxShortDataSize);\n      PushFront(std::move(block));\n      block = RawBlock::NewInternal(\n          NewBlockCapacity(0, min_length, recommended_length, options));\n    } else {\n      block = RawBlock::NewInternal(\n          NewBlockCapacity(size_, min_length, recommended_length, options));\n      block->Prepend(short_data());\n    }\n    PushFront(std::move(block));\n  } else {\n    if (front()->can_prepend(min_length)) {\n      // New space can be prepended in place.\n    } else if (min_length == 0) {\n      return absl::Span<char>();\n    } else if (front()->tiny() &&\n               ABSL_PREDICT_TRUE(min_length <=\n                                 RawBlock::kMaxCapacity - front()->size())) {\n      // The first block must be rewritten. Merge it with the new space to a\n      // new block.\n      IntrusiveSharedPtr<RawBlock> block =\n          RawBlock::NewInternal(NewBlockCapacity(front()->size(), min_length,\n                                                 recommended_length, options));\n      block->Prepend(*front());\n      SetFront(std::move(block));\n    } else {\n      IntrusiveSharedPtr<RawBlock> block;\n      if (front()->wasteful()) {\n        // The first block must be rewritten. Rewrite it separately from the new\n        // block to avoid rewriting the same data again if the new block gets\n        // only partially filled.\n        IntrusiveSharedPtr<RawBlock> first = SetFrontSameSize(front()->Copy());\n        if (first->TryClear() && first->can_prepend(min_length)) {\n          // Reuse this block.\n          block = std::move(first);\n        }\n      }\n      if (block == nullptr) {\n        // Prepend a new block.\n        block = RawBlock::NewInternal(\n            NewBlockCapacity(0, min_length, recommended_length, options));\n      }\n      PushFront(std::move(block));\n    }\n  }\n  const absl::Span<char> buffer = front()->PrependBuffer(\n      UnsignedMin(max_length, std::numeric_limits<size_t>::max() - size_));\n  RIEGELI_ASSERT_GE(buffer.size(), min_length)\n      << \"Chain::RawBlock::PrependBuffer() returned less than the free space\";\n  RefreshFront();\n  size_ += buffer.size();\n  return buffer;\n}\n\nvoid Chain::Append(BytesRef src, Options options) {\n  while (!src.empty()) {\n    const absl::Span<char> buffer =\n        AppendBuffer(1, src.size(), src.size(), options);\n    std::memcpy(buffer.data(), src.data(), buffer.size());\n    src.remove_prefix(buffer.size());\n  }\n}\n\nvoid Chain::Append(const Chain& src, Options options) {\n  AppendChain<ShareOwnership>(src, options);\n}\n\nvoid Chain::Append(Chain&& src, Options options) {\n  AppendChain<PassOwnership>(std::move(src), options);\n}\n\ntemplate <typename Ownership, typename ChainRef>\ninline void Chain::AppendChain(ChainRef&& src, Options options) {\n  if (src.begin_ == src.end_) {\n    Append(src.short_data(), options);\n    return;\n  }\n  RIEGELI_CHECK_LE(src.size(), std::numeric_limits<size_t>::max() - size_)\n      << \"Failed precondition of Chain::Append(Chain): \"\n         \"Chain size overflow\";\n  const BlockPtr* src_iter = src.begin_;\n  // If the first block of `src` is handled specially,\n  // `(src_iter++)->block_ptr->Unref<Ownership>()` skips it so that\n  // `AppendBlocks<Ownership>()` does not append it again.\n  if (begin_ == end_) {\n    if (src.front()->tiny() ||\n        (src.end_ - src.begin_ > 1 && src.front()->wasteful())) {\n      // The first block of `src` must be rewritten. Merge short data with it to\n      // a new block.\n      if (!short_data().empty() || !src.front()->empty()) {\n        RIEGELI_ASSERT_LE(src.front()->size(), RawBlock::kMaxCapacity - size_)\n            << \"Sum of sizes of short data and a tiny or wasteful block \"\n               \"exceeds RawBlock::kMaxCapacity\";\n        const size_t capacity =\n            src.end_ - src.begin_ == 1\n                ? NewBlockCapacity(size_,\n                                   UnsignedMax(src.front()->size(),\n                                               kMaxShortDataSize - size_),\n                                   0, options)\n                : UnsignedMax(size_ + src.front()->size(), kMaxShortDataSize);\n        IntrusiveSharedPtr<RawBlock> merged = RawBlock::NewInternal(capacity);\n        merged->AppendWithExplicitSizeToCopy(short_data(), kMaxShortDataSize);\n        merged->Append(*src.front());\n        PushBack(std::move(merged));\n      }\n      (src_iter++)->block_ptr->Unref<Ownership>();\n    } else if (!empty()) {\n      // Copy short data to a real block.\n      IntrusiveSharedPtr<RawBlock> real =\n          RawBlock::NewInternal(kMaxShortDataSize);\n      real->AppendWithExplicitSizeToCopy(short_data(), kMaxShortDataSize);\n      PushBack(std::move(real));\n    }\n  } else {\n    if (back()->tiny() && src.front()->tiny()) {\n    merge:\n      // Boundary blocks must be merged, or they are both empty or wasteful so\n      // merging them is cheaper than rewriting them separately.\n      if (back()->empty() && src.front()->empty()) {\n        PopBack();\n      } else if (back()->can_append(src.front()->size()) &&\n                 (src.end_ - src.begin_ == 1 ||\n                  !back()->wasteful(src.front()->size()))) {\n        // Boundary blocks can be appended in place; this is always cheaper than\n        // merging them to a new block.\n        back()->Append(*src.front());\n      } else {\n        // Boundary blocks cannot be appended in place. Merge them to a new\n        // block.\n        RIEGELI_ASSERT_LE(src.front()->size(),\n                          RawBlock::kMaxCapacity - back()->size())\n            << \"Sum of sizes of two tiny or wasteful blocks exceeds \"\n               \"RawBlock::kMaxCapacity\";\n        const size_t capacity =\n            src.end_ - src.begin_ == 1\n                ? NewBlockCapacity(back()->size(), src.front()->size(), 0,\n                                   options)\n                : back()->size() + src.front()->size();\n        IntrusiveSharedPtr<RawBlock> merged = RawBlock::NewInternal(capacity);\n        merged->Append(*back());\n        merged->Append(*src.front());\n        SetBack(std::move(merged));\n      }\n      (src_iter++)->block_ptr->Unref<Ownership>();\n    } else if (back()->empty()) {\n      if (src.end_ - src.begin_ > 1 && src.front()->wasteful()) goto merge;\n      // The last block is empty and must be removed.\n      PopBack();\n    } else if (back()->wasteful()) {\n      if (src.end_ - src.begin_ > 1 &&\n          (src.front()->empty() || src.front()->wasteful())) {\n        goto merge;\n      }\n      // The last block must reduce waste.\n      if (back()->can_append(src.front()->size()) &&\n          (src.end_ - src.begin_ == 1 ||\n           !back()->wasteful(src.front()->size())) &&\n          src.front()->size() <= kAllocationCost + back()->size()) {\n        // Appending in place is possible and is cheaper than rewriting the last\n        // block.\n        back()->Append(*src.front());\n        (src_iter++)->block_ptr->Unref<Ownership>();\n      } else {\n        // Appending in place is not possible, or rewriting the last block is\n        // cheaper.\n        SetBack(back()->Copy());\n      }\n    } else if (src.end_ - src.begin_ > 1) {\n      if (src.front()->empty()) {\n        // The first block of `src` is empty and must be skipped.\n        (src_iter++)->block_ptr->Unref<Ownership>();\n      } else if (src.front()->wasteful()) {\n        // The first block of `src` must reduce waste.\n        if (back()->can_append(src.front()->size()) &&\n            !back()->wasteful(src.front()->size())) {\n          // Appending in place is possible; this is always cheaper than\n          // rewriting the first block of `src`.\n          back()->Append(*src.front());\n        } else {\n          // Appending in place is not possible.\n          PushBack(src.front()->Copy());\n        }\n        (src_iter++)->block_ptr->Unref<Ownership>();\n      }\n    }\n  }\n  size_ += src.size_;\n  AppendBlocks<Ownership>(src_iter, src.end_);\n  src.DropPassedBlocks(Ownership());\n}\n\nvoid Chain::Append(const Block& src, Options options) {\n  if (src.raw_block() != nullptr) AppendRawBlock(src.raw_block(), options);\n}\n\nvoid Chain::Append(Block&& src, Options options) {\n  if (src.raw_block() != nullptr) {\n    AppendRawBlock(std::move(src).raw_block(), options);\n  }\n}\n\ntemplate <typename RawBlockPtrRef>\ninline void Chain::AppendRawBlock(RawBlockPtrRef&& block, Options options) {\n  RIEGELI_CHECK_LE(block->size(), std::numeric_limits<size_t>::max() - size_)\n      << \"Failed precondition of Chain::Append(Block): \"\n         \"Chain size overflow\";\n  if (begin_ == end_) {\n    if (!short_data().empty()) {\n      if (block->tiny()) {\n        // The block must be rewritten. Merge short data with it to a new block.\n        RIEGELI_ASSERT_LE(block->size(), RawBlock::kMaxCapacity - size_)\n            << \"Sum of sizes of short data and a tiny block exceeds \"\n               \"RawBlock::kMaxCapacity\";\n        const size_t capacity = NewBlockCapacity(\n            size_, UnsignedMax(block->size(), kMaxShortDataSize - size_), 0,\n            options);\n        IntrusiveSharedPtr<RawBlock> merged = RawBlock::NewInternal(capacity);\n        merged->AppendWithExplicitSizeToCopy(short_data(), kMaxShortDataSize);\n        merged->Append(*block);\n        PushBack(std::move(merged));\n        size_ += block->size();\n        return;\n      }\n      // Copy short data to a real block.\n      IntrusiveSharedPtr<RawBlock> real =\n          RawBlock::NewInternal(kMaxShortDataSize);\n      real->AppendWithExplicitSizeToCopy(short_data(), kMaxShortDataSize);\n      PushBack(std::move(real));\n    }\n  } else {\n    if (back()->tiny() && block->tiny()) {\n      // Boundary blocks must be merged.\n      if (back()->can_append(block->size())) {\n        // Boundary blocks can be appended in place; this is always cheaper than\n        // merging them to a new block.\n        back()->Append(*block);\n      } else {\n        // Boundary blocks cannot be appended in place. Merge them to a new\n        // block.\n        RIEGELI_ASSERT_LE(block->size(),\n                          RawBlock::kMaxCapacity - back()->size())\n            << \"Sum of sizes of two tiny blocks exceeds RawBlock::kMaxCapacity\";\n        IntrusiveSharedPtr<RawBlock> merged = RawBlock::NewInternal(\n            NewBlockCapacity(back()->size(), block->size(), 0, options));\n        merged->Append(*back());\n        merged->Append(*block);\n        SetBack(std::move(merged));\n      }\n      size_ += block->size();\n      return;\n    }\n    if (back()->empty()) {\n      // The last block is empty and must be removed.\n      size_ += block->size();\n      SetBack(std::forward<RawBlockPtrRef>(block));\n      return;\n    }\n    if (back()->wasteful()) {\n      // The last block must reduce waste.\n      if (back()->can_append(block->size()) &&\n          block->size() <= kAllocationCost + back()->size()) {\n        // Appending in place is possible and is cheaper than rewriting the last\n        // block.\n        back()->Append(*block);\n        size_ += block->size();\n        return;\n      }\n      // Appending in place is not possible, or rewriting the last block is\n      // cheaper.\n      SetBack(back()->Copy());\n    }\n  }\n  size_ += block->size();\n  PushBack(std::forward<RawBlockPtrRef>(block));\n}\n\nvoid Chain::Append(const absl::Cord& src, Options options) {\n  AppendCord(src, options);\n}\n\nvoid Chain::Append(absl::Cord&& src, Options options) {\n  AppendCord(std::move(src), options);\n}\n\ntemplate <typename CordRef>\nvoid Chain::AppendCord(CordRef&& src, Options options) {\n  if (const std::optional<absl::string_view> flat = src.TryFlat();\n      flat != std::nullopt) {\n    if (flat->size() <= MaxBytesToCopy(options)) {\n      Append(*flat, options);\n    } else {\n      Append(Block(riegeli::Maker<FlatCordBlock>(std::forward<CordRef>(src))),\n             options);\n    }\n    return;\n  }\n  AppendCordSlow(std::forward<CordRef>(src), options);\n}\n\ntemplate <typename CordRef>\ninline void Chain::AppendCordSlow(CordRef&& src, Options options) {\n  // Avoid creating wasteful blocks and then rewriting them: append copied\n  // fragments when their accumulated size is known, tweaking `size_hint` for\n  // block sizing.\n  absl::InlinedVector<absl::string_view, 16> copied_fragments;\n  Options copy_options = options;\n  copy_options.set_size_hint(size());\n  absl::Cord::CharIterator iter = src.char_begin();\n  while (iter != src.char_end()) {\n    const absl::string_view fragment = absl::Cord::ChunkRemaining(iter);\n    if (fragment.size() <= kMaxBytesToCopy) {\n      copied_fragments.push_back(fragment);\n      copy_options.set_size_hint(*copy_options.size_hint() + fragment.size());\n      absl::Cord::Advance(&iter, fragment.size());\n    } else {\n      for (const absl::string_view copied_fragment : copied_fragments) {\n        Append(copied_fragment, copy_options);\n      }\n      copied_fragments.clear();\n      Append(Block(riegeli::Maker<FlatCordBlock>(\n                 riegeli::Invoker([&iter, size = fragment.size()]() {\n                   return absl::Cord::AdvanceAndRead(&iter, size);\n                 }))),\n             options);\n      copy_options.set_size_hint(size());\n    }\n  }\n  for (const absl::string_view copied_fragment : copied_fragments) {\n    Append(copied_fragment, options);\n  }\n}\n\nvoid Chain::Prepend(BytesRef src, Options options) {\n  while (!src.empty()) {\n    const absl::Span<char> buffer =\n        PrependBuffer(1, src.size(), src.size(), options);\n    std::memcpy(buffer.data(), src.data() + (src.size() - buffer.size()),\n                buffer.size());\n    src.remove_suffix(buffer.size());\n  }\n}\n\nvoid Chain::Prepend(const Chain& src, Options options) {\n  PrependChain<ShareOwnership>(src, options);\n}\n\nvoid Chain::Prepend(Chain&& src, Options options) {\n  PrependChain<PassOwnership>(std::move(src), options);\n}\n\ntemplate <typename Ownership, typename ChainRef>\ninline void Chain::PrependChain(ChainRef&& src, Options options) {\n  if (src.begin_ == src.end_) {\n    Prepend(src.short_data(), options);\n    return;\n  }\n  RIEGELI_CHECK_LE(src.size(), std::numeric_limits<size_t>::max() - size_)\n      << \"Failed precondition of Chain::Prepend(Chain): \"\n         \"Chain size overflow\";\n  const BlockPtr* src_iter = src.end_;\n  // If the last block of src is handled specially,\n  // `(--src_iter)->block_ptr->Unref<Ownership>()` skips it so that\n  // `PrependBlocks<Ownership>()` does not prepend it again.\n  if (begin_ == end_) {\n    if (src.back()->tiny() ||\n        (src.end_ - src.begin_ > 1 && src.back()->wasteful())) {\n      // The last block of `src` must be rewritten. Merge short data with it to\n      // a new block.\n      if (!short_data().empty() || !src.back()->empty()) {\n        RIEGELI_ASSERT_LE(src.back()->size(), RawBlock::kMaxCapacity - size_)\n            << \"Sum of sizes of short data and a tiny or wasteful block \"\n               \"exceeds RawBlock::kMaxCapacity\";\n        const size_t capacity =\n            src.end_ - src.begin_ == 1\n                ? NewBlockCapacity(size_, src.back()->size(), 0, options)\n                : size_ + src.back()->size();\n        IntrusiveSharedPtr<RawBlock> merged = RawBlock::NewInternal(capacity);\n        merged->Prepend(short_data());\n        merged->Prepend(*src.back());\n        PushFront(std::move(merged));\n      }\n      (--src_iter)->block_ptr->Unref<Ownership>();\n    } else if (!empty()) {\n      // Copy short data to a real block.\n      IntrusiveSharedPtr<RawBlock> real =\n          RawBlock::NewInternal(kMaxShortDataSize);\n      real->AppendWithExplicitSizeToCopy(short_data(), kMaxShortDataSize);\n      PushFront(std::move(real));\n    }\n  } else {\n    if (front()->tiny() && src.back()->tiny()) {\n    merge:\n      // Boundary blocks must be merged, or they are both empty or wasteful so\n      // merging them is cheaper than rewriting them separately.\n      if (src.back()->empty() && front()->empty()) {\n        PopFront();\n      } else if (front()->can_prepend(src.back()->size()) &&\n                 (src.end_ - src.begin_ == 1 ||\n                  !front()->wasteful(src.back()->size()))) {\n        // Boundary blocks can be prepended in place; this is always cheaper\n        // than merging them to a new block.\n        front()->Prepend(*src.back());\n        RefreshFront();\n      } else {\n        // Boundary blocks cannot be prepended in place. Merge them to a new\n        // block.\n        RIEGELI_ASSERT_LE(src.back()->size(),\n                          RawBlock::kMaxCapacity - front()->size())\n            << \"Sum of sizes of two tiny or wasteful blocks exceeds \"\n               \"RawBlock::kMaxCapacity\";\n        const size_t capacity =\n            src.end_ - src.begin_ == 1\n                ? NewBlockCapacity(front()->size(), src.back()->size(), 0,\n                                   options)\n                : front()->size() + src.back()->size();\n        IntrusiveSharedPtr<RawBlock> merged = RawBlock::NewInternal(capacity);\n        merged->Prepend(*front());\n        merged->Prepend(*src.back());\n        SetFront(std::move(merged));\n      }\n      (--src_iter)->block_ptr->Unref<Ownership>();\n    } else if (front()->empty()) {\n      if (src.end_ - src.begin_ > 1 && src.back()->wasteful()) goto merge;\n      // The first block is empty and must be removed.\n      PopFront();\n    } else if (front()->wasteful()) {\n      if (src.end_ - src.begin_ > 1 &&\n          (src.back()->empty() || src.back()->wasteful())) {\n        goto merge;\n      }\n      // The first block must reduce waste.\n      if (front()->can_prepend(src.back()->size()) &&\n          (src.end_ - src.begin_ == 1 ||\n           !front()->wasteful(src.back()->size())) &&\n          src.back()->size() <= kAllocationCost + front()->size()) {\n        // Prepending in place is possible and is cheaper than rewriting the\n        // first block.\n        front()->Prepend(*src.back());\n        RefreshFront();\n        (--src_iter)->block_ptr->Unref<Ownership>();\n      } else {\n        // Prepending in place is not possible, or rewriting the first block is\n        // cheaper.\n        SetFrontSameSize(front()->Copy());\n      }\n    } else if (src.end_ - src.begin_ > 1) {\n      if (src.back()->empty()) {\n        // The last block of `src` is empty and must be skipped.\n        (--src_iter)->block_ptr->Unref<Ownership>();\n      } else if (src.back()->wasteful()) {\n        // The last block of `src` must reduce waste.\n        if (front()->can_prepend(src.back()->size()) &&\n            !front()->wasteful(src.back()->size())) {\n          // Prepending in place is possible; this is always cheaper than\n          // rewriting the last block of `src`.\n          front()->Prepend(*src.back());\n          RefreshFront();\n        } else {\n          // Prepending in place is not possible.\n          PushFront(src.back()->Copy());\n        }\n        (--src_iter)->block_ptr->Unref<Ownership>();\n      }\n    }\n  }\n  size_ += src.size_;\n  PrependBlocks<Ownership>(src.begin_, src_iter);\n  src.DropPassedBlocks(Ownership());\n}\n\nvoid Chain::Prepend(const Block& src, Options options) {\n  if (src.raw_block() != nullptr) PrependRawBlock(src.raw_block(), options);\n}\n\nvoid Chain::Prepend(Block&& src, Options options) {\n  if (src.raw_block() != nullptr) {\n    PrependRawBlock(std::move(src).raw_block(), options);\n  }\n}\n\ntemplate <typename RawBlockPtrRef>\ninline void Chain::PrependRawBlock(RawBlockPtrRef&& block, Options options) {\n  RIEGELI_CHECK_LE(block->size(), std::numeric_limits<size_t>::max() - size_)\n      << \"Failed precondition of Chain::Prepend(Block): \"\n         \"Chain size overflow\";\n  if (begin_ == end_) {\n    if (!short_data().empty()) {\n      if (block->tiny()) {\n        // The block must be rewritten. Merge short data with it to a new block.\n        RIEGELI_ASSERT_LE(block->size(), RawBlock::kMaxCapacity - size_)\n            << \"Sum of sizes of short data and a tiny block exceeds \"\n               \"RawBlock::kMaxCapacity\";\n        const size_t capacity =\n            NewBlockCapacity(size_, block->size(), 0, options);\n        IntrusiveSharedPtr<RawBlock> merged = RawBlock::NewInternal(capacity);\n        merged->Prepend(short_data());\n        merged->Prepend(*block);\n        PushFront(std::move(merged));\n        size_ += block->size();\n        return;\n      }\n      // Copy short data to a real block.\n      IntrusiveSharedPtr<RawBlock> real =\n          RawBlock::NewInternal(kMaxShortDataSize);\n      real->AppendWithExplicitSizeToCopy(short_data(), kMaxShortDataSize);\n      PushFront(std::move(real));\n    }\n  } else {\n    if (front()->tiny() && block->tiny()) {\n      // Boundary blocks must be merged.\n      if (front()->can_prepend(block->size())) {\n        // Boundary blocks can be prepended in place; this is always cheaper\n        // than merging them to a new block.\n        front()->Prepend(*block);\n        RefreshFront();\n      } else {\n        // Boundary blocks cannot be prepended in place. Merge them to a new\n        // block.\n        RIEGELI_ASSERT_LE(block->size(),\n                          RawBlock::kMaxCapacity - front()->size())\n            << \"Sum of sizes of two tiny blocks exceeds RawBlock::kMaxCapacity\";\n        IntrusiveSharedPtr<RawBlock> merged = RawBlock::NewInternal(\n            NewBlockCapacity(front()->size(), block->size(), 0, options));\n        merged->Prepend(*front());\n        merged->Prepend(*block);\n        SetFront(std::move(merged));\n      }\n      size_ += block->size();\n      return;\n    }\n    if (front()->empty()) {\n      // The first block is empty and must be removed.\n      size_ += block->size();\n      SetFront(std::forward<RawBlockPtrRef>(block));\n      return;\n    }\n    if (front()->wasteful()) {\n      // The first block must reduce waste.\n      if (front()->can_prepend(block->size()) &&\n          block->size() <= kAllocationCost + front()->size()) {\n        // Prepending in place is possible and is cheaper than rewriting the\n        // first block.\n        front()->Prepend(*block);\n        RefreshFront();\n        size_ += block->size();\n        return;\n      }\n      // Prepending in place is not possible, or rewriting the first block is\n      // cheaper.\n      SetFrontSameSize(front()->Copy());\n    }\n  }\n  size_ += block->size();\n  PushFront(std::forward<RawBlockPtrRef>(block));\n}\n\nvoid Chain::Prepend(const absl::Cord& src, Options options) {\n  PrependCord(src, options);\n}\n\nvoid Chain::Prepend(absl::Cord&& src, Options options) {\n  PrependCord(std::move(src), options);\n}\n\ntemplate <typename CordRef>\ninline void Chain::PrependCord(CordRef&& src, Options options) {\n  if (src.size() <= MaxBytesToCopy(options)) {\n    if (const std::optional<absl::string_view> flat = src.TryFlat();\n        flat != std::nullopt) {\n      Prepend(*flat, options);\n      return;\n    }\n  }\n  Prepend(Chain(std::forward<CordRef>(src)), options);\n}\n\nvoid Chain::AppendFrom(absl::Cord::CharIterator& iter, size_t length,\n                       Options options) {\n  // Avoid creating wasteful blocks and then rewriting them: append copied\n  // fragments when their accumulated size is known, tweaking `size_hint` for\n  // block sizing.\n  absl::InlinedVector<absl::string_view, 16> copied_fragments;\n  Options copy_options = options;\n  copy_options.set_size_hint(size());\n  while (length > 0) {\n    absl::string_view fragment = absl::Cord::ChunkRemaining(iter);\n    fragment = absl::string_view(fragment.data(),\n                                 UnsignedMin(fragment.size(), length));\n    if (fragment.size() <= kMaxBytesToCopy) {\n      copied_fragments.push_back(fragment);\n      copy_options.set_size_hint(*copy_options.size_hint() + fragment.size());\n      absl::Cord::Advance(&iter, fragment.size());\n    } else {\n      for (const absl::string_view copied_fragment : copied_fragments) {\n        Append(copied_fragment, copy_options);\n      }\n      copied_fragments.clear();\n      Append(Block(riegeli::Maker<FlatCordBlock>(\n                 riegeli::Invoker([&iter, size = fragment.size()]() {\n                   return absl::Cord::AdvanceAndRead(&iter, size);\n                 }))),\n             options);\n      copy_options.set_size_hint(size());\n    }\n    length -= fragment.size();\n  }\n  for (const absl::string_view copied_fragment : copied_fragments) {\n    Append(copied_fragment, options);\n  }\n}\n\nvoid Chain::RemoveSuffix(size_t length, Options options) {\n  if (length == 0) return;\n  RIEGELI_CHECK_LE(length, size())\n      << \"Failed precondition of Chain::RemoveSuffix(): \"\n      << \"length to remove greater than current size\";\n  size_ -= length;\n  if (begin_ == end_) {\n    // `Chain` has short data which have suffix removed in place.\n    return;\n  }\n  while (length > back()->size()) {\n    length -= back()->size();\n    PopBack();\n    RIEGELI_ASSERT_NE(begin_, end_)\n        << \"Failed invariant of Chain: \"\n           \"sum of block sizes smaller than Chain size\";\n  }\n  if (back()->TryRemoveSuffix(length)) {\n    if (end_ - begin_ > 1 && back()->tiny() && end_[-2].block_ptr->tiny()) {\n      // Last two blocks must be merged.\n      IntrusiveSharedPtr<RawBlock> last = PopBack();\n      if (!last->empty()) {\n        RIEGELI_ASSERT_LE(last->size(), RawBlock::kMaxCapacity - back()->size())\n            << \"Sum of sizes of two tiny blocks exceeds \"\n               \"RawBlock::kMaxCapacity\";\n        IntrusiveSharedPtr<RawBlock> merged = RawBlock::NewInternal(\n            NewBlockCapacity(back()->size() + last->size(), 0, 0, options));\n        merged->Append(*back());\n        merged->Append(*last);\n        SetBack(std::move(merged));\n      }\n    }\n    return;\n  }\n  IntrusiveSharedPtr<RawBlock> last = PopBack();\n  if (length == last->size()) return;\n  absl::string_view data = *last;\n  data.remove_suffix(length);\n  // Compensate for increasing `size_` by `Append()`.\n  size_ -= data.size();\n  Append(ExternalRef(riegeli::Invoker(MakeBlock(), std::move(last)), data),\n         options);\n}\n\nvoid Chain::RemovePrefix(size_t length, Options options) {\n  if (length == 0) return;\n  RIEGELI_CHECK_LE(length, size())\n      << \"Failed precondition of Chain::RemovePrefix(): \"\n      << \"length to remove greater than current size\";\n  size_ -= length;\n  if (begin_ == end_) {\n    // `Chain` has short data which have prefix removed by shifting the rest.\n    std::memmove(short_data_begin(), short_data_begin() + length, size_);\n    return;\n  }\n  while (length > front()->size()) {\n    length -= front()->size();\n    PopFront();\n    RIEGELI_ASSERT_NE(begin_, end_)\n        << \"Failed invariant of Chain: \"\n           \"sum of block sizes smaller than Chain size\";\n  }\n  if (front()->TryRemovePrefix(length)) {\n    RefreshFront();\n    if (end_ - begin_ > 1 && front()->tiny() && begin_[1].block_ptr->tiny()) {\n      // First two blocks must be merged.\n      IntrusiveSharedPtr<RawBlock> first = PopFront();\n      if (!first->empty()) {\n        RIEGELI_ASSERT_LE(first->size(),\n                          RawBlock::kMaxCapacity - front()->size())\n            << \"Sum of sizes of two tiny blocks exceeds \"\n               \"RawBlock::kMaxCapacity\";\n        IntrusiveSharedPtr<RawBlock> merged = RawBlock::NewInternal(\n            NewBlockCapacity(first->size() + front()->size(), 0, 0, options));\n        merged->Prepend(*front());\n        merged->Prepend(*first);\n        SetFront(std::move(merged));\n      }\n    }\n    return;\n  }\n  IntrusiveSharedPtr<RawBlock> first = PopFront();\n  if (length == first->size()) return;\n  absl::string_view data = *first;\n  data.remove_prefix(length);\n  // Compensate for increasing `size_` by `Prepend()`.\n  size_ -= data.size();\n  Prepend(ExternalRef(riegeli::Invoker(MakeBlock(), std::move(first)), data),\n          options);\n}\n\nvoid swap(Chain& a, Chain& b) noexcept {\n  using std::swap;\n  if (a.has_here()) {\n    a.begin_ = b.block_ptrs_.here + (a.begin_ - a.block_ptrs_.here);\n    a.end_ = b.block_ptrs_.here + (a.end_ - a.block_ptrs_.here);\n  }\n  if (b.has_here()) {\n    b.begin_ = a.block_ptrs_.here + (b.begin_ - b.block_ptrs_.here);\n    b.end_ = a.block_ptrs_.here + (b.end_ - b.block_ptrs_.here);\n  }\n  swap(a.block_ptrs_, b.block_ptrs_);\n  swap(a.begin_, b.begin_);\n  swap(a.end_, b.end_);\n  swap(a.size_, b.size_);\n}\n\nStrongOrdering Chain::Compare(const Chain& a, const Chain& b) {\n  BlockIterator a_iter = a.blocks().cbegin();\n  BlockIterator b_iter = b.blocks().cbegin();\n  size_t this_pos = 0;\n  size_t that_pos = 0;\n  while (a_iter != a.blocks().cend()) {\n    if (b_iter == b.blocks().cend()) {\n      do {\n        if (!a_iter->empty()) return StrongOrdering::greater;\n        ++a_iter;\n      } while (a_iter != a.blocks().cend());\n      return StrongOrdering::equal;\n    }\n    const size_t length =\n        UnsignedMin(a_iter->size() - this_pos, b_iter->size() - that_pos);\n    if (const int ordering = std::memcmp(a_iter->data() + this_pos,\n                                         b_iter->data() + that_pos, length);\n        ordering != 0) {\n      return AsStrongOrdering(ordering);\n    }\n    this_pos += length;\n    if (this_pos == a_iter->size()) {\n      ++a_iter;\n      this_pos = 0;\n    }\n    that_pos += length;\n    if (that_pos == b_iter->size()) {\n      ++b_iter;\n      that_pos = 0;\n    }\n  }\n  while (b_iter != b.blocks().cend()) {\n    if (!b_iter->empty()) return StrongOrdering::less;\n    ++b_iter;\n  }\n  return StrongOrdering::equal;\n}\n\nStrongOrdering Chain::Compare(const Chain& a, absl::string_view b) {\n  BlockIterator a_iter = a.blocks().cbegin();\n  size_t this_pos = 0;\n  size_t that_pos = 0;\n  while (a_iter != a.blocks().cend()) {\n    if (that_pos == b.size()) {\n      do {\n        if (!a_iter->empty()) return StrongOrdering::greater;\n        ++a_iter;\n      } while (a_iter != a.blocks().cend());\n      return StrongOrdering::equal;\n    }\n    const size_t length =\n        UnsignedMin(a_iter->size() - this_pos, b.size() - that_pos);\n    if (const int ordering =\n            std::memcmp(a_iter->data() + this_pos, b.data() + that_pos, length);\n        ordering != 0) {\n      return AsStrongOrdering(ordering);\n    }\n    this_pos += length;\n    if (this_pos == a_iter->size()) {\n      ++a_iter;\n      this_pos = 0;\n    }\n    that_pos += length;\n  }\n  return that_pos == b.size() ? StrongOrdering::equal : StrongOrdering::less;\n}\n\nvoid Chain::Output(std::ostream& dest) const {\n  WriteWithPadding(dest, size(), [&] {\n    for (const absl::string_view fragment : blocks()) {\n      dest.write(fragment.data(), IntCast<std::streamsize>(fragment.size()));\n    }\n  });\n}\n\nvoid Chain::VerifyInvariants() const {\n#if RIEGELI_DEBUG\n  if (begin_ == end_) {\n    if (has_here()) {\n      RIEGELI_CHECK_LE(size(), kMaxShortDataSize);\n    } else {\n      RIEGELI_CHECK_EQ(size(), 0u);\n    }\n  } else {\n    RIEGELI_CHECK_LE(begin_, end_);\n    if (has_here()) {\n      RIEGELI_CHECK_LE(PtrDistance(begin_, end_), 2u);\n    } else {\n      RIEGELI_CHECK_GE(begin_, block_ptrs_.allocated.begin);\n      RIEGELI_CHECK_LE(end_, block_ptrs_.allocated.end);\n    }\n    bool is_tiny = false;\n    size_t offset =\n        has_allocated() ? begin_[block_offsets()].block_offset : size_t{0};\n    const BlockPtr* iter = begin_;\n    do {\n      if (is_tiny) {\n        RIEGELI_CHECK(!iter->block_ptr->tiny());\n        is_tiny = false;\n      } else {\n        is_tiny = iter->block_ptr->tiny();\n      }\n      if (iter != begin_ && iter != end_ - 1) {\n        RIEGELI_CHECK(!iter->block_ptr->empty());\n        RIEGELI_CHECK(!iter->block_ptr->wasteful());\n      }\n      if (has_allocated()) {\n        RIEGELI_CHECK_EQ(iter[block_offsets()].block_offset, offset);\n      }\n      offset += iter->block_ptr->size();\n      ++iter;\n    } while (iter != end_);\n    if (has_allocated()) offset -= begin_[block_offsets()].block_offset;\n    RIEGELI_CHECK_EQ(size(), offset);\n  }\n#endif\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/chain.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_CHAIN_H_\n#define RIEGELI_BASE_CHAIN_H_\n\n#include \"riegeli/base/chain_base.h\"            // IWYU pragma: export\n#include \"riegeli/base/chain_details.h\"         // IWYU pragma: export\n#include \"riegeli/base/external_ref_base.h\"     // IWYU pragma: keep\n#include \"riegeli/base/external_ref_support.h\"  // IWYU pragma: keep\n\n#endif  // RIEGELI_BASE_CHAIN_H_\n"
  },
  {
    "path": "riegeli/base/chain_base.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_CHAIN_BASE_H_\n#define RIEGELI_BASE_CHAIN_BASE_H_\n\n// IWYU pragma: private, include \"riegeli/base/chain.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <cstring>\n#include <functional>\n#include <iosfwd>\n#include <limits>\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/macros.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/external_data.h\"\n#include \"riegeli/base/external_ref_support.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/intrusive_shared_ptr.h\"\n#include \"riegeli/base/memory_estimator.h\"\n#include \"riegeli/base/ownership.h\"\n#include \"riegeli/base/ref_count.h\"\n#include \"riegeli/base/type_traits.h\"\n\nnamespace riegeli {\n\nclass ExternalRef;\n\n// A `Chain` represents a sequence of bytes. It supports efficient appending and\n// prepending, and sharing memory with other `Chain`s and other types. It does\n// not support efficient random access.\n//\n// A `Chain` can be written using `ChainWriter` and `ChainBackwardWriter`,\n// and can be read using `ChainReader`. `Chain` itself exposes lower level\n// appending/prepending and iteration functions.\n//\n// A `Chain` is implemented with a sequence of blocks holding flat data\n// fragments.\nclass Chain : public WithCompare<Chain> {\n private:\n  class RawBlock;\n\n  // A union of either a block pointer or a block offset. Having a union makes\n  // easier to allocate an array containing both kinds of data, with block\n  // offsets following block pointers.\n  union BlockPtr {\n    RawBlock* block_ptr;\n    size_t block_offset;\n  };\n\n  static constexpr size_t kMaxShortDataSize = 2 * sizeof(BlockPtr);\n\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // Expected final size, or `std::nullopt` if unknown. This may improve\n    // performance and memory usage.\n    //\n    // If the size hint turns out to not match reality, nothing breaks.\n    Options& set_size_hint(std::optional<size_t> size_hint) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      if (size_hint == std::nullopt) {\n        size_hint_ = std::numeric_limits<size_t>::max();\n      } else {\n        size_hint_ =\n            UnsignedMin(*size_hint, std::numeric_limits<size_t>::max() - 1);\n      }\n      return *this;\n    }\n    Options&& set_size_hint(std::optional<size_t> size_hint) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_size_hint(size_hint));\n    }\n    std::optional<size_t> size_hint() const {\n      if (size_hint_ == std::numeric_limits<size_t>::max()) {\n        return std::nullopt;\n      } else {\n        return size_hint_;\n      }\n    }\n\n    // Minimal size of a block of allocated data.\n    //\n    // This is used initially, while the destination is small.\n    //\n    // Default: `kDefaultMinBlockSize` (512).\n    Options& set_min_block_size(size_t min_block_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      min_block_size_ = UnsignedMin(min_block_size, uint32_t{1} << 31);\n      return *this;\n    }\n    Options&& set_min_block_size(size_t min_block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_min_block_size(min_block_size));\n    }\n    size_t min_block_size() const { return min_block_size_; }\n\n    // Maximal size of a block of allocated data.\n    //\n    // This is for performance tuning, not a guarantee: does not apply to\n    // objects allocated separately and then appended to this `Chain`.\n    //\n    // Default: `kDefaultMaxBlockSize` (64K).\n    Options& set_max_block_size(size_t max_block_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GT(max_block_size, 0u)\n          << \"Failed precondition of Chain::Options::set_max_block_size(): \"\n             \"zero block size\";\n      max_block_size_ = UnsignedMin(max_block_size, uint32_t{1} << 31);\n      return *this;\n    }\n    Options&& set_max_block_size(size_t max_block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_block_size(max_block_size));\n    }\n    size_t max_block_size() const { return max_block_size_; }\n\n    // A shortcut for `set_min_block_size(block_size)` with\n    // `set_max_block_size(block_size)`.\n    Options& set_block_size(size_t block_size) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_min_block_size(block_size).set_max_block_size(block_size);\n    }\n    Options&& set_block_size(size_t block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_block_size(block_size));\n    }\n\n   private:\n    // `std::nullopt` is encoded as `std::numeric_limits<size_t>::max()` to\n    // reduce object size.\n    size_t size_hint_ = std::numeric_limits<size_t>::max();\n    // Use `uint32_t` instead of `size_t` to reduce the object size.\n    uint32_t min_block_size_ = uint32_t{kDefaultMinBlockSize};\n    uint32_t max_block_size_ = uint32_t{kDefaultMaxBlockSize};\n  };\n\n  class Block;\n  class BlockRef;\n  class BlockIterator;\n  class Blocks;\n  struct BlockAndChar;\n\n  // A sentinel value for the `max_length` parameter of\n  // `AppendBuffer()`/`PrependBuffer()`.\n  static constexpr size_t kAnyLength = std::numeric_limits<size_t>::max();\n\n  static constexpr size_t kMaxBytesToCopyToEmpty = kMaxShortDataSize;\n\n  size_t MaxBytesToCopy(Options options = Options()) const {\n    if (options.size_hint() != std::nullopt && size() < *options.size_hint()) {\n      return UnsignedClamp(*options.size_hint() - size() - 1,\n                           kMaxBytesToCopyToEmpty, kMaxBytesToCopy);\n    }\n    if (empty()) return kMaxBytesToCopyToEmpty;\n    return kMaxBytesToCopy;\n  }\n\n  // Allocated size of an external block containing an external object of type\n  // `T`.\n  template <typename T>\n  static constexpr size_t kExternalAllocatedSize();\n\n  constexpr Chain() = default;\n\n  // Converts from a string-like type.\n  explicit Chain(BytesRef src);\n  explicit Chain(ExternalRef src);\n  template <typename Src,\n            std::enable_if_t<SupportsExternalRefWhole<Src>::value, int> = 0>\n  explicit Chain(Src&& src);\n  explicit Chain(Block src);\n  explicit Chain(const absl::Cord& src);\n  explicit Chain(absl::Cord&& src);\n\n  Chain(const Chain& that);\n  Chain& operator=(const Chain& that);\n\n  // The source `Chain` is left cleared.\n  //\n  // Moving a `Chain` invalidates its `BlockIterator`s and data pointers, but\n  // the shape of blocks (their number and sizes) remains unchanged.\n  Chain(Chain&& that) noexcept;\n  Chain& operator=(Chain&& that) noexcept;\n\n  ~Chain();\n\n  // Makes `*this` equivalent to a newly constructed `Chain`. This avoids\n  // constructing a temporary `Chain` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(BytesRef src);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(ExternalRef src);\n  template <typename Src,\n            std::enable_if_t<SupportsExternalRefWhole<Src>::value, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Src&& src);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Block src);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(const absl::Cord& src);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(absl::Cord&& src);\n\n  // Removes all data.\n  ABSL_ATTRIBUTE_REINITIALIZES void Clear();\n\n  // A container of `absl::string_view` blocks comprising data of the `Chain`.\n  Blocks blocks() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  bool empty() const { return size_ == 0; }\n  size_t size() const { return size_; }\n\n  void CopyTo(char* dest) const;\n  void AppendTo(std::string& dest) const&;\n  void AppendTo(std::string& dest) &&;\n  void AppendTo(absl::Cord& dest) const&;\n  void AppendTo(absl::Cord& dest) &&;\n  void PrependTo(absl::Cord& dest) const&;\n  void PrependTo(absl::Cord& dest) &&;\n  explicit operator std::string() const&;\n  explicit operator std::string() &&;\n  explicit operator absl::Cord() const&;\n  explicit operator absl::Cord() &&;\n\n  // If the `Chain` contents are flat, returns them, otherwise returns\n  // `std::nullopt`.\n  std::optional<absl::string_view> TryFlat() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // If the `Chain` contents are not flat, flattens them in place. Returns flat\n  // contents.\n  absl::string_view Flatten() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Locates the block containing the given character position, and the\n  // character index within the block.\n  //\n  // The opposite conversion is `Chain::BlockIterator::CharIndexInChain()`.\n  //\n  // Precondition: `char_index_in_chain <= size()`\n  BlockAndChar BlockAndCharIndex(size_t char_index_in_chain) const;\n\n  // Shows internal structure in a human-readable way, for debugging.\n  void DumpStructure(std::ostream& dest) const;\n\n  // Supports `MemoryEstimator`.\n  friend void RiegeliRegisterSubobjects(const Chain* self,\n                                        MemoryEstimator& memory_estimator) {\n    self->RegisterSubobjects(memory_estimator);\n  }\n\n  // Appends/prepends some uninitialized space. The buffer will have length at\n  // least `min_length`, preferably `recommended_length`, and at most\n  // `max_length`.\n  //\n  // If `min_length == 0`, returns whatever space was already allocated\n  // (possibly an empty buffer) without invalidating existing pointers. If the\n  // `Chain` was empty then the empty contents can be moved.\n  //\n  // If `recommended_length < min_length`, `recommended_length` is assumed to be\n  // `min_length`.\n  //\n  // If `max_length == kAnyLength`, there is no maximum.\n  //\n  // Precondition: `min_length <= max_length`\n  absl::Span<char> AppendBuffer(\n      size_t min_length, size_t recommended_length = 0,\n      size_t max_length = kAnyLength,\n      Options options = Options()) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  absl::Span<char> PrependBuffer(\n      size_t min_length, size_t recommended_length = 0,\n      size_t max_length = kAnyLength,\n      Options options = Options()) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Equivalent to `AppendBuffer()`/`PrependBuffer()` with\n  // `min_length == max_length`.\n  absl::Span<char> AppendFixedBuffer(size_t length, Options options = Options())\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  absl::Span<char> PrependFixedBuffer(\n      size_t length, Options options = Options()) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Appends/prepends a string-like type.\n  void Append(BytesRef src, Options options = Options());\n  void Append(ExternalRef src);\n  void Append(ExternalRef src, Options options);\n  template <typename Src,\n            std::enable_if_t<SupportsExternalRefWhole<Src>::value, int> = 0>\n  void Append(Src&& src);\n  template <typename Src,\n            std::enable_if_t<SupportsExternalRefWhole<Src>::value, int> = 0>\n  void Append(Src&& src, Options options);\n  void Append(const Chain& src, Options options = Options());\n  void Append(Chain&& src, Options options = Options());\n  void Append(const Block& src, Options options = Options());\n  void Append(Block&& src, Options options = Options());\n  void Append(const absl::Cord& src, Options options = Options());\n  void Append(absl::Cord&& src, Options options = Options());\n  void Prepend(BytesRef src, Options options = Options());\n  void Prepend(ExternalRef src);\n  void Prepend(ExternalRef src, Options options);\n  template <typename Src,\n            std::enable_if_t<SupportsExternalRefWhole<Src>::value, int> = 0>\n  void Prepend(Src&& src);\n  template <typename Src,\n            std::enable_if_t<SupportsExternalRefWhole<Src>::value, int> = 0>\n  void Prepend(Src&& src, Options options);\n  void Prepend(const Chain& src, Options options = Options());\n  void Prepend(Chain&& src, Options options = Options());\n  void Prepend(const Block& src, Options options = Options());\n  void Prepend(Block&& src, Options options = Options());\n  void Prepend(const absl::Cord& src, Options options = Options());\n  void Prepend(absl::Cord&& src, Options options = Options());\n\n  // `AppendFrom(iter, length)` is equivalent to\n  // `Append(absl::Cord::AdvanceAndRead(&iter, length))` but more efficient.\n  void AppendFrom(absl::Cord::CharIterator& iter, size_t length,\n                  Options options = Options());\n\n  // Removes suffix/prefix of the given length.\n  //\n  // Precondition: `length <= size()`\n  void RemoveSuffix(size_t length, Options options = Options());\n  void RemovePrefix(size_t length, Options options = Options());\n\n  friend void swap(Chain& a, Chain& b) noexcept;\n\n  friend bool operator==(const Chain& a, const Chain& b) {\n    return a.size() == b.size() && Compare(a, b) == 0;\n  }\n  friend StrongOrdering RIEGELI_COMPARE(const Chain& a, const Chain& b) {\n    return Compare(a, b);\n  }\n\n  friend bool operator==(const Chain& a, absl::string_view b) {\n    return a.size() == b.size() && Compare(a, b) == 0;\n  }\n  friend StrongOrdering RIEGELI_COMPARE(const Chain& a, absl::string_view b) {\n    return Compare(a, b);\n  }\n\n  template <typename HashState>\n  friend HashState AbslHashValue(HashState hash_state, const Chain& self) {\n    return self.HashValue(std::move(hash_state));\n  }\n\n  // Default stringification by `absl::StrCat()` etc.\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const Chain& src) {\n    src.Stringify(dest);\n  }\n\n  friend std::ostream& operator<<(std::ostream& dest, const Chain& src) {\n    src.Output(dest);\n    return dest;\n  }\n\n  // Supports `riegeli::Debug()`.\n  template <typename DebugStream>\n  friend void RiegeliDebug(const Chain& src, DebugStream& dest) {\n    src.Debug(dest);\n  }\n\n  // Supports `absl::Format(&chain, format, args...)`.\n  friend void AbslFormatFlush(Chain* dest, absl::string_view src) {\n    dest->Append(src);\n  }\n\n  // For testing. If `RIEGELI_DEBUG` is defined, verifies internal invariants,\n  // otherwise does nothing.\n  void VerifyInvariants() const;\n\n private:\n  class BlockPtrPtr;\n  struct MakeBlock;\n  struct ExternalMethods;\n  template <typename T>\n  struct ExternalMethodsFor;\n\n  struct Empty {};\n\n  struct Allocated {\n    // The extent of the allocated array of block pointers. This array is\n    // immediately followed by the array of block offsets of the same size,\n    // used for efficient finding of the block covering the given position.\n    // Only some middle portion of each array is filled.\n    //\n    // The offset of the first block is not necessarily 0 but an arbitrary value\n    // (with possible wrapping around the `size_t` range), to avoid having to\n    // update all offsets in `Prepend()` or `RemovePrefix()`.\n    BlockPtr* begin;\n    BlockPtr* end;\n  };\n\n  union BlockPtrs {\n    constexpr BlockPtrs() noexcept : empty() {}\n\n    // If the `Chain` is empty, no block pointers are needed. Some union member\n    // is needed though for the default constructor to be constexpr.\n    Empty empty;\n    // If `begin_ == end_`, `size_` characters.\n    //\n    // If also `has_here()`, then there are 0 pointers in `here` so `short_data`\n    // can safely contain `size_` characters. If also `has_allocated()`, then\n    // `size_ == 0`, and `EnsureHasHere()` must be called before writing to\n    // `short_data`.\n    char short_data[kMaxShortDataSize];\n    // If `has_here()`, array of block pointers between `begin_` i.e. `here` and\n    // `end_` (0 to 2 pointers). In this case block offsets are implicit.\n    BlockPtr here[2];\n    // If `has_allocated()`, pointers to a heap-allocated array of block\n    // pointers and block offsets.\n    Allocated allocated;\n  };\n\n  // When deciding whether to copy an array of bytes or perform a small memory\n  // allocation, prefer copying up to this length.\n  static constexpr size_t kAllocationCost = 512;\n\n  bool ClearSlow();\n  absl::string_view FlattenSlow();\n\n  bool has_here() const { return begin_ == block_ptrs_.here; }\n  bool has_allocated() const { return begin_ != block_ptrs_.here; }\n\n  absl::string_view short_data() const;\n  char* short_data_begin();\n  const char* short_data_begin() const;\n\n  static BlockPtr* NewBlockPtrs(size_t capacity);\n  void DeleteBlockPtrs();\n  // If `has_allocated()`, delete the block pointer array and make `has_here()`\n  // `true`. This is used before appending to `short_data`.\n  //\n  // Precondition: `begin_ == end_`\n  void EnsureHasHere();\n\n  void UnrefBlocks();\n  static void UnrefBlocks(const BlockPtr* begin, const BlockPtr* end);\n  static void UnrefBlocksSlow(const BlockPtr* begin, const BlockPtr* end);\n\n  void DropPassedBlocks(PassOwnership);\n  void DropPassedBlocks(ShareOwnership) const;\n\n  // The offset of the block offsets part of the block pointer array, in array\n  // elements.\n  size_t block_offsets() const {\n    RIEGELI_ASSERT(has_allocated())\n        << \"Failed precondition of block_offsets(): \"\n           \"block pointer array is not allocated\";\n    return PtrDistance(block_ptrs_.allocated.begin, block_ptrs_.allocated.end);\n  }\n\n  // Returns the last block. Can be changed in place (if its own constraints\n  // allow that).\n  RawBlock* const& back() const { return end_[-1].block_ptr; }\n  // Returns the first block. If its size changes, this must be reflected in the\n  // array of block offset, e.g. with `RefreshFront()`.\n  RawBlock* const& front() const { return begin_[0].block_ptr; }\n\n  void Initialize(absl::string_view src);\n  void InitializeSlow(absl::string_view src);\n  void Initialize(Block src);\n  void Initialize(const absl::Cord& src);\n  void Initialize(absl::Cord&& src);\n  // This template is defined and used only in chain.cc.\n  template <typename CordRef>\n  void InitializeFromCord(CordRef&& src);\n  void Initialize(const Chain& src);\n  void CopyToSlow(char* dest) const;\n  std::string ToString() const;\n  void AppendToSlow(absl::Cord& dest) const&;\n  void AppendToSlow(absl::Cord& dest) &&;\n  void PrependToSlow(absl::Cord& dest) const&;\n  void PrependToSlow(absl::Cord& dest) &&;\n\n  IntrusiveSharedPtr<RawBlock> SetBack(IntrusiveSharedPtr<RawBlock> block);\n  IntrusiveSharedPtr<RawBlock> SetFront(IntrusiveSharedPtr<RawBlock> block);\n  // Like `SetFront()`, but skips the `RefreshFront()` step. This is enough if\n  // the block has the same size as the block being replaced.\n  IntrusiveSharedPtr<RawBlock> SetFrontSameSize(\n      IntrusiveSharedPtr<RawBlock> block);\n  // Recomputes the block offset of the first block if needed.\n  void RefreshFront();\n  void PushBack(IntrusiveSharedPtr<RawBlock> block);\n  void PushFront(IntrusiveSharedPtr<RawBlock> block);\n  IntrusiveSharedPtr<RawBlock> PopBack();\n  IntrusiveSharedPtr<RawBlock> PopFront();\n  // This template is defined and used only in chain.cc.\n  template <typename Ownership>\n  void AppendBlocks(const BlockPtr* begin, const BlockPtr* end);\n  // This template is defined and used only in chain.cc.\n  template <typename Ownership>\n  void PrependBlocks(const BlockPtr* begin, const BlockPtr* end);\n  void ReserveBack(size_t extra_capacity);\n  void ReserveFront(size_t extra_capacity);\n  void ReserveBackSlow(size_t extra_capacity);\n  void ReserveFrontSlow(size_t extra_capacity);\n\n  // Decides about the capacity of a new block to be appended/prepended.\n  //\n  // If `replaced_length > 0`, the block will replace an existing block of that\n  // size. In addition to `replaced_length`, it requires the capacity of at\n  // least `min_length`, preferably `recommended_length`.\n  size_t NewBlockCapacity(size_t replaced_length, size_t min_length,\n                          size_t recommended_length, Options options) const;\n\n  // This template is defined and used only in chain.cc.\n  template <typename Ownership, typename ChainRef>\n  void AppendChain(ChainRef&& src, Options options);\n  // This template is defined and used only in chain.cc.\n  template <typename Ownership, typename ChainRef>\n  void PrependChain(ChainRef&& src, Options options);\n\n  // This template is defined and used only in chain.cc.\n  template <typename RawBlockPtrRef>\n  void AppendRawBlock(RawBlockPtrRef&& block, Options options = Options());\n  // This template is defined and used only in chain.cc.\n  template <typename RawBlockPtrRef>\n  void PrependRawBlock(RawBlockPtrRef&& block, Options options = Options());\n\n  // This template is defined and used only in chain.cc.\n  template <typename CordRef>\n  void AppendCord(CordRef&& src, Options options);\n  // This template is defined and used only in chain.cc.\n  template <typename CordRef>\n  void AppendCordSlow(CordRef&& src, Options options);\n  // This template is defined and used only in chain.cc.\n  template <typename CordRef>\n  void PrependCord(CordRef&& src, Options options);\n\n  void RegisterSubobjects(MemoryEstimator& memory_estimator) const;\n  static StrongOrdering Compare(const Chain& a, const Chain& b);\n  static StrongOrdering Compare(const Chain& a, absl::string_view b);\n  template <typename HashState>\n  HashState HashValue(HashState hash_state) const;\n  template <typename Sink>\n  void Stringify(Sink& dest) const;\n  void Output(std::ostream& dest) const;\n  template <typename DebugStream>\n  void Debug(DebugStream& dest) const;\n\n  BlockPtrs block_ptrs_;\n\n  // The range of the block pointers array which is actually used.\n  //\n  // Invariants:\n  //   `begin_ <= end_`\n  //   if `has_here()` then `begin_ == block_ptrs_.here`\n  //                    and `end_ <= block_ptrs_.here + 2`\n  //   if `has_allocated()` then `begin_ >= block_ptrs_.allocated.begin`\n  //                         and `end_ <= block_ptrs_.allocated.end`\n  BlockPtr* begin_ = block_ptrs_.here;\n  BlockPtr* end_ = block_ptrs_.here;\n\n  // Invariants:\n  //   if `begin_ == end_` then `size_ <= kMaxShortDataSize`\n  //   if `begin_ == end_ && has_allocated()` then `size_ == 0`\n  //   if `begin_ != end_` then\n  //       `size_` is the sum of sizes of blocks in the range [`begin_`..`end_`)\n  size_t size_ = 0;\n};\n\n// Implementation details follow.\n\n// `Chain` representation consists of blocks.\n//\n// An internal block holds an allocated array which consists of free space\n// before data, data, and free space after data. Block size is the size of\n// data; block capacity is the size of the allocated array.\n//\n// An external block holds some object which keeps a data array alive, the\n// destructor of the object, and the address of the data array.\n//\n// Definitions:\n//  - empty block: a block with size == 0\n//  - tiny block: a block with size < `kDefaultMinBlockSize`\n//  - wasteful block: a block with free space > size + `kDefaultMinBlockSize`\n//\n// Invariants of a `Chain`:\n//  - A block can be empty or wasteful only if it is the first or last block.\n//  - Tiny blocks must not be adjacent.\nclass Chain::RawBlock {\n public:\n  static constexpr size_t kInternalAllocatedOffset();\n  static constexpr size_t kMaxCapacity =\n      size_t{std::numeric_limits<ptrdiff_t>::max()};\n\n  // Creates an internal block.\n  static IntrusiveSharedPtr<RawBlock> NewInternal(size_t min_capacity);\n\n  // Constructs an internal block. This constructor is public for\n  // `SizeReturningNewAligned()`.\n  explicit RawBlock(const size_t* raw_capacity);\n\n  // Constructs an external block containing an external object of type `T`,\n  // and sets block data to `BytesRef(new_object)`. This constructor is public\n  // for `NewAligned()`.\n  template <typename T>\n  explicit RawBlock(Initializer<T> object);\n\n  // Constructs an external block containing an external object of type `T`, and\n  // sets block data to `data`. This constructor is public for `NewAligned()`.\n  template <typename T>\n  explicit RawBlock(Initializer<T> object, absl::string_view substr);\n\n  // Allocated size of an external block containing an external object of type\n  // `T`.\n  template <typename T>\n  static constexpr size_t kExternalAllocatedSize();\n\n  template <typename Ownership = ShareOwnership>\n  RawBlock* Ref();\n\n  template <typename Ownership = PassOwnership>\n  void Unref();\n\n  IntrusiveSharedPtr<RawBlock> Copy();\n\n  bool TryClear();\n\n  /*implicit*/ operator absl::string_view() const { return substr_; }\n  bool empty() const { return substr_.empty(); }\n  size_t size() const { return substr_.size(); }\n  const char* data_begin() const { return substr_.data(); }\n  const char* data_end() const { return substr_.data() + substr_.size(); }\n\n  // Returns a reference to the external object, assuming that this is an\n  // external block holding an object of type `T`.\n  template <typename T>\n  T& unchecked_external_object();\n  template <typename T>\n  const T& unchecked_external_object() const;\n\n  // Returns a pointer to the external object if this is an external block\n  // holding an object of type `T`, otherwise returns `nullptr`.\n  template <typename T>\n  const T* checked_external_object() const;\n\n  // Returns a pointer to the external object if this is an external block\n  // holding an object of type `T` and the block has a unique owner, otherwise\n  // returns `nullptr`.\n  template <typename T>\n  T* checked_external_object_with_unique_owner();\n\n  bool tiny(size_t extra_size = 0) const;\n  bool wasteful(size_t extra_size = 0) const;\n\n  // Shows internal structure in a human-readable way, for debugging.\n  void DumpStructure(std::ostream& dest) const;\n\n  // Supports `MemoryEstimator`.\n  friend size_t RiegeliDynamicSizeOf(const RawBlock* self) {\n    return self->DynamicSizeOf();\n  }\n\n  // Supports `MemoryEstimator`.\n  friend void RiegeliRegisterSubobjects(const RawBlock* self,\n                                        MemoryEstimator& memory_estimator) {\n    self->RegisterSubobjects(memory_estimator);\n  }\n\n  bool can_append(size_t length) const;\n  bool can_prepend(size_t length) const;\n  absl::Span<char> AppendBuffer(size_t max_length);\n  absl::Span<char> PrependBuffer(size_t max_length);\n  void Append(absl::string_view src, size_t space_before = 0);\n  // Reads `size_to_copy` from `src.data()` but accounts for `src.size()`.\n  // Faster than `Append()` if `size_to_copy` is a compile time constant, but\n  // requires `size_to_copy` bytes to be readable, possibly past the end of src.\n  //\n  // Precondition: `size_to_copy >= src.size()`\n  void AppendWithExplicitSizeToCopy(absl::string_view src, size_t size_to_copy);\n  void Prepend(absl::string_view src, size_t space_after = 0);\n  bool TryRemoveSuffix(size_t length);\n  bool TryRemovePrefix(size_t length);\n\n private:\n  template <typename T>\n  friend struct ExternalMethodsFor;\n\n  struct External {\n    // Type-erased methods of the object.\n    const ExternalMethods* methods;\n    // Lowest possible beginning of the object (actual object has a different\n    // type and can begin at a higher address due to alignment).\n    char object_lower_bound[1];\n  };\n\n  template <typename T>\n  static constexpr size_t kExternalObjectOffset();\n\n#if RIEGELI_DEBUG\n  template <typename T, std::enable_if_t<\n                            std::is_convertible_v<const T&, BytesRef>, int> = 0>\n  static void AssertSubstr(const T& object, absl::string_view substr) {\n    if (!substr.empty()) {\n      const BytesRef whole = object;\n      RIEGELI_ASSERT(std::greater_equal<>()(substr.data(), whole.data()))\n          << \"Failed precondition of Chain::Block::Block(): \"\n             \"substring not contained in whole data\";\n      RIEGELI_ASSERT(std::less_equal<>()(substr.data() + substr.size(),\n                                         whole.data() + whole.size()))\n          << \"Failed precondition of Chain::Block::Block(): \"\n             \"substring not contained in whole data\";\n    }\n  }\n  template <\n      typename T,\n      std::enable_if_t<!std::is_convertible_v<const T&, BytesRef>, int> = 0>\n#else\n  template <typename T>\n#endif\n  static void AssertSubstr(ABSL_ATTRIBUTE_UNUSED const T& object,\n                           ABSL_ATTRIBUTE_UNUSED absl::string_view substr) {\n  }\n\n  bool is_mutable() const { return is_internal() && has_unique_owner(); }\n\n  bool has_unique_owner() const;\n\n  bool is_internal() const { return allocated_end_ != nullptr; }\n  bool is_external() const { return allocated_end_ == nullptr; }\n\n  size_t capacity() const;\n  size_t space_before() const;\n  size_t space_after() const;\n\n  size_t DynamicSizeOf() const;\n  void RegisterSubobjects(MemoryEstimator& memory_estimator) const;\n\n  RefCount ref_count_;\n  absl::string_view substr_;\n  // If `is_internal()`, end of allocated space. If `is_external()`, `nullptr`.\n  // This distinguishes internal from external blocks.\n  char* allocated_end_ = nullptr;\n  union {\n    // If `is_internal()`, beginning of data (actual allocated size is larger).\n    char allocated_begin_[1];\n    // If `is_external()`, the remaining fields.\n    External external_;\n  };\n};\n\n// Represents a reference counted pointer to a single block of a `Chain`.\nclass Chain::Block {\n public:\n  // Creates an empty `Block`.\n  Block() = default;\n\n  // Given an object which owns a byte array, converts it to a `Block` by\n  // attaching the object, avoiding copying the bytes.\n  //\n  // `ExternalRef` is a higher level mechanism which chooses between sharing the\n  // object and copying the data.\n  //\n  // The `object` parameter supports `riegeli::Maker<T>(args...)` to construct\n  // `T` in-place.\n  //\n  // If the `substr` parameter is given, `substr` must be owned by the object\n  // after it gets created or moved.\n  //\n  // If the `substr` parameter is not given, `T` must be convertible to\n  // `BytesRef`.\n  //\n  // `T` may also support the following member functions, either with or without\n  // the `substr` parameter, with the following definitions assumed by default:\n  // ```\n  //   // Called once before the destructor, except on a moved-from object.\n  //   // If only this function is needed, `T` can be a lambda.\n  //   void operator()(absl::string_view substr) && {}\n  //\n  //   // Shows internal structure in a human-readable way, for debugging.\n  //   friend void RiegeliDumpStructure(const T* self, absl::string_view substr,\n  //                                    std::ostream& dest) {\n  //     out << \"[external] { }\";\n  //   }\n  //\n  //   // Registers this object with `MemoryEstimator`.\n  //   //\n  //   // By default calls `memory_estimator.RegisterUnknownType<T>()` and\n  //   // as an approximation of memory usage of an unknown type, registers just\n  //   // the stored `substr` if unique.\n  //   friend void RiegeliRegisterSubobjects(\n  //       const T* self, riegeli::MemoryEstimator& memory_estimator);\n  // ```\n  //\n  // The `substr` parameter of these member functions, if present, will get the\n  // `substr` parameter passed to `FromExternal()`. Having `substr` available in\n  // these functions might avoid storing `substr` in the external object.\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<Block, TargetT<T>>,\n                                   std::is_convertible<TargetT<T>, BytesRef>>,\n                int> = 0>\n  explicit Block(T&& object);\n  template <typename T>\n  explicit Block(T&& object, absl::string_view substr);\n\n  Block(const Block& that) = default;\n  Block& operator=(const Block& that) = default;\n\n  Block(Block&& that) = default;\n  Block& operator=(Block&& that) = default;\n\n  /*implicit*/ operator absl::string_view() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    if (block_ == nullptr) return absl::string_view();\n    return *block_;\n  }\n\n  bool empty() const { return block_ == nullptr || block_->empty(); }\n  size_t size() const {\n    if (block_ == nullptr) return 0;\n    return block_->size();\n  }\n  const char* data() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    if (block_ == nullptr) return nullptr;\n    return block_->data_begin();\n  }\n\n  // Indicates support for:\n  //  * `ExternalRef(const Block&)`\n  //  * `ExternalRef(Block&&)`\n  //  * `ExternalRef(const Block&, substr)`\n  //  * `ExternalRef(Block&&, substr)`\n  friend void RiegeliSupportsExternalRef(const Block*) {}\n\n  // Supports `ExternalRef`.\n  friend Block RiegeliToChainBlock(Block* self, absl::string_view substr) {\n    return std::move(*self).ToChainBlock(substr);\n  }\n\n  // Supports `ExternalRef`.\n  friend absl::Cord RiegeliToCord(Block* self, absl::string_view substr) {\n    return std::move(*self).ToCord(substr);\n  }\n  friend absl::Cord RiegeliToCord(const Block* self, absl::string_view substr) {\n    return self->ToCord(substr);\n  }\n\n  // Supports `ExternalRef`.\n  friend ExternalStorage RiegeliToExternalStorage(Block* self) {\n    return std::move(*self).ToExternalStorage();\n  }\n\n  // Supports `ExternalRef` and `Chain::Block`.\n  friend void RiegeliDumpStructure(const Block* self, absl::string_view substr,\n                                   std::ostream& dest) {\n    self->DumpStructure(substr, dest);\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const Block* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->block_);\n  }\n\n private:\n  friend class Chain;  // For `Block()` and `raw_block()`.\n\n  explicit Block(RawBlock* block);\n  explicit Block(RawBlock* block, absl::string_view substr);\n  explicit Block(IntrusiveSharedPtr<RawBlock> block);\n\n  const IntrusiveSharedPtr<RawBlock>& raw_block() const& { return block_; }\n  IntrusiveSharedPtr<RawBlock>&& raw_block() && { return std::move(block_); }\n\n  Block ToChainBlock(absl::string_view substr) &&;\n  absl::Cord ToCord(absl::string_view substr) &&;\n  absl::Cord ToCord(absl::string_view substr) const&;\n  ExternalStorage ToExternalStorage() &&;\n  void DumpStructure(absl::string_view substr, std::ostream& dest) const;\n\n  IntrusiveSharedPtr<RawBlock> block_;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_CHAIN_BASE_H_\n"
  },
  {
    "path": "riegeli/base/chain_details.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_CHAIN_DETAILS_H_\n#define RIEGELI_BASE_CHAIN_DETAILS_H_\n\n// IWYU pragma: private, include \"riegeli/base/chain.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <algorithm>\n#include <cstring>\n#include <iosfwd>\n#include <iterator>\n#include <memory>\n#include <new>\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain_base.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/external_data.h\"\n#include \"riegeli/base/external_ref_base.h\"\n#include \"riegeli/base/external_ref_support.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/intrusive_shared_ptr.h\"\n#include \"riegeli/base/iterable.h\"\n#include \"riegeli/base/memory_estimator.h\"\n#include \"riegeli/base/new_aligned.h\"\n#include \"riegeli/base/ownership.h\"\n#include \"riegeli/base/type_traits.h\"\n\nnamespace riegeli {\n\n// Represents either `const BlockPtr*`, or one of two special values\n// (`kBeginShortData` and `kEndShortData`) behaving as if they were pointers in\n// a single-element `BlockPtr` array.\nclass Chain::BlockPtrPtr : public WithCompare<BlockPtrPtr> {\n public:\n  explicit constexpr BlockPtrPtr(uintptr_t repr) : repr_(repr) {}\n  static BlockPtrPtr from_ptr(const BlockPtr* ptr);\n\n  bool is_special() const;\n  const BlockPtr* as_ptr() const;\n\n  BlockPtrPtr operator+(ptrdiff_t n) const;\n  BlockPtrPtr operator-(ptrdiff_t n) const;\n  friend ptrdiff_t operator-(BlockPtrPtr a, BlockPtrPtr b) {\n    return Subtract(a, b);\n  }\n\n  friend bool operator==(BlockPtrPtr a, BlockPtrPtr b) {\n    return a.repr_ == b.repr_;\n  }\n  friend StrongOrdering RIEGELI_COMPARE(BlockPtrPtr a, BlockPtrPtr b) {\n    RIEGELI_ASSERT_EQ(a.is_special(), b.is_special())\n        << \"Incompatible BlockPtrPtr values\";\n    if (a.is_special()) return riegeli::Compare(a.repr_, b.repr_);\n    return riegeli::Compare(a.as_ptr(), b.as_ptr());\n  }\n\n private:\n  // `operator-` body is defined in a member function to gain access to private\n  // `Chain::RawBlock` under gcc.\n  static ptrdiff_t Subtract(BlockPtrPtr a, BlockPtrPtr b) {\n    RIEGELI_ASSERT_EQ(a.is_special(), b.is_special())\n        << \"Incompatible BlockPtrPtr values\";\n    if (a.is_special()) {\n      const ptrdiff_t byte_diff =\n          static_cast<ptrdiff_t>(a.repr_) - static_cast<ptrdiff_t>(b.repr_);\n      // Pointer subtraction with the element size being a power of 2 typically\n      // rounds in the same way as right shift (towards -inf), not as division\n      // (towards zero), so the right shift allows the compiler to eliminate the\n      // `is_special()` check.\n      switch (sizeof(RawBlock*)) {\n        case 1 << 2:\n          return byte_diff >> 2;\n        case 1 << 3:\n          return byte_diff >> 3;\n        default:\n          return byte_diff / ptrdiff_t{sizeof(RawBlock*)};\n      }\n    }\n    return a.as_ptr() - b.as_ptr();\n  }\n\n  uintptr_t repr_;\n};\n\n// Access private constructors of `Chain::Block`.\nstruct Chain::MakeBlock {\n  Block operator()(IntrusiveSharedPtr<RawBlock> block) const {\n    return Block(std::move(block));\n  }\n  Block operator()(RawBlock* block) const { return Block(block); }\n};\n\nclass Chain::BlockRef {\n public:\n  BlockRef(const BlockRef& that) = default;\n  BlockRef& operator=(const BlockRef& that) = default;\n\n  /*implicit*/ operator absl::string_view() const;\n\n  bool empty() const;\n  const char* data() const;\n  size_t size() const;\n\n  // Indicates support for:\n  //  * `ExternalRef(BlockRef)`\n  //  * `ExternalRef(BlockRef, substr)`\n  friend void RiegeliSupportsExternalRef(const BlockRef*) {}\n\n  // Supports `ExternalRef`.\n  friend bool RiegeliExternalCopy(const BlockRef* self) {\n    return self->ExternalCopy();\n  }\n\n  // Supports `ExternalRef`.\n  friend Chain::Block RiegeliToChainBlock(const BlockRef* self,\n                                          absl::string_view substr) {\n    return self->ToChainBlock(substr);\n  }\n\n  // Supports `ExternalRef`.\n  template <typename Callback>\n  friend void RiegeliExternalDelegate(const BlockRef* self,\n                                      absl::string_view substr,\n                                      Callback&& delegate_to) {\n    self->ExternalDelegate(substr, std::forward<Callback>(delegate_to));\n  }\n\n  // Returns a pointer to the external object if this is an external block\n  // holding an object of type `T`, otherwise returns `nullptr`.\n  template <typename T>\n  const T* external_object() const;\n\n private:\n  friend class Chain;  // For `BlockRef()`.\n\n  explicit BlockRef(const Chain* chain, BlockPtrPtr ptr)\n      : chain_(chain), ptr_(ptr) {}\n\n  bool ExternalCopy() const;\n  Chain::Block ToChainBlock(absl::string_view substr) const;\n  template <typename Callback>\n  void ExternalDelegate(absl::string_view substr, Callback&& delegate_to) const;\n\n  const Chain* chain_;\n  // If `*chain_` has short data, `kBeginShortData`.\n  // If `*chain_` has block pointers, a pointer to an element of the block\n  // pointer array.\n  BlockPtrPtr ptr_;\n};\n\nclass Chain::BlockIterator : public WithCompare<BlockIterator> {\n public:\n  using iterator_concept = std::random_access_iterator_tag;\n  // `iterator_category` is only `std::input_iterator_tag` because the\n  // `LegacyForwardIterator` requirement and above require `reference` to be\n  // a true reference type.\n  using iterator_category = std::input_iterator_tag;\n  using value_type = BlockRef;\n  using reference = value_type;\n  using pointer = ArrowProxy<reference>;\n  using difference_type = ptrdiff_t;\n\n  BlockIterator() = default;\n\n  explicit BlockIterator(const Chain* chain ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                         size_t block_index);\n\n  BlockIterator(const BlockIterator& that) = default;\n  BlockIterator& operator=(const BlockIterator& that) = default;\n\n  const Chain* chain() const { return chain_; }\n  size_t block_index() const;\n\n  // Returns the char index relative to the beginning of the chain, given the\n  // corresponding char index relative to the beginning of the block.\n  //\n  // The opposite conversion is `Chain::BlockAndCharIndex()`.\n  size_t CharIndexInChain(size_t char_index_in_block = 0) const;\n\n  reference operator*() const;\n  pointer operator->() const;\n  BlockIterator& operator++();\n  BlockIterator operator++(int);\n  BlockIterator& operator--();\n  BlockIterator operator--(int);\n  BlockIterator& operator+=(difference_type n);\n  BlockIterator operator+(difference_type n) const;\n  BlockIterator& operator-=(difference_type n);\n  BlockIterator operator-(difference_type n) const;\n  reference operator[](difference_type n) const;\n\n  friend bool operator==(BlockIterator a, BlockIterator b) {\n    RIEGELI_ASSERT_EQ(a.chain_, b.chain_)\n        << \"Failed precondition of operator==(Chain::BlockIterator): \"\n           \"incomparable iterators\";\n    return a.ptr_ == b.ptr_;\n  }\n  friend StrongOrdering RIEGELI_COMPARE(BlockIterator a, BlockIterator b) {\n    RIEGELI_ASSERT_EQ(a.chain_, b.chain_)\n        << \"Failed precondition of operator<=>(Chain::BlockIterator): \"\n           \"incomparable iterators\";\n    return riegeli::Compare(a.ptr_, b.ptr_);\n  }\n  friend difference_type operator-(BlockIterator a, BlockIterator b) {\n    RIEGELI_ASSERT_EQ(a.chain_, b.chain_)\n        << \"Failed precondition of operator-(Chain::BlockIterator): \"\n           \"incomparable iterators\";\n    return a.ptr_ - b.ptr_;\n  }\n  friend BlockIterator operator+(difference_type n, BlockIterator a) {\n    return a + n;\n  }\n\n private:\n  friend class Chain;\n\n  static constexpr BlockPtrPtr kBeginShortData{0};\n  static constexpr BlockPtrPtr kEndShortData{sizeof(BlockPtr)};\n\n  explicit BlockIterator(const Chain* chain, BlockPtrPtr ptr);\n\n  size_t CharIndexInChainInternal() const;\n\n  const Chain* chain_ = nullptr;\n  // If `chain_ == nullptr`, `kBeginShortData`.\n  // If `*chain_` has no block pointers and no short data, `kEndShortData`.\n  // If `*chain_` has short data, `kBeginShortData` or `kEndShortData`.\n  // If `*chain_` has block pointers, a pointer to an element of the block\n  // pointer array.\n  BlockPtrPtr ptr_ = kBeginShortData;\n};\n\nclass Chain::Blocks {\n public:\n  using value_type = BlockRef;\n  using reference = value_type;\n  using const_reference = reference;\n  using iterator = BlockIterator;\n  using const_iterator = iterator;\n  using reverse_iterator = std::reverse_iterator<iterator>;\n  using const_reverse_iterator = reverse_iterator;\n  using size_type = size_t;\n  using difference_type = ptrdiff_t;\n\n  Blocks() = default;\n\n  Blocks(const Blocks& that) = default;\n  Blocks& operator=(const Blocks& that) = default;\n\n  iterator begin() const;\n  iterator cbegin() const { return begin(); }\n  iterator end() const;\n  iterator cend() const { return end(); }\n\n  reverse_iterator rbegin() const { return reverse_iterator(end()); }\n  reverse_iterator crbegin() const { return rbegin(); }\n  reverse_iterator rend() const { return reverse_iterator(begin()); }\n  reverse_iterator crend() const { return rend(); }\n\n  bool empty() const;\n  size_type size() const;\n\n  reference operator[](size_type n) const;\n  reference at(size_type n) const;\n  reference front() const;\n  reference back() const;\n\n private:\n  friend class Chain;\n\n  explicit Blocks(const Chain* chain) noexcept : chain_(chain) {}\n\n  const Chain* chain_ = nullptr;\n};\n\n// Represents the position of a character in a `Chain`.\n//\n// A `CharIterator` is not provided because it is more efficient to iterate by\n// blocks and process character ranges within a block.\nstruct Chain::BlockAndChar {\n  // Intended invariant:\n  //   if `block_iter == block_iter.chain()->blocks().cend()`\n  //       then `char_index == 0`\n  //       else `char_index < block_iter->size()`\n  BlockIterator block_iter;\n  size_t char_index;\n};\n\n// Implementation details follow.\n\nstruct Chain::ExternalMethods {\n  void (*delete_block)(RawBlock* block);\n  void (*dump_structure)(const RawBlock& block, std::ostream& dest);\n  size_t dynamic_sizeof;\n  void (*register_subobjects)(const RawBlock* block,\n                              MemoryEstimator& memory_estimator);\n};\n\nnamespace chain_internal {\n\ntemplate <typename T, typename Enable = void>\nstruct HasCallOperatorSubstr : std::false_type {};\n\ntemplate <typename T>\nstruct HasCallOperatorSubstr<T, std::void_t<decltype(std::declval<T&&>()(\n                                    std::declval<absl::string_view>()))>>\n    : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct HasCallOperatorWhole : std::false_type {};\n\ntemplate <typename T>\nstruct HasCallOperatorWhole<T, std::void_t<decltype(std::declval<T&&>()())>>\n    : std::true_type {};\n\ntemplate <typename T>\nstruct HasCallOperator\n    : std::disjunction<HasCallOperatorSubstr<T>, HasCallOperatorWhole<T>> {};\n\ntemplate <typename T,\n          std::enable_if_t<HasCallOperatorSubstr<T>::value, int> = 0>\ninline void CallOperator(T&& object, absl::string_view substr) {\n  std::forward<T>(object)(substr);\n}\n\ntemplate <\n    typename T,\n    std::enable_if_t<std::conjunction_v<std::negation<HasCallOperatorSubstr<T>>,\n                                        HasCallOperatorWhole<T>>,\n                     int> = 0>\ninline void CallOperator(T&& object,\n                         ABSL_ATTRIBUTE_UNUSED absl::string_view substr) {\n  std::forward<T>(object)();\n}\n\ntemplate <\n    typename T,\n    std::enable_if_t<std::conjunction_v<std::negation<HasCallOperatorSubstr<T>>,\n                                        std::negation<HasCallOperatorWhole<T>>>,\n                     int> = 0>\ninline void CallOperator(ABSL_ATTRIBUTE_UNUSED T&& object,\n                         ABSL_ATTRIBUTE_UNUSED absl::string_view substr) {}\n\ntemplate <typename T,\n          std::enable_if_t<MemoryEstimator::RegisterSubobjectsIsGood<T>::value,\n                           int> = 0>\ninline void RegisterSubobjects(const T* object,\n                               ABSL_ATTRIBUTE_UNUSED absl::string_view substr,\n                               MemoryEstimator& memory_estimator) {\n  memory_estimator.RegisterSubobjects(object);\n}\n\ntemplate <typename T,\n          std::enable_if_t<!MemoryEstimator::RegisterSubobjectsIsGood<T>::value,\n                           int> = 0>\ninline void RegisterSubobjects(ABSL_ATTRIBUTE_UNUSED const T* object,\n                               absl::string_view substr,\n                               MemoryEstimator& memory_estimator) {\n  memory_estimator.RegisterUnknownType<T>();\n  // As an approximation of memory usage of an unknown type, register just the\n  // stored data if unique.\n  if (memory_estimator.RegisterNode(substr.data())) {\n    memory_estimator.RegisterDynamicMemory(substr.size());\n  }\n}\n\ntemplate <typename T, typename Enable = void>\nstruct HasRiegeliDumpStructureWithSubstr : std::false_type {};\n\ntemplate <typename T>\nstruct HasRiegeliDumpStructureWithSubstr<\n    T, std::void_t<decltype(RiegeliDumpStructure(\n           std::declval<const T*>(), std::declval<absl::string_view>(),\n           std::declval<std::ostream&>()))>> : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct HasRiegeliDumpStructureWithoutData : std::false_type {};\n\ntemplate <typename T>\nstruct HasRiegeliDumpStructureWithoutData<\n    T, std::void_t<decltype(RiegeliDumpStructure(\n           std::declval<const T*>(), std::declval<std::ostream&>()))>>\n    : std::true_type {};\n\nvoid DumpStructureDefault(std::ostream& dest);\n\ntemplate <typename T, std::enable_if_t<\n                          HasRiegeliDumpStructureWithSubstr<T>::value, int> = 0>\ninline void DumpStructure(const T* object, absl::string_view substr,\n                          std::ostream& dest) {\n  RiegeliDumpStructure(object, substr, dest);\n}\n\ntemplate <\n    typename T,\n    std::enable_if_t<\n        std::conjunction_v<std::negation<HasRiegeliDumpStructureWithSubstr<T>>,\n                           HasRiegeliDumpStructureWithoutData<T>>,\n        int> = 0>\ninline void DumpStructure(const T* object,\n                          ABSL_ATTRIBUTE_UNUSED absl::string_view substr,\n                          std::ostream& dest) {\n  RiegeliDumpStructure(object, dest);\n}\n\ntemplate <\n    typename T,\n    std::enable_if_t<std::conjunction_v<\n                         std::negation<HasRiegeliDumpStructureWithSubstr<T>>,\n                         std::negation<HasRiegeliDumpStructureWithoutData<T>>>,\n                     int> = 0>\ninline void DumpStructure(ABSL_ATTRIBUTE_UNUSED const T* object,\n                          ABSL_ATTRIBUTE_UNUSED absl::string_view substr,\n                          std::ostream& dest) {\n  chain_internal::DumpStructureDefault(dest);\n}\n\n}  // namespace chain_internal\n\n// Supports `ExternalRef` and `Chain::Block`.\nvoid RiegeliDumpStructure(const std::string* self, std::ostream& dest);\n\ntemplate <typename T>\nstruct Chain::ExternalMethodsFor {\n  // Creates an external block containing an external object constructed from\n  // `object`, and sets block data to `BytesRef(new_object)`.\n  static IntrusiveSharedPtr<RawBlock> NewBlock(Initializer<T> object);\n\n  // Creates an external block containing an external object constructed from\n  // `object`, and sets block data to `data`.\n  static IntrusiveSharedPtr<RawBlock> NewBlock(Initializer<T> object,\n                                               absl::string_view substr);\n\n private:\n  static void DeleteBlock(RawBlock* block);\n  static void DumpStructure(const RawBlock& block, std::ostream& dest);\n  static constexpr size_t DynamicSizeOf();\n  static void RegisterSubobjects(const RawBlock* block,\n                                 MemoryEstimator& memory_estimator);\n\n public:\n  static constexpr ExternalMethods kMethods = {\n      DeleteBlock, DumpStructure, DynamicSizeOf(), RegisterSubobjects};\n};\n\ntemplate <typename T>\ninline IntrusiveSharedPtr<Chain::RawBlock>\nChain::ExternalMethodsFor<T>::NewBlock(Initializer<T> object) {\n  return IntrusiveSharedPtr<RawBlock>(\n      NewAligned<RawBlock, UnsignedMax(alignof(RawBlock), alignof(T))>(\n          RawBlock::kExternalAllocatedSize<T>(), std::move(object)));\n}\n\ntemplate <typename T>\ninline IntrusiveSharedPtr<Chain::RawBlock>\nChain::ExternalMethodsFor<T>::NewBlock(Initializer<T> object,\n                                       absl::string_view substr) {\n  return IntrusiveSharedPtr<RawBlock>(\n      NewAligned<RawBlock, UnsignedMax(alignof(RawBlock), alignof(T))>(\n          RawBlock::kExternalAllocatedSize<T>(), std::move(object), substr));\n}\n\ntemplate <typename T>\nvoid Chain::ExternalMethodsFor<T>::DeleteBlock(RawBlock* block) {\n  chain_internal::CallOperator(std::move(block->unchecked_external_object<T>()),\n                               *block);\n  block->unchecked_external_object<T>().~T();\n  DeleteAligned<RawBlock, UnsignedMax(alignof(RawBlock), alignof(T))>(\n      block, RawBlock::kExternalAllocatedSize<T>());\n}\n\ntemplate <typename T>\nvoid Chain::ExternalMethodsFor<T>::DumpStructure(const RawBlock& block,\n                                                 std::ostream& dest) {\n  chain_internal::DumpStructure(&block.unchecked_external_object<T>(), block,\n                                dest);\n}\n\ntemplate <typename T>\nconstexpr size_t Chain::ExternalMethodsFor<T>::DynamicSizeOf() {\n  return RawBlock::kExternalAllocatedSize<T>();\n}\n\ntemplate <typename T>\nvoid Chain::ExternalMethodsFor<T>::RegisterSubobjects(\n    const RawBlock* block, MemoryEstimator& memory_estimator) {\n  chain_internal::RegisterSubobjects(&block->unchecked_external_object<T>(),\n                                     *block, memory_estimator);\n}\n\ntemplate <typename T>\ninline Chain::RawBlock::RawBlock(Initializer<T> object) {\n  external_.methods = &ExternalMethodsFor<T>::kMethods;\n  new (&unchecked_external_object<T>()) T(std::move(object));\n  substr_ = BytesRef(unchecked_external_object<T>());\n  RIEGELI_ASSERT(is_external()) << \"A RawBlock with allocated_end_ == nullptr \"\n                                   \"should be considered external\";\n}\n\ntemplate <typename T>\ninline Chain::RawBlock::RawBlock(Initializer<T> object,\n                                 absl::string_view substr)\n    : substr_(substr) {\n  external_.methods = &ExternalMethodsFor<T>::kMethods;\n  new (&unchecked_external_object<T>()) T(std::move(object));\n  RIEGELI_ASSERT(is_external()) << \"A RawBlock with allocated_end_ == nullptr \"\n                                   \"should be considered external\";\n  AssertSubstr(unchecked_external_object<T>(), substr);\n}\n\nconstexpr size_t Chain::RawBlock::kInternalAllocatedOffset() {\n  return offsetof(RawBlock, allocated_begin_);\n}\n\ntemplate <typename T>\nconstexpr size_t Chain::RawBlock::kExternalObjectOffset() {\n  return RoundUp<alignof(T)>(offsetof(RawBlock, external_) +\n                             offsetof(External, object_lower_bound));\n}\n\ntemplate <typename T>\nconstexpr size_t Chain::RawBlock::kExternalAllocatedSize() {\n  return kExternalObjectOffset<T>() + sizeof(T);\n}\n\ntemplate <typename Ownership>\ninline Chain::RawBlock* Chain::RawBlock::Ref() {\n  ref_count_.Ref<Ownership>();\n  return this;\n}\n\ntemplate <typename Ownership>\ninline void Chain::RawBlock::Unref() {\n  if (ref_count_.Unref<Ownership>()) {\n    if (is_internal()) {\n      DeleteAligned<RawBlock>(this, kInternalAllocatedOffset() + capacity());\n    } else {\n      external_.methods->delete_block(this);\n    }\n  }\n}\n\ninline bool Chain::RawBlock::has_unique_owner() const {\n  return ref_count_.HasUniqueOwner();\n}\n\ninline size_t Chain::RawBlock::capacity() const {\n  RIEGELI_ASSERT(is_internal())\n      << \"Failed precondition of Chain::RawBlock::capacity(): \"\n         \"block not internal\";\n  return PtrDistance(allocated_begin_, allocated_end_);\n}\n\ntemplate <typename T>\ninline T& Chain::RawBlock::unchecked_external_object() {\n  RIEGELI_ASSERT(is_external())\n      << \"Failed precondition of Chain::RawBlock::unchecked_external_object(): \"\n      << \"block not external\";\n  return *std::launder(reinterpret_cast<T*>(reinterpret_cast<char*>(this) +\n                                            kExternalObjectOffset<T>()));\n}\n\ntemplate <typename T>\ninline const T& Chain::RawBlock::unchecked_external_object() const {\n  RIEGELI_ASSERT(is_external())\n      << \"Failed precondition of Chain::RawBlock::unchecked_external_object(): \"\n      << \"block not external\";\n  return *std::launder(reinterpret_cast<const T*>(\n      reinterpret_cast<const char*>(this) + kExternalObjectOffset<T>()));\n}\n\ntemplate <typename T>\ninline const T* Chain::RawBlock::checked_external_object() const {\n  return is_external() && external_.methods == &ExternalMethodsFor<T>::kMethods\n             ? &unchecked_external_object<T>()\n             : nullptr;\n}\n\ntemplate <typename T>\ninline T* Chain::RawBlock::checked_external_object_with_unique_owner() {\n  return is_external() &&\n                 external_.methods == &ExternalMethodsFor<T>::kMethods &&\n                 has_unique_owner()\n             ? &unchecked_external_object<T>()\n             : nullptr;\n}\n\ninline bool Chain::RawBlock::TryClear() {\n  if (is_mutable()) {\n    substr_ = substr_.substr(0, 0);\n    return true;\n  }\n  return false;\n}\n\ninline bool Chain::RawBlock::TryRemoveSuffix(size_t length) {\n  RIEGELI_ASSERT_LE(length, size())\n      << \"Failed precondition of Chain::RawBlock::TryRemoveSuffix(): \"\n      << \"length to remove greater than current size\";\n  if (is_mutable()) {\n    substr_.remove_suffix(length);\n    return true;\n  }\n  return false;\n}\n\ninline bool Chain::RawBlock::TryRemovePrefix(size_t length) {\n  RIEGELI_ASSERT_LE(length, size())\n      << \"Failed precondition of Chain::RawBlock::TryRemovePrefix(): \"\n      << \"length to remove greater than current size\";\n  if (is_mutable()) {\n    substr_.remove_prefix(length);\n    return true;\n  }\n  return false;\n}\n\ninline Chain::BlockPtrPtr Chain::BlockPtrPtr::from_ptr(const BlockPtr* ptr) {\n  return BlockPtrPtr(reinterpret_cast<uintptr_t>(ptr));\n}\n\ninline bool Chain::BlockPtrPtr::is_special() const {\n  return repr_ <= sizeof(BlockPtr);\n}\n\ninline const Chain::BlockPtr* Chain::BlockPtrPtr::as_ptr() const {\n  RIEGELI_ASSERT(!is_special()) << \"Unexpected special BlockPtrPtr value\";\n  return reinterpret_cast<const BlockPtr*>(repr_);\n}\n\n// Code conditional on `is_special()` is written such that both branches\n// typically compile to the same code, allowing the compiler eliminate the\n// `is_special()` checks.\n\ninline Chain::BlockPtrPtr Chain::BlockPtrPtr::operator+(ptrdiff_t n) const {\n  if (is_special()) {\n    return BlockPtrPtr(IntCast<uintptr_t>(IntCast<ptrdiff_t>(repr_) +\n                                          n * ptrdiff_t{sizeof(RawBlock*)}));\n  }\n  return BlockPtrPtr::from_ptr(as_ptr() + n);\n}\n\ninline Chain::BlockPtrPtr Chain::BlockPtrPtr::operator-(ptrdiff_t n) const {\n  if (is_special()) {\n    return BlockPtrPtr(IntCast<uintptr_t>(IntCast<ptrdiff_t>(repr_) -\n                                          n * ptrdiff_t{sizeof(RawBlock*)}));\n  }\n  return BlockPtrPtr::from_ptr(as_ptr() - n);\n}\n\ninline Chain::BlockRef::operator absl::string_view() const {\n  if (ptr_ == BlockIterator::kBeginShortData) {\n    return chain_->short_data();\n  } else {\n    return *ptr_.as_ptr()->block_ptr;\n  }\n}\n\ninline bool Chain::BlockRef::empty() const {\n  return ptr_ != BlockIterator::kBeginShortData &&\n         ptr_.as_ptr()->block_ptr->empty();\n}\n\ninline const char* Chain::BlockRef::data() const {\n  if (ptr_ == BlockIterator::kBeginShortData) {\n    return chain_->short_data_begin();\n  } else {\n    return ptr_.as_ptr()->block_ptr->data_begin();\n  }\n}\n\ninline size_t Chain::BlockRef::size() const {\n  if (ptr_ == BlockIterator::kBeginShortData) {\n    return chain_->size_;\n  } else {\n    return ptr_.as_ptr()->block_ptr->size();\n  }\n}\n\ninline bool Chain::BlockRef::ExternalCopy() const {\n  return ptr_ == BlockIterator::kBeginShortData;\n}\n\ninline Chain::Block Chain::BlockRef::ToChainBlock(\n    absl::string_view substr) const {\n  RIEGELI_ASSERT(ptr_ != BlockIterator::kBeginShortData)\n      << \"Failed precondition of RiegeliToChainBlock(const Chain::BlockRef*): \"\n         \"case excluded by RiegeliExternalCopy()\";\n  return Block(ptr_.as_ptr()->block_ptr, substr);\n}\n\ntemplate <typename Callback>\ninline void Chain::BlockRef::ExternalDelegate(absl::string_view substr,\n                                              Callback&& delegate_to) const {\n  RIEGELI_ASSERT(ptr_ != BlockIterator::kBeginShortData)\n      << \"Failed precondition of \"\n         \"RiegeliExternalDelegate(const Chain::BlockRef*): \"\n         \"case excluded by RiegeliExternalCopy()\";\n  std::forward<Callback>(delegate_to)(Block(ptr_.as_ptr()->block_ptr), substr);\n}\n\ntemplate <typename T>\ninline const T* Chain::BlockRef::external_object() const {\n  if (ptr_ == BlockIterator::kBeginShortData) {\n    return nullptr;\n  } else {\n    return ptr_.as_ptr()->block_ptr->checked_external_object<T>();\n  }\n}\n\ninline Chain::BlockIterator::BlockIterator(\n    const Chain* chain ABSL_ATTRIBUTE_LIFETIME_BOUND, size_t block_index)\n    : chain_(chain),\n      ptr_((ABSL_PREDICT_FALSE(chain_ == nullptr) ? kBeginShortData\n            : chain_->begin_ == chain_->end_\n                ? (chain_->empty() ? kEndShortData : kBeginShortData)\n                : BlockPtrPtr::from_ptr(chain_->begin_)) +\n           IntCast<ptrdiff_t>(block_index)) {}\n\ninline Chain::BlockIterator::BlockIterator(const Chain* chain, BlockPtrPtr ptr)\n    : chain_(chain), ptr_(ptr) {}\n\ninline size_t Chain::BlockIterator::block_index() const {\n  if (ptr_ == kBeginShortData) {\n    return 0;\n  } else if (ptr_ == kEndShortData) {\n    return chain_->empty() ? 0 : 1;\n  } else {\n    return PtrDistance(chain_->begin_, ptr_.as_ptr());\n  }\n}\n\ninline size_t Chain::BlockIterator::CharIndexInChain(\n    size_t char_index_in_block) const {\n  return CharIndexInChainInternal() + char_index_in_block;\n}\n\ninline Chain::BlockIterator::reference Chain::BlockIterator::operator*() const {\n  RIEGELI_ASSERT(ptr_ != kEndShortData)\n      << \"Failed precondition of Chain::BlockIterator::operator*: \"\n         \"iterator is end()\";\n  return BlockRef(chain_, ptr_);\n}\n\ninline Chain::BlockIterator::pointer Chain::BlockIterator::operator->() const {\n  return pointer(**this);\n}\n\ninline Chain::BlockIterator& Chain::BlockIterator::operator++() {\n  ptr_ = ptr_ + 1;\n  return *this;\n}\n\ninline Chain::BlockIterator Chain::BlockIterator::operator++(int) {\n  const BlockIterator tmp = *this;\n  ++*this;\n  return tmp;\n}\n\ninline Chain::BlockIterator& Chain::BlockIterator::operator--() {\n  ptr_ = ptr_ - 1;\n  return *this;\n}\n\ninline Chain::BlockIterator Chain::BlockIterator::operator--(int) {\n  const BlockIterator tmp = *this;\n  --*this;\n  return tmp;\n}\n\ninline Chain::BlockIterator& Chain::BlockIterator::operator+=(\n    difference_type n) {\n  ptr_ = ptr_ + n;\n  return *this;\n}\n\ninline Chain::BlockIterator Chain::BlockIterator::operator+(\n    difference_type n) const {\n  return BlockIterator(*this) += n;\n}\n\ninline Chain::BlockIterator& Chain::BlockIterator::operator-=(\n    difference_type n) {\n  ptr_ = ptr_ - n;\n  return *this;\n}\n\ninline Chain::BlockIterator Chain::BlockIterator::operator-(\n    difference_type n) const {\n  return BlockIterator(*this) -= n;\n}\n\ninline Chain::BlockIterator::reference Chain::BlockIterator::operator[](\n    difference_type n) const {\n  return *(*this + n);\n}\n\ntemplate <typename T,\n          std::enable_if_t<\n              std::conjunction_v<NotSameRef<Chain::Block, TargetT<T>>,\n                                 std::is_convertible<TargetT<T>, BytesRef>>,\n              int>>\ninline Chain::Block::Block(T&& object)\n    : block_(\n          ExternalMethodsFor<TargetT<T>>::NewBlock(std::forward<T>(object))) {}\n\ntemplate <typename T>\ninline Chain::Block::Block(T&& object, absl::string_view substr)\n    : block_(ExternalMethodsFor<TargetT<T>>::NewBlock(std::forward<T>(object),\n                                                      substr)) {}\n\ninline Chain::Block::Block(RawBlock* block, absl::string_view substr) {\n  if (block->size() == substr.size()) {\n    block_.Reset(block, kShareOwnership);\n    return;\n  }\n  if (const Block* const block_ptr = block->checked_external_object<Block>()) {\n    // `block` is already a `Block`. Refer to its target instead.\n    block = block_ptr->block_.get();\n  }\n  block_.Reset(block, kShareOwnership);\n  block_ = ExternalMethodsFor<Block>::NewBlock(std::move(*this), substr);\n}\n\ninline Chain::Block::Block(RawBlock* block) {\n  if (const Block* const block_ptr = block->checked_external_object<Block>()) {\n    // `block` is already a `Block`. Refer to its target instead.\n    block = block_ptr->block_.get();\n  }\n  block_.Reset(block, kShareOwnership);\n}\n\ninline Chain::Block::Block(IntrusiveSharedPtr<RawBlock> block) {\n  if (const Block* const block_ptr = block->checked_external_object<Block>()) {\n    // `block` is already a `Block`. Refer to its target instead.\n    block = block_ptr->block_;\n  }\n  block_ = std::move(block);\n}\n\ninline ExternalStorage Chain::Block::ToExternalStorage() && {\n  return ExternalStorage(block_.Release(), [](void* ptr) {\n    static_cast<RawBlock*>(ptr)->Unref();\n  });\n}\n\ninline Chain::Blocks::iterator Chain::Blocks::begin() const {\n  return BlockIterator(chain_,\n                       chain_->begin_ == chain_->end_\n                           ? (chain_->empty() ? BlockIterator::kEndShortData\n                                              : BlockIterator::kBeginShortData)\n                           : BlockPtrPtr::from_ptr(chain_->begin_));\n}\n\ninline Chain::Blocks::iterator Chain::Blocks::end() const {\n  return BlockIterator(chain_, chain_->begin_ == chain_->end_\n                                   ? BlockIterator::kEndShortData\n                                   : BlockPtrPtr::from_ptr(chain_->end_));\n}\n\ninline Chain::Blocks::size_type Chain::Blocks::size() const {\n  if (chain_->begin_ == chain_->end_) {\n    return chain_->empty() ? 0 : 1;\n  } else {\n    return PtrDistance(chain_->begin_, chain_->end_);\n  }\n}\n\ninline bool Chain::Blocks::empty() const {\n  return chain_->begin_ == chain_->end_ && chain_->empty();\n}\n\ninline Chain::Blocks::reference Chain::Blocks::operator[](size_type n) const {\n  RIEGELI_ASSERT_LT(n, size())\n      << \"Failed precondition of Chain::Blocks::operator[]: \"\n         \"block index out of range\";\n  return BlockRef(chain_, chain_->begin_ == chain_->end_\n                              ? BlockIterator::kBeginShortData\n                              : BlockPtrPtr::from_ptr(chain_->begin_ + n));\n}\n\ninline Chain::Blocks::reference Chain::Blocks::at(size_type n) const {\n  RIEGELI_CHECK_LT(n, size()) << \"Failed precondition of Chain::Blocks::at(): \"\n                                 \"block index out of range\";\n  return BlockRef(chain_, chain_->begin_ == chain_->end_\n                              ? BlockIterator::kBeginShortData\n                              : BlockPtrPtr::from_ptr(chain_->begin_ + n));\n}\n\ninline Chain::Blocks::reference Chain::Blocks::front() const {\n  RIEGELI_ASSERT(!empty())\n      << \"Failed precondition of Chain::Blocks::front(): no blocks\";\n  return BlockRef(chain_, chain_->begin_ == chain_->end_\n                              ? BlockIterator::kBeginShortData\n                              : BlockPtrPtr::from_ptr(chain_->begin_));\n}\n\ninline Chain::Blocks::reference Chain::Blocks::back() const {\n  RIEGELI_ASSERT(!empty())\n      << \"Failed precondition of Chain::Blocks::back(): no blocks\";\n  return BlockRef(chain_, chain_->begin_ == chain_->end_\n                              ? BlockIterator::kBeginShortData\n                              : BlockPtrPtr::from_ptr(chain_->end_ - 1));\n}\n\ntemplate <typename T>\nconstexpr size_t Chain::kExternalAllocatedSize() {\n  return RawBlock::kExternalAllocatedSize<T>();\n}\n\ninline Chain::Chain(BytesRef src) { Initialize(src); }\n\ninline Chain::Chain(ExternalRef src) { std::move(src).InitializeTo(*this); }\n\ntemplate <typename Src,\n          std::enable_if_t<SupportsExternalRefWhole<Src>::value, int>>\ninline Chain::Chain(Src&& src) {\n  ExternalRef(std::forward<Src>(src)).InitializeTo(*this);\n}\n\ninline Chain::Chain(Block src) {\n  if (src.raw_block() != nullptr) Initialize(std::move(src));\n}\n\ninline Chain::Chain(Chain&& that) noexcept\n    : size_(std::exchange(that.size_, 0)) {\n  // Use `std::memcpy()` instead of copy constructor to silence\n  // `-Wmaybe-uninitialized` in gcc.\n  std::memcpy(&block_ptrs_, &that.block_ptrs_, sizeof(BlockPtrs));\n  if (that.has_here()) {\n    // `that.has_here()` implies that `that.begin_ == that.block_ptrs_.here`\n    // already.\n    begin_ = block_ptrs_.here;\n    end_ = block_ptrs_.here + (std::exchange(that.end_, that.block_ptrs_.here) -\n                               that.block_ptrs_.here);\n  } else {\n    begin_ = std::exchange(that.begin_, that.block_ptrs_.here);\n    end_ = std::exchange(that.end_, that.block_ptrs_.here);\n  }\n  // It does not matter what is left in `that.block_ptrs_` because `that.begin_`\n  // and `that.end_` point to the empty prefix of `that.block_ptrs_.here[]`.\n}\n\ninline Chain& Chain::operator=(Chain&& that) noexcept {\n  // Exchange `that.begin_` and `that.end_` early to support self-assignment.\n  BlockPtr* begin;\n  BlockPtr* end;\n  if (that.has_here()) {\n    // `that.has_here()` implies that `that.begin_ == that.block_ptrs_.here`\n    // already.\n    begin = block_ptrs_.here;\n    end = block_ptrs_.here + (std::exchange(that.end_, that.block_ptrs_.here) -\n                              that.block_ptrs_.here);\n  } else {\n    begin = std::exchange(that.begin_, that.block_ptrs_.here);\n    end = std::exchange(that.end_, that.block_ptrs_.here);\n  }\n  UnrefBlocks();\n  DeleteBlockPtrs();\n  // It does not matter what is left in `that.block_ptrs_` because `that.begin_`\n  // and `that.end_` point to the empty prefix of `that.block_ptrs_.here[]`. Use\n  // `std::memcpy()` instead of assignment to silence `-Wmaybe-uninitialized` in\n  // gcc.\n  std::memcpy(&block_ptrs_, &that.block_ptrs_, sizeof(BlockPtrs));\n  begin_ = begin;\n  end_ = end;\n  size_ = std::exchange(that.size_, 0);\n  return *this;\n}\n\ninline Chain::~Chain() {\n  UnrefBlocks();\n  DeleteBlockPtrs();\n}\n\ninline void Chain::Reset() { Clear(); }\n\ninline void Chain::Reset(ExternalRef src) { std::move(src).AssignTo(*this); }\n\ntemplate <typename Src,\n          std::enable_if_t<SupportsExternalRefWhole<Src>::value, int>>\ninline void Chain::Reset(Src&& src) {\n  ExternalRef(std::forward<Src>(src)).AssignTo(*this);\n}\n\ninline void Chain::Clear() {\n  size_ = 0;\n  if (begin_ != end_) ClearSlow();\n}\n\ninline void Chain::Initialize(absl::string_view src) {\n  RIEGELI_ASSERT_EQ(size_, 0u)\n      << \"Failed precondition of Chain::Initialize(string_view): \"\n         \"size not reset\";\n  if (src.size() <= kMaxShortDataSize) {\n    if (src.empty()) return;\n    EnsureHasHere();\n    size_ = src.size();\n    std::memcpy(short_data_begin(), src.data(), src.size());\n    return;\n  }\n  InitializeSlow(src);\n}\n\ninline void Chain::Initialize(Block src) {\n  size_ = src.raw_block()->size();\n  (end_++)->block_ptr = std::move(src).raw_block().Release();\n}\n\ninline absl::string_view Chain::short_data() const {\n  return absl::string_view(short_data_begin(), size_);\n}\n\ninline char* Chain::short_data_begin() {\n  RIEGELI_ASSERT_EQ(begin_, end_)\n      << \"Failed precondition of Chain::short_data_begin(): blocks exist\";\n  RIEGELI_ASSERT(empty() || has_here())\n      << \"Failed precondition of Chain::short_data_begin(): \"\n         \"block pointer array is allocated\";\n  return block_ptrs_.short_data;\n}\n\ninline const char* Chain::short_data_begin() const {\n  RIEGELI_ASSERT_EQ(begin_, end_)\n      << \"Failed precondition of Chain::short_data_begin(): blocks exist\";\n  RIEGELI_ASSERT(empty() || has_here())\n      << \"Failed precondition of Chain::short_data_begin(): \"\n         \"block pointer array is allocated\";\n  return block_ptrs_.short_data;\n}\n\ninline void Chain::DeleteBlockPtrs() {\n  if (has_allocated()) {\n    std::allocator<BlockPtr>().deallocate(\n        block_ptrs_.allocated.begin,\n        2 * PtrDistance(block_ptrs_.allocated.begin,\n                        block_ptrs_.allocated.end));\n  }\n}\n\ninline void Chain::EnsureHasHere() {\n  RIEGELI_ASSERT_EQ(begin_, end_)\n      << \"Failed precondition of Chain::EnsureHasHere(): blocks exist\";\n  if (ABSL_PREDICT_FALSE(has_allocated())) {\n    DeleteBlockPtrs();\n    begin_ = block_ptrs_.here;\n    end_ = block_ptrs_.here;\n  }\n}\n\ninline void Chain::UnrefBlocks() { UnrefBlocks(begin_, end_); }\n\ninline void Chain::UnrefBlocks(const BlockPtr* begin, const BlockPtr* end) {\n  if (begin != end) UnrefBlocksSlow(begin, end);\n}\n\ninline Chain::Blocks Chain::blocks() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return Blocks(this);\n}\n\ninline std::optional<absl::string_view> Chain::TryFlat() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  switch (end_ - begin_) {\n    case 0:\n      return short_data();\n    case 1:\n      return *front();\n    default:\n      return std::nullopt;\n  }\n}\n\ninline absl::string_view Chain::Flatten() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  switch (end_ - begin_) {\n    case 0:\n      return short_data();\n    case 1:\n      return *front();\n    default:\n      return FlattenSlow();\n  }\n}\n\ninline absl::Span<char> Chain::AppendFixedBuffer(size_t length, Options options)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return AppendBuffer(length, length, length, options);\n}\n\ninline absl::Span<char> Chain::PrependFixedBuffer(\n    size_t length, Options options) ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return PrependBuffer(length, length, length, options);\n}\n\ninline void Chain::Append(ExternalRef src) { std::move(src).AppendTo(*this); }\n\ninline void Chain::Append(ExternalRef src, Options options) {\n  std::move(src).AppendTo(*this, options);\n}\n\ninline void Chain::Prepend(ExternalRef src) { std::move(src).PrependTo(*this); }\n\ninline void Chain::Prepend(ExternalRef src, Options options) {\n  std::move(src).PrependTo(*this, options);\n}\n\ntemplate <typename Src,\n          std::enable_if_t<SupportsExternalRefWhole<Src>::value, int>>\ninline void Chain::Append(Src&& src) {\n  ExternalRef(std::forward<Src>(src)).AppendTo(*this);\n}\n\ntemplate <typename Src,\n          std::enable_if_t<SupportsExternalRefWhole<Src>::value, int>>\ninline void Chain::Append(Src&& src, Options options) {\n  ExternalRef(std::forward<Src>(src)).AppendTo(*this, options);\n}\n\ntemplate <typename Src,\n          std::enable_if_t<SupportsExternalRefWhole<Src>::value, int>>\ninline void Chain::Prepend(Src&& src) {\n  ExternalRef(std::forward<Src>(src)).PrependTo(*this);\n}\n\ntemplate <typename Src,\n          std::enable_if_t<SupportsExternalRefWhole<Src>::value, int>>\ninline void Chain::Prepend(Src&& src, Options options) {\n  ExternalRef(std::forward<Src>(src)).PrependTo(*this, options);\n}\n\ntemplate <typename HashState>\nHashState Chain::HashValue(HashState hash_state) const {\n  if (empty()) return HashState::combine(std::move(hash_state), size_t{0});\n  RIEGELI_ASSERT(!blocks().empty());\n  constexpr size_t kChunkSize = 256;\n  char chunk[kChunkSize];\n\n  // Hash chunks of size `kChunkSize` using `HashState::combine_contiguous()`.\n  // The last chunk can be smaller; no chunk is empty. Then combine the size.\n  size_t position = 0;\n  for (size_t block_index = 0; block_index < blocks().size() - 1;\n       ++block_index) {\n    absl::string_view block = blocks()[block_index];\n    if (block.size() < kChunkSize - position) {\n      std::memcpy(chunk + position, block.data(), block.size());\n      position += block.size();\n      continue;\n    }\n    if (position > 0) {\n      const size_t remaining = kChunkSize - position;\n      std::memcpy(chunk + position, block.data(), remaining);\n      hash_state = HashState::combine_contiguous(std::move(hash_state), chunk,\n                                                 kChunkSize);\n      block.remove_prefix(remaining);\n    }\n    while (block.size() >= kChunkSize) {\n      hash_state = HashState::combine_contiguous(std::move(hash_state),\n                                                 block.data(), kChunkSize);\n      block.remove_prefix(kChunkSize);\n    }\n    std::memcpy(chunk, block.data(), block.size());\n    position = block.size();\n  }\n\n  // The last block can be hashed without copying its last chunk if there are no\n  // buffered data from the previous blocks.\n  absl::string_view block = blocks().back();\n  if (block.size() <= kChunkSize - position) {\n    if (position > 0) {\n      std::memcpy(chunk + position, block.data(), block.size());\n      position += block.size();\n      hash_state =\n          HashState::combine_contiguous(std::move(hash_state), chunk, position);\n    } else if (!block.empty()) {\n      hash_state = HashState::combine_contiguous(std::move(hash_state),\n                                                 block.data(), block.size());\n    }\n    return HashState::combine(std::move(hash_state), size());\n  }\n  if (position > 0) {\n    const size_t remaining = kChunkSize - position;\n    std::memcpy(chunk + position, block.data(), remaining);\n    hash_state =\n        HashState::combine_contiguous(std::move(hash_state), chunk, kChunkSize);\n    block.remove_prefix(remaining);\n  }\n  while (block.size() > kChunkSize) {\n    hash_state = HashState::combine_contiguous(std::move(hash_state),\n                                               block.data(), kChunkSize);\n    block.remove_prefix(kChunkSize);\n  }\n  RIEGELI_ASSERT(!block.empty());\n  hash_state = HashState::combine_contiguous(std::move(hash_state),\n                                             block.data(), block.size());\n  return HashState::combine(std::move(hash_state), size());\n}\n\ntemplate <typename Sink>\nvoid Chain::Stringify(Sink& dest) const {\n  for (const absl::string_view block : blocks()) dest.Append(block);\n}\n\ntemplate <typename DebugStream>\nvoid Chain::Debug(DebugStream& dest) const {\n  dest.DebugStringQuote();\n  for (const absl::string_view fragment : blocks()) {\n    dest.DebugStringFragment(fragment);\n  }\n  dest.DebugStringQuote();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_CHAIN_DETAILS_H_\n"
  },
  {
    "path": "riegeli/base/closing_ptr.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_CLOSING_PTR_H_\n#define RIEGELI_BASE_CLOSING_PTR_H_\n\n#include <memory>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// A deleter for `std::unique_ptr` which does nothing.\nstruct NullDeleter {\n  template <typename T>\n  void operator()(ABSL_ATTRIBUTE_UNUSED T* ptr) const {}\n};\n\n// Marks the pointer with the intent to transfer the responsibility to close the\n// object when done with the pointer, even though the object is not moved nor\n// destroyed.\n//\n// In the context of `Dependency` and `Any`, passing `ClosingPtr(&m)`\n// instead of `std::move(m)` avoids moving `m`, but the caller must ensure that\n// the dependent object is valid while the host object needs it.\n\ntemplate <typename T>\nusing ClosingPtrType = std::unique_ptr<T, NullDeleter>;\n\ntemplate <typename T>\ninline ClosingPtrType<T> ClosingPtr(T* ptr) {\n  return ClosingPtrType<T>(ptr);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_CLOSING_PTR_H_\n"
  },
  {
    "path": "riegeli/base/compact_string.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/compact_string.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <cstring>\n#include <ostream>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/estimated_allocated_size.h\"\n\nnamespace riegeli {\n\nvoid CompactString::AssignSlow(absl::string_view src) {\n  const size_t old_capacity = capacity();\n  DeleteRepr(std::exchange(\n      repr_,\n      MakeRepr(src, UnsignedMax(src.size(), old_capacity + old_capacity / 2))));\n}\n\nvoid CompactString::AssignSlow(const CompactString& that) {\n  const uintptr_t that_tag = that.repr_ & kTagMask;\n  const size_t that_size = that.allocated_size_for_tag(that_tag);\n  if (ABSL_PREDICT_TRUE(that_size <= capacity())) {\n    set_size(that_size);\n    // Use `std::memmove()` to support assigning from `*this`.\n    std::memmove(data(), that.allocated_data(), that_size);\n  } else {\n    AssignSlow(absl::string_view(that.allocated_data(), that_size));\n  }\n}\n\nuintptr_t CompactString::MakeReprSlow(size_t size, size_t capacity) {\n  RIEGELI_ASSERT_LE(size, capacity)\n      << \"Failed precondition of CompactString::MakeReprSlow(): \"\n         \"size exceeds capacity\";\n  RIEGELI_ASSERT_GT(capacity, kInlineCapacity)\n      << \"Failed precondition of CompactString::MakeReprSlow(): \"\n         \"representation is inline, use MakeRepr() instead\";\n  uintptr_t repr;\n  if (capacity <= 0xff) {\n    const size_t requested =\n        UnsignedMin(EstimatedAllocatedSize(capacity + 2), size_t{0xff + 2});\n    repr = reinterpret_cast<uintptr_t>(Allocate(requested) + 2);\n    set_allocated_capacity<uint8_t>(requested - 2, repr);\n    set_allocated_size<uint8_t>(size, repr);\n  } else if (capacity <= 0xffff) {\n    const size_t requested =\n        UnsignedMin(EstimatedAllocatedSize(capacity + 4), size_t{0xffff + 4});\n    repr = reinterpret_cast<uintptr_t>(Allocate(requested) + 4);\n    set_allocated_capacity<uint16_t>(requested - 4, repr);\n    set_allocated_size<uint16_t>(size, repr);\n  } else {\n    static_assert(sizeof(size_t) % 4 == 0, \"Unsupported size_t size\");\n    RIEGELI_CHECK_LE(capacity, max_size()) << \"CompactString capacity overflow\";\n    const size_t requested =\n        EstimatedAllocatedSize(capacity + 2 * sizeof(size_t));\n    repr =\n        reinterpret_cast<uintptr_t>(Allocate(requested) + 2 * sizeof(size_t));\n    set_allocated_capacity<size_t>(requested - 2 * sizeof(size_t), repr);\n    set_allocated_size<size_t>(size, repr);\n  }\n  return repr;\n}\n\nchar* CompactString::ResizeSlow(size_t new_size, size_t min_capacity,\n                                size_t used_size) {\n  RIEGELI_ASSERT_LE(new_size, min_capacity)\n      << \"Failed precondition of CompactString::ResizeSlow(): \"\n         \"size exceeds capacity\";\n  RIEGELI_ASSERT_LE(used_size, size())\n      << \"Failed precondition of CompactString::ResizeSlow(): \"\n         \"used size exceeds old size\";\n  RIEGELI_ASSERT_LE(used_size, new_size)\n      << \"Failed precondition of CompactString::ResizeSlow(): \"\n         \"used size exceeds new size\";\n  const size_t old_capacity = capacity();\n  RIEGELI_ASSERT_GT(min_capacity, kInlineCapacity)\n      << \"Inline representation has a fixed capacity, so reallocation is never \"\n         \"needed when the new capacity can use inline representation\";\n  const uintptr_t new_repr = MakeReprSlow(\n      new_size, UnsignedMax(min_capacity, old_capacity + old_capacity / 2));\n  char* ptr = allocated_data(new_repr);\n  std::memcpy(ptr, data(), used_size);\n  ptr += used_size;\n  DeleteRepr(std::exchange(repr_, new_repr));\n  return ptr;\n}\n\nvoid CompactString::ShrinkToFitSlow() {\n  const uintptr_t tag = repr_ & kTagMask;\n  RIEGELI_ASSERT_NE(tag, kInlineTag)\n      << \"Failed precondition of CompactString::ShrinkToFitSlow(): \"\n         \"representation is inline, use shrink_to_fit() instead\";\n  size_t size;\n  if (tag == 2) {\n    size = allocated_size<uint8_t>();\n    if (size > kInlineCapacity &&\n        allocated_capacity<uint8_t>() + 2 <=\n            UnsignedMin(EstimatedAllocatedSize(size + 2), size_t{0xff + 2})) {\n      return;\n    }\n  } else if (tag == 4) {\n    size = allocated_size<uint16_t>();\n    if (size > 0xff &&\n        allocated_capacity<uint16_t>() + 4 <=\n            UnsignedMin(EstimatedAllocatedSize(size + 4), size_t{0xffff + 4})) {\n      return;\n    }\n  } else if (tag == 0) {\n    size = allocated_size<size_t>();\n    if (size > 0xffff &&\n        allocated_capacity<size_t>() + 2 * sizeof(size_t) <=\n            EstimatedAllocatedSize(size + 2 * sizeof(size_t))) {\n      return;\n    }\n  } else {\n    RIEGELI_ASSUME_UNREACHABLE() << \"Impossible tag: \" << tag;\n  }\n  DeleteRepr(std::exchange(\n      repr_, MakeRepr(absl::string_view(allocated_data(), size))));\n}\n\nchar* CompactString::AppendSlow(size_t length) {\n  const size_t old_size = size();\n  RIEGELI_CHECK_LE(length, max_size() - old_size)\n      << \"CompactString size overflow\";\n  const size_t new_size = old_size + length;\n  return ResizeSlow(new_size, new_size, old_size);\n}\n\nvoid CompactString::AppendSlow(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of CompactString::AppendSlow(): \"\n         \"nothing to append\";\n  const size_t old_size = size();\n  const size_t old_capacity = capacity();\n  RIEGELI_CHECK_LE(src.size(), max_size() - old_size)\n      << \"CompactString size overflow\";\n  const size_t new_size = old_size + src.size();\n  RIEGELI_ASSERT_GT(new_size, kInlineCapacity)\n      << \"Inline representation has a fixed capacity, so reallocation is never \"\n         \"needed when the new capacity can use inline representation\";\n  const uintptr_t new_repr = MakeReprSlow(\n      new_size, UnsignedMax(new_size, old_capacity + old_capacity / 2));\n  char* ptr = allocated_data(new_repr);\n  std::memcpy(ptr, data(), old_size);\n  ptr += old_size;\n  // Copy from `src` before deleting `repr_` to support appending from a\n  // substring of `*this`.\n  std::memcpy(ptr, src.data(), src.size());\n  DeleteRepr(std::exchange(repr_, new_repr));\n}\n\nvoid CompactString::ReserveOneMoreByteSlow() {\n  const size_t used_size = size();\n  RIEGELI_ASSERT_GT(used_size + 1, kInlineCapacity)\n      << \"Inline representation has a fixed capacity, so reallocation is never \"\n         \"needed when the new capacity can use inline representation\";\n  const uintptr_t new_repr = MakeReprSlow(used_size, used_size + 1);\n  char* const ptr = allocated_data(new_repr);\n  std::memcpy(ptr, data(), used_size);\n  DeleteRepr(std::exchange(repr_, new_repr));\n}\n\nvoid CompactString::DumpStructure(absl::string_view substr,\n                                  std::ostream& dest) const {\n  dest << \"[compact_string] {\";\n  if (!substr.empty()) {\n    if (substr.data() != data()) {\n      dest << \" space_before: \" << PtrDistance(data(), substr.data());\n    }\n    dest << \" space_after: \"\n         << PtrDistance(substr.data() + substr.size(), data() + capacity());\n  }\n  dest << \" }\";\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/compact_string.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_COMPACT_STRING_H_\n#define RIEGELI_BASE_COMPACT_STRING_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <cstring>\n#include <iosfwd>\n#include <limits>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/config.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/hash/hash.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/external_data.h\"\n#include \"riegeli/base/new_aligned.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/type_traits.h\"\n\nnamespace riegeli {\n\n// `CompactString` provides a subset of functionality of `std::string`, while\n// having less space overhead. It is useful for storing many short strings for\n// a long time where each string owns its memory.\n//\n// A `CompactString` object internally consists of a pointer to heap-allocated\n// data. The representation has 4 cases, distinguished by how the pointer is\n// aligned modulo 8:\n//  * 6 - not really a pointer but short string optimization: the size is\n//        stored in bits [3..8), the data are stored in the remaining bytes\n//  * 2 - the size is stored before the data as `uint8_t`\n//  * 4 - the size is stored before the data as `uint16_t`\n//  * 0 - the size is stored before the data as `size_t`\n//\n// In the last three cases the capacity is stored before the size in the same\n// width as the size.\n//\n// The data are not necessarily NUL-terminated.\n//\n// Since `data()`, `size()`, `operator[]` etc. involve branches, for iteration\n// it is faster to store the result of conversion to `absl::string_view` and\n// iterate over that, or use `StringReader`, and for repeated appending it is\n// faster to use `CompactStringWriter`.\n//\n// Memory usage of a `CompactString` of capacity c, assuming 8-byte pointers,\n// where H(n) is memory usage of a heap-allocated block of length n:\n//\n//          c        | `CompactString` memory usage\n//   ----------------|------------------------------\n//        0 .. 7     | 8\n//        8 .. 255   | 8 + H(c + 2)\n//      256 .. 65535 | 8 + H(c + 4)\n//    65536 .. max   | 8 + H(c + 16)\n//\n// For sizes up to 255 this is less than libc++ `std::string` by about 15, and\n// less than libstdc++ `std::string` by about 23.\nclass ABSL_ATTRIBUTE_TRIVIAL_ABI CompactString\n    : public WithCompare<CompactString> {\n public:\n  static constexpr size_t max_size() {\n    return std::numeric_limits<size_t>::max() - 2 * sizeof(size_t);\n  }\n\n  // Creates an empty `CompactString`.\n  CompactString() = default;\n\n  // Creates a `CompactString` with the given size and uninitialized data.\n  explicit CompactString(size_t size) : repr_(MakeRepr(size)) {}\n\n  // Creates a `CompactString` which holds a copy of `src`.\n  explicit CompactString(BytesRef src) : repr_(MakeRepr(src)) {}\n  CompactString& operator=(BytesRef src);\n\n  // Creates a `CompactString` which holds a copy of `src`. Reserves one extra\n  // char so that `c_str()` does not need reallocation.\n  static CompactString ForCStr(BytesRef src) {\n    return CompactString(FromReprTag(), MakeRepr(src, src.size() + 1));\n  }\n\n  CompactString(const CompactString& that);\n  CompactString& operator=(const CompactString& that);\n\n  // The source `CompactString` is left empty.\n  CompactString(CompactString&& that) noexcept\n      : repr_(std::exchange(that.repr_, kInlineTag)) {}\n  CompactString& operator=(CompactString&& that) {\n    DeleteRepr(std::exchange(repr_, std::exchange(that.repr_, kInlineTag)));\n    return *this;\n  }\n\n  ~CompactString() { DeleteRepr(repr_); }\n\n  // Views the value as an `absl::string_view`.\n  /*implicit*/ operator absl::string_view() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  bool empty() const { return size() == 0; }\n  char* data() ABSL_ATTRIBUTE_LIFETIME_BOUND;              // Never `nullptr`.\n  const char* data() const ABSL_ATTRIBUTE_LIFETIME_BOUND;  // Never `nullptr`.\n  size_t size() const;\n  size_t capacity() const;\n\n  char& operator[](size_t index) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const char& operator[](size_t index) const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  char& at(size_t index) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const char& at(size_t index) const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  char& front() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const char& front() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  char& back() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const char& back() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  void clear() { set_size(0); }\n\n  // Sets the size to `new_size` without reallocation.\n  //\n  // If `new_size <= size()`, the prefix of data with `new_size` is preserved.\n  //\n  // If `new_size >= size()`, all existing data are preserved and new data are\n  // uninitialized.\n  //\n  // Precondition: `new_size <= capacity()`\n  void set_size(size_t new_size);\n\n  // Sets the size to `new_size`, reallocating if needed, ensuring that repeated\n  // growth has the cost proportional to the final size.\n  //\n  // If `new_size <= size()`, the prefix of data with `new_size` is preserved.\n  //\n  // If `new_size >= size()`, all existing data are preserved and new data are\n  // uninitialized.\n  //\n  // `resize(new_size)` is equivalent to `reserve(new_size)` followed by\n  // `set_size(new_size)`.\n  void resize(size_t new_size);\n\n  // Sets the size to `new_size`, ensuring that repeated growth has the cost\n  // proportional to the final size.\n  //\n  // The prefix of data with `used_size` is preserved.\n  //\n  // If `new_size > size()`, new data are uninitialized.\n  //\n  // Returns `data() + used_size`, for convenience of appending to previously\n  // used data.\n  //\n  // `resize(new_size, used_size)` is equivalent to `set_size(used_size)`\n  // followed by `resize(new_size)` and returning `data() + used_size`.\n  // `resize(new_size)` is equivalent to `resize(new_size, size())`.\n  //\n  // Preconditions:\n  //   `used_size <= size()`\n  //   `used_size <= new_size`\n  char* resize(size_t new_size, size_t used_size) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Ensures that `capacity() >= min_capacity`, ensuring that repeated growth\n  // has the cost proportional to the final size.\n  void reserve(size_t min_capacity);\n\n  void shrink_to_fit();\n\n  // Appends `length` uninitialized data.\n  //\n  // Returns `data() + used_size` where `used_size` is `size()` before the call,\n  // for convenience of appending to previously used data.\n  //\n  // `append(length)` is equivalent to `resize(size() + length, size())` with\n  // a check against overflow of `size() + length`.\n  char* append(size_t length) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Appends `src`.\n  void append(absl::string_view src);\n\n  // Ensures that `data()` are NUL-terminated after `size()` and returns\n  // `data()`.\n  //\n  // In contrast to `std::string::c_str()`, this is a non-const operation.\n  // It may reallocate the string and it writes the NUL each time.\n  const char* c_str() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns the representation of the `CompactString` as `uintptr_t`.\n  //\n  // Ownership is transferred to the `uintptr_t`, the `CompactString` is\n  // left empty. The `uintptr_t` must be passed exactly once to\n  // `CompactString::MoveFromRaw()` to recover the `CompactString` and free its\n  // memory.\n  //\n  // The returned `uintptr_t` is always even and never zero.\n  uintptr_t RawMove() && { return std::exchange(repr_, kInlineTag); }\n\n  // Returns a pointer to the representation of the `CompactString` as\n  // `uintptr_t`.\n  //\n  // Ownership is not transferred and the `CompactString` is unchanged.\n  //\n  // The returned `uintptr_t` is always even and never zero.\n  const uintptr_t* RawView() const { return &repr_; }\n\n  // Recovers a `CompactString` from the representation returned by\n  // `CompactString::RawMove()`.\n  //\n  // Ownership is transferred to the `CompactString`, `raw` must not be read\n  // again.\n  //\n  // Calling `MoveFromRaw()` and dropping its result frees the memory of the\n  // `CompactString`.\n  static CompactString MoveFromRaw(const uintptr_t& raw) {\n    RIEGELI_ASSERT_NE(raw, 0u)\n        << \"Failed precondition of CompactString::MoveFromRaw(): \"\n           \"representation is zero\";\n    RIEGELI_ASSERT_EQ(raw & 1, 0u)\n        << \"Failed precondition of CompactString::MoveFromRaw(): \"\n           \"representation is not even\";\n    const uintptr_t raw_copy = raw;\n    // The original `raw` will possibly hold a pointer which had ownership\n    // transferred and thus might no longer be valid. Hence reading `raw` again\n    // is most likely a bug.\n    MarkPoisoned(reinterpret_cast<const char*>(&raw), sizeof(uintptr_t));\n    return CompactString(FromReprTag(), raw_copy);\n  }\n\n  // Views contents of a `CompactString` from the representation returned by\n  // `CompactString::RawMove()` or `CompactString::RawView()`.\n  //\n  // Ownership is not transferred and `*raw` is unchanged.\n  static absl::string_view ViewFromRaw(\n      const uintptr_t* raw ABSL_ATTRIBUTE_LIFETIME_BOUND) {\n    RIEGELI_ASSERT_NE(*raw, 0u)\n        << \"Failed precondition of CompactString::ViewFromRaw(): \"\n           \"representation is zero\";\n    RIEGELI_ASSERT_EQ(*raw & 1, 0u)\n        << \"Failed precondition of CompactString::ViewFromRaw(): \"\n           \"representation is not even\";\n    const uintptr_t tag = *raw & kTagMask;\n    if (tag == kInlineTag) {\n      return absl::string_view(inline_data(raw), inline_size(*raw));\n    }\n    return absl::string_view(allocated_data(*raw),\n                             allocated_size_for_tag(tag, *raw));\n  }\n\n  // Returns the representation of a copy of the `CompactString` viewed from\n  // the representation returned by `CompactString::RawMove()`.\n  //\n  // Equivalent to `RawMove(CompactString(ViewFromRaw(&raw)))`.\n  static uintptr_t CopyRaw(uintptr_t raw) {\n    RIEGELI_ASSERT_NE(raw, 0u)\n        << \"Failed precondition of CompactString::CopyRaw(): \"\n           \"representation is zero\";\n    RIEGELI_ASSERT_EQ(raw & 1, 0u)\n        << \"Failed precondition of CompactString::CopyRaw(): \"\n           \"representation is not even\";\n    const uintptr_t tag = raw & kTagMask;\n    if (tag == kInlineTag) return raw;\n    return MakeRepr(absl::string_view(allocated_data(raw),\n                                      allocated_size_for_tag(tag, raw)));\n  }\n\n  static const char* CStrFromRaw(uintptr_t* raw);\n\n  friend bool operator==(const CompactString& a, const CompactString& b) {\n    return a.repr_ == b.repr_ || absl::string_view(a) == absl::string_view(b);\n  }\n  friend StrongOrdering RIEGELI_COMPARE(const CompactString& a,\n                                        const CompactString& b) {\n    if (a.repr_ == b.repr_) return StrongOrdering::equal;\n    return riegeli::Compare(absl::string_view(a), absl::string_view(b));\n  }\n\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<NotSameRef<CompactString, T>,\n                                          std::is_convertible<T&&, BytesRef>>,\n                       int> = 0>\n  friend bool operator==(const CompactString& a, T&& b) {\n    return absl::string_view(a) == BytesRef(std::forward<T>(b));\n  }\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<NotSameRef<CompactString, T>,\n                                          std::is_convertible<T&&, BytesRef>>,\n                       int> = 0>\n  friend StrongOrdering RIEGELI_COMPARE(const CompactString& a, T&& b) {\n    return riegeli::Compare(absl::string_view(a), BytesRef(std::forward<T>(b)));\n  }\n\n  template <typename HashState>\n  friend HashState AbslHashValue(HashState hash_state,\n                                 const CompactString& self) {\n    return HashState::combine(std::move(hash_state), absl::string_view(self));\n  }\n\n  // Default stringification by `absl::StrCat()` etc.\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const CompactString& src) {\n    dest.Append(absl::string_view(src));\n  }\n\n  friend std::ostream& operator<<(std::ostream& dest,\n                                  const CompactString& src) {\n    return dest << absl::string_view(src);\n  }\n\n  // Supports `absl::Format(&compact_string, format, args...)`.\n  friend void AbslFormatFlush(CompactString* dest, absl::string_view src) {\n    dest->append(src);\n  }\n\n  // Indicates support for:\n  //  * `ExternalRef(CompactString&&)`\n  //  * `ExternalRef(CompactString&&, substr)`\n  friend void RiegeliSupportsExternalRef(CompactString*) {}\n\n  // Supports `ExternalRef`.\n  friend bool RiegeliExternalCopy(const CompactString* self) {\n    return (self->repr_ & kTagMask) == kInlineTag;\n  }\n\n  // Supports `ExternalRef`.\n  friend ExternalStorage RiegeliToExternalStorage(CompactString* self) {\n    return ExternalStorage(\n        reinterpret_cast<void*>(std::exchange(self->repr_, kInlineTag)),\n        [](void* ptr) {\n          const uintptr_t repr = reinterpret_cast<uintptr_t>(ptr);\n          RIEGELI_ASSUME_NE(repr & kTagMask, kInlineTag)\n              << \"Failed precondition of \"\n                 \"RiegeliToExternalStorage(CompactString*): \"\n                 \"case excluded by RiegeliExternalCopy()\";\n          DeleteRepr(repr);\n        });\n  }\n\n  // Supports `ExternalRef` and `Chain::Block`.\n  friend void RiegeliDumpStructure(const CompactString* self,\n                                   absl::string_view substr,\n                                   std::ostream& dest) {\n    self->DumpStructure(substr, dest);\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const CompactString* self,\n                                        MemoryEstimator& memory_estimator) {\n    self->RegisterSubobjects(memory_estimator);\n  }\n\n private:\n  struct FromReprTag {\n    explicit FromReprTag() = default;\n  };\n\n  explicit CompactString(FromReprTag, uintptr_t raw) : repr_(raw) {}\n\n  static constexpr size_t kTagBits = 3;\n  static constexpr uintptr_t kTagMask = (1u << kTagBits) - 1;\n  static constexpr uintptr_t kInlineTag = 6;\n\n  static constexpr size_t kInlineCapacity =\n      UnsignedMin(sizeof(uintptr_t) - 1, size_t{0xff >> kTagBits});\n\n#if ABSL_IS_LITTLE_ENDIAN\n  static constexpr size_t kInlineDataOffset = 1;\n#elif ABSL_IS_BIG_ENDIAN\n  static constexpr size_t kInlineDataOffset = 0;\n#else\n#error Unknown endianness\n#endif\n\n  char* inline_data() { return inline_data(&repr_); }\n  const char* inline_data() const { return inline_data(&repr_); }\n\n  static char* inline_data(uintptr_t* repr) {\n    RIEGELI_ASSERT_EQ(*repr & kTagMask, kInlineTag)\n        << \"Failed precondition of CompactString::inline_data(): \"\n           \"representation not inline\";\n    return reinterpret_cast<char*>(repr) + kInlineDataOffset;\n  }\n  static const char* inline_data(const uintptr_t* repr) {\n    RIEGELI_ASSERT_EQ(*repr & kTagMask, kInlineTag)\n        << \"Failed precondition of CompactString::inline_data(): \"\n           \"representation not inline\";\n    return reinterpret_cast<const char*>(repr) + kInlineDataOffset;\n  }\n\n  size_t inline_size() const { return inline_size(repr_); }\n\n  static size_t inline_size(uintptr_t repr) {\n    RIEGELI_ASSERT_EQ(repr & kTagMask, kInlineTag)\n        << \"Failed precondition of CompactString::inline_size(): \"\n           \"representation not inline\";\n    const size_t size = IntCast<size_t>((repr & 0xff) >> kTagBits);\n    // This assumption helps the compiler to reason about comparisons with\n    // `size()`.\n    RIEGELI_ASSUME_LE(size, kInlineCapacity)\n        << \"Failed invariant of CompactString: \"\n           \"inline size never exceeds kInlineCapacity\";\n    return size;\n  }\n\n  char* allocated_data() const { return allocated_data(repr_); }\n\n  static char* allocated_data(uintptr_t repr) {\n    RIEGELI_ASSERT_NE(repr & kTagMask, kInlineTag)\n        << \"Failed precondition of CompactString::allocated_data(): \"\n           \"representation not allocated\";\n    return reinterpret_cast<char*>(repr);\n  }\n\n  size_t allocated_size_for_tag(uintptr_t tag) const {\n    return allocated_size_for_tag(tag, repr_);\n  }\n\n  static size_t allocated_size_for_tag(uintptr_t tag, uintptr_t repr) {\n    if (tag == 2) return allocated_size<uint8_t>(repr);\n    if (tag == 4) return allocated_size<uint16_t>(repr);\n    if (tag == 0) return allocated_size<size_t>(repr);\n    RIEGELI_ASSUME_UNREACHABLE() << \"Impossible tag: \" << tag;\n  }\n\n  template <typename T>\n  size_t allocated_size() const {\n    return allocated_size<T>(repr_);\n  }\n\n  template <typename T>\n  static size_t allocated_size(uintptr_t repr) {\n    const uintptr_t tag = repr & kTagMask;\n    RIEGELI_ASSERT_EQ(tag == 0 ? 2 * sizeof(size_t) : tag, 2 * sizeof(T))\n        << \"Failed precondition of CompactString::allocated_size(): \"\n           \"tag does not match size representation\";\n    T stored_size;\n    std::memcpy(&stored_size, allocated_data(repr) - sizeof(T), sizeof(T));\n    return size_t{stored_size};\n  }\n\n  void set_inline_size(size_t size) { set_inline_size(size, repr_); }\n\n  static void set_inline_size(size_t size, uintptr_t& repr) {\n    RIEGELI_ASSERT_EQ(repr & kTagMask, kInlineTag)\n        << \"Failed precondition of CompactString::set_inline_size(): \"\n           \"representation not inline\";\n    repr = (repr & ~(0xff & ~kTagMask)) | (size << kTagBits);\n  }\n\n  template <typename T>\n  void set_allocated_size(size_t size) {\n    set_allocated_size<T>(size, repr_);\n  }\n\n  template <typename T>\n  static void set_allocated_size(size_t size, uintptr_t repr) {\n    const uintptr_t tag = repr & kTagMask;\n    RIEGELI_ASSERT_EQ(tag == 0 ? 2 * sizeof(size_t) : tag, 2 * sizeof(T))\n        << \"Failed precondition of CompactString::set_allocated_size(): \"\n           \"tag does not match size representation\";\n    const T stored_size = IntCast<T>(size);\n    std::memcpy(allocated_data(repr) - sizeof(T), &stored_size, sizeof(T));\n  }\n\n  void set_allocated_size_for_tag(uintptr_t tag, size_t new_size);\n\n  size_t allocated_capacity_for_tag(uintptr_t tag) const {\n    return allocated_capacity_for_tag(tag, repr_);\n  }\n\n  static size_t allocated_capacity_for_tag(uintptr_t tag, uintptr_t repr) {\n    if (tag == 2) return allocated_capacity<uint8_t>(repr);\n    if (tag == 4) return allocated_capacity<uint16_t>(repr);\n    if (tag == 0) return allocated_capacity<size_t>(repr);\n    RIEGELI_ASSUME_UNREACHABLE() << \"Impossible tag: \" << tag;\n  }\n\n  template <typename T>\n  size_t allocated_capacity() const {\n    return allocated_capacity<T>(repr_);\n  }\n\n  template <typename T>\n  static size_t allocated_capacity(uintptr_t repr) {\n    const uintptr_t tag = repr & kTagMask;\n    RIEGELI_ASSERT_EQ(tag == 0 ? 2 * sizeof(size_t) : tag, 2 * sizeof(T))\n        << \"Failed precondition of CompactString::allocated_capacity(): \"\n           \"tag does not match capacity representation\";\n    T stored_capacity;\n    std::memcpy(&stored_capacity, allocated_data(repr) - 2 * sizeof(T),\n                sizeof(T));\n    // This assumption helps the compiler to reason about comparisons with\n    // `capacity()`.\n    RIEGELI_ASSUME_GT(stored_capacity, kInlineCapacity)\n        << \"Failed invariant of CompactString: \"\n           \"allocated capacity always exceeds kInlineCapacity\";\n    return size_t{stored_capacity};\n  }\n\n  template <typename T>\n  static void set_allocated_capacity(size_t capacity, uintptr_t repr) {\n    const uintptr_t tag = repr & kTagMask;\n    RIEGELI_ASSERT_EQ(tag == 0 ? 2 * sizeof(size_t) : tag, 2 * sizeof(T))\n        << \"Failed precondition of CompactString::set_allocated_capacity(): \"\n           \"tag does not match capacity representation\";\n    const T stored_capacity = IntCast<T>(capacity);\n    std::memcpy(allocated_data(repr) - 2 * sizeof(T), &stored_capacity,\n                sizeof(T));\n  }\n\n  static char* Allocate(size_t size) {\n    return static_cast<char*>(NewAligned<void, 8>(size));\n  }\n  static void Free(char* ptr, size_t size) {\n    DeleteAligned<void, 8>(ptr, size);\n  }\n\n  static uintptr_t MakeRepr(size_t size, size_t capacity);\n  static uintptr_t MakeReprSlow(size_t size, size_t capacity);\n  static uintptr_t MakeRepr(size_t size);\n  static uintptr_t MakeRepr(absl::string_view src, size_t capacity);\n  static uintptr_t MakeRepr(absl::string_view src);\n  static void DeleteRepr(uintptr_t repr);\n\n  void AssignSlow(absl::string_view src);\n  void AssignSlow(const CompactString& that);\n  char* ResizeSlow(size_t new_size, size_t min_capacity, size_t used_size);\n  void ShrinkToFitSlow();\n  char* AppendSlow(size_t length);\n  void AppendSlow(absl::string_view src);\n  void ReserveOneMoreByteSlow();\n\n  void DumpStructure(absl::string_view substr, std::ostream& dest) const;\n  template <typename MemoryEstimator>\n  void RegisterSubobjects(MemoryEstimator& memory_estimator) const;\n\n  uintptr_t repr_ = kInlineTag;\n};\n\n// Hash and equality which support heterogeneous lookup.\nstruct CompactStringHash {\n  using is_transparent = void;\n  size_t operator()(const CompactString& value) const {\n    return absl::Hash<CompactString>()(value);\n  }\n  size_t operator()(absl::string_view value) const {\n    return absl::Hash<absl::string_view>()(value);\n  }\n};\nstruct CompactStringEq {\n  using is_transparent = void;\n  bool operator()(const CompactString& a, const CompactString& b) const {\n    return a == b;\n  }\n  bool operator()(const CompactString& a, absl::string_view b) const {\n    return a == b;\n  }\n  bool operator()(absl::string_view a, const CompactString& b) const {\n    return a == b;\n  }\n  bool operator()(absl::string_view a, absl::string_view b) const {\n    return a == b;\n  }\n};\n\n// Implementation details follow.\n\ninline uintptr_t CompactString::MakeRepr(size_t size, size_t capacity) {\n  RIEGELI_ASSERT_LE(size, capacity)\n      << \"Failed precondition of CompactString::MakeRepr(): \"\n         \"size greater than capacity\";\n  if (capacity <= kInlineCapacity) {\n    return uintptr_t{(size << kTagBits) + kInlineTag};\n  }\n  return MakeReprSlow(size, capacity);\n}\n\ninline uintptr_t CompactString::MakeRepr(size_t size) {\n  return MakeRepr(size, size);\n}\n\ninline uintptr_t CompactString::MakeRepr(absl::string_view src,\n                                         size_t capacity) {\n  uintptr_t repr = MakeRepr(src.size(), capacity);\n  riegeli::null_safe_memcpy(\n      capacity <= kInlineCapacity ? inline_data(&repr) : allocated_data(repr),\n      src.data(), src.size());\n  return repr;\n}\n\ninline uintptr_t CompactString::MakeRepr(absl::string_view src) {\n  return MakeRepr(src, src.size());\n}\n\ninline void CompactString::DeleteRepr(uintptr_t repr) {\n  const uintptr_t tag = repr & kTagMask;\n  if (tag == kInlineTag) return;\n  const size_t offset = tag == 0 ? 2 * sizeof(size_t) : IntCast<size_t>(tag);\n  Free(allocated_data(repr) - offset,\n       allocated_capacity_for_tag(tag, repr) + offset);\n}\n\ninline CompactString& CompactString::operator=(BytesRef src) {\n  if (ABSL_PREDICT_TRUE(src.size() <= capacity())) {\n    set_size(src.size());\n    // Use `memmove()` to support assigning from a substring of `*this`.\n    riegeli::null_safe_memmove(data(), src.data(), src.size());\n  } else {\n    AssignSlow(src);\n  }\n  return *this;\n}\n\ninline CompactString::CompactString(const CompactString& that) {\n  const uintptr_t that_tag = that.repr_ & kTagMask;\n  if (that_tag == kInlineTag) {\n    repr_ = that.repr_;\n  } else {\n    repr_ = MakeRepr(absl::string_view(that.allocated_data(),\n                                       that.allocated_size_for_tag(that_tag)));\n  }\n}\n\ninline CompactString& CompactString::operator=(const CompactString& that) {\n  const uintptr_t that_tag = that.repr_ & kTagMask;\n  if (that_tag == kInlineTag) {\n    const uintptr_t tag = repr_ & kTagMask;\n    if (tag == kInlineTag) {\n      repr_ = that.repr_;\n    } else {\n      set_allocated_size_for_tag(tag, that.inline_size());\n      RIEGELI_ASSERT_LE(kInlineCapacity, capacity())\n          << \"Failed invariant of CompactString: \"\n             \"inline capacity always fits in a capacity\";\n      // Copy fixed `kInlineCapacity` instead of variable `that.inline_size()`.\n      std::memcpy(allocated_data(), that.inline_data(), kInlineCapacity);\n      // The `#ifdef` helps the compiler to realize that computing the arguments\n      // is unnecessary if `MarkPoisoned()` does nothing.\n#ifdef MEMORY_SANITIZER\n      // This part got unpoisoned by copying `kInlineCapacity` instead of\n      // `that.inline_size()`. Poison it again.\n      MarkPoisoned(allocated_data() + that.inline_size(),\n                   kInlineCapacity - that.inline_size());\n#endif\n    }\n  } else {\n    AssignSlow(that);\n  }\n  return *this;\n}\n\ninline char* CompactString::data() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  const uintptr_t tag = repr_ & kTagMask;\n  if (tag == kInlineTag) return inline_data();\n  return allocated_data();\n}\n\ninline const char* CompactString::data() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  const uintptr_t tag = repr_ & kTagMask;\n  if (tag == kInlineTag) return inline_data();\n  return allocated_data();\n}\n\ninline size_t CompactString::size() const {\n  const uintptr_t tag = repr_ & kTagMask;\n  if (tag == kInlineTag) return inline_size();\n  return allocated_size_for_tag(tag);\n}\n\ninline size_t CompactString::capacity() const {\n  const uintptr_t tag = repr_ & kTagMask;\n  if (tag == kInlineTag) return kInlineCapacity;\n  return allocated_capacity_for_tag(tag);\n}\n\ninline CompactString::operator absl::string_view() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return ViewFromRaw(&repr_);\n}\n\ninline char& CompactString::operator[](size_t index)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT_LT(index, size())\n      << \"Failed precondition of CompactString::operator[]: index out of range\";\n  return data()[index];\n}\n\ninline const char& CompactString::operator[](size_t index) const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT_LT(index, size())\n      << \"Failed precondition of CompactString::operator[]: index out of range\";\n  return data()[index];\n}\n\ninline char& CompactString::at(size_t index) ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_CHECK_LT(index, size())\n      << \"Failed precondition of CompactString::at(): index out of range\";\n  return data()[index];\n}\n\ninline const char& CompactString::at(size_t index) const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_CHECK_LT(index, size())\n      << \"Failed precondition of CompactString::at(): index out of range\";\n  return data()[index];\n}\n\ninline char& CompactString::front() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT(!empty())\n      << \"Failed precondition of CompactString::front(): empty string\";\n  return data()[0];\n}\n\ninline const char& CompactString::front() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT(!empty())\n      << \"Failed precondition of CompactString::front(): empty string\";\n  return data()[0];\n}\n\ninline char& CompactString::back() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT(!empty())\n      << \"Failed precondition of CompactString::back(): empty string\";\n  return data()[size() - 1];\n}\n\ninline const char& CompactString::back() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT(!empty())\n      << \"Failed precondition of CompactString::back(): empty string\";\n  return data()[size() - 1];\n}\n\ninline void CompactString::set_size(size_t new_size) {\n  RIEGELI_ASSERT_LE(new_size, capacity())\n      << \"Failed precondition of CompactString::SetSize(): size out of range\";\n  const uintptr_t tag = repr_ & kTagMask;\n  if (tag == kInlineTag) {\n    set_inline_size(new_size);\n    return;\n  }\n  set_allocated_size_for_tag(tag, new_size);\n}\n\ninline void CompactString::set_allocated_size_for_tag(uintptr_t tag,\n                                                      size_t new_size) {\n  // The `#ifdef` helps the compiler to realize that computing the arguments is\n  // unnecessary if `MarkPoisoned()` does nothing.\n#ifdef MEMORY_SANITIZER\n  if (new_size < allocated_size_for_tag(tag)) {\n    MarkPoisoned(allocated_data() + new_size,\n                 allocated_size_for_tag(tag) - new_size);\n  }\n#endif\n  if (tag == 2) {\n    set_allocated_size<uint8_t>(new_size);\n  } else if (tag == 4) {\n    set_allocated_size<uint16_t>(new_size);\n  } else if (tag == 0) {\n    set_allocated_size<size_t>(new_size);\n  } else {\n    RIEGELI_ASSUME_UNREACHABLE() << \"Impossible tag: \" << tag;\n  }\n}\n\ninline void CompactString::resize(size_t new_size) {\n  if (ABSL_PREDICT_TRUE(new_size <= capacity())) {\n    set_size(new_size);\n    return;\n  }\n  ResizeSlow(new_size, new_size, size());\n}\n\ninline char* CompactString::resize(size_t new_size, size_t used_size)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT_LE(used_size, size())\n      << \"Failed precondition of CompactString::resize(): \"\n         \"used size exceeds old size\";\n  RIEGELI_ASSERT_LE(used_size, new_size)\n      << \"Failed precondition of CompactString::resize(): \"\n         \"used size exceeds new size\";\n  if (ABSL_PREDICT_TRUE(new_size <= capacity())) {\n    // The `#ifdef` helps the compiler to realize that computing the arguments\n    // is unnecessary if `MarkPoisoned()` does nothing.\n#ifdef MEMORY_SANITIZER\n    const uintptr_t tag = repr_ & kTagMask;\n    if (tag != kInlineTag) {\n      MarkPoisoned(\n          allocated_data() + used_size,\n          UnsignedMin(allocated_size_for_tag(tag), new_size) - used_size);\n    }\n#endif\n    set_size(new_size);\n    return data() + used_size;\n  }\n  return ResizeSlow(new_size, new_size, used_size);\n}\n\ninline void CompactString::reserve(size_t min_capacity) {\n  if (ABSL_PREDICT_TRUE(min_capacity <= capacity())) return;\n  const size_t used_size = size();\n  ResizeSlow(used_size, min_capacity, used_size);\n}\n\ninline void CompactString::shrink_to_fit() {\n  const uintptr_t tag = repr_ & kTagMask;\n  if (tag == kInlineTag) return;\n  ShrinkToFitSlow();\n}\n\ninline char* CompactString::append(size_t length)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  const size_t old_size = size();\n  const size_t old_capacity = capacity();\n  if (ABSL_PREDICT_TRUE(length <= old_capacity - old_size)) {\n    set_size(old_size + length);\n    return data() + old_size;\n  }\n  return AppendSlow(length);\n}\n\ninline void CompactString::append(absl::string_view src) {\n  const size_t old_size = size();\n  const size_t old_capacity = capacity();\n  if (ABSL_PREDICT_TRUE(src.size() <= old_capacity - old_size)) {\n    set_size(old_size + src.size());\n    riegeli::null_safe_memcpy(data() + old_size, src.data(), src.size());\n    return;\n  }\n  AppendSlow(src);\n}\n\ninline const char* CompactString::c_str() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  const size_t used_size = size();\n  // Allocate just enough for NUL, do not call `reserve(used_size + 1)` here\n  // because that could overallocate by 50%. In `c_str()` it is likely that the\n  // string already has its final value.\n  if (ABSL_PREDICT_FALSE(used_size == capacity())) ReserveOneMoreByteSlow();\n  char* const ptr = data();\n  ptr[used_size] = '\\0';\n  return ptr;\n}\n\ninline const char* CompactString::CStrFromRaw(uintptr_t* raw) {\n  RIEGELI_ASSERT_NE(*raw, 0u)\n      << \"Failed precondition of CompactString::CStrFromRaw(): \"\n         \"representation is zero\";\n  RIEGELI_ASSERT_EQ(*raw & 1, 0u)\n      << \"Failed precondition of CompactString::CStrFromRaw(): \"\n         \"representation is not even\";\n  uintptr_t tag = *raw & kTagMask;\n  char* ptr;\n  size_t used_size;\n  size_t capacity;\n  if (tag == kInlineTag) {\n    ptr = inline_data(raw);\n    used_size = inline_size(*raw);\n    capacity = kInlineCapacity;\n  } else {\n    ptr = allocated_data(*raw);\n    used_size = allocated_size_for_tag(tag, *raw);\n    capacity = allocated_capacity_for_tag(tag, *raw);\n  }\n  if (ABSL_PREDICT_FALSE(used_size == capacity)) {\n    CompactString str = CompactString::MoveFromRaw(*raw);\n    str.ReserveOneMoreByteSlow();\n    *raw = std::move(str).RawMove();\n    tag = *raw & kTagMask;\n    ptr = allocated_data(*raw);\n    used_size = allocated_size_for_tag(tag, *raw);\n  }\n  ptr[used_size] = '\\0';\n  return ptr;\n}\n\ntemplate <typename MemoryEstimator>\ninline void CompactString::RegisterSubobjects(\n    MemoryEstimator& memory_estimator) const {\n  const uintptr_t tag = repr_ & kTagMask;\n  if (tag == kInlineTag) return;\n  const size_t offset = tag == 0 ? 2 * sizeof(size_t) : IntCast<size_t>(tag);\n  memory_estimator.RegisterDynamicMemory(\n      allocated_data() - offset, offset + allocated_capacity_for_tag(tag));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_COMPACT_STRING_H_\n"
  },
  {
    "path": "riegeli/base/compare.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_COMPARE_H_\n#define RIEGELI_BASE_COMPARE_H_\n\n#include <type_traits>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/strings/string_view.h\"  // IWYU pragma: keep\n#if !__cpp_impl_three_way_comparison\n#include \"absl/types/compare.h\"\n#endif\n\nABSL_POINTERS_DEFAULT_NONNULL\n\n// Emulate C++20 `operator<=>` machinery for earlier C++ versions.\n\nnamespace riegeli {\n\n// `PartialOrdering` is `std::partial_ordering` in C++20 or\n// `absl::partial_ordering` in earlier C++ versions.\n#if __cpp_impl_three_way_comparison\nusing PartialOrdering = decltype(0.0 <=> 0.0);\n#else\nusing PartialOrdering = absl::partial_ordering;\n#endif\n\n// `WeakOrdering` is not provided because it cannot be implemented without\n// conditionally including `<compare>`.\n\n// `StrongOrdering` is `std::strong_ordering` in C++20 or\n// `absl::strong_ordering` in earlier C++ versions.\n#if __cpp_impl_three_way_comparison\nusing StrongOrdering = decltype(0 <=> 0);\n#else\nusing StrongOrdering = absl::strong_ordering;\n#endif\n\n// Define `friend auto RIEGELI_COMPARE` instead of C++20\n// `friend auto operator<=>`.\n//\n// It should return `PartialOrdering` or `StrongOrdering`.\n//\n// It is meant to be called by `riegeli::Compare(a, b)`, not directly as\n// `RIEGELI_COMPARE(a, b)`.\n#if __cpp_impl_three_way_comparison\n#define RIEGELI_COMPARE operator<=>\n#else\n#define RIEGELI_COMPARE RiegeliCompare\n#endif\n\n// `IsOrdering<T>::value` is `true` if values of type `T` can be assumed to\n// indicate an ordering: they are comparable with literal 0.\n//\n// This includes `{std,absl}::{partial,weak,strong}_ordering`, and `int` being\n// the result of `std::memcmp()` or `absl::string_view::compare()`.\n\ntemplate <typename T, typename Enable = void>\nstruct IsOrdering : std::false_type {};\n\ntemplate <typename T>\nstruct IsOrdering<T, std::void_t<decltype(std::declval<T>() < 0),\n                                 decltype(std::declval<T>() > 0),\n                                 decltype(std::declval<T>() == 0)>>\n    : std::true_type {};\n\n// `IsTotalOrdering<T>::value` is `true` if values of type `T` can be assumed to\n// indicate a total ordering: they are comparable with literal 0, and\n// `T::unordered` is not defined.\n//\n// This includes `{std,absl}::{weak,strong}_ordering`, and `int` being the\n// result of `std::memcmp()` or `absl::string_view::compare()`.\n\ntemplate <typename T, typename Enable = void>\nstruct IsTotalOrdering : IsOrdering<T> {};\n\ntemplate <typename T>\nstruct IsTotalOrdering<T, std::void_t<decltype(T::unordered)>>\n    : std::false_type {};\n\nnamespace compare_internal {\n\ntemplate <typename T, typename Enable = void>\nstruct IsTotalOrderingWithEqual : std::false_type {};\n\ntemplate <typename T>\nstruct IsTotalOrderingWithEqual<T, std::void_t<decltype(T::equal)>>\n    : IsTotalOrdering<T> {};\n\n}  // namespace compare_internal\n\n// `IsStrongOrdering<T>::value` is `true` if values of type `T` can be assumed\n// to indicate a strong ordering: they are comparable with literal 0,\n// `T::unordered` is not defined, and either `T::equivalent` is not defined or\n// `T::equal` is defined too.\n//\n// This includes `{std,absl}::strong_ordering`, and `int` being the result of\n// `std::memcmp()` or `absl::string_view::compare()`.\n\ntemplate <typename T, typename Enable = void>\nstruct IsStrongOrdering : IsTotalOrdering<T> {};\n\ntemplate <typename T>\nstruct IsStrongOrdering<T, std::void_t<decltype(T::equivalent)>>\n    : compare_internal::IsTotalOrderingWithEqual<T> {};\n\n// Converts a value indicating an ordering to `PartialOrdering`.\n\ntemplate <typename T,\n          std::enable_if_t<\n              std::conjunction_v<IsOrdering<T>,\n                                 std::is_convertible<T, PartialOrdering>>,\n              int> = 0>\ninline PartialOrdering AsPartialOrdering(T ordering) {\n  return ordering;\n}\n\ntemplate <typename T,\n          std::enable_if_t<std::conjunction_v<IsOrdering<T>,\n                                              std::negation<std::is_convertible<\n                                                  T, PartialOrdering>>>,\n                           int> = 0>\ninline PartialOrdering AsPartialOrdering(T ordering) {\n  return ordering < 0    ? PartialOrdering::less\n         : ordering > 0  ? PartialOrdering::greater\n         : ordering == 0 ? PartialOrdering::equivalent\n                         : PartialOrdering::unordered;\n}\n\n// Converts a value indicating a strong ordering to `StrongOrdering`.\n\ntemplate <\n    typename T,\n    std::enable_if_t<std::conjunction_v<IsStrongOrdering<T>,\n                                        std::is_convertible<T, StrongOrdering>>,\n                     int> = 0>\ninline StrongOrdering AsStrongOrdering(T ordering) {\n  return ordering;\n}\n\ntemplate <\n    typename T,\n    std::enable_if_t<std::conjunction_v<\n                         IsStrongOrdering<T>,\n                         std::negation<std::is_convertible<T, StrongOrdering>>>,\n                     int> = 0>\ninline StrongOrdering AsStrongOrdering(T ordering) {\n  return ordering < 0   ? StrongOrdering::less\n         : ordering > 0 ? StrongOrdering::greater\n                        : StrongOrdering::equal;\n}\n\n#if !__cpp_impl_three_way_comparison\n\n// Definitions of `RIEGELI_COMPARE` which in C++20 are provided automatically.\n\ntemplate <\n    typename A, typename B,\n    std::enable_if_t<\n        std::conjunction_v<std::is_integral<A>, std::is_integral<B>>, int> = 0>\ninline StrongOrdering RIEGELI_COMPARE(A a, B b) {\n  return a < b   ? StrongOrdering::less\n         : a > b ? StrongOrdering::greater\n                 : StrongOrdering::equal;\n}\n\ntemplate <typename A, typename B,\n          std::enable_if_t<\n              std::conjunction_v<std::negation<std::conjunction<\n                                     std::is_integral<A>, std::is_integral<B>>>,\n                                 std::is_arithmetic<A>, std::is_arithmetic<B>>,\n              int> = 0>\ninline PartialOrdering RIEGELI_COMPARE(A a, B b) {\n  static_assert(std::is_floating_point_v<A> || std::is_floating_point_v<B>,\n                \"Arithmetic types which are not integral types \"\n                \"must be floating point types\");\n  return a < b    ? PartialOrdering::less\n         : a > b  ? PartialOrdering::greater\n         : a == b ? PartialOrdering::equivalent\n                  : PartialOrdering::unordered;\n}\n\ntemplate <typename T, std::enable_if_t<std::is_enum_v<T>, int> = 0>\ninline StrongOrdering RIEGELI_COMPARE(T a, T b) {\n  return a < b   ? StrongOrdering::less\n         : a > b ? StrongOrdering::greater\n                 : StrongOrdering::equal;\n}\n\ntemplate <typename T>\ninline StrongOrdering RIEGELI_COMPARE(T* a, T* b) {\n  return a < b   ? StrongOrdering::less\n         : a > b ? StrongOrdering::greater\n                 : StrongOrdering::equal;\n}\n\ninline StrongOrdering RIEGELI_COMPARE(absl::string_view a,\n                                      absl::string_view b) {\n  return AsStrongOrdering(a.compare(b));\n}\n\n#endif\n\nnamespace compare_internal {\n\n#if !__cpp_impl_three_way_comparison\n\ntemplate <typename A, typename B, typename Enable = void>\nstruct HasEqual : std::false_type {};\n\ntemplate <typename A, typename B>\nstruct HasEqual<\n    A, B,\n    std::void_t<decltype(std::declval<const A&>() == std::declval<const B&>())>>\n    : std::true_type {};\n\n#endif\n\ntemplate <typename A, typename B, typename Enable = void>\nstruct HasCompare : std::false_type {};\n\ntemplate <typename A, typename B>\nstruct HasCompare<A, B,\n                  std::void_t<decltype(\n#if __cpp_impl_three_way_comparison\n                      std::declval<const A&>() <=> std::declval<const B&>()\n#else\n                      RIEGELI_COMPARE(std::declval<const A&>(),\n                                      std::declval<const B&>())\n#endif\n                          )>> : std::true_type {\n};\n\ntemplate <typename T, typename Enable = void>\nstruct IsDedicatedOrdering : std::false_type {};\n\ntemplate <typename T>\nstruct IsDedicatedOrdering<T,\n                           std::void_t<decltype(T::less), decltype(T::greater)>>\n    : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct HasCompareWithLiteral0 : std::false_type {};\n\ntemplate <typename T>\nstruct HasCompareWithLiteral0<T, std::void_t<decltype(\n#if __cpp_impl_three_way_comparison\n                                     0 <=> std::declval<T>()\n#else\n                                     RIEGELI_COMPARE(0, std::declval<T>())\n#endif\n                                         )>> : std::true_type {\n};\n\n}  // namespace compare_internal\n\n// Call `riegeli::Compare(a, b)` instead of C++20 `a <=> b`.\ntemplate <typename A, typename B,\n          std::enable_if_t<compare_internal::HasCompare<A, B>::value, int> = 0>\ninline auto Compare(const A& a, const B& b) {\n#if __cpp_impl_three_way_comparison\n  return a <=> b;\n#else\n  return RIEGELI_COMPARE(a, b);\n#endif\n}\n\n// Call `NegateOrdering(ordering)` instead of C++20 `0 <=> ordering`.\n//\n// `riegeli::Compare(0, ordering)` does not work because it does not properly\n// forward to `<=>` the property that the argument is a literal 0.\n\ntemplate <\n    typename Ordering,\n    std::enable_if_t<\n        std::conjunction_v<compare_internal::IsDedicatedOrdering<Ordering>,\n                           compare_internal::HasCompareWithLiteral0<Ordering>>,\n        int> = 0>\ninline Ordering NegateOrdering(Ordering ordering) {\n#if __cpp_impl_three_way_comparison\n  return 0 <=> ordering;\n#else\n  return RIEGELI_COMPARE(0, ordering);\n#endif\n}\n\ntemplate <\n    typename Ordering,\n    std::enable_if_t<\n        std::conjunction_v<\n            compare_internal::IsDedicatedOrdering<Ordering>,\n            std::negation<compare_internal::HasCompareWithLiteral0<Ordering>>>,\n        int> = 0>\ninline Ordering NegateOrdering(Ordering ordering) {\n  if (0 < ordering) return Ordering::less;\n  if (0 > ordering) return Ordering::greater;\n  return ordering;\n}\n\n// For types which support equality, derive `T` from `WithEqual<T>`, and define\n// `friend bool operator==` with the first parameter of type `const T&` or `T`,\n// and the second parameter of the same type, or possibly also of other types.\n//\n// `WithEqual` provides `!=`. For heterogeneous equality it provides `==` and\n// `!=` with swapped parameters.\n//\n// In C++20 this is automatic.\ntemplate <typename T>\nclass WithEqual {\n public:\n#if !__cpp_impl_three_way_comparison\n  template <\n      typename Other,\n      std::enable_if_t<compare_internal::HasEqual<T, Other>::value, int> = 0>\n  friend bool operator!=(const T& a, const Other& b) {\n    return !(a == b);\n  }\n\n  template <\n      typename Other,\n      std::enable_if_t<std::conjunction_v<std::negation<std::is_same<Other, T>>,\n                                          compare_internal::HasEqual<T, Other>>,\n                       int> = 0>\n  friend bool operator==(const Other& a, const T& b) {\n    return b == a;\n  }\n  template <\n      typename Other,\n      std::enable_if_t<std::conjunction_v<std::negation<std::is_same<Other, T>>,\n                                          compare_internal::HasEqual<T, Other>>,\n                       int> = 0>\n  friend bool operator!=(const Other& a, const T& b) {\n    return !(b == a);\n  }\n#endif\n};\n\n// For types which support comparison, derive `T` from `WithCompare<T>`. and\n// define `friend bool operator==` and `friend auto RIEGELI_COMPARE` with the\n// first parameter of type `const T&` or `T`, and the second parameter of the\n// same type, or possibly also of other types.\n//\n// `WithCompare` provides `!=`, `<`, `>`, `<=`, and `>=`. For heterogeneous\n// comparison it provides `==`, `!=`, `RIEGELI_COMPARE, `<`, `>`, `<=`, and `>=`\n// with swapped parameters.\n//\n// In C++20 this is automatic.\ntemplate <typename T>\nclass WithCompare : public WithEqual<T> {\n public:\n#if !__cpp_impl_three_way_comparison\n  template <\n      typename Other,\n      std::enable_if_t<compare_internal::HasCompare<T, Other>::value, int> = 0>\n  friend bool operator<(const T& a, const Other& b) {\n    return RIEGELI_COMPARE(a, b) < 0;\n  }\n  template <\n      typename Other,\n      std::enable_if_t<compare_internal::HasCompare<T, Other>::value, int> = 0>\n  friend bool operator>(const T& a, const Other& b) {\n    return RIEGELI_COMPARE(a, b) > 0;\n  }\n  template <\n      typename Other,\n      std::enable_if_t<compare_internal::HasCompare<T, Other>::value, int> = 0>\n  friend bool operator<=(const T& a, const Other& b) {\n    return RIEGELI_COMPARE(a, b) <= 0;\n  }\n  template <\n      typename Other,\n      std::enable_if_t<compare_internal::HasCompare<T, Other>::value, int> = 0>\n  friend bool operator>=(const T& a, const Other& b) {\n    return RIEGELI_COMPARE(a, b) >= 0;\n  }\n\n  template <typename Other,\n            std::enable_if_t<\n                std::conjunction_v<std::negation<std::is_same<Other, T>>,\n                                   compare_internal::HasCompare<T, Other>>,\n                int> = 0>\n  friend auto RIEGELI_COMPARE(const Other& a, const T& b) {\n    return NegateOrdering(RIEGELI_COMPARE(b, a));\n  }\n  template <typename Other,\n            std::enable_if_t<\n                std::conjunction_v<std::negation<std::is_same<Other, T>>,\n                                   compare_internal::HasCompare<T, Other>>,\n                int> = 0>\n  friend bool operator<(const Other& a, const T& b) {\n    return 0 < RIEGELI_COMPARE(b, a);\n  }\n  template <typename Other,\n            std::enable_if_t<\n                std::conjunction_v<std::negation<std::is_same<Other, T>>,\n                                   compare_internal::HasCompare<T, Other>>,\n                int> = 0>\n  friend bool operator>(const Other& a, const T& b) {\n    return 0 > RIEGELI_COMPARE(b, a);\n  }\n  template <typename Other,\n            std::enable_if_t<\n                std::conjunction_v<std::negation<std::is_same<Other, T>>,\n                                   compare_internal::HasCompare<T, Other>>,\n                int> = 0>\n  friend bool operator<=(const Other& a, const T& b) {\n    return 0 <= RIEGELI_COMPARE(b, a);\n  }\n  template <typename Other,\n            std::enable_if_t<\n                std::conjunction_v<std::negation<std::is_same<Other, T>>,\n                                   compare_internal::HasCompare<T, Other>>,\n                int> = 0>\n  friend bool operator>=(const Other& a, const T& b) {\n    return 0 >= RIEGELI_COMPARE(b, a);\n  }\n#endif\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_COMPARE_H_\n"
  },
  {
    "path": "riegeli/base/constexpr.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_CONSTEXPR_H_\n#define RIEGELI_BASE_CONSTEXPR_H_\n\n#include \"absl/base/nullability.h\"\n#include \"riegeli/base/port.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// Returns `true` if the value of the expression is known at compile time.\n#if RIEGELI_INTERNAL_HAS_BUILTIN(__builtin_constant_p) || \\\n    RIEGELI_INTERNAL_IS_GCC_VERSION(3, 1)\n#define RIEGELI_IS_CONSTANT(expr) __builtin_constant_p(expr)\n#else\n#define RIEGELI_IS_CONSTANT(expr) false\n#endif\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_CONSTEXPR_H_\n"
  },
  {
    "path": "riegeli/base/cord_iterator_span.cc",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/cord_iterator_span.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <string>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/resize_and_overwrite.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/string_utils.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nvoid CordIteratorSpan::ReadSlow(absl::Cord::CharIterator& src, size_t length,\n                                char* dest) {\n  absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  RIEGELI_ASSERT_LT(chunk.size(), length)\n      << \"Failed precondition of CordIteratorSpan::ReadSlow(): \"\n         \"enough data available, use Read() instead\";\n  do {\n    std::memcpy(dest, chunk.data(), chunk.size());\n    absl::Cord::Advance(&src, chunk.size());\n    dest += chunk.size();\n    length -= chunk.size();\n    chunk = absl::Cord::ChunkRemaining(src);\n  } while (chunk.size() < length);\n  std::memcpy(dest, chunk.data(), length);\n  absl::Cord::Advance(&src, length);\n}\n\nabsl::string_view CordIteratorSpan::ToStringView(std::string& scratch) && {\n  absl::Cord::CharIterator& iter = *iterator_;\n  size_t length = length_;\n  if (length == 0) return absl::string_view();\n  absl::string_view chunk = absl::Cord::ChunkRemaining(iter);\n  if (ABSL_PREDICT_TRUE(chunk.size() >= length)) {\n    absl::Cord::Advance(&iter, length);\n    return chunk.substr(0, length);\n  }\n  scratch.clear();\n  riegeli::StringResizeAndOverwriteAmortized(scratch, length,\n                                             [&](char* data, size_t size) {\n                                               ReadSlow(iter, size, data);\n                                               return size;\n                                             });\n  return scratch;\n}\n\nvoid CordIteratorSpan::ToString(std::string& dest) && {\n  absl::Cord::CharIterator& iter = *iterator_;\n  size_t length = length_;\n  dest.clear();\n  absl::StringResizeAndOverwrite(dest, length, [&](char* data, size_t size) {\n    Read(iter, size, data);\n    return size;\n  });\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/cord_iterator_span.h",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_CORD_ITERATOR_SPAN_H_\n#define RIEGELI_BASE_CORD_ITERATOR_SPAN_H_\n\n#include <stddef.h>\n\n#include <cstring>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `CordIteratorSpan` specifies a span of `absl::Cord::CharIterator` contents\n// from the current position with the given length.\n//\n// This can express the span as a single object, which is sometimes convenient.\nclass CordIteratorSpan {\n public:\n  // Returns the number of bytes from `src` to the end of the `absl::Cord`.\n  static size_t Remaining(const absl::Cord::CharIterator& src) {\n    return IntCast<size_t>(\n        absl::Cord::Distance(src, absl::Cord::CharIterator()));\n  }\n\n  // Copies `length` bytes from `src` to `dest[]`.\n  static void Read(absl::Cord::CharIterator& src, size_t length,\n                   char* absl_nullable dest);\n\n  // Specifies the span from the current position of `*src` with `length`.\n  explicit CordIteratorSpan(absl::Cord::CharIterator* src\n                                ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                            size_t length)\n      : iterator_(src), length_(length) {\n    RIEGELI_ASSERT_LE(length, Remaining(*iterator_))\n        << \"Failed precondition of CordIteratorSpan: not enough remaining data\";\n  }\n\n  CordIteratorSpan(CordIteratorSpan&& that) = default;\n  CordIteratorSpan& operator=(CordIteratorSpan&& that) = default;\n\n  absl::Cord::CharIterator& iterator() const { return *iterator_; }\n  size_t length() const { return length_; }\n\n  // Destructively reads the contents of the span to an `absl::Cord`.\n  //\n  // An implicit conversion allows to use a `CordIteratorSpan` when an\n  // `absl::Cord` is expected. Some functions treat a parameter of type\n  // `CordIteratorSpan` specially to enable a more efficient implementation.\n  /*implicit*/ operator absl::Cord() && { return std::move(*this).ToCord(); }\n\n  // Destructively reads the contents of the span to an `absl::Cord`.\n  absl::Cord ToCord() && {\n    return absl::Cord::AdvanceAndRead(iterator_, length_);\n  }\n\n  // Destructively reads the contents of the span to an `absl::string_view`.\n  //\n  // May use `scratch` for storage for the result.\n  absl::string_view ToStringView(std::string& scratch) &&;\n\n  // Destructively reads the contents of the span to an existing `std::string`.\n  void ToString(std::string& dest) &&;\n\n  // Returns the contents of the span as an `absl::string_view` if it is flat.\n  // Otherwise returns `std::nullopt`.\n  std::optional<absl::string_view> TryFlat() const;\n\n private:\n  static void ReadSlow(absl::Cord::CharIterator& src, size_t length,\n                       char* dest);\n\n  absl::Cord::CharIterator* iterator_;\n  size_t length_;\n};\n\n// Specialization of `DependencyImpl<const absl::Cord*, CordIteratorSpan>`.\n//\n// This allows to pass a `CordIteratorSpan` as a parameter of `CordReader`.\ntemplate <>\nclass DependencyImpl<const absl::Cord*, CordIteratorSpan> {\n public:\n  explicit DependencyImpl(CordIteratorSpan span)\n      : span_(std::move(span)), cord_(std::move(span_)) {}\n\n  CordIteratorSpan& manager() ABSL_ATTRIBUTE_LIFETIME_BOUND { return span_; }\n  const CordIteratorSpan& manager() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return span_;\n  }\n\n  const absl::Cord* get() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return &cord_; }\n\n  bool IsOwning() const { return false; }\n\n  static constexpr bool kIsStable = false;\n\n protected:\n  DependencyImpl(DependencyImpl&& that) = default;\n  DependencyImpl& operator=(DependencyImpl&& that) = default;\n\n  ~DependencyImpl() = default;\n\n private:\n  CordIteratorSpan span_;\n  const absl::Cord cord_;\n};\n\n// Implementation details follow.\n\ninline void CordIteratorSpan::Read(absl::Cord::CharIterator& src, size_t length,\n                                   char* absl_nullable dest) {\n  RIEGELI_ASSERT_LE(length, Remaining(src))\n      << \"Failed precondition of CordIteratorSpan::Read(): \"\n         \"not enough remaining data\";\n  if (length == 0) return;\n  RIEGELI_ASSERT(dest != nullptr)\n      << \"Failed precondition of CordIteratorSpan::Read(): \"\n         \"non-empty span from nullptr\";\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  if (ABSL_PREDICT_FALSE(chunk.size() < length)) {\n    ReadSlow(src, length, dest);\n    return;\n  }\n  std::memcpy(dest, chunk.data(), length);\n  absl::Cord::Advance(&src, length);\n}\n\ninline std::optional<absl::string_view> CordIteratorSpan::TryFlat() const {\n  if (length_ == 0) return absl::string_view();\n  absl::string_view chunk = absl::Cord::ChunkRemaining(*iterator_);\n  if (ABSL_PREDICT_FALSE(chunk.size() < length_)) return std::nullopt;\n  return chunk.substr(0, length_);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_CORD_ITERATOR_SPAN_H_\n"
  },
  {
    "path": "riegeli/base/cord_utils.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/cord_utils.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <utility>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/cord_buffer.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/string_utils.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli::cord_internal {\n\nvoid CopyCordToArray(const absl::Cord& src, char* absl_nullable dest) {\n  for (const absl::string_view fragment : src.Chunks()) {\n    std::memcpy(dest, fragment.data(), fragment.size());\n    dest += fragment.size();\n  }\n}\n\nabsl::Cord MakeBlockyCord(absl::string_view src) {\n  absl::Cord dest;\n  AppendToBlockyCord(src, dest);\n  return dest;\n}\n\nvoid AssignToBlockyCord(absl::string_view src, absl::Cord& dest) {\n  if (src.size() <= absl::CordBuffer::kDefaultLimit) {\n    dest = src;\n    return;\n  }\n  dest.Clear();\n  AppendToBlockyCord(src, dest);\n}\n\nvoid AppendToBlockyCord(absl::string_view src, absl::Cord& dest) {\n  if (src.empty()) return;\n  {\n    absl::CordBuffer buffer = dest.GetAppendBuffer(0, 1);\n    const size_t existing_length = buffer.length();\n    if (existing_length > 0) {\n      buffer.SetLength(\n          UnsignedMin(existing_length + src.size(), buffer.capacity()));\n      std::memcpy(buffer.data() + existing_length, src.data(),\n                  buffer.length() - existing_length);\n      src.remove_prefix(buffer.length() - existing_length);\n      dest.Append(std::move(buffer));\n      if (src.empty()) return;\n    }\n  }\n  do {\n    absl::CordBuffer buffer = absl::CordBuffer::CreateWithCustomLimit(\n        kCordBufferBlockSize, src.size());\n    buffer.SetLength(UnsignedMin(src.size(), buffer.capacity()));\n    std::memcpy(buffer.data(), src.data(), buffer.length());\n    src.remove_prefix(buffer.length());\n    dest.Append(std::move(buffer));\n  } while (!src.empty());\n}\n\nvoid PrependToBlockyCord(absl::string_view src, absl::Cord& dest) {\n  while (!src.empty()) {\n    absl::CordBuffer buffer = absl::CordBuffer::CreateWithCustomLimit(\n        kCordBufferBlockSize, src.size());\n    buffer.SetLength(UnsignedMin(src.size(), buffer.capacity()));\n    std::memcpy(buffer.data(), src.data() + src.size() - buffer.length(),\n                buffer.length());\n    src.remove_suffix(buffer.length());\n    dest.Prepend(std::move(buffer));\n  }\n}\n\n}  // namespace riegeli::cord_internal\n"
  },
  {
    "path": "riegeli/base/cord_utils.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_CORD_UTILS_H_\n#define RIEGELI_BASE_CORD_UTILS_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/numeric/bits.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/cord_buffer.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/buffering.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli::cord_internal {\n\n// `absl::cord_internal::kFlatOverhead`. Does not have to be accurate.\ninline constexpr size_t kFlatOverhead =\n    sizeof(size_t) + sizeof(uint32_t) + sizeof(uint8_t);\n\n// The `block_size` parameter for `absl::CordBuffer::CreateWithCustomLimit()`.\ninline constexpr size_t kCordBufferBlockSize =\n    UnsignedMin(kDefaultMaxBlockSize, absl::CordBuffer::kCustomLimit);\n\n// Maximum usable size supported by `absl::CordBuffer`.\ninline constexpr size_t kCordBufferMaxSize =\n    absl::CordBuffer::MaximumPayload(kCordBufferBlockSize);\n\n// When deciding whether to copy an array of bytes or share memory to an\n// `absl::Cord`, prefer copying up to this length when creating a new\n// `absl::Cord`.\n//\n// This is `absl::cord_internal::kMaxInline`. Does not have to be accurate.\ninline constexpr size_t kMaxBytesToCopyToEmptyCord = 15;\n\n// When deciding whether to copy an array of bytes or share memory to an\n// `absl::Cord`, prefer copying up to this length when appending to a non-empty\n// `absl::Cord`.\n//\n// This is `absl::cord_internal::kMaxBytesToCopy`. Does not have to be accurate.\ninline constexpr size_t kMaxBytesToCopyToNonEmptyCord = 511;\n\n// When deciding whether to copy an array of bytes or share memory to an\n// `absl::Cord`, prefer copying up to this length when appending to `dest`.\n//\n// `absl::Cord::Append(absl::Cord)` chooses to copy bytes from a source up to\n// this length, so it is better to avoid constructing the source as `absl::Cord`\n// if it will not be shared anyway.\ninline size_t MaxBytesToCopyToCord(absl::Cord& dest) {\n  if (dest.empty()) return kMaxBytesToCopyToEmptyCord;\n  return kMaxBytesToCopyToNonEmptyCord;\n}\n\n// Copies `src` to `dest[]`.\n//\n// `dest[]` must have sufficient size for `src.size()`, and `dest` may be\n// `nullptr` only if `src.empty()`.\nvoid CopyCordToArray(const absl::Cord& src, char* absl_nullable dest);\n\n// Variants of `absl::Cord` operations with different block sizing tradeoffs:\n//  * `MakeBlockyCord(src)` is like `absl::Cord(src)`.\n//  * `AssignToBlockyCord(src, dest)` is like `dest = src`.\n//  * `AppendToBlockyCord(src, dest)` is like `dest.Append(src)`.\n//  * `PrependToBlockyCord(src, dest)` is like `dest.Prepend(src)`.\n//\n// They avoid splitting `src` into 4083-byte fragments and avoid overallocation,\n// without guarantees.\nabsl::Cord MakeBlockyCord(absl::string_view src);\nvoid AssignToBlockyCord(absl::string_view src, absl::Cord& dest);\nvoid AppendToBlockyCord(absl::string_view src, absl::Cord& dest);\nvoid PrependToBlockyCord(absl::string_view src, absl::Cord& dest);\n\n// Returns usable size provided by `absl::CordBuffer::CreateWithCustomLimit()`\n// called with `kCordBufferBlockSize` and `capacity`. Does not have to be\n// accurate.\ninline size_t CordBufferSizeForCapacity(size_t capacity) {\n  if (capacity >= kCordBufferMaxSize) return kCordBufferMaxSize;\n  if (capacity <= absl::CordBuffer::kDefaultLimit) return capacity;\n  if (!absl::has_single_bit(capacity)) {\n    static constexpr size_t kMaxPageSlop = 128;\n    const size_t rounded_up = size_t{1} << absl::bit_width(capacity - 1);\n    const size_t slop = rounded_up - capacity;\n    if (slop >= kFlatOverhead && slop <= kMaxPageSlop + kFlatOverhead) {\n      capacity = rounded_up;\n    } else {\n      const size_t rounded_down = size_t{1} << (absl::bit_width(capacity) - 1);\n      capacity = rounded_down;\n    }\n  }\n  return capacity - kFlatOverhead;\n}\n\n}  // namespace riegeli::cord_internal\n\n#endif  // RIEGELI_BASE_CORD_UTILS_H_\n"
  },
  {
    "path": "riegeli/base/debug.cc",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/debug.h\"\n\n#include <stdint.h>\n\n#include <cstddef>\n#include <optional>\n#include <type_traits>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nnamespace {\n\ninline void WriteHex1(uint8_t src, DebugStream& dest) {\n  dest.Write(static_cast<char>(src + (src < 10 ? '0' : 'a' - 10)));\n}\n\ninline void WriteHex2(uint8_t src, DebugStream& dest) {\n  WriteHex1(static_cast<uint8_t>(src >> 4), dest);\n  WriteHex1(static_cast<uint8_t>(src & 0x0f), dest);\n}\n\ninline void WriteHex4(uint16_t src, DebugStream& dest) {\n  WriteHex2(static_cast<uint8_t>(src >> 8), dest);\n  WriteHex2(static_cast<uint8_t>(src & 0xff), dest);\n}\n\ninline void WriteHex8(uint32_t src, DebugStream& dest) {\n  WriteHex4(static_cast<uint16_t>(src >> 16), dest);\n  WriteHex4(static_cast<uint16_t>(src & 0xffff), dest);\n}\n\ntemplate <char quote, typename IntType, typename CharType>\nvoid WriteChar(CharType src, DebugStream& dest) {\n  if (src >= 32 && src <= 126) {\n    if (src == quote || src == '\\\\') dest.Write('\\\\');\n    dest.Write(static_cast<char>(src));\n    return;\n  }\n  switch (src) {\n    case '\\t':\n      dest.Write(\"\\\\t\");\n      break;\n    case '\\n':\n      dest.Write(\"\\\\n\");\n      break;\n    case '\\r':\n      dest.Write(\"\\\\r\");\n      break;\n    default: {\n      const auto unsigned_src = static_cast<IntType>(src);\n      if (unsigned_src <= 0xff) {\n        dest.Write(\"\\\\x{\");\n        WriteHex2(static_cast<uint8_t>(unsigned_src), dest);\n      } else {\n        dest.Write(\"\\\\u{\");\n        if (unsigned_src <= 0xffff) {\n          WriteHex4(static_cast<uint16_t>(unsigned_src), dest);\n        } else {\n          WriteHex8(unsigned_src, dest);\n        }\n      }\n      dest.Write('}');\n      break;\n    }\n  }\n}\n\ntemplate <typename IntType, typename CharType>\nvoid WriteQuotedChar(CharType src, DebugStream& dest) {\n  dest.Write('\\'');\n  WriteChar<'\\'', IntType>(src, dest);\n  dest.Write('\\'');\n}\n\ntemplate <typename IntType, typename CharType>\nvoid WriteQuotedString(absl::Span<const CharType> src, DebugStream& dest) {\n  dest.Write('\"');\n  for (const CharType ch : src) {\n    WriteChar<'\"', IntType>(ch, dest);\n  }\n  dest.Write('\"');\n}\n\n}  // namespace\n\nvoid DebugStream::DebugStringFragment(absl::string_view src) {\n  for (const char ch : src) {\n    WriteChar<'\"', uint8_t>(ch, *this);\n  }\n}\n\nvoid RiegeliDebug(bool src, DebugStream& dest) {\n  dest.Write(src ? absl::string_view(\"true\") : absl::string_view(\"false\"));\n}\n\nvoid RiegeliDebug(char src, DebugStream& dest) {\n  WriteQuotedChar<uint8_t>(src, dest);\n}\n\nvoid RiegeliDebug(wchar_t src, DebugStream& dest) {\n  WriteQuotedChar<std::conditional_t<sizeof(wchar_t) == 2, uint16_t, uint32_t>>(\n      src, dest);\n}\n\n#if __cpp_char8_t\nvoid RiegeliDebug(char8_t src, DebugStream& dest) {\n  WriteQuotedChar<uint8_t>(src, dest);\n}\n#endif  // __cpp_char8_t\n\nvoid RiegeliDebug(char16_t src, DebugStream& dest) {\n  WriteQuotedChar<uint16_t>(src, dest);\n}\n\nvoid RiegeliDebug(char32_t src, DebugStream& dest) {\n  WriteQuotedChar<uint32_t>(src, dest);\n}\n\nvoid RiegeliDebug(absl::string_view src, DebugStream& dest) {\n  WriteQuotedString<uint8_t>(absl::MakeConstSpan(src), dest);\n}\n\nvoid RiegeliDebug(std::wstring_view src, DebugStream& dest) {\n  WriteQuotedString<\n      std::conditional_t<sizeof(wchar_t) == 2, uint16_t, uint32_t>>(\n      absl::MakeConstSpan(src), dest);\n}\n\n#if __cpp_char8_t\nvoid RiegeliDebug(std::u8string_view src, DebugStream& dest) {\n  WriteQuotedString<uint8_t>(absl::MakeConstSpan(src), dest);\n}\n#endif  // __cpp_char8_t\n\nvoid RiegeliDebug(std::u16string_view src, DebugStream& dest) {\n  WriteQuotedString<uint16_t>(absl::MakeConstSpan(src), dest);\n}\n\nvoid RiegeliDebug(std::u32string_view src, DebugStream& dest) {\n  WriteQuotedString<uint32_t>(absl::MakeConstSpan(src), dest);\n}\n\nvoid RiegeliDebug(const absl::Cord& src, DebugStream& dest) {\n  dest.DebugStringQuote();\n  for (const absl::string_view fragment : src.Chunks()) {\n    dest.DebugStringFragment(fragment);\n  }\n  dest.DebugStringQuote();\n}\n\nvoid RiegeliDebug(const void* absl_nullable src, DebugStream& dest) {\n  if (src == nullptr) {\n    dest.Write(\"nullptr\");\n  } else {\n    dest << src;\n  }\n}\n\nvoid RiegeliDebug(ABSL_ATTRIBUTE_UNUSED std::nullptr_t src, DebugStream& dest) {\n  dest.Write(\"nullptr\");\n}\n\nvoid RiegeliDebug(ABSL_ATTRIBUTE_UNUSED std::nullopt_t src, DebugStream& dest) {\n  dest.Write(\"nullopt\");\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/debug.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_DEBUG_H_\n#define RIEGELI_BASE_DEBUG_H_\n\n#include <stddef.h>\n\n#include <cstddef>\n#include <ios>\n#include <memory>\n#include <optional>\n#include <ostream>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/has_absl_stringify.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/stream_utils.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nclass DebugStream;\n\nnamespace debug_internal {\n\ntemplate <typename T, typename Enable = void>\nstruct HasRiegeliDebug : std::false_type {};\n\ntemplate <typename T>\nstruct HasRiegeliDebug<\n    T, std::void_t<decltype(RiegeliDebug(std::declval<const T&>(),\n                                         std::declval<DebugStream&>()))>>\n    : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct HasDebugString : std::false_type {};\n\ntemplate <typename T>\nstruct HasDebugString<\n    T,\n    std::enable_if_t<std::is_convertible_v<\n        decltype(std::declval<const T&>().DebugString()), absl::string_view>>>\n    : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct HasOperatorOutput : std::false_type {};\n\ntemplate <typename T>\nstruct HasOperatorOutput<T, std::void_t<decltype(std::declval<std::ostream&>()\n                                                 << std::declval<T>())>>\n    : std::true_type {};\n\n}  // namespace debug_internal\n\n// `SupportsDebug<T>::value` is `true` if `T` supports `riegeli::Debug()`:\n// writing the value in a format suitable for error messages.\n//\n// The value is generally written in a way which reflects as much as is compared\n// by `operator==`, without indicating the type nor internal structure, using\n// syntax similar to C++ expressions.\ntemplate <typename T>\nstruct SupportsDebug\n    : std::disjunction<\n          debug_internal::HasRiegeliDebug<T>, debug_internal::HasDebugString<T>,\n          absl::HasAbslStringify<T>, debug_internal::HasOperatorOutput<T>> {};\n\n// To customize `riegeli::Debug()` for a class `T`, define a free function\n// `friend void RiegeliDebug(const T& src, DebugStream& dest)` as a friend of\n// `T` inside class definition or in the same namespace as `T`, so that it can\n// be found via ADL. `DebugStream` in the parameter type can also be a template\n// parameter to reduce library dependencies.\n//\n// `riegeli::Debug(src)` uses the first defined form among the following:\n//  * `RiegeliDebug(src, dest)`\n//  * `src.DebugString()`\n//  * `dest << src`\n//  * `AbslStringify(dest, src)`\nclass DebugStream {\n public:\n  // Will write to `dest`.\n  explicit DebugStream(std::ostream* dest ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : dest_(dest) {}\n\n  DebugStream(const DebugStream& that) = default;\n  DebugStream& operator=(const DebugStream& that) = default;\n\n  // Writes a character using `std::ostream::write()`.\n  void Write(char src) { dest_->write(&src, 1); }\n\n  // Writes a string using `std::ostream::write()`.\n  void Write(absl::string_view src) {\n    dest_->write(src.data(), static_cast<std::streamsize>(src.size()));\n  }\n\n  // Writes a value formatted using `operator<<`.\n  //\n  // Using stream manipulators is supported, but if the stream state is not\n  // reset to the default before calling `Debug()`, then the results can be\n  // inconsistent, depending on the type being written.\n  template <typename T>\n  DebugStream& operator<<(T&& src) {\n    *dest_ << std::forward<T>(src);\n    return *this;\n  }\n\n  // Writes a value in a format suitable for error messages. This calls the\n  // first defined form among the following:\n  //  * `RiegeliDebug(src, *this)`\n  //  * `Write(src.DebugString())`\n  //  * `AbslStringify(sink, src)` for `OStreamStringifySink(dest)`\n  //  * `*dest << src`\n  //\n  // This is used to implement `riegeli::Debug()`, and to write subobjects by\n  // implementations of `RiegeliDebug()` for objects containing them.\n  template <typename T, std::enable_if_t<SupportsDebug<T>::value, int> = 0>\n  void Debug(const T& src) {\n    if constexpr (debug_internal::HasRiegeliDebug<T>::value) {\n      RiegeliDebug(src, *this);\n    } else if constexpr (debug_internal::HasDebugString<T>::value) {\n      Write(src.DebugString());\n    } else if constexpr (debug_internal::HasOperatorOutput<T>::value) {\n      *dest_ << src;\n    } else {\n      static_assert(absl::HasAbslStringify<T>::value);\n      OStreamStringifySink sink(dest_);\n      AbslStringify(sink, src);\n    }\n  }\n\n  // To implement `RiegeliDebug()` for string-like types which are not\n  // represented as one fragment, the following pattern can be used:\n  //\n  // ```\n  //   dest.DebugStringQuote();\n  //   for (const absl::string_view fragment : fragments) {\n  //     dest.DebugStringFragment(fragment);\n  //   }\n  //   dest.DebugStringQuote();\n  // ```\n  //\n  // If the representation is always flat, relying on `Debug()` for\n  // `absl::string_view` is sufficient.\n  void DebugStringQuote() { Write('\"'); }\n  void DebugStringFragment(absl::string_view src);\n\n private:\n  std::ostream* dest_;\n};\n\n// The following overloads cover supported types which do not define\n// `RiegeliDebug()` themselves.\n\n// `bool` is written as `true` or `false`.\nvoid RiegeliDebug(bool src, DebugStream& dest);\n\n// Non-bool and non-character numeric types, including `signed char` and\n// `unsigned char`, are written as numbers.\ninline void RiegeliDebug(signed char src, DebugStream& dest) {\n  dest << int{src};\n}\ninline void RiegeliDebug(unsigned char src, DebugStream& dest) {\n  dest << unsigned{src};\n}\ninline void RiegeliDebug(short src, DebugStream& dest) { dest << src; }\ninline void RiegeliDebug(unsigned short src, DebugStream& dest) { dest << src; }\ninline void RiegeliDebug(int src, DebugStream& dest) { dest << src; }\ninline void RiegeliDebug(unsigned src, DebugStream& dest) { dest << src; }\ninline void RiegeliDebug(long src, DebugStream& dest) { dest << src; }\ninline void RiegeliDebug(unsigned long src, DebugStream& dest) { dest << src; }\ninline void RiegeliDebug(long long src, DebugStream& dest) { dest << src; }\ninline void RiegeliDebug(unsigned long long src, DebugStream& dest) {\n  dest << src;\n}\ninline void RiegeliDebug(float src, DebugStream& dest) { dest << src; }\ninline void RiegeliDebug(double src, DebugStream& dest) { dest << src; }\ninline void RiegeliDebug(long double src, DebugStream& dest) { dest << src; }\n\n// Character types are written in C++ character literal format.\nvoid RiegeliDebug(char src, DebugStream& dest);\nvoid RiegeliDebug(wchar_t src, DebugStream& dest);\n#if __cpp_char8_t\nvoid RiegeliDebug(char8_t src, DebugStream& dest);\n#endif\nvoid RiegeliDebug(char16_t src, DebugStream& dest);\nvoid RiegeliDebug(char32_t src, DebugStream& dest);\n\n// Enumeration types are written like their underlying types.\ntemplate <typename T, std::enable_if_t<std::is_enum_v<T>, int> = 0>\nvoid RiegeliDebug(T src, DebugStream& dest) {\n  dest.Debug(static_cast<std::underlying_type_t<T>>(src));\n}\n\n// `absl::string_view` is written in C++ string literal format.\n//\n// This covers types implicitly convertible to `absl::string_view` like\n// `std::string` and `CompactString`.\nvoid RiegeliDebug(absl::string_view src, DebugStream& dest);\n\nvoid RiegeliDebug(std::wstring_view src, DebugStream& dest);\n#if __cpp_char8_t\nvoid RiegeliDebug(std::u8string_view src, DebugStream& dest);\n#endif\nvoid RiegeliDebug(std::u16string_view src, DebugStream& dest);\nvoid RiegeliDebug(std::u32string_view src, DebugStream& dest);\n\n// `absl::Cord` is written in C++ string literal format.\nvoid RiegeliDebug(const absl::Cord& src, DebugStream& dest);\n\n// A null pointer is written as \"nullptr\". Other data pointers, including char\n// pointers, as well as function pointers, are written using `operator<<` for\n// `const void*`.\nvoid RiegeliDebug(std::nullptr_t src, DebugStream& dest);\nvoid RiegeliDebug(const void* absl_nullable src, DebugStream& dest);\ntemplate <typename T, std::enable_if_t<std::is_function_v<T>, int> = 0>\nvoid RiegeliDebug(T* absl_nullable src, DebugStream& dest) {\n  dest.Debug(reinterpret_cast<void*>(src));\n}\n\n// `std::unique_ptr` and `std::shared_ptr` are written like pointers.\ntemplate <typename T, typename Deleter>\nvoid RiegeliDebug(const absl_nullable std::unique_ptr<T, Deleter>& src,\n                  DebugStream& dest) {\n  dest.Debug(src.get());\n}\ntemplate <typename T>\nvoid RiegeliDebug(const absl_nullable std::shared_ptr<T>& src,\n                  DebugStream& dest) {\n  dest.Debug(src.get());\n}\n\n// `std::optional` values are written as \"nullopt\" when absent, or as the\n// underlying data wrapped in braces when present.\nvoid RiegeliDebug(std::nullopt_t src, DebugStream& dest);\ntemplate <typename T, std::enable_if_t<SupportsDebug<T>::value, int> = 0>\nvoid RiegeliDebug(const std::optional<T>& src, DebugStream& dest) {\n  if (src == std::nullopt) {\n    dest.Debug(std::nullopt);\n  } else {\n    dest.Write('{');\n    dest.Debug(*src);\n    dest.Write('}');\n  }\n}\n\n// The type returned by `riegeli::Debug()`.\ntemplate <typename T>\nclass DebugType {\n public:\n  template <typename DependentT = T,\n            std::enable_if_t<!std::is_rvalue_reference_v<DependentT>, int> = 0>\n  explicit DebugType(const T& src) : src_(src) {}\n  template <typename DependentT = T,\n            std::enable_if_t<!std::is_lvalue_reference_v<DependentT>, int> = 0>\n  explicit DebugType(T&& src) : src_(std::forward<T>(src)) {}\n\n  DebugType(const DebugType& that) = default;\n  DebugType& operator=(const DebugType& that) = default;\n\n  DebugType(DebugType&& that) = default;\n  DebugType& operator=(DebugType&& that) = default;\n\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const DebugType& src) {\n    StringifyOStream stream(&dest);\n    DebugStream(&stream).Debug(src.src_);\n  }\n\n  // Faster implementation if `Sink` is `OStreamStringifySink`.\n  friend void AbslStringify(OStreamStringifySink& dest, const DebugType& src) {\n    DebugStream(dest.dest()).Debug(src.src_);\n  }\n\n  friend std::ostream& operator<<(std::ostream& dest, const DebugType& src) {\n    DebugStream(&dest).Debug(src.src_);\n    return dest;\n  }\n\n  std::string ToString() const {\n    std::string dest;\n    StringOStream stream(&dest);\n    DebugStream(&stream).Debug(src_);\n    return dest;\n  }\n\n private:\n  T src_;\n};\n\ntemplate <typename T>\nexplicit DebugType(T&& src) -> DebugType<std::decay_t<T>>;\n\n// `riegeli::Debug()` wraps an object such that it is formatted using\n// `DebugStream::Debug()` when explicitly converted to `std::string` or written\n// using `AbslStringify()` or `operator<<`.\n//\n// `riegeli::Debug()` does not own the object, even if it involves temporaries,\n// hence it should be stringified by the same expression which constructed it,\n// so that the temporaries outlive its usage. For storing a `DebugType` in a\n// variable or returning it from a function, construct `DebugType` directly.\ntemplate <typename T, std::enable_if_t<SupportsDebug<T>::value, int> = 0>\ninline DebugType<const T&> Debug(const T& src ABSL_ATTRIBUTE_LIFETIME_BOUND) {\n  return DebugType<const T&>(src);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_DEBUG_H_\n"
  },
  {
    "path": "riegeli/base/dependency.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_DEPENDENCY_H_\n#define RIEGELI_BASE_DEPENDENCY_H_\n\n#include <cstddef>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/meta/type_traits.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/dependency_manager.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/type_traits.h\"\n\nnamespace riegeli {\n\n// `Dependency<Handle, Manager>` stores or refers to an optionally owned object\n// which is stored as type `Manager` and accessed through type `Handle`.\n//\n// When a dependent object is said to be owned by a host object or function, the\n// host is responsible for closing it when done, and certain other operations\n// are propagated to it. The host is usually also responsible for destroying the\n// owned object.\n//\n// Often `Handle` is some pointer `Base*`, and then `Manager` can be e.g.\n// `T` (owned), `T*` (not owned), `std::unique_ptr<T, Deleter>` (owned),\n// or `Any<T*>` (maybe owned), with some `T` derived from `Base`.\n//\n// Often `Dependency<Handle, Manager>` is a member of a host class template\n// parameterized by `Manager`, with `Handle` fixed by the host class. The member\n// is initialized from an argument of a constructor or a resetting function.\n// A user of the host class specifies ownership of the dependent object and\n// possibly narrows its type by choosing the `Manager` template argument of the\n// host class. The `Manager` type can be deduced from a constructor argument\n// using CTAD, which is usually done by removing any toplevel references and\n// `const` qualifiers using `std::decay`.\n//\n// As an alternative to passing `std::move(manager)`, passing\n// `ClosingPtr(&manager)` avoids moving `manager`, but the caller must ensure\n// that the dependent object is valid while the host object needs it.\n//\n// `Manager` can also be `T&` (not owned) or `T&&` (owned). They are primarily\n// meant to be used with a host function rather than a host object, because such\n// a dependency stores only a reference to the dependent object. By convention a\n// reference argument is expected to be valid for the duration of the function\n// call but not necessarily after the function returns. The `Manager` type is\n// usually deduced from a function argument as a reference type rather than\n// using `std::decay`.\n//\n// `Manager` being `T&` is functionally equivalent to `T*`, but offers a more\n// idiomatic API for passing an object which does not need to be valid after the\n// function returns.\n//\n// `Manager` being `T&&` is similar to `ClosingPtrType<T>`. In contrast to a\n// host class, a host function does not decay `T&&` to `T` and avoids moving\n// the `Manager`, because the dependent object can be expected to be valid for\n// the duration of the function call.\n\n// `Dependency<Handle, Manager>` derives from `DependencyImpl<Handle, Manager>`\n// which has specializations for various combinations of `Handle` and `Manager`\n// types. Some operations of `Dependency` are provided by `DependencyImpl`,\n// others are added by `Dependency` in a uniform way.\n//\n// `DependencyImpl<Handle, Manager>` specializations often derive from\n// `DependencyManager<Manager>` or `DependencyBase<Manager>`.\n//\n// `DependencyManager<Manager>` provides a preliminary interpretation of\n// `Manager` independently from `Handle`. This interpretation is then refined by\n// `DependencyImpl`.\n\n// Operations of `Dependency<Handle, Manager>`:\n//\n// ```\n//   // Constructs a dummy `Manager` from\n//   // `RiegeliDependencySentinel(static_cast<Manager*>(nullptr))`. Used\n//   // when the host object is closed and does not need a dependent object.\n//   //\n//   // Supported optionally.\n//   //\n//   // Provided by `DependencyBase` and explicitly inherited.\n//   Dependency();\n//\n//   // Copies or moves a `Manager`. Used to specify the initial value of the\n//   // dependent object.\n//   //\n//   // Provided by `DependencyBase` and explicitly inherited.\n//   explicit Dependency(Initializer<Manager> manager);\n//\n//   // Copies the dependency.\n//   //\n//   // Supported optionally.\n//   Dependency(const Dependency& that) noexcept;\n//   Dependency& operator=(const Dependency& that) noexcept;\n//\n//   // Moves the dependency.\n//   //\n//   // Supported optionally.\n//   Dependency(Dependency&& that) noexcept;\n//   Dependency& operator=(Dependency&& that) noexcept;\n//\n//   // Makes `*this` equivalent to a newly constructed Dependency. This avoids\n//   // constructing a temporary Dependency and moving from it.\n//   //\n//   // The overload with no parameters is supported when the corresponding\n//   // constructor is supported.\n//   //\n//   // Provided by `DependencyBase`.\n//   ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n//   ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Manager> manager);\n//\n//   // Exposes the stored `Manager`.\n//   //\n//   // Provided by `DependencyBase` or `DependencyImpl`.\n//   Manager& manager();\n//   const Manager& manager() const;\n//\n//   // The type returned by `get()`.\n//   //\n//   // Provided by `Dependency`, not `DependencyImpl`.\n//   using Subhandle = ...;\n//\n//   // Returns a `Handle` to the `Manager`.\n//   //\n//   // `get()` might return a subtype of `Handle` which retains more static\n//   // type information about `Manager`, e.g. a pointer to a class derived from\n//   // what `Handle` points to, or a class derived from `Handle`.\n//   //\n//   // The result is non-const even if the `Manager` is stored inside the\n//   // `Dependency`.\n//   //\n//   // Provided by `DependencyImpl`.\n//   Handle get() const;\n//\n//   // If `Handle` is `Base*` or another dereferenceable type, `Dependency` can\n//   // be used as a smart pointer to `Base`, for convenience.\n//   //\n//   // Provided by `Dependency`, not `DependencyImpl`.\n//   Base& operator*() const { return *get(); }\n//   Base* operator->() const { return get(); }\n//\n//   // If `Handle` is `Base*` or another type comparable against `nullptr`,\n//   // `Dependency` can be compared against `nullptr`.\n//   //\n//   // Provided by `Dependency`, not `DependencyImpl`.\n//   friend bool operator==(const Dependency& a, std::nullptr_t) {\n//     return a.get() == nullptr;\n//   }\n//\n//   // If `true`, the `Dependency` owns the dependent object, i.e. closing the\n//   // host object should close the dependent object.\n//   //\n//   // Provided by `DependencyManagerImpl`, `DependencyImpl`, or `Dependency`.\n//   // In `Dependency` implemented in terms of `kIsOwning`.\n//   bool IsOwning() const;\n//\n//   // The value of `IsOwning()` if known statically or mostly statically.\n//   //\n//   // This constant is optional.\n//   //\n//   // If `IsOwning()` returns a statically known constant, `kIsOwning` should\n//   // be defined. `Dependency` will provide `IsOwning()`.\n//   //\n//   // If `IsOwning()` returns `true` except for a sentinel value like\n//   // `nullptr`, e.g. for `std::unique_ptr`, `kIsOwning` can still be defined\n//   // in addition to `IsOwning()`. This allows to use the static\n//   // approximatimation when static selection is needed, with the caveat that\n//   // it will return `true` also for the sentinel value.\n//   //\n//   // Provided by `DependencyManagerImpl` or `DependencyImpl`.\n//   static constexpt bool kIsOwning;\n//\n//   // If `true`, `get()` stays unchanged when a `Dependency` is moved.\n//   //\n//   // This can be used as an optimization to avoid recomputing values derived\n//   // from them when a `Dependency` is moved.\n//   //\n//   // Provided by `DependencyBase`, `DependencyManagerImpl`, or\n//   // `DependencyImpl`.\n//   static constexpr bool kIsStable;\n// ```\n\n// `DependencyImpl` specializations provide what `DependencyBase` provides\n// (constructors, `Reset()`, `manager()`, and `kIsStable`), and also `get()`,\n// `IsOwning()`, and `kIsOwning`.\n\n// This template is specialized but does not have a primary definition.\ntemplate <typename Handle, typename Manager, typename Enable = void>\nclass DependencyImpl;\n\n// Specialization of `DependencyImpl<T*, Manager>` when\n// `DependencyManagerPtr<Manager>` is a pointer convertible to `T*`.\ntemplate <typename T, typename Manager>\nclass DependencyImpl<\n    T*, Manager,\n    std::enable_if_t<std::conjunction_v<\n        std::disjunction<\n            std::is_pointer<DependencyManagerPtr<Manager>>,\n            std::is_same<DependencyManagerPtr<Manager>, std::nullptr_t>>,\n        std::is_convertible<DependencyManagerPtr<Manager>, T*>>>>\n    : public DependencyManager<Manager> {\n public:\n  using DependencyImpl::DependencyManager::DependencyManager;\n\n  DependencyManagerPtr<Manager> get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return this->ptr();\n  }\n\n protected:\n  DependencyImpl(const DependencyImpl& that) = default;\n  DependencyImpl& operator=(const DependencyImpl& that) = default;\n\n  DependencyImpl(DependencyImpl&& that) = default;\n  DependencyImpl& operator=(DependencyImpl&& that) = default;\n\n  ~DependencyImpl() = default;\n};\n\n// Specialization of `DependencyImpl<absl::Span<T>, Manager>` when\n// `DependencyManagerRef<Manager>` is explicitly convertible to `absl::Span<T>`.\n//\n// Specialized separately for `get()` to return\n// `absl::Span<std::remove_const_t<T>>` if possible.\ntemplate <typename T, typename Manager>\nclass DependencyImpl<\n    absl::Span<T>, Manager,\n    std::enable_if_t<std::conjunction_v<\n        std::is_pointer<DependencyManagerPtr<Manager>>,\n        std::is_constructible<absl::Span<T>, DependencyManagerRef<Manager>>>>>\n    : public DependencyManager<Manager> {\n public:\n  using DependencyImpl::DependencyManager::DependencyManager;\n\n  // Return `absl::Span<std::remove_const_t<T>>` when\n  // `DependencyManagerRef<Manager>` is convertible to it.\n  template <typename DependentManager = Manager,\n            std::enable_if_t<\n                std::is_constructible_v<absl::Span<std::remove_const_t<T>>,\n                                        DependencyManagerRef<DependentManager>>,\n                int> = 0>\n  absl::Span<std::remove_const_t<T>> get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return absl::Span<std::remove_const_t<T>>(*this->ptr());\n  }\n  template <typename DependentManager = Manager,\n            std::enable_if_t<!std::is_constructible_v<\n                                 absl::Span<std::remove_const_t<T>>,\n                                 DependencyManagerRef<DependentManager>>,\n                             int> = 0>\n  absl::Span<T> get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return absl::Span<T>(*this->ptr());\n  }\n\n  static constexpr bool kIsStable =\n      DependencyImpl::DependencyManager::kIsStable ||\n      std::is_same_v<Manager, absl::Span<T>> ||\n      std::is_same_v<Manager, absl::Span<std::remove_const_t<T>>>;\n\n protected:\n  DependencyImpl(const DependencyImpl& that) = default;\n  DependencyImpl& operator=(const DependencyImpl& that) = default;\n\n  DependencyImpl(DependencyImpl&& that) = default;\n  DependencyImpl& operator=(DependencyImpl&& that) = default;\n\n  ~DependencyImpl() = default;\n};\n\n// Specialization of `DependencyImpl<absl::Span<T>, Manager>` when\n// `DependencyManagerPtr<Manager>` is `absl::Span<T>` or\n// `absl::Span<std::remove_const_t<T>>`.\n//\n// Specialized separately for `get()` to return\n// `absl::Span<std::remove_const_t<T>>` if possible.\ntemplate <typename T, typename Manager>\nclass DependencyImpl<\n    absl::Span<T>, Manager,\n    std::enable_if_t<std::disjunction_v<\n        std::is_same<DependencyManagerPtr<Manager>, absl::Span<T>>,\n        std::is_same<DependencyManagerPtr<Manager>,\n                     absl::Span<std::remove_const_t<T>>>>>>\n    : public DependencyManager<Manager> {\n public:\n  using DependencyImpl::DependencyManager::DependencyManager;\n\n  DependencyManagerPtr<Manager> get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return this->ptr();\n  }\n\n protected:\n  DependencyImpl(const DependencyImpl& that) = default;\n  DependencyImpl& operator=(const DependencyImpl& that) = default;\n\n  DependencyImpl(DependencyImpl&& that) = default;\n  DependencyImpl& operator=(DependencyImpl&& that) = default;\n\n  ~DependencyImpl() = default;\n};\n\n// Specialization of `DependencyImpl<absl::string_view, Manager>` when\n// `DependencyManagerRef<Manager>` is convertible to `BytesRef`.\ntemplate <typename Manager>\nclass DependencyImpl<\n    absl::string_view, Manager,\n    std::enable_if_t<std::conjunction_v<\n        std::is_pointer<DependencyManagerPtr<Manager>>,\n        std::is_convertible<DependencyManagerRef<Manager>, BytesRef>>>>\n    : public DependencyManager<Manager> {\n public:\n  using DependencyImpl::DependencyManager::DependencyManager;\n\n  absl::string_view get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return BytesRef(*this->ptr());\n  }\n\n  static constexpr bool kIsStable =\n      DependencyImpl::DependencyManager::kIsStable ||\n      std::is_same_v<Manager, absl::string_view> ||\n      std::is_same_v<Manager, absl::Span<const char>> ||\n      std::is_same_v<Manager, absl::Span<char>>;\n\n protected:\n  DependencyImpl(const DependencyImpl& that) = default;\n  DependencyImpl& operator=(const DependencyImpl& that) = default;\n\n  DependencyImpl(DependencyImpl&& that) = default;\n  DependencyImpl& operator=(DependencyImpl&& that) = default;\n\n  ~DependencyImpl() = default;\n};\n\n// Specialization of `DependencyImpl<absl::string_view, Manager>` when\n// `DependencyManagerPtr<Manager>` is `absl::Span<const char>` or\n// `absl::Span<char>`.\n//\n// Specialized separately because `absl::Span<const char>` is not convertible\n// to `absl::string_view` in the regular way.\ntemplate <typename Manager>\nclass DependencyImpl<\n    absl::string_view, Manager,\n    std::enable_if_t<std::disjunction_v<\n        std::is_same<DependencyManagerPtr<Manager>, absl::Span<const char>>,\n        std::is_same<DependencyManagerPtr<Manager>, absl::Span<char>>>>>\n    : public DependencyManager<Manager> {\n public:\n  using DependencyImpl::DependencyManager::DependencyManager;\n\n  absl::string_view get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    const absl::Span<const char> span = this->ptr();\n    return absl::string_view(span.data(), span.size());\n  }\n\n protected:\n  DependencyImpl(const DependencyImpl& that) = default;\n  DependencyImpl& operator=(const DependencyImpl& that) = default;\n\n  DependencyImpl(DependencyImpl&& that) = default;\n  DependencyImpl& operator=(DependencyImpl&& that) = default;\n\n  ~DependencyImpl() = default;\n};\n\nnamespace dependency_internal {\n\n// `SupportsDependencyImpl<Handle, Manager>::value` is `true` when\n// `DependencyImpl<Handle, Manager>` is defined.\n\ntemplate <typename Handle, typename Manager, typename Enable = void>\nstruct SupportsDependencyImpl : std::false_type {};\n\ntemplate <typename Handle, typename Manager>\nstruct SupportsDependencyImpl<\n    Handle, Manager,\n    std::void_t<\n        decltype(std::declval<const DependencyImpl<Handle, Manager>&>().get())>>\n    : std::true_type {};\n\n// `DependencyDefault<Handle, Manager>` extends\n// `DependencyImpl<Handle, Manager>` with the basic cases when\n// `DependencyManagerRef<Manager>` or `DependencyManagerPtr<Manager>` is\n// explicitly convertible to `Handle`.\n\n// This template is specialized but does not have a primary definition.\ntemplate <typename Handle, typename Manager, typename Enable = void>\nclass DependencyDefault;\n\n// Specialization of `DependencyDefault<Handle, Manager>` when\n// `DependencyImpl<Handle, Manager>` is defined: delegate to it.\ntemplate <typename Handle, typename Manager>\nclass DependencyDefault<\n    Handle, Manager,\n    std::enable_if_t<SupportsDependencyImpl<Handle, Manager>::value>>\n    : public DependencyImpl<Handle, Manager> {\n public:\n  using DependencyDefault::DependencyImpl::DependencyImpl;\n\n  static_assert(\n      std::is_convertible_v<\n          decltype(std::declval<\n                       const typename DependencyDefault::DependencyImpl&>()\n                       .get()),\n          Handle>,\n      \"DependencyImpl<Handle, Manager>::get() must return a subtype of Handle\");\n\n protected:\n  DependencyDefault(const DependencyDefault& that) = default;\n  DependencyDefault& operator=(const DependencyDefault& that) = default;\n\n  DependencyDefault(DependencyDefault&& that) = default;\n  DependencyDefault& operator=(DependencyDefault&& that) = default;\n\n  ~DependencyDefault() = default;\n};\n\n// Specialization of `DependencyDefault<Handle, Manager>` when\n// `DependencyImpl<Handle, Manager>` is not defined and\n// `DependencyManagerRef<Manager>` is explicitly convertible to `Handle`:\n// let `get()` return `*ptr()`, as its original type if possible.\ntemplate <typename Handle, typename Manager>\nclass DependencyDefault<\n    Handle, Manager,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<SupportsDependencyImpl<Handle, Manager>>,\n        std::is_pointer<DependencyManagerPtr<Manager>>,\n        std::is_constructible<Handle, DependencyManagerRef<Manager>>>>>\n    : public DependencyManager<Manager> {\n public:\n  using DependencyDefault::DependencyManager::DependencyManager;\n\n  // Return `DependencyManagerRef<Manager>` when it is a subclass of `Handle`.\n  template <\n      typename DependentManager = Manager,\n      std::enable_if_t<std::is_convertible_v<\n                           DependencyManagerPtr<DependentManager>, Handle*>,\n                       int> = 0>\n  DependencyManagerRef<DependentManager> get() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return *this->ptr();\n  }\n  template <\n      typename DependentManager = Manager,\n      std::enable_if_t<!std::is_convertible_v<\n                           DependencyManagerPtr<DependentManager>, Handle*>,\n                       int> = 0>\n  Handle get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return Handle(*this->ptr());\n  }\n\n  static constexpr bool kIsStable =\n      DependencyDefault::DependencyManager::kIsStable ||\n      std::is_convertible_v<DependencyManagerPtr<Manager>, Handle*>;\n\n protected:\n  DependencyDefault(const DependencyDefault& that) = default;\n  DependencyDefault& operator=(const DependencyDefault& that) = default;\n\n  DependencyDefault(DependencyDefault&& that) = default;\n  DependencyDefault& operator=(DependencyDefault&& that) = default;\n\n  ~DependencyDefault() = default;\n};\n\n// Specialization of `DependencyDefault<Handle, Manager>` when\n// `DependencyImpl<Handle, Manager>` is not defined,\n// `DependencyManagerRef<Manager>` is not convertible to `Handle`, and\n// `DependencyManagerPtr<Manager>` is explicitly convertible to `Handle`:\n// let `get()` return `ptr()`, as its original type if possible.\ntemplate <typename Handle, typename Manager>\nclass DependencyDefault<\n    Handle, Manager,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<SupportsDependencyImpl<Handle, Manager>>,\n        std::negation<std::conjunction<\n            std::is_pointer<DependencyManagerPtr<Manager>>,\n            std::is_constructible<Handle, DependencyManagerRef<Manager>>>>,\n        std::is_constructible<Handle, DependencyManagerPtr<Manager>>>>>\n    : public DependencyManager<Manager> {\n public:\n  using DependencyDefault::DependencyManager::DependencyManager;\n\n  // Return `DependencyManagerPtr<Manager>` when it is a subclass of `Handle`.\n  template <\n      typename DependentManager = Manager,\n      std::enable_if_t<std::is_convertible_v<\n                           DependencyManagerPtr<DependentManager>*, Handle*>,\n                       int> = 0>\n  DependencyManagerPtr<DependentManager> get() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return this->ptr();\n  }\n  template <\n      typename DependentManager = Manager,\n      std::enable_if_t<!std::is_convertible_v<\n                           DependencyManagerPtr<DependentManager>*, Handle*>,\n                       int> = 0>\n  Handle get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return Handle(this->ptr());\n  }\n\n  static constexpr bool kIsStable =\n      DependencyDefault::DependencyManager::kIsStable ||\n      std::is_convertible_v<DependencyManagerPtr<Manager>*, Handle*>;\n\n protected:\n  DependencyDefault(const DependencyDefault& that) = default;\n  DependencyDefault& operator=(const DependencyDefault& that) = default;\n\n  DependencyDefault(DependencyDefault&& that) = default;\n  DependencyDefault& operator=(DependencyDefault&& that) = default;\n\n  ~DependencyDefault() = default;\n};\n\n// `SupportsDependencyDefault<Handle, Manager>::value` is `true` when\n// `DependencyDefault<Handle, Manager, Manager&>` is defined.\ntemplate <typename Handle, typename Manager>\nstruct SupportsDependencyDefault\n    : std::disjunction<\n          dependency_internal::SupportsDependencyImpl<Handle, Manager>,\n          std::conjunction<\n              std::is_pointer<DependencyManagerPtr<Manager>>,\n              std::is_constructible<Handle, DependencyManagerRef<Manager>>>,\n          std::is_constructible<Handle, DependencyManagerPtr<Manager>>> {};\n\n// `DependencyDeref<Handle, Manager>` extends\n// `DependencyDefault<Handle, Manager>` with cases where `Manager` is\n// a reference, if `DependencyImpl<Handle, Manager>` is not defined.\n//\n// If `DependencyImpl<Handle, Manager>` uses `DependencyManager<Manager>`, then\n// this is already covered. Custom specializations might not cover this.\n\n// This template is specialized but does not have a primary definition.\ntemplate <typename Handle, typename Manager, typename Enable = void>\nclass DependencyDeref;\n\n// Specialization of `DependencyDeref<Handle, Manager>` when\n// `DependencyDefault<Handle, Manager>` is defined: delegate to it.\ntemplate <typename Handle, typename Manager>\nclass DependencyDeref<\n    Handle, Manager,\n    std::enable_if_t<SupportsDependencyDefault<Handle, Manager>::value>>\n    : public DependencyDefault<Handle, Manager> {\n public:\n  using DependencyDeref::DependencyDefault::DependencyDefault;\n\n protected:\n  DependencyDeref(const DependencyDeref& that) = default;\n  DependencyDeref& operator=(const DependencyDeref& that) = default;\n\n  DependencyDeref(DependencyDeref&& that) = default;\n  DependencyDeref& operator=(DependencyDeref&& that) = default;\n\n  ~DependencyDeref() = default;\n};\n\n// Specialization of `DependencyDeref<Handle, Manager>` when\n// `DependencyDefault<Handle, Manager>` is not defined,\n// `Manager` is a reference, and\n// `DependencyDefault<Handle, absl::remove_cvref_t<Manager>>` is defined:\n// delegate to the latter.\ntemplate <typename Handle, typename Manager>\nclass DependencyDeref<\n    Handle, Manager,\n    std::enable_if_t<std::conjunction_v<\n        std::is_reference<Manager>,\n        std::negation<SupportsDependencyDefault<Handle, Manager>>,\n        SupportsDependencyDefault<Handle, absl::remove_cvref_t<Manager>>>>>\n    : public DependencyDefault<Handle, absl::remove_cvref_t<Manager>> {\n public:\n  using DependencyDeref::DependencyDefault::DependencyDefault;\n\n protected:\n  DependencyDeref(const DependencyDeref& that) = default;\n  DependencyDeref& operator=(const DependencyDeref& that) = default;\n\n  DependencyDeref(DependencyDeref&& that) = default;\n  DependencyDeref& operator=(DependencyDeref&& that) = default;\n\n  ~DependencyDeref() = default;\n};\n\n// `SupportsDependencyDeref<Handle, Manager>::value` is `true` when\n// `DependencyDeref<Handle, Manager>` is defined.\ntemplate <typename Handle, typename Manager>\nstruct SupportsDependencyDeref\n    : std::disjunction<\n          SupportsDependencyDefault<Handle, Manager>,\n          std::conjunction<std::is_reference<Manager>,\n                           SupportsDependencyDefault<\n                               Handle, absl::remove_cvref_t<Manager>>>> {};\n\n}  // namespace dependency_internal\n\n// `SupportsDependency<Handle, Manager>::value` is `true` when\n// `Dependency<Handle, Manager>` is defined and usable, i.e. constructible from\n// `Initializer<Manager>`.\n//\n// An immovable `Manager` is usable when the `Initializer<Manager>` has been\n// constructed from `riegeli::Maker()` or `riegeli::Invoker()`, not from an\n// already constructed object.\ntemplate <typename Handle, typename Manager>\nstruct SupportsDependency\n    : std::conjunction<\n          dependency_internal::SupportsDependencyDeref<Handle, Manager>> {};\n\n// `TargetSupportsDependency<Handle, Manager>::value` is `true` when\n// `Dependency<Handle, TargetT<Manager>>` is defined and constructible from\n// `Manager&&`.\n//\n// An immovable `TargetT<Manager>` is usable when the `Dependency` has been\n// initialized with `riegeli::Maker()` or `riegeli::Invoker()`, possibly behind\n// `Initializer`, not from an already constructed object.\ntemplate <typename Handle, typename Manager>\nstruct TargetSupportsDependency\n    : std::conjunction<\n          SupportsDependency<Handle, TargetT<Manager>>,\n          std::is_convertible<Manager&&, Initializer<TargetT<Manager>>>> {};\n\n// `TargetRefSupportsDependency<Handle, Manager>::value` is `true` when\n// `DependencyRef<Handle, Manager>` i.e.\n// `Dependency<Handle, TargetRefT<Manager>>` is defined and constructible from\n// `Manager&&`.\n//\n// An immovable `TargetRefT<Manager>` is usable when the `Dependency` has been\n// initialized with `riegeli::Maker()` or `riegeli::Invoker()`, possibly behind\n// `Initializer`, not from an already constructed object.\ntemplate <typename Handle, typename Manager>\nstruct TargetRefSupportsDependency\n    : std::conjunction<\n          SupportsDependency<Handle, TargetRefT<Manager>>,\n          std::is_convertible<Manager&&, Initializer<TargetRefT<Manager>>>> {};\n\nnamespace dependency_internal {\n\ntemplate <bool value>\nstruct IsConstexprBool : std::true_type {};\n\n}  // namespace dependency_internal\n\n// `HasStaticIsOwning<T>::value` is `true` if `T` defines\n// `static constexpr bool kIsOwning`.\n\ntemplate <typename T, typename Enable = void>\nstruct HasStaticIsOwning : std::false_type {};\n\ntemplate <typename T>\nstruct HasStaticIsOwning<\n    T,\n    std::enable_if_t<dependency_internal::IsConstexprBool<T::kIsOwning>::value>>\n    : std::true_type {};\n\n// Deriving a class from `PropagateStaticIsOwning<T>` defines\n// `static constexpr bool kIsOwning = T::kIsOwning` if `T` defines `kIsOwning`.\n\ntemplate <typename T, typename Enable = void>\nclass PropagateStaticIsOwning {};\n\ntemplate <typename T>\nclass PropagateStaticIsOwning<T,\n                              std::enable_if_t<HasStaticIsOwning<T>::value>> {\n public:\n  static constexpr bool kIsOwning = T::kIsOwning;\n};\n\nnamespace dependency_internal {\n\ntemplate <typename T, typename Enable = void>\nstruct HasDynamicIsOwning : std::false_type {};\n\ntemplate <typename T>\nstruct HasDynamicIsOwning<\n    T, std::enable_if_t<std::is_convertible_v<\n           decltype(std::declval<const T&>().IsOwning()), bool>>>\n    : std::true_type {};\n\n// `DependencyDerived` adds `Dependency` and `StableDependency` operations\n// uniformly implemented in terms of other operations: `operator*`,\n// `operator->`, and comparisons against `nullptr`.\n//\n// It derives from the template parameter `Base` so that it can be used in\n// `Dependency` (applied to `DependencyDeref`) and `StableDependency`\n// (applied to `StableDependencyImpl`).\ntemplate <typename Base, typename Handle, typename Manager>\nclass DependencyDerived\n    : public Base,\n      public WithEqual<DependencyDerived<Base, Handle, Manager>> {\n public:\n  using Base::Base;\n\n  using Subhandle = decltype(std::declval<const Base&>().get());\n\n  template <\n      typename DependentSubhandle = Subhandle,\n      std::enable_if_t<HasDereference<DependentSubhandle>::value, int> = 0>\n  decltype(*std::declval<DependentSubhandle>()) operator*() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    Subhandle handle = this->get();\n    AssertNotNull(handle,\n                  \"Failed precondition of Dependency::operator*: null handle\");\n    return *std::move(handle);\n  }\n\n  template <typename DependentSubhandle = Subhandle,\n            std::enable_if_t<HasArrow<DependentSubhandle>::value, int> = 0>\n  Subhandle operator->() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    Subhandle handle = this->get();\n    AssertNotNull(handle,\n                  \"Failed precondition of Dependency::operator->: null handle\");\n    return handle;\n  }\n\n  template <typename DependentSubhandle = Subhandle,\n            std::enable_if_t<\n                IsComparableAgainstNullptr<DependentSubhandle>::value, int> = 0>\n  friend bool operator==(const DependencyDerived& a, std::nullptr_t) {\n    return a.get() == nullptr;\n  }\n\n  template <typename DependentBase = Base,\n            std::enable_if_t<HasDynamicIsOwning<DependentBase>::value, int> = 0>\n  bool IsOwning() const {\n    return Base::IsOwning();\n  }\n  template <\n      typename DependentBase = Base,\n      std::enable_if_t<\n          std::conjunction_v<std::negation<HasDynamicIsOwning<DependentBase>>,\n                             HasStaticIsOwning<DependentBase>>,\n          int> = 0>\n  bool IsOwning() const {\n    return Base::kIsOwning;\n  }\n\n protected:\n  DependencyDerived(const DependencyDerived& that) = default;\n  DependencyDerived& operator=(const DependencyDerived& that) = default;\n\n  DependencyDerived(DependencyDerived&& that) = default;\n  DependencyDerived& operator=(DependencyDerived&& that) = default;\n\n  ~DependencyDerived() = default;\n\n private:\n  template <typename DependentSubhandle = Subhandle,\n            std::enable_if_t<\n                IsComparableAgainstNullptr<DependentSubhandle>::value, int> = 0>\n  static void AssertNotNull(Subhandle handle, absl::string_view message) {\n    RIEGELI_ASSERT(handle != nullptr) << message;\n  }\n  template <\n      typename DependentSubhandle = Subhandle,\n      std::enable_if_t<!IsComparableAgainstNullptr<DependentSubhandle>::value,\n                       int> = 0>\n  static void AssertNotNull(ABSL_ATTRIBUTE_UNUSED Subhandle handle,\n                            ABSL_ATTRIBUTE_UNUSED absl::string_view message) {}\n};\n\n}  // namespace dependency_internal\n\ntemplate <typename Handle, typename Manager>\nclass Dependency : public dependency_internal::DependencyDerived<\n                       dependency_internal::DependencyDeref<Handle, Manager>,\n                       Handle, Manager> {\n public:\n  using Dependency::DependencyDerived::DependencyDerived;\n\n  Dependency(const Dependency& that) = default;\n  Dependency& operator=(const Dependency& that) = default;\n\n  Dependency(Dependency&& that) = default;\n  Dependency& operator=(Dependency&& that) = default;\n};\n\n// `DependencyRef<Handle, Manager>` is an alias for\n// `Dependency<Handle, TargetRefT<Manager>>`.\ntemplate <typename Handle, typename Manager>\nusing DependencyRef = Dependency<Handle, TargetRefT<Manager>>;\n\nnamespace dependency_internal {\n\n// `AlwaysFalse<T...>::value` is `false`, but formally depends on `T...`.\n// This is useful for `static_assert()`.\n\ntemplate <typename... T>\nstruct AlwaysFalse : std::false_type {};\n\n}  // namespace dependency_internal\n\n// A placeholder `Dependency` manager to be deduced by CTAD, used to delete CTAD\n// for particular constructor argument types.\n//\n// It takes `ConstructorArgTypes` so that an error message from the\n// `static_assert()` can show them.\n\ntemplate <typename... ConstructorArgTypes>\nstruct DeleteCtad {\n  DeleteCtad() = delete;\n};\n\ntemplate <typename Handle, typename... ConstructorArgTypes>\nclass Dependency<Handle, DeleteCtad<ConstructorArgTypes...>> {\n  static_assert(dependency_internal::AlwaysFalse<ConstructorArgTypes...>::value,\n                \"Template arguments must be written explicitly \"\n                \"with these constructor argument types\");\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_DEPENDENCY_H_\n"
  },
  {
    "path": "riegeli/base/dependency_base.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_DEPENDENCY_BASE_H_\n#define RIEGELI_BASE_DEPENDENCY_BASE_H_\n\n#include <stddef.h>\n\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/reset.h\"\n\nnamespace riegeli {\n\n// `RiegeliDependencySentinel(T*)` specifies how to initialize a default\n// `Manager` (for `Dependency`) or `Handle` (for `Any`) of type `T`.\n//\n// To customize that for a class `T`, define a free function\n// `friend Result RiegeliDependencySentinel(T*)` as a friend of `T` inside class\n// definition or in the same namespace as `T`, so that it can be found via ADL.\n//\n// `RiegeliDependencySentinel(T*)` returns a value convertible to\n// `Initializer<T>`, usually a `MakerType<Args...>`.\n//\n// The argument of `RiegeliDependencySentinel(T*)` is always a null pointer,\n// used to choose the right overload based on the type.\n\ninline MakerType<> RiegeliDependencySentinel(void*) { return {}; }\n\n// Implementation shared between most specializations of `DependencyManagerImpl`\n// and `DependencyImpl` which store `manager()` in a member variable.\n//\n// `DependencyBase` provides constructors, `Reset()`, `manager()`, `kIsStable`,\n// and protected `mutable_manager()`.\ntemplate <typename Manager>\nclass DependencyBase {\n public:\n  template <typename DependentManager = Manager,\n            std::enable_if_t<std::is_convertible_v<\n                                 decltype(RiegeliDependencySentinel(\n                                     static_cast<DependentManager*>(nullptr))),\n                                 Initializer<DependentManager>>,\n                             int> = 0>\n  DependencyBase() noexcept\n      : DependencyBase(\n            RiegeliDependencySentinel(static_cast<Manager*>(nullptr))) {}\n\n  explicit DependencyBase(Initializer<Manager> manager)\n      : manager_(std::move(manager)) {}\n\n  template <\n      typename DependentManager = Manager,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::is_convertible<decltype(RiegeliDependencySentinel(\n                                      static_cast<DependentManager*>(nullptr))),\n                                  Initializer<DependentManager>>,\n              std::is_move_assignable<DependentManager>>,\n          int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset() {\n    Reset(RiegeliDependencySentinel(static_cast<Manager*>(nullptr)));\n  }\n  template <\n      typename DependentManager = Manager,\n      std::enable_if_t<std::is_move_assignable_v<DependentManager>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Manager> manager) {\n    riegeli::Reset(manager_, std::move(manager));\n  }\n\n  Manager& manager() ABSL_ATTRIBUTE_LIFETIME_BOUND { return manager_; }\n  const Manager& manager() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return manager_;\n  }\n\n  static constexpr bool kIsStable = false;\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const DependencyBase* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->manager_);\n  }\n\n protected:\n  DependencyBase(const DependencyBase& that) = default;\n  DependencyBase& operator=(const DependencyBase& that) = default;\n\n  DependencyBase(DependencyBase&& that) = default;\n  DependencyBase& operator=(DependencyBase&& that) = default;\n\n  ~DependencyBase() = default;\n\n  Manager& mutable_manager() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return manager_;\n  }\n\n private:\n  mutable Manager manager_;\n};\n\n// Specialization of `DependencyBase` for lvalue references.\n//\n// Only a subset of operations is provided: the dependency must be initialized,\n// and assignment is not supported.\ntemplate <typename Manager>\nclass DependencyBase<Manager&> {\n public:\n  explicit DependencyBase(Initializer<Manager&> manager) noexcept\n      : manager_(std::move(manager)) {}\n\n  Manager& manager() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return manager_; }\n\n  static constexpr bool kIsStable = true;\n\n protected:\n  DependencyBase(const DependencyBase& that) = default;\n  DependencyBase& operator=(const DependencyBase&) = delete;\n\n  ~DependencyBase() = default;\n\n  Manager& mutable_manager() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return manager_;\n  }\n\n private:\n  Manager& manager_;\n};\n\n// Specialization of `DependencyBase` for rvalue references.\n//\n// Only a subset of operations is provided: the dependency must be initialized,\n// and assignment is not supported.\ntemplate <typename Manager>\nclass DependencyBase<Manager&&> {\n public:\n  explicit DependencyBase(Initializer<Manager&&> manager) noexcept\n      : manager_(std::move(manager)) {}\n\n  Manager& manager() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return manager_; }\n\n  static constexpr bool kIsStable = true;\n\n protected:\n  DependencyBase(DependencyBase&& that) = default;\n  DependencyBase& operator=(DependencyBase&&) = delete;\n\n  ~DependencyBase() = default;\n\n  Manager& mutable_manager() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return manager_;\n  }\n\n private:\n  Manager&& manager_;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_DEPENDENCY_BASE_H_\n"
  },
  {
    "path": "riegeli/base/dependency_manager.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_DEPENDENCY_MANAGER_H_\n#define RIEGELI_BASE_DEPENDENCY_MANAGER_H_\n\n#include <cstddef>\n#include <memory>\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/meta/type_traits.h\"\n#include \"riegeli/base/dependency_base.h\"\n\nnamespace riegeli {\n\n// `DependencyManager<Manager>` provides a preliminary interpretation of\n// `Manager` as a pointer or pointer-like type, in the form of a protected\n// member function `ptr()`. It is used by `DependencyImpl` specializations to\n// infer `get()`, which often returns `*ptr()` or `ptr()`, depending on which\n// of them is convertible to `Handle`.\n//\n// Examples:\n//  * `T* DependencyManager<T>::ptr()`\n//  * `T* DependencyManager<T&>::ptr()`\n//  * `T* DependencyManager<T&&>::ptr()`\n//  * `T* DependencyManager<T*>::ptr()`\n//  * `std::nullptr_t DependencyManager<std::nullptr_t>::ptr()`\n//  * `T* DependencyManager<std::unique_ptr<T, Deleter>>::ptr()`\n//  * `T* DependencyManager<std::optional<T>>::ptr()`\n//  * `Handle DependencyManager<Any<Handle>>::ptr()`\n//\n// `DependencyManager<Manager>` derives from\n// `DependencyManagerImpl<Manager, ManagerStorage>` (where `ManagerStorage` is\n// `Manager`, `Manager&`, or `Manager&&`) which has specializations for various\n// `Manager` types.\n//\n// `DependencyManagerImpl<Manager, ManagerStorage>` specializations often derive\n// from `DependencyBase<ManagerStorage>` (or from `DependencyBase<Manager>` if\n// `Manager` is cheap to move).\n//\n// `DependencyManager` provides what `DependencyBase` provides (constructors,\n// `Reset()`, `manager()`, and `kIsStable`), and also `ptr()`, `IsOwning()`,\n// and `kIsOwning`.\n\n// This template is specialized but does not have a primary definition.\ntemplate <typename Manager, typename ManagerStorage, typename Enable = void>\nclass DependencyManagerImpl;\n\n// Specialization of `DependencyManagerImpl<T*, ManagerStorage>`: an unowned\n// dependency stored by pointer.\ntemplate <typename T, typename ManagerStorage>\nclass DependencyManagerImpl<T*, ManagerStorage> : public DependencyBase<T*> {\n public:\n  using DependencyManagerImpl::DependencyBase::DependencyBase;\n\n  static constexpr bool kIsOwning = false;\n\n  static constexpr bool kIsStable = true;\n\n protected:\n  DependencyManagerImpl(const DependencyManagerImpl& that) = default;\n  DependencyManagerImpl& operator=(const DependencyManagerImpl& that) = default;\n\n  DependencyManagerImpl(DependencyManagerImpl&& that) = default;\n  DependencyManagerImpl& operator=(DependencyManagerImpl&& that) = default;\n\n  ~DependencyManagerImpl() = default;\n\n  T* ptr() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return this->manager(); }\n};\n\n// Specialization of `DependencyManagerImpl<std::nullptr_t, ManagerStorage>`:\n// an unowned dependency stored by pointer, always missing. This is useful for\n// `Any` and `AnyRef`.\ntemplate <typename ManagerStorage>\nclass DependencyManagerImpl<std::nullptr_t, ManagerStorage>\n    : public DependencyBase<std::nullptr_t> {\n public:\n  using DependencyManagerImpl::DependencyBase::DependencyBase;\n\n  static constexpr bool kIsOwning = false;\n\n  static constexpr bool kIsStable = true;\n\n protected:\n  DependencyManagerImpl(const DependencyManagerImpl& that) = default;\n  DependencyManagerImpl& operator=(const DependencyManagerImpl& that) = default;\n\n  DependencyManagerImpl(DependencyManagerImpl&& that) = default;\n  DependencyManagerImpl& operator=(DependencyManagerImpl&& that) = default;\n\n  ~DependencyManagerImpl() = default;\n\n  std::nullptr_t ptr() const { return nullptr; }\n};\n\n// Specialization of\n// `DependencyManagerImpl<std::unique_ptr<T, Deleter>, ManagerStorage>`:\n// an owned dependency stored by `std::unique_ptr`.\ntemplate <typename T, typename Deleter, typename ManagerStorage>\nclass DependencyManagerImpl<std::unique_ptr<T, Deleter>, ManagerStorage>\n    : public DependencyBase<\n          std::conditional_t<std::is_empty_v<Deleter>,\n                             std::unique_ptr<T, Deleter>, ManagerStorage>> {\n public:\n  using DependencyManagerImpl::DependencyBase::DependencyBase;\n\n  bool IsOwning() const { return this->manager() != nullptr; }\n\n  static constexpr bool kIsOwning = true;\n\n  static constexpr bool kIsStable = true;\n\n protected:\n  DependencyManagerImpl(DependencyManagerImpl&& that) = default;\n  DependencyManagerImpl& operator=(DependencyManagerImpl&& that) = default;\n\n  ~DependencyManagerImpl() = default;\n\n  T* ptr() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return this->manager().get(); }\n};\n\n// Specialization of\n// `DependencyManagerImpl<std::optional<T>, ManagerStorage>`:\n// an owned dependency stored by `std::optional`.\ntemplate <typename T, typename ManagerStorage>\nclass DependencyManagerImpl<std::optional<T>, ManagerStorage>\n    : public DependencyBase<ManagerStorage> {\n public:\n  using DependencyManagerImpl::DependencyBase::DependencyBase;\n\n  bool IsOwning() const { return this->manager() != std::nullopt; }\n\n  static constexpr bool kIsOwning = true;\n\n protected:\n  DependencyManagerImpl(const DependencyManagerImpl& that) = default;\n  DependencyManagerImpl& operator=(const DependencyManagerImpl& that) = default;\n\n  DependencyManagerImpl(DependencyManagerImpl&& that) = default;\n  DependencyManagerImpl& operator=(DependencyManagerImpl&& that) = default;\n\n  ~DependencyManagerImpl() = default;\n\n  T* ptr() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    if (this->mutable_manager() == std::nullopt) return nullptr;\n    return &*this->mutable_manager();\n  }\n};\n\nnamespace dependency_manager_internal {\n\n// `SupportsDependencyManagerImpl<Manager>::value` is `true` when\n// `DependencyManagerImpl<Manager, Manager>` is defined.\n\ntemplate <typename Manager, typename Enable = void>\nstruct SupportsDependencyManagerImpl : std::false_type {};\n\ntemplate <typename Manager>\nstruct SupportsDependencyManagerImpl<\n    Manager,\n    std::void_t<\n        decltype(std::declval<const DependencyManagerImpl<Manager, Manager>&>()\n                     .manager())>> : std::true_type {};\n\n}  // namespace dependency_manager_internal\n\n// `DependencyManager<Manager>` extends\n// `DependencyManagerImpl<Manager, ManagerStorage>` with the basic case when\n// `Manager` is an owned dependency stored by value, and with specializations\n// when `Manager` is `T&` or `T&&`.\n\ntemplate <typename Manager, typename Enable = void>\nclass DependencyManager;\n\n// Specialization of `DependencyManager<Manager>` when\n// `DependencyManagerImpl<Manager>` is defined: delegate to it.\ntemplate <typename Manager>\nclass DependencyManager<\n    Manager,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<std::is_reference<Manager>>,\n        dependency_manager_internal::SupportsDependencyManagerImpl<Manager>>>>\n    : public DependencyManagerImpl<Manager, Manager> {\n public:\n  using DependencyManager::DependencyManagerImpl::DependencyManagerImpl;\n\n  static_assert(\n      std::is_convertible_v<\n          decltype(std::declval<\n                       typename DependencyManager::DependencyManagerImpl&>()\n                       .manager()),\n          Manager&>,\n      \"DependencyManagerImpl<Manager, Manager>::manager() \"\n      \"must return Manager&\");\n\n protected:\n  DependencyManager(const DependencyManager& that) = default;\n  DependencyManager& operator=(const DependencyManager& that) = default;\n\n  DependencyManager(DependencyManager&& that) = default;\n  DependencyManager& operator=(DependencyManager&& that) = default;\n\n  ~DependencyManager() = default;\n};\n\n// Specialization of `DependencyManager<Manager>` when\n// `DependencyManagerImpl<Manager>` is not defined: an owned dependency stored\n// by value.\ntemplate <typename Manager>\nclass DependencyManager<\n    Manager, std::enable_if_t<std::conjunction_v<\n                 std::negation<std::is_reference<Manager>>,\n                 std::negation<dependency_manager_internal::\n                                   SupportsDependencyManagerImpl<Manager>>>>>\n    : public DependencyBase<Manager> {\n public:\n  using DependencyManager::DependencyBase::DependencyBase;\n\n  static constexpr bool kIsOwning = true;\n\n protected:\n  DependencyManager(const DependencyManager& that) = default;\n  DependencyManager& operator=(const DependencyManager& that) = default;\n\n  DependencyManager(DependencyManager&& that) = default;\n  DependencyManager& operator=(DependencyManager&& that) = default;\n\n  ~DependencyManager() = default;\n\n  Manager* ptr() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return &this->mutable_manager();\n  }\n};\n\n// Specialization of `DependencyManager<Manager&>` when\n// `DependencyManagerImpl<absl::remove_cvref_t<Manager>>` is defined:\n// delegate to it, but store `absl::remove_cvref_t<Manager>` by reference\n// to avoid moving it.\n//\n// This handles cases where `Manager` is deduced from a function parameter\n// as a reference type, but the type under the reference determines the\n// interpretation, e.g. `T*&`.\ntemplate <typename Manager>\nclass DependencyManager<\n    Manager&,\n    std::enable_if_t<dependency_manager_internal::SupportsDependencyManagerImpl<\n        absl::remove_cvref_t<Manager>>::value>>\n    : public DependencyManagerImpl<absl::remove_cvref_t<Manager>,\n                                   absl::remove_cvref_t<Manager>&> {\n public:\n  using DependencyManager::DependencyManagerImpl::DependencyManagerImpl;\n\n  static_assert(\n      std::is_convertible_v<\n          decltype(std::declval<\n                       typename DependencyManager::DependencyManagerImpl&>()\n                       .manager()),\n          Manager&>,\n      \"DependencyManagerImpl<Manager, Manager&>::manager() \"\n      \"must return Manager&\");\n\n protected:\n  DependencyManager(const DependencyManager& that) = default;\n  DependencyManager& operator=(const DependencyManager& that) = default;\n\n  DependencyManager(DependencyManager&& that) = default;\n  DependencyManager& operator=(DependencyManager&& that) = default;\n\n  ~DependencyManager() = default;\n};\n\n// Specialization of `DependencyManager<Manager&>` when\n// `DependencyManagerImpl<absl::remove_cvref_t<Manager>>` is not defined:\n// an unowned dependency stored by lvalue reference.\ntemplate <typename Manager>\nclass DependencyManager<\n    Manager&, std::enable_if_t<\n                  !dependency_manager_internal::SupportsDependencyManagerImpl<\n                      absl::remove_cvref_t<Manager>>::value>>\n    : public DependencyBase<Manager&> {\n public:\n  using DependencyManager::DependencyBase::DependencyBase;\n\n  static constexpr bool kIsOwning = false;\n\n protected:\n  DependencyManager(const DependencyManager& that) = default;\n  DependencyManager& operator=(const DependencyManager&) = delete;\n\n  ~DependencyManager() = default;\n\n  Manager* ptr() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return &this->manager();\n  }\n};\n\n// Specialization of `DependencyManager<Manager&&>` when\n// `DependencyManagerImpl<absl::remove_cvref_t<Manager>>` is defined:\n// delegate to it, but store `absl::remove_cvref_t<Manager>` by reference\n// to avoid moving it.\n//\n// This handles cases where `Manager` is deduced from a function parameter\n// as a reference type, but the type under the reference determines the\n// interpretation, e.g. `std::unique_ptr<T>&&`.\ntemplate <typename Manager>\nclass DependencyManager<\n    Manager&&,\n    std::enable_if_t<dependency_manager_internal::SupportsDependencyManagerImpl<\n        absl::remove_cvref_t<Manager>>::value>>\n    : public DependencyManagerImpl<absl::remove_cvref_t<Manager>,\n                                   absl::remove_cvref_t<Manager>&&> {\n public:\n  using DependencyManager::DependencyManagerImpl::DependencyManagerImpl;\n\n  static_assert(\n      std::is_convertible_v<\n          decltype(std::declval<\n                       typename DependencyManager::DependencyManagerImpl&>()\n                       .manager()),\n          Manager&>,\n      \"DependencyManagerImpl<Manager, Manager&&>::manager() \"\n      \"must return Manager&\");\n\n protected:\n  DependencyManager(const DependencyManager& that) = default;\n  DependencyManager& operator=(const DependencyManager& that) = default;\n\n  DependencyManager(DependencyManager&& that) = default;\n  DependencyManager& operator=(DependencyManager&& that) = default;\n\n  ~DependencyManager() = default;\n};\n\n// Specialization of `DependencyManager<Manager&&>` when\n// `DependencyManagerImpl<absl::remove_cvref_t<Manager>>` is not defined: an\n// owned dependency stored by rvalue reference.\ntemplate <typename Manager>\nclass DependencyManager<\n    Manager&&, std::enable_if_t<\n                   !dependency_manager_internal::SupportsDependencyManagerImpl<\n                       absl::remove_cvref_t<Manager>>::value>>\n    : public DependencyBase<Manager&&> {\n public:\n  using DependencyManager::DependencyBase::DependencyBase;\n\n  static constexpr bool kIsOwning = true;\n\n protected:\n  DependencyManager(DependencyManager&& that) = default;\n  DependencyManager& operator=(DependencyManager&&) = delete;\n\n  ~DependencyManager() = default;\n\n  Manager* ptr() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return &this->manager();\n  }\n};\n\nnamespace dependency_manager_internal {\n\n// Expose protected `DependencyManager::ptr()` for `DependencyManagerPtr`.\ntemplate <typename Manager>\nstruct DependencyManagerAccess : DependencyManager<Manager> {\n  using DependencyManagerAccess::DependencyManager::ptr;\n};\n\n// `DependencyManagerPtrImpl<Manager>::type` is the type returned by\n// `DependencyManager<Manager>::ptr()`.\ntemplate <typename Manager, typename Enable = void>\nstruct DependencyManagerPtrImpl {\n  using type =\n      decltype(std::declval<const DependencyManagerAccess<Manager>&>().ptr());\n};\n\n// In `DependencyManagerPtrImpl<Manager>` for `Manager` stored by value, avoid\n// instantiating `DependencyManager<Manager>` just to see what its `ptr()` would\n// return. This could lead to subtle compile errors, causing the following chain\n// of template instantiations:\n//\n//  * `TargetRefSupportsDependency<X*, Abstract&>`\n//  * `SupportsDependencyInit<X*, Abstract&, Abstract&>`\n//  * `SupportsDependencyDeref<X*, Abstract&>`\n//  * `SupportsDependencyDefault<X*, Abstract>`\n//  * `DependencyManagerPtr<Abstract>`\n//  * `DependencyManager<Abstract>`\n//  * `DependencyBase<Abstract>`\n//\n// which contains a member variable of an abstract type.\ntemplate <typename Manager>\nstruct DependencyManagerPtrImpl<\n    Manager, std::enable_if_t<std::conjunction_v<\n                 std::negation<std::is_reference<Manager>>,\n                 std::negation<SupportsDependencyManagerImpl<Manager>>>>> {\n  using type = Manager*;\n};\n\ntemplate <typename Manager>\nstruct DependencyManagerRefImpl {\n  using type = Manager;\n};\n\ntemplate <typename Manager>\nstruct DependencyManagerRefImpl<Manager*> {\n  using type = Manager&;\n};\n\n}  // namespace dependency_manager_internal\n\n// `DependencyManagerPtr<Manager>` is the type returned by\n// `DependencyManager<Manager>::ptr()`.\ntemplate <typename Manager>\nusing DependencyManagerPtr =\n    typename dependency_manager_internal::DependencyManagerPtrImpl<\n        Manager>::type;\n\n// `DependencyManagerRef<Manager>` is `DependencyManagerPtr<Manager>` with the\n// toplevel pointer changed to lvalue reference, if any.\n//\n// This should normally be used under the condition that\n// `std::is_pointer_v<DependencyManagerPtr<Manager>>`.\ntemplate <typename Manager>\nusing DependencyManagerRef =\n    typename dependency_manager_internal::DependencyManagerRefImpl<\n        DependencyManagerPtr<Manager>>::type;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_DEPENDENCY_MANAGER_H_\n"
  },
  {
    "path": "riegeli/base/errno_mapping.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifdef _WIN32\n#define WIN32_LEAN_AND_MEAN\n#endif\n\n#include \"riegeli/base/errno_mapping.h\"\n\n#ifdef _WIN32\n#include <stdint.h>\n#include <windows.h>\n#endif\n\n#include <cerrno>\n\n#include \"absl/status/status.h\"\n#ifdef _WIN32\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/unicode.h\"\n#endif\n\nnamespace riegeli {\n\n#ifdef _WIN32\n\nnamespace {\n\nabsl::StatusCode WindowsErrorToStatusCode(DWORD error_number) {\n  switch (error_number) {\n    case ERROR_SUCCESS:\n      return absl::StatusCode::kOk;\n    case ERROR_OPERATION_ABORTED:\n      return absl::StatusCode::kCancelled;\n    case ERROR_INVALID_HANDLE:\n    case ERROR_INVALID_PARAMETER:\n    case ERROR_BUFFER_OVERFLOW:\n    case ERROR_INVALID_NAME:\n    case ERROR_NEGATIVE_SEEK:\n    case ERROR_DIRECTORY:\n    case ERROR_REPARSE_TAG_INVALID:\n    case WSAEFAULT:\n    case WSAEINVAL:\n    case WSAENAMETOOLONG:\n      return absl::StatusCode::kInvalidArgument;\n    case ERROR_FILE_NOT_FOUND:\n    case ERROR_PATH_NOT_FOUND:\n    case ERROR_INVALID_DRIVE:\n    case ERROR_BAD_UNIT:\n    case ERROR_BAD_NETPATH:\n    case ERROR_DEV_NOT_EXIST:\n    case ERROR_BAD_PATHNAME:\n      return absl::StatusCode::kNotFound;\n    case ERROR_FILE_EXISTS:\n    case ERROR_ALREADY_EXISTS:\n      return absl::StatusCode::kAlreadyExists;\n    case ERROR_ACCESS_DENIED:\n    case ERROR_INVALID_ACCESS:\n    case ERROR_CURRENT_DIRECTORY:\n    case ERROR_WRITE_PROTECT:\n    case ERROR_SHARING_VIOLATION:\n    case ERROR_CANNOT_MAKE:\n    case ERROR_NOACCESS:\n    case WSAEACCES:\n      return absl::StatusCode::kPermissionDenied;\n    case ERROR_TOO_MANY_OPEN_FILES:\n    case ERROR_NOT_ENOUGH_MEMORY:\n    case ERROR_OUTOFMEMORY:\n    case ERROR_HANDLE_DISK_FULL:\n    case ERROR_DISK_FULL:\n    case WSAEMFILE:\n      return absl::StatusCode::kResourceExhausted;\n    case ERROR_BROKEN_PIPE:\n    case ERROR_BUSY_DRIVE:\n    case ERROR_DIR_NOT_EMPTY:\n    case ERROR_BUSY:\n    case ERROR_OPEN_FILES:\n    case ERROR_DEVICE_IN_USE:\n    case WSAEBADF:\n      return absl::StatusCode::kFailedPrecondition;\n    case ERROR_HANDLE_EOF:\n      return absl::StatusCode::kOutOfRange;\n    case ERROR_INVALID_FUNCTION:\n    case ERROR_NOT_SUPPORTED:\n      return absl::StatusCode::kUnimplemented;\n    case ERROR_NOT_READY:\n    case ERROR_LOCK_VIOLATION:\n    case ERROR_LOCKED:\n    case ERROR_RETRY:\n    case WSAEINTR:\n      return absl::StatusCode::kUnavailable;\n    default:\n      return absl::StatusCode::kUnknown;\n  }\n}\n\n}  // namespace\n\n#endif  // _WIN32\n\nint StatusCodeToErrno(absl::StatusCode status_code) {\n  switch (status_code) {\n    case absl::StatusCode::kOk:\n      return 0;\n    case absl::StatusCode::kCancelled:\n      return ECANCELED;\n    case absl::StatusCode::kUnknown:\n      return EIO;\n    case absl::StatusCode::kInvalidArgument:\n      return EINVAL;\n    case absl::StatusCode::kDeadlineExceeded:\n      return ETIMEDOUT;\n    case absl::StatusCode::kNotFound:\n      return ENOENT;\n    case absl::StatusCode::kAlreadyExists:\n      return EEXIST;\n    case absl::StatusCode::kPermissionDenied:\n      return EACCES;\n    case absl::StatusCode::kResourceExhausted:\n      return ENOSPC;\n    case absl::StatusCode::kFailedPrecondition:\n      // Does not round trip:\n      // `absl::ErrnoToStatusCode(EINVAL) == absl::StatusCode::kInvalidArgument`\n      return EINVAL;\n    case absl::StatusCode::kAborted:\n      return EDEADLK;\n    case absl::StatusCode::kOutOfRange:\n      return ERANGE;\n    case absl::StatusCode::kUnimplemented:\n      return ENOTSUP;\n    case absl::StatusCode::kInternal:\n      // Does not round trip:\n      // `absl::ErrnoToStatusCode(EIO) == absl::StatusCode::kUnknown`\n      return EIO;\n    case absl::StatusCode::kUnavailable:\n      return EAGAIN;\n    case absl::StatusCode::kDataLoss:\n      // Does not round trip:\n      // `absl::ErrnoToStatusCode(EIO) == absl::StatusCode::kUnknown`\n      return EIO;\n    case absl::StatusCode::kUnauthenticated:\n      // Does not round trip:\n      // `absl::ErrnoToStatusCode(EACCES) ==\n      //      absl::StatusCode::kPermissionDenied`\n      return EACCES;\n    default:\n      // Does not round trip:\n      // `absl::ErrnoToStatusCode(EIO) == absl::StatusCode::kUnknown`\n      return EIO;\n  }\n}\n\n#ifdef _WIN32\nabsl::Status WindowsErrorToStatus(uint32_t error_number,\n                                  absl::string_view message) {\n  LPWSTR os_message;\n  const DWORD length = FormatMessageW(\n      FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM |\n          FORMAT_MESSAGE_IGNORE_INSERTS,\n      nullptr, IntCast<DWORD>(error_number), 0,\n      reinterpret_cast<LPWSTR>(&os_message), 0, nullptr);\n  const absl::Status status(\n      WindowsErrorToStatusCode(IntCast<DWORD>(error_number)),\n      absl::StrCat(message, \": \",\n                   WideToUtf8Lossy(absl::MakeConstSpan(\n                       os_message, IntCast<size_t>(length)))));\n  LocalFree(os_message);\n  return status;\n}\n#endif  // _WIN32\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/errno_mapping.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_ERRNO_MAPPING_H_\n#define RIEGELI_BASE_ERRNO_MAPPING_H_\n\n#ifdef _WIN32\n#include <stdint.h>\n#endif\n\n#include \"absl/status/status.h\"\n#ifdef _WIN32\n#include \"absl/strings/string_view.h\"\n#endif\n\nnamespace riegeli {\n\n// Converts `absl::StatusCode` to `errno` value.\nint StatusCodeToErrno(absl::StatusCode status_code);\n\n#ifdef _WIN32\nabsl::Status WindowsErrorToStatus(uint32_t error_number,\n                                  absl::string_view message);\n#endif  // _WIN32\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_ERRNO_MAPPING_H_\n"
  },
  {
    "path": "riegeli/base/estimated_allocated_size.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_ESTIMATED_ALLOCATED_SIZE_H_\n#define RIEGELI_BASE_ESTIMATED_ALLOCATED_SIZE_H_\n\n#include <stddef.h>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"riegeli/base/arithmetic.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// Returns the estimated size which will be allocated when requesting to\n// allocate `requested_size`.\ninline size_t EstimatedAllocatedSize(size_t requested_size) {\n  // Placeholder for asking the memory manager, which might be possible on some\n  // platforms.\n  return RoundUp<2 * sizeof(void*)>(\n      UnsignedMax(requested_size, 4 * sizeof(void*)));\n}\n\n// Returns the estimated size which was allocated at `ptr` when requested to\n// allocate `requested_size`.\ninline size_t EstimatedAllocatedSize(ABSL_ATTRIBUTE_UNUSED const void* ptr,\n                                     size_t requested_size) {\n  // Placeholder for using `ptr`, which might be possible on some platforms.\n  return EstimatedAllocatedSize(requested_size);\n}\n\n// A deterministic variant of `EstimatedAllocatedSize()`, useful for testing.\ninline size_t EstimatedAllocatedSizeForTesting(size_t requested_size) {\n  return 16 + requested_size;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_ESTIMATED_ALLOCATED_SIZE_H_\n"
  },
  {
    "path": "riegeli/base/external_data.cc",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/external_data.h\"\n\n#include <cstring>\n\n#include \"absl/strings/string_view.h\"\n\nnamespace riegeli {\n\nExternalData ExternalDataCopy(absl::string_view data) {\n  char* storage = nullptr;\n  if (!data.empty()) {\n    storage = static_cast<char*>(operator new(data.size()));\n    std::memcpy(storage, data.data(), data.size());\n  }\n  return ExternalData{ExternalStorage(storage, operator delete),\n                      absl::string_view(storage, data.size())};\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/external_data.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_EXTERNAL_DATA_H_\n#define RIEGELI_BASE_EXTERNAL_DATA_H_\n\n#include <memory>\n#include <utility>\n\n#include \"absl/strings/string_view.h\"\n\nnamespace riegeli {\n\n// Type-erased external object with its deleter.\n//\n// `ExternalStorage` can be decomposed with `void* ExternalStorage::release()`\n// and `ExternalStorage::get_deleter() -> void (*)(void*)`.\nusing ExternalStorage = std::unique_ptr<void, void (*)(void*)>;\n\n// Supports `ExternalRef`.\ninline ExternalStorage RiegeliToExternalStorage(ExternalStorage* self) {\n  return std::move(*self);\n}\n\n// Type-erased external object with its deleter and a substring of a byte array\n// it owns.\nstruct ExternalData {\n  /*implicit*/ operator absl::string_view() const { return substr; }\n\n  // Indicates support for:\n  //  * `ExternalRef(ExternalData&&)`\n  //  * `ExternalRef(ExternalData&&, substr)`\n  friend void RiegeliSupportsExternalRef(ExternalData*) {}\n\n  // Supports `ExternalRef`.\n  friend ExternalStorage RiegeliToExternalStorage(ExternalData* self) {\n    return std::move(self->storage);\n  }\n\n  ExternalStorage storage;  // Must outlive usages of `substr`.\n  absl::string_view substr;\n};\n\n// Creates `ExternalData` holding a copy of `data`.\nExternalData ExternalDataCopy(absl::string_view data);\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_EXTERNAL_DATA_H_\n"
  },
  {
    "path": "riegeli/base/external_ref.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_EXTERNAL_REF_H_\n#define RIEGELI_BASE_EXTERNAL_REF_H_\n\n#include \"riegeli/base/chain_base.h\"            // IWYU pragma: keep\n#include \"riegeli/base/chain_details.h\"         // IWYU pragma: keep\n#include \"riegeli/base/external_ref_base.h\"     // IWYU pragma: export\n#include \"riegeli/base/external_ref_support.h\"  // IWYU pragma: export\n\n#endif  // RIEGELI_BASE_EXTERNAL_REF_H_\n"
  },
  {
    "path": "riegeli/base/external_ref_base.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_EXTERNAL_REF_BASE_H_\n#define RIEGELI_BASE_EXTERNAL_REF_BASE_H_\n\n// IWYU pragma: private, include \"riegeli/base/external_ref.h\"\n\n#include <stddef.h>\n\n#include <functional>\n#include <memory>\n#include <new>  // IWYU pragma: keep\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/meta/type_traits.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain_base.h\"\n#include \"riegeli/base/cord_utils.h\"\n#include \"riegeli/base/external_data.h\"\n#include \"riegeli/base/external_ref_support.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/temporary_storage.h\"\n\nnamespace riegeli {\n\n// `ExternalRef` specifies a byte array in a way which allows sharing it with\n// other objects without copying if that is considered more efficient than\n// copying. It mediates between the producer and the consumer of the data during\n// transfer; it is not suitable for longer storage. Creating an `ExternalRef` is\n// usually more efficient than creating a `Chain` or `absl::Cord` if the data\n// will ultimately be copied rather than shared.\n//\n// `ExternalRef` is constructed from an object of some type `T` which owns the\n// data, or from `Initializer<T>`. The latter allows to skip constructing the\n// object if the data are known beforehand, will ultimately be copied, and the\n// constructed object is not needed otherwise.\n//\n// `ExternalRef` can be converted to `absl::string_view`, `Chain`, `absl::Cord`,\n// or `ExternalData`, or assigned, appended, or prepended to a `Chain` or\n// `absl::Cord`. Apart from conversion to `absl::string_view` it can be consumed\n// at most once.\n//\n// In contrast to `Chain::Block` and `absl::MakeCordFromExternal()`,\n// `ExternalRef` chooses between sharing the object and copying the data,\n// depending on the size of the data, the method of consuming the data,\n// and the state of the destination for appending or prepending.\n//\n// `ExternalRef` itself does not own the object description nor the data, and is\n// efficiently movable. The state is stored in a storage object passed as a\n// default argument to the original `ExternalRef` constructor.\n//\n// The expected interface of the object which owns the data is a superset of the\n// interfaces expected by `Chain::Block` and `absl::MakeCordFromExternal()`.\n//\n// `ExternalRef` constructors require the external object type to indicate\n// that it supports `ExternalRef` by providing one of the following functions\n// (only their presence is checked, they are never called):\n// ```\n//   // Indicates support for `ExternalRef(T&&, substr)`.\n//   //\n//   // `substr` must be owned by the object if it gets created or moved, unless\n//   // `RiegeliExternalCopy()` (see below) recognizes cases when it is not.\n//   //\n//   // If `T` is convertible to `BytesRef`, then also indicates support for\n//   // `ExternalRef(T&&)`.\n//   //\n//   // The parameter can also have type `const T*`. This also indicates\n//   // support for `ExternalRef(const T&, substr)` and possibly\n//   // `ExternalRef(const T&)`, i.e. that `T` is copyable and copying it is\n//   // more efficient than copying the data.\n//   //\n//   // If the `ExternalRef` is later converted to `absl::Cord` and\n//   // `absl::MakeCordFromExternal()` gets used, then this avoids an allocation\n//   // by taking advantage of the promise that `substr` will be owned also by\n//   // the moved object (`absl::MakeCordFromExternal()` requires knowing the\n//   // data before specifying the object to be moved).\n//   friend void RiegeliSupportsExternalRef(T*) {}\n//\n//   // Indicates support for `ExternalRef(T&&)`, as long as `T` is convertible\n//   // to `BytesRef`.\n//   //\n//   // The parameter can also have type `const T*`. This also indicates support\n//   // for `ExternalRef(const T&)`, i.e. that `T` is copyable and copying it is\n//   // more efficient than copying the data.\n//   friend void RiegeliSupportsExternalRefWhole(T*) {}\n// ```\n//\n// `ExternalRef::From()` are like `ExternalRef` constructors, but\n// `RiegeliSupportsExternalRef()` or `RiegeliSupportsExternalRefWhole()` is not\n// needed. The caller is responsible for using an appropriate type of the\n// external object.\n//\n// `T` may also support the following member functions, either with or without\n// the `substr` parameter, with the following definitions assumed by default:\n// ```\n//   // Called once before the destructor, except on a moved-from object.\n//   // If only this function is needed, `T` can be a lambda.\n//   //\n//   // If this is present, the object will be created unconditionally, because\n//   // calling this might be needed to delete resources which already exist.\n//   //\n//   // This can also be a const method. If this is not a const method and the\n//   // object is passed by const or lvalue reference, this will be called on a\n//   // mutable copy of the object.\n//   void operator()(absl::string_view substr) && {}\n//\n//   // If this returns `true`, the data will be copied instead of wrapping the\n//   // object. The data does not need to be stable while the object is moved.\n//   // `RiegeliToChainBlock()`, `RiegeliToCord()`, `RiegeliToExternalData()`,\n//   // `RiegeliToExternalStorage()`, nor `RiegeliExternalDelegate()` will not\n//   // be called.\n//   //\n//   // Typically this indicates an object with short data stored inline.\n//   friend bool RiegeliExternalCopy(const T* self) { return false; }\n//\n//   // Converts `*self` or its `substr` to `Chain::Block`, if this can be done\n//   // more efficiently than with `Chain::Block` constructor. Can modify\n//   // `*self`. `operator()` will no longer be called.\n//   //\n//   // The `self` parameter can also have type `const T*`. If it has type `T*`\n//   // and the object is passed by const or lvalue reference, this will be\n//   // called on a mutable copy of the object.\n//   //\n//   // If the `substr` parameter was given to `ExternalRef` constructor, the\n//   // `substr` parameter is required here, otherwise it is optional.\n//   friend Chain::Block RiegeliToChainBlock(T* self, absl::string_view substr);\n//\n//   // Converts `*self` or its `substr` to `absl::Cord`, if this can be done\n//   // more efficiently than with `absl::MakeCordFromExternal()`. Can modify\n//   // `*self`. `operator()` will no longer be called.\n//   //\n//   // The `self` parameter can also have type `const T*`. If it has type `T*`\n//   // and the object is passed by const or lvalue reference, this will be\n//   // called on a mutable copy of the object.\n//   //\n//   // If the `substr` parameter was given to `ExternalRef` constructor, the\n//   // `substr` parameter is required here, otherwise it is optional.\n//   friend absl::Cord RiegeliToCord(T* self, absl::string_view substr);\n//\n//   // Converts `*self` to `ExternalData`, if this can be done more efficiently\n//   // than allocating the object on the heap, e.g. if the object fits in a\n//   // pointer. Can modify `*self`. `operator()` will no longer be called.\n//   //\n//   // The `self` parameter can also have type `const T*`. If it has type `T*`\n//   // and the object is passed by const or lvalue reference, this will be\n//   // called on a mutable copy of the object.\n//   //\n//   // If the `substr` parameter was given to `ExternalRef` constructor, the\n//   // `substr` parameter is required here, otherwise it is optional.\n//   friend ExternalData RiegeliToExternalData(T* self,\n//                                             absl::string_view substr);\n//\n//   // This can be defined instead of `RiegeliToExternalData()` with `substr`,\n//   // which would return `ExternalData` with the same `substr`.\n//   friend ExternalStorage RiegeliToExternalStorage(T* self);\n//\n//   // If defined, indicates a subobject to wrap instead of the whole object.\n//   // It must call `std::forward<Callback>(delegate_to)(subobject)` or\n//   // `std::forward<Callback>(delegate_to)(subobject, substr)`, preferably\n//   // with `std::move(subobject)`. `delegate_to` must be called exactly once.\n//   //\n//   // Typically this indicates a smaller object which is sufficient to keep\n//   // the data alive, or the active variant if the object stores one of\n//   // multiple subobjects.\n//   //\n//   // `RiegeliToChainBlock()`, `RiegeliToCord()`, `RiegeliToExternalData()`,\n//   // and `RiegeliToExternalStorage()`, if defined, are used in preference to\n//   // this.\n//   //\n//   // The subobject will be processed like by `ExternalRef::From()`, including\n//   // the possibility of further delegation, except that `Initializer` is not\n//   // supported. The subobject must not be `*self`.\n//   //\n//   // The `self` parameter can also have type `const T*`. If it has type `T*`\n//   // and the object is passed by const or lvalue reference, this will be\n//   // called on a mutable copy of the object.\n//   //\n//   // The `substr` parameter is optional here. If absent here while the\n//   // `substr` parameter was given to `ExternalRef` constructor, then it is\n//   // propagated. If absent here while the `substr` parameter was not given\n//   // to `ExternalRef` constructor, then `subobject` must be convertible to\n//   // `BytesRef`. If present here, it can be used to specify the data if\n//   // `subobject` is not convertible to `BytesRef`.\n//   template <typename Callback>\n//   friend void RiegeliExternalDelegate(T* self, absl::string_view substr,\n//                                       Callback&& delegate_to);\n//\n//   // Shows internal structure in a human-readable way, for debugging.\n//   //\n//   // Used for conversion to `Chain`.\n//   friend void RiegeliDumpStructure(const T* self, absl::string_view substr,\n//                                    std::ostream& dest) {\n//     dest << \"[external] { }\";\n//   }\n//\n//   // Registers this object with `MemoryEstimator`.\n//   //\n//   // By default calls `memory_estimator.RegisterUnknownType<T>()` and\n//   // as an approximation of memory usage of an unknown type, registers just\n//   // the stored `substr` if unique.\n//   //\n//   // Used for conversion to `Chain`.\n//   friend void RiegeliRegisterSubobjects(\n//       const T* self, riegeli::MemoryEstimator& memory_estimator);\n// ```\n//\n// The `substr` parameter of these member functions, if present, will get the\n// `substr` parameter passed to `ExternalRef` constructor. Having `substr`\n// available in these functions might avoid storing `substr` in the external\n// object.\nclass ExternalRef {\n private:\n  using UseStringViewFunction = void (*)(void* context, absl::string_view data);\n  using UseChainBlockFunction = void (*)(void* context, Chain::Block data);\n  using UseCordFunction = void (*)(void* context, absl::Cord data);\n  using UseExternalDataFunction = void (*)(void* context, ExternalData data);\n\n  template <typename T, typename Enable = void>\n  struct HasCallOperatorSubstr : std::false_type {};\n\n  template <typename T>\n  struct HasCallOperatorSubstr<T, std::void_t<decltype(std::declval<T&&>()(\n                                      std::declval<absl::string_view>()))>>\n      : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct HasCallOperatorWhole : std::false_type {};\n\n  template <typename T>\n  struct HasCallOperatorWhole<T, std::void_t<decltype(std::declval<T&&>()())>>\n      : std::true_type {};\n\n  template <typename T>\n  struct HasCallOperator\n      : std::disjunction<HasCallOperatorSubstr<T>, HasCallOperatorWhole<T>> {};\n\n  template <typename T,\n            std::enable_if_t<HasCallOperatorSubstr<T>::value, int> = 0>\n  static void CallOperatorWhole(T&& object) {\n    const absl::string_view data = BytesRef(object);\n    std::forward<T>(object)(data);\n  }\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<std::negation<HasCallOperatorSubstr<T>>,\n                                   HasCallOperatorWhole<T>>,\n                int> = 0>\n  static void CallOperatorWhole(T&& object) {\n    std::forward<T>(object)();\n  }\n  template <\n      typename T,\n      std::enable_if_t<\n          std::conjunction_v<std::negation<HasCallOperator<T>>,\n                             HasCallOperatorSubstr<absl::remove_cvref_t<T>>>,\n          int> = 0>\n  static void CallOperatorWhole(T&& object) {\n    absl::remove_cvref_t<T> copy(object);\n    const absl::string_view data = BytesRef(copy);\n    std::move(copy)(data);\n  }\n  template <\n      typename T,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::negation<HasCallOperator<T>>,\n              std::negation<HasCallOperatorSubstr<absl::remove_cvref_t<T>>>,\n              HasCallOperatorWhole<absl::remove_cvref_t<T>>>,\n          int> = 0>\n  static void CallOperatorWhole(T&& object) {\n    (absl::remove_cvref_t<T>(object))();\n  }\n  template <typename T,\n            std::enable_if_t<!HasCallOperator<absl::remove_cvref_t<T>>::value,\n                             int> = 0>\n  static void CallOperatorWhole(ABSL_ATTRIBUTE_UNUSED T&& object) {}\n\n  template <typename T,\n            std::enable_if_t<HasCallOperatorSubstr<T>::value, int> = 0>\n  static void CallOperatorSubstr(T&& object, absl::string_view substr) {\n    std::forward<T>(object)(substr);\n  }\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<std::negation<HasCallOperatorSubstr<T>>,\n                                   HasCallOperatorWhole<T>>,\n                int> = 0>\n  static void CallOperatorSubstr(\n      T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view substr) {\n    std::forward<T>(object)();\n  }\n  template <\n      typename T,\n      std::enable_if_t<\n          std::conjunction_v<std::negation<HasCallOperator<T>>,\n                             HasCallOperatorSubstr<absl::remove_cvref_t<T>>>,\n          int> = 0>\n  static void CallOperatorSubstr(\n      T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view substr) {\n    absl::remove_cvref_t<T> copy(object);\n    const absl::string_view data = BytesRef(copy);\n    std::move(copy)(data);\n  }\n  template <\n      typename T,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::negation<HasCallOperator<T>>,\n              std::negation<HasCallOperatorSubstr<absl::remove_cvref_t<T>>>,\n              HasCallOperatorWhole<absl::remove_cvref_t<T>>>,\n          int> = 0>\n  static void CallOperatorSubstr(\n      T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view substr) {\n    (absl::remove_cvref_t<T>(object))();\n  }\n  template <typename T,\n            std::enable_if_t<!HasCallOperator<absl::remove_cvref_t<T>>::value,\n                             int> = 0>\n  static void CallOperatorSubstr(\n      ABSL_ATTRIBUTE_UNUSED T&& object,\n      ABSL_ATTRIBUTE_UNUSED absl::string_view substr) {}\n\n  template <typename T>\n  static external_ref_internal::PointerTypeT<T> Pointer(T&& object) {\n    return &object;\n  }\n\n#if RIEGELI_DEBUG\n  template <typename T, std::enable_if_t<\n                            std::is_convertible_v<const T&, BytesRef>, int> = 0>\n  static void AssertSubstr(const T& object, absl::string_view substr) {\n    if (!substr.empty()) {\n      const BytesRef whole = object;\n      RIEGELI_ASSERT(std::greater_equal<>()(substr.data(), whole.data()))\n          << \"Failed precondition of ExternalRef: \"\n             \"substring not contained in whole data\";\n      RIEGELI_ASSERT(std::less_equal<>()(substr.data() + substr.size(),\n                                         whole.data() + whole.size()))\n          << \"Failed precondition of ExternalRef: \"\n             \"substring not contained in whole data\";\n    }\n  }\n  template <\n      typename T,\n      std::enable_if_t<!std::is_convertible_v<const T&, BytesRef>, int> = 0>\n#else\n  template <typename T>\n#endif\n  static void AssertSubstr(ABSL_ATTRIBUTE_UNUSED const T& object,\n                           ABSL_ATTRIBUTE_UNUSED absl::string_view substr) {\n  }\n\n  template <typename T, typename Callback, typename Enable = void>\n  struct HasRiegeliExternalDelegateWhole : std::false_type {};\n\n  template <typename T, typename Callback>\n  struct HasRiegeliExternalDelegateWhole<\n      T, Callback,\n      std::void_t<decltype(RiegeliExternalDelegate(\n          std::declval<external_ref_internal::PointerTypeT<T>>(),\n          std::declval<Callback>()))>> : std::true_type {};\n\n  template <typename T, typename Callback, typename Enable = void>\n  struct HasRiegeliExternalDelegateSubstr : std::false_type {};\n\n  template <typename T, typename Callback>\n  struct HasRiegeliExternalDelegateSubstr<\n      T, Callback,\n      std::void_t<decltype(RiegeliExternalDelegate(\n          std::declval<external_ref_internal::PointerTypeT<T>>(),\n          std::declval<absl::string_view>(), std::declval<Callback>()))>>\n      : std::true_type {};\n\n  template <typename T, typename Callback>\n  struct HasExternalDelegateWhole\n      : std::disjunction<HasRiegeliExternalDelegateWhole<T, Callback>,\n                         HasRiegeliExternalDelegateSubstr<T, Callback>> {};\n\n  template <typename T, typename Callback,\n            std::enable_if_t<\n                HasRiegeliExternalDelegateWhole<T, Callback>::value, int> = 0>\n  static void ExternalDelegateWhole(T&& object, Callback&& delegate_to) {\n    RiegeliExternalDelegate(ExternalRef::Pointer(std::forward<T>(object)),\n                            std::forward<Callback>(delegate_to));\n  }\n  template <typename T, typename Callback,\n            std::enable_if_t<\n                std::conjunction_v<\n                    std::negation<HasRiegeliExternalDelegateWhole<T, Callback>>,\n                    HasRiegeliExternalDelegateSubstr<T, Callback>>,\n                int> = 0>\n  static void ExternalDelegateWhole(T&& object, Callback&& delegate_to) {\n    const absl::string_view data = BytesRef(object);\n    RiegeliExternalDelegate(ExternalRef::Pointer(std::forward<T>(object)), data,\n                            std::forward<Callback>(delegate_to));\n  }\n\n  template <typename T, typename Callback,\n            std::enable_if_t<\n                HasRiegeliExternalDelegateWhole<T, Callback>::value, int> = 0>\n  static void ExternalDelegateWhole(\n      T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view data,\n      Callback&& delegate_to) {\n    RiegeliExternalDelegate(ExternalRef::Pointer(std::forward<T>(object)),\n                            std::forward<Callback>(delegate_to));\n  }\n  template <typename T, typename Callback,\n            std::enable_if_t<\n                std::conjunction_v<\n                    std::negation<HasRiegeliExternalDelegateWhole<T, Callback>>,\n                    HasRiegeliExternalDelegateSubstr<T, Callback>>,\n                int> = 0>\n  static void ExternalDelegateWhole(T&& object, absl::string_view data,\n                                    Callback&& delegate_to) {\n    RiegeliExternalDelegate(ExternalRef::Pointer(std::forward<T>(object)), data,\n                            std::forward<Callback>(delegate_to));\n  }\n\n  template <typename T, typename Callback>\n  struct HasExternalDelegateSubstr\n      : std::disjunction<HasRiegeliExternalDelegateSubstr<T, Callback>,\n                         HasRiegeliExternalDelegateWhole<T, Callback>> {};\n\n  template <typename T, typename Callback,\n            std::enable_if_t<\n                HasRiegeliExternalDelegateSubstr<T, Callback>::value, int> = 0>\n  static void ExternalDelegateSubstr(T&& object, absl::string_view substr,\n                                     Callback&& delegate_to) {\n    RiegeliExternalDelegate(ExternalRef::Pointer(std::forward<T>(object)),\n                            substr, std::forward<Callback>(delegate_to));\n  }\n  template <\n      typename T, typename Callback,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::negation<HasRiegeliExternalDelegateSubstr<T, Callback>>,\n              HasRiegeliExternalDelegateWhole<T, Callback>>,\n          int> = 0>\n  static void ExternalDelegateSubstr(\n      T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view substr,\n      Callback&& delegate_to) {\n    RiegeliExternalDelegate(ExternalRef::Pointer(std::forward<T>(object)),\n                            std::forward<Callback>(delegate_to));\n  }\n\n  template <typename T, typename Enable = void>\n  struct HasRiegeliToChainBlockWhole : std::false_type {};\n\n  template <typename T>\n  struct HasRiegeliToChainBlockWhole<\n      T, std::enable_if_t<std::is_convertible_v<\n             decltype(RiegeliToChainBlock(\n                 std::declval<external_ref_internal::PointerTypeT<T>>())),\n             Chain::Block>>> : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct HasRiegeliToChainBlockSubstr : std::false_type {};\n\n  template <typename T>\n  struct HasRiegeliToChainBlockSubstr<\n      T, std::enable_if_t<std::is_convertible_v<\n             decltype(RiegeliToChainBlock(\n                 std::declval<external_ref_internal::PointerTypeT<T>>(),\n                 std::declval<absl::string_view>())),\n             Chain::Block>>> : std::true_type {};\n\n  template <typename T>\n  struct HasToChainBlockWhole\n      : std::disjunction<HasRiegeliToChainBlockWhole<T>,\n                         HasRiegeliToChainBlockSubstr<T>> {};\n\n  template <typename T,\n            std::enable_if_t<HasRiegeliToChainBlockWhole<T>::value, int> = 0>\n  static Chain::Block ToChainBlockWhole(T&& object) {\n    return RiegeliToChainBlock(ExternalRef::Pointer(std::forward<T>(object)));\n  }\n  template <typename T,\n            std::enable_if_t<std::conjunction_v<\n                                 std::negation<HasRiegeliToChainBlockWhole<T>>,\n                                 HasRiegeliToChainBlockSubstr<T>>,\n                             int> = 0>\n  static Chain::Block ToChainBlockWhole(T&& object) {\n    const absl::string_view data = BytesRef(object);\n    return RiegeliToChainBlock(ExternalRef::Pointer(std::forward<T>(object)),\n                               data);\n  }\n\n  template <typename T,\n            std::enable_if_t<HasRiegeliToChainBlockWhole<T>::value, int> = 0>\n  static Chain::Block ToChainBlockWhole(\n      T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view data) {\n    return RiegeliToChainBlock(ExternalRef::Pointer(std::forward<T>(object)));\n  }\n  template <typename T,\n            std::enable_if_t<std::conjunction_v<\n                                 std::negation<HasRiegeliToChainBlockWhole<T>>,\n                                 HasRiegeliToChainBlockSubstr<T>>,\n                             int> = 0>\n  static Chain::Block ToChainBlockWhole(T&& object, absl::string_view data) {\n    return RiegeliToChainBlock(ExternalRef::Pointer(std::forward<T>(object)),\n                               data);\n  }\n\n  template <typename T>\n  using HasToChainBlockSubstr = HasRiegeliToChainBlockSubstr<T>;\n\n  template <typename T,\n            std::enable_if_t<HasRiegeliToChainBlockSubstr<T>::value, int> = 0>\n  static Chain::Block ToChainBlockSubstr(T&& object, absl::string_view substr) {\n    return RiegeliToChainBlock(ExternalRef::Pointer(std::forward<T>(object)),\n                               substr);\n  }\n\n  template <typename T>\n  class ConverterToChainBlockWhole {\n   public:\n    ConverterToChainBlockWhole(const ConverterToChainBlockWhole&) = delete;\n    ConverterToChainBlockWhole& operator=(const ConverterToChainBlockWhole&) =\n        delete;\n\n    template <\n        typename SubT,\n        std::enable_if_t<std::is_convertible_v<const SubT&, BytesRef>, int> = 0>\n    void operator()(SubT&& subobject) && {\n      // The constructor processes the subobject.\n      const absl::string_view data = BytesRef(subobject);\n      ConverterToChainBlockWhole<SubT> converter(\n          std::forward<SubT>(subobject), data, context_, use_string_view_,\n          use_chain_block_);\n    }\n\n    template <typename SubT>\n    void operator()(SubT&& subobject, absl::string_view substr) && {\n      // The constructor processes the subobject.\n      ConverterToChainBlockSubstr<SubT> converter(\n          std::forward<SubT>(subobject), substr, context_, use_string_view_,\n          use_chain_block_);\n    }\n\n   private:\n    friend class ExternalRef;\n\n    ABSL_ATTRIBUTE_ALWAYS_INLINE\n    explicit ConverterToChainBlockWhole(T&& object, absl::string_view data,\n                                        void* context,\n                                        UseStringViewFunction use_string_view,\n                                        UseChainBlockFunction use_chain_block)\n        : context_(context),\n          use_string_view_(use_string_view),\n          use_chain_block_(use_chain_block) {\n      if (RiegeliExternalCopy(&object)) {\n        use_string_view_(context_, data);\n        ExternalRef::CallOperatorWhole(std::forward<T>(object));\n        return;\n      }\n      std::move(*this).Callback(std::forward<T>(object), data);\n    }\n\n    template <\n        typename DependentT = T,\n        std::enable_if_t<HasToChainBlockWhole<DependentT>::value, int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object,\n                                               absl::string_view data) && {\n      use_chain_block_(context_, ExternalRef::ToChainBlockWhole(\n                                     std::forward<T>(object), data));\n    }\n    template <typename DependentT = T,\n              std::enable_if_t<\n                  std::conjunction_v<\n                      std::negation<HasToChainBlockWhole<DependentT>>,\n                      HasToChainBlockWhole<absl::remove_cvref_t<DependentT>>>,\n                  int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(\n        T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view data) && {\n      use_chain_block_(context_, ExternalRef::ToChainBlockWhole(\n                                     absl::remove_cvref_t<T>(object)));\n    }\n    template <typename DependentT = T,\n              std::enable_if_t<std::conjunction_v<\n                                   std::negation<HasToChainBlockWhole<\n                                       absl::remove_cvref_t<DependentT>>>,\n                                   HasExternalDelegateWhole<\n                                       DependentT, ConverterToChainBlockWhole>>,\n                               int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object,\n                                               absl::string_view data) && {\n      ExternalRef::ExternalDelegateWhole(std::forward<T>(object), data,\n                                         std::move(*this));\n    }\n    template <typename DependentT = T,\n              std::enable_if_t<\n                  std::conjunction_v<\n                      std::negation<HasToChainBlockWhole<\n                          absl::remove_cvref_t<DependentT>>>,\n                      std::negation<HasExternalDelegateWhole<\n                          DependentT, ConverterToChainBlockWhole>>,\n                      HasExternalDelegateWhole<absl::remove_cvref_t<DependentT>,\n                                               ConverterToChainBlockWhole>>,\n                  int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(\n        T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view data) && {\n      ExternalRef::ExternalDelegateWhole(absl::remove_cvref_t<T>(object),\n                                         std::move(*this));\n    }\n    template <typename DependentT = T,\n              std::enable_if_t<\n                  std::conjunction_v<std::negation<HasToChainBlockWhole<\n                                         absl::remove_cvref_t<DependentT>>>,\n                                     std::negation<HasExternalDelegateWhole<\n                                         absl::remove_cvref_t<DependentT>,\n                                         ConverterToChainBlockWhole>>>,\n                  int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(\n        T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view data) && {\n      use_chain_block_(context_, Chain::Block(std::forward<T>(object)));\n    }\n\n    void* context_;\n    UseStringViewFunction use_string_view_;\n    UseChainBlockFunction use_chain_block_;\n  };\n\n  template <typename T>\n  class ConverterToChainBlockSubstr {\n   public:\n    ConverterToChainBlockSubstr(const ConverterToChainBlockSubstr&) = delete;\n    ConverterToChainBlockSubstr& operator=(const ConverterToChainBlockSubstr&) =\n        delete;\n\n    template <typename SubT>\n    void operator()(SubT&& subobject) && {\n      std::move (*this)(std::forward<SubT>(subobject), substr_);\n    }\n\n    template <typename SubT>\n    void operator()(SubT&& subobject, absl::string_view substr) && {\n      RIEGELI_ASSERT_EQ(substr_.size(), substr.size())\n          << \"ExternalRef: size mismatch\";\n      // The constructor processes the subobject.\n      ConverterToChainBlockSubstr<SubT> converter(\n          std::forward<SubT>(subobject), substr, context_, use_string_view_,\n          use_chain_block_);\n    }\n\n   private:\n    friend class ExternalRef;\n\n    ABSL_ATTRIBUTE_ALWAYS_INLINE\n    explicit ConverterToChainBlockSubstr(T&& object, absl::string_view substr,\n                                         void* context,\n                                         UseStringViewFunction use_string_view,\n                                         UseChainBlockFunction use_chain_block)\n        : substr_(substr),\n          context_(context),\n          use_string_view_(use_string_view),\n          use_chain_block_(use_chain_block) {\n      AssertSubstr(object, substr_);\n      if (RiegeliExternalCopy(&object)) {\n        use_string_view_(context_, substr_);\n        ExternalRef::CallOperatorSubstr(std::forward<T>(object), substr_);\n        return;\n      }\n      std::move(*this).Callback(std::forward<T>(object));\n    }\n\n    template <\n        typename DependentT = T,\n        std::enable_if_t<HasToChainBlockSubstr<DependentT>::value, int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      use_chain_block_(context_, ExternalRef::ToChainBlockSubstr(\n                                     std::forward<T>(object), substr_));\n    }\n    template <typename DependentT = T,\n              std::enable_if_t<\n                  std::conjunction_v<\n                      std::negation<HasToChainBlockSubstr<DependentT>>,\n                      HasToChainBlockSubstr<absl::remove_cvref_t<DependentT>>>,\n                  int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      use_chain_block_(context_, ExternalRef::ToChainBlockSubstr(\n                                     absl::remove_cvref_t<T>(object), substr_));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<\n            std::conjunction_v<std::negation<HasToChainBlockSubstr<\n                                   absl::remove_cvref_t<DependentT>>>,\n                               HasExternalDelegateSubstr<\n                                   DependentT, ConverterToChainBlockSubstr>>,\n            int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      ExternalRef::ExternalDelegateSubstr(std::forward<T>(object), substr_,\n                                          std::move(*this));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<\n            std::conjunction_v<\n                std::negation<\n                    HasToChainBlockSubstr<absl::remove_cvref_t<DependentT>>>,\n                std::negation<HasExternalDelegateSubstr<\n                    DependentT, ConverterToChainBlockSubstr>>,\n                HasExternalDelegateSubstr<absl::remove_cvref_t<DependentT>,\n                                          ConverterToChainBlockSubstr>>,\n            int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      ExternalRef::ExternalDelegateSubstr(absl::remove_cvref_t<T>(object),\n                                          substr_, std::move(*this));\n    }\n    template <typename DependentT = T,\n              std::enable_if_t<\n                  std::conjunction_v<std::negation<HasToChainBlockSubstr<\n                                         absl::remove_cvref_t<DependentT>>>,\n                                     std::negation<HasExternalDelegateSubstr<\n                                         absl::remove_cvref_t<DependentT>,\n                                         ConverterToChainBlockSubstr>>>,\n                  int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      use_chain_block_(context_,\n                       Chain::Block(std::forward<T>(object), substr_));\n    }\n\n    absl::string_view substr_;\n    void* context_;\n    UseStringViewFunction use_string_view_;\n    UseChainBlockFunction use_chain_block_;\n  };\n\n  template <typename T>\n  class ObjectForCordWhole {\n   public:\n    explicit ObjectForCordWhole(Initializer<T> object)\n        : ptr_(std::move(object)) {}\n\n    ObjectForCordWhole(ObjectForCordWhole&& that) = default;\n    ObjectForCordWhole& operator=(ObjectForCordWhole&& that) = default;\n\n    void operator()(absl::string_view substr) && {\n      ExternalRef::CallOperatorSubstr(std::move(*ptr_), substr);\n    }\n\n    T& operator*() { return *ptr_; }\n    const T& operator*() const { return *ptr_; }\n\n   private:\n    // Wrapped in `std::unique_ptr` so that the data are stable.\n    // `absl::MakeCordFromExternal()` requires the data to be known beforehand\n    // and valid for the moved external object.\n    std::unique_ptr<T> ptr_;\n  };\n\n  template <typename T>\n  class ObjectForCordSubstr {\n   public:\n    explicit ObjectForCordSubstr(Initializer<T> object,\n                                 absl::string_view substr)\n        : object_(std::move(object)) {\n      AssertSubstr(**this, substr);\n    }\n\n    ObjectForCordSubstr(ObjectForCordSubstr&& that) = default;\n    ObjectForCordSubstr& operator=(ObjectForCordSubstr&& that) = default;\n\n    void operator()(absl::string_view substr) && {\n      ExternalRef::CallOperatorSubstr(std::move(object_), substr);\n    }\n\n    T& operator*() { return object_; }\n    const T& operator*() const { return object_; }\n\n   private:\n    ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS T object_;\n  };\n\n  template <typename T, typename Enable = void>\n  struct HasRiegeliToCordWhole : std::false_type {};\n\n  template <typename T>\n  struct HasRiegeliToCordWhole<\n      T, std::enable_if_t<std::is_convertible_v<\n             decltype(RiegeliToCord(\n                 std::declval<external_ref_internal::PointerTypeT<T>>())),\n             absl::Cord>>> : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct HasRiegeliToCordSubstr : std::false_type {};\n\n  template <typename T>\n  struct HasRiegeliToCordSubstr<\n      T, std::enable_if_t<std::is_convertible_v<\n             decltype(RiegeliToCord(\n                 std::declval<external_ref_internal::PointerTypeT<T>>(),\n                 std::declval<absl::string_view>())),\n             absl::Cord>>> : std::true_type {};\n\n  template <typename T>\n  struct HasToCordWhole\n      : std::disjunction<HasRiegeliToCordWhole<T>, HasRiegeliToCordSubstr<T>> {\n  };\n\n  template <typename T,\n            std::enable_if_t<HasRiegeliToCordWhole<T>::value, int> = 0>\n  static absl::Cord ToCordWhole(T&& object) {\n    return RiegeliToCord(ExternalRef::Pointer(std::forward<T>(object)));\n  }\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<std::negation<HasRiegeliToCordWhole<T>>,\n                                   HasRiegeliToCordSubstr<T>>,\n                int> = 0>\n  static absl::Cord ToCordWhole(T&& object) {\n    const absl::string_view data = BytesRef(object);\n    return RiegeliToCord(ExternalRef::Pointer(std::forward<T>(object)), data);\n  }\n\n  template <typename T,\n            std::enable_if_t<HasRiegeliToCordWhole<T>::value, int> = 0>\n  static absl::Cord ToCordWhole(T&& object,\n                                ABSL_ATTRIBUTE_UNUSED absl::string_view data) {\n    return RiegeliToCord(ExternalRef::Pointer(std::forward<T>(object)));\n  }\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<std::negation<HasRiegeliToCordWhole<T>>,\n                                   HasRiegeliToCordSubstr<T>>,\n                int> = 0>\n  static absl::Cord ToCordWhole(T&& object, absl::string_view data) {\n    return RiegeliToCord(ExternalRef::Pointer(std::forward<T>(object)), data);\n  }\n\n  template <typename T>\n  using HasToCordSubstr = HasRiegeliToCordSubstr<T>;\n\n  template <typename T,\n            std::enable_if_t<HasRiegeliToCordSubstr<T>::value, int> = 0>\n  static absl::Cord ToCordSubstr(T&& object, absl::string_view substr) {\n    return RiegeliToCord(ExternalRef::Pointer(std::forward<T>(object)), substr);\n  }\n\n  template <typename T>\n  class ConverterToCordWhole {\n   public:\n    ConverterToCordWhole(const ConverterToCordWhole&) = delete;\n    ConverterToCordWhole& operator=(const ConverterToCordWhole&) = delete;\n\n    template <\n        typename SubT,\n        std::enable_if_t<std::is_convertible_v<const SubT&, BytesRef>, int> = 0>\n    void operator()(SubT&& subobject) && {\n      // The constructor processes the subobject.\n      const absl::string_view data = BytesRef(subobject);\n      ConverterToCordWhole<SubT> converter(std::forward<SubT>(subobject), data,\n                                           context_, use_string_view_,\n                                           use_cord_);\n    }\n\n    template <typename SubT>\n    void operator()(SubT&& subobject, absl::string_view substr) && {\n      // The constructor processes the subobject.\n      ConverterToCordSubstr<SubT> converter(std::forward<SubT>(subobject),\n                                            substr, context_, use_string_view_,\n                                            use_cord_);\n    }\n\n   private:\n    friend class ExternalRef;\n\n    ABSL_ATTRIBUTE_ALWAYS_INLINE\n    explicit ConverterToCordWhole(T&& object, absl::string_view data,\n                                  void* context,\n                                  UseStringViewFunction use_string_view,\n                                  UseCordFunction use_cord)\n        : context_(context),\n          use_string_view_(use_string_view),\n          use_cord_(use_cord) {\n      if (RiegeliExternalCopy(&object)) {\n        use_string_view_(context_, data);\n        ExternalRef::CallOperatorWhole(std::forward<T>(object));\n        return;\n      }\n      std::move(*this).Callback(std::forward<T>(object), data);\n    }\n\n    template <typename DependentT = T,\n              std::enable_if_t<HasToCordWhole<DependentT>::value, int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object,\n                                               absl::string_view data) && {\n      use_cord_(context_,\n                ExternalRef::ToCordWhole(std::forward<T>(object), data));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<std::conjunction_v<\n                             std::negation<HasToCordWhole<DependentT>>,\n                             HasToCordWhole<absl::remove_cvref_t<DependentT>>>,\n                         int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(\n        T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view data) && {\n      use_cord_(context_, ExternalRef::ToCordWhole(absl::remove_cvref_t<T>(\n                              std::forward<T>(object))));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<\n            std::conjunction_v<\n                std::negation<HasToCordWhole<absl::remove_cvref_t<DependentT>>>,\n                HasExternalDelegateWhole<DependentT, ConverterToCordWhole>>,\n            int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object,\n                                               absl::string_view data) && {\n      ExternalRef::ExternalDelegateWhole(std::forward<T>(object), data,\n                                         std::move(*this));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<\n            std::conjunction_v<\n                std::negation<HasToCordWhole<absl::remove_cvref_t<DependentT>>>,\n                std::negation<\n                    HasExternalDelegateWhole<DependentT, ConverterToCordWhole>>,\n                HasExternalDelegateWhole<absl::remove_cvref_t<DependentT>,\n                                         ConverterToCordWhole>>,\n            int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(\n        T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view data) && {\n      ExternalRef::ExternalDelegateWhole(absl::remove_cvref_t<T>(object),\n                                         std::move(*this));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<\n            std::conjunction_v<\n                std::negation<HasToCordWhole<absl::remove_cvref_t<DependentT>>>,\n                std::negation<HasExternalDelegateWhole<\n                    absl::remove_cvref_t<DependentT>, ConverterToCordWhole>>,\n                SupportsExternalRefSubstr<std::decay_t<DependentT>>>,\n            int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object,\n                                               absl::string_view data) && {\n      // If the type indicates that substrings are stable, then\n      // `ObjectForCordSubstr` can be used instead of `ObjectForCordWhole`.\n      use_cord_(context_, absl::MakeCordFromExternal(\n                              data, ObjectForCordSubstr<std::decay_t<T>>(\n                                        std::forward<T>(object), data)));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<\n            std::conjunction_v<\n                std::negation<HasToCordWhole<absl::remove_cvref_t<DependentT>>>,\n                std::negation<HasExternalDelegateWhole<\n                    absl::remove_cvref_t<DependentT>, ConverterToCordWhole>>,\n                std::negation<\n                    SupportsExternalRefSubstr<std::decay_t<DependentT>>>>,\n            int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(\n        T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view data) && {\n      ObjectForCordWhole<std::decay_t<T>> object_for_cord(\n          std::forward<T>(object));\n      const absl::string_view moved_data = BytesRef(*object_for_cord);\n      use_cord_(context_, absl::MakeCordFromExternal(\n                              moved_data, std::move(object_for_cord)));\n    }\n\n    void* context_;\n    UseStringViewFunction use_string_view_;\n    UseCordFunction use_cord_;\n  };\n\n  template <typename T>\n  class ConverterToCordSubstr {\n   public:\n    ConverterToCordSubstr(const ConverterToCordSubstr&) = delete;\n    ConverterToCordSubstr& operator=(const ConverterToCordSubstr&) = delete;\n\n    template <typename SubT>\n    void operator()(SubT&& subobject) && {\n      std::move (*this)(std::forward<SubT>(subobject), substr_);\n    }\n\n    template <typename SubT>\n    void operator()(SubT&& subobject, absl::string_view substr) && {\n      RIEGELI_ASSERT_EQ(substr_.size(), substr.size())\n          << \"ExternalRef: size mismatch\";\n      // The constructor processes the subobject.\n      ConverterToCordSubstr<SubT> converter(std::forward<SubT>(subobject),\n                                            substr, context_, use_string_view_,\n                                            use_cord_);\n    }\n\n   private:\n    friend class ExternalRef;\n\n    ABSL_ATTRIBUTE_ALWAYS_INLINE\n    explicit ConverterToCordSubstr(T&& object, absl::string_view substr,\n                                   void* context,\n                                   UseStringViewFunction use_string_view,\n                                   UseCordFunction use_cord)\n        : substr_(substr),\n          context_(context),\n          use_string_view_(use_string_view),\n          use_cord_(use_cord) {\n      AssertSubstr(object, substr_);\n      if (RiegeliExternalCopy(&object)) {\n        use_string_view_(context_, substr_);\n        ExternalRef::CallOperatorSubstr(std::forward<T>(object), substr_);\n        return;\n      }\n      std::move(*this).Callback(std::forward<T>(object));\n    }\n\n    template <typename DependentT = T,\n              std::enable_if_t<HasToCordSubstr<DependentT>::value, int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      use_cord_(context_,\n                ExternalRef::ToCordSubstr(std::forward<T>(object), substr_));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<std::conjunction_v<\n                             std::negation<HasToCordSubstr<DependentT>>,\n                             HasToCordSubstr<absl::remove_cvref_t<DependentT>>>,\n                         int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      use_cord_(context_, ExternalRef::ToCordSubstr(\n                              absl::remove_cvref_t<T>(object), substr_));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<\n            std::conjunction_v<\n                std::negation<\n                    HasToCordSubstr<absl::remove_cvref_t<DependentT>>>,\n                HasExternalDelegateSubstr<DependentT, ConverterToCordSubstr>>,\n            int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      ExternalRef::ExternalDelegateSubstr(std::forward<T>(object), substr_,\n                                          std::move(*this));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<\n            std::conjunction_v<\n                std::negation<\n                    HasToCordSubstr<absl::remove_cvref_t<DependentT>>>,\n                std::negation<HasExternalDelegateSubstr<DependentT,\n                                                        ConverterToCordSubstr>>,\n                HasExternalDelegateSubstr<absl::remove_cvref_t<DependentT>,\n                                          ConverterToCordSubstr>>,\n            int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      ExternalRef::ExternalDelegateSubstr(absl::remove_cvref_t<T>(object),\n                                          substr_, std::move(*this));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<\n            std::conjunction_v<\n                std::negation<\n                    HasToCordSubstr<absl::remove_cvref_t<DependentT>>>,\n                std::negation<HasExternalDelegateSubstr<\n                    absl::remove_cvref_t<DependentT>, ConverterToCordSubstr>>>,\n            int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      use_cord_(context_, absl::MakeCordFromExternal(\n                              substr_, ObjectForCordSubstr<std::decay_t<T>>(\n                                           std::forward<T>(object), substr_)));\n    }\n\n    absl::string_view substr_;\n    void* context_;\n    UseStringViewFunction use_string_view_;\n    UseCordFunction use_cord_;\n  };\n\n  template <typename T, typename Enable = void>\n  class ExternalObjectWhole {\n   public:\n    explicit ExternalObjectWhole(Initializer<T> object)\n        : object_(std::move(object)) {}\n\n    ~ExternalObjectWhole() {\n      ExternalRef::CallOperatorWhole(std::move(object_));\n    }\n\n    T& operator*() { return object_; }\n    const T& operator*() const { return object_; }\n\n   private:\n    ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS T object_;\n  };\n\n  template <typename T, typename Enable = void>\n  class ExternalObjectSubstr;\n\n  template <typename T>\n  class ExternalObjectSubstr<\n      T, std::enable_if_t<HasCallOperatorSubstr<T>::value>> {\n   public:\n    explicit ExternalObjectSubstr(Initializer<T> object,\n                                  absl::string_view substr)\n        : object_(std::move(object)), substr_(substr) {\n      AssertSubstr(**this, substr);\n    }\n\n    ~ExternalObjectSubstr() { std::move(object_)(substr_); }\n\n    T& operator*() { return object_; }\n    const T& operator*() const { return object_; }\n\n   private:\n    ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS T object_;\n    absl::string_view substr_;\n  };\n\n  template <typename T>\n  class ExternalObjectSubstr<T,\n                             std::enable_if_t<!HasCallOperatorSubstr<T>::value>>\n      : public ExternalObjectWhole<T> {\n   public:\n    explicit ExternalObjectSubstr(Initializer<T> object,\n                                  absl::string_view substr)\n        : ExternalObjectSubstr::ExternalObjectWhole(std::move(object)) {\n      AssertSubstr(**this, substr);\n    }\n  };\n\n  template <typename T, typename Enable = void>\n  struct HasRiegeliToExternalDataWhole : std::false_type {};\n\n  template <typename T>\n  struct HasRiegeliToExternalDataWhole<\n      T, std::enable_if_t<std::is_convertible_v<\n             decltype(RiegeliToExternalData(\n                 std::declval<external_ref_internal::PointerTypeT<T>>())),\n             ExternalData>>> : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct HasRiegeliToExternalDataSubstr : std::false_type {};\n\n  template <typename T>\n  struct HasRiegeliToExternalDataSubstr<\n      T, std::enable_if_t<std::is_convertible_v<\n             decltype(RiegeliToExternalData(\n                 std::declval<external_ref_internal::PointerTypeT<T>>(),\n                 std::declval<absl::string_view>())),\n             ExternalData>>> : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct HasRiegeliToExternalStorage : std::false_type {};\n\n  template <typename T>\n  struct HasRiegeliToExternalStorage<\n      T, std::enable_if_t<std::is_convertible_v<\n             decltype(RiegeliToExternalStorage(\n                 std::declval<external_ref_internal::PointerTypeT<T>>())),\n             ExternalStorage>>> : std::true_type {};\n\n  template <typename T>\n  struct HasToExternalDataSubstr\n      : std::disjunction<HasRiegeliToExternalDataSubstr<T>,\n                         HasRiegeliToExternalStorage<T>> {};\n\n  template <typename T,\n            std::enable_if_t<HasRiegeliToExternalDataSubstr<T>::value, int> = 0>\n  static ExternalData ToExternalDataSubstr(T&& object,\n                                           absl::string_view substr) {\n    return RiegeliToExternalData(ExternalRef::Pointer(std::forward<T>(object)),\n                                 substr);\n  }\n\n  template <\n      typename T,\n      std::enable_if_t<\n          std::disjunction_v<std::negation<HasRiegeliToExternalDataSubstr<T>>,\n                             HasRiegeliToExternalStorage<T>>,\n          int> = 0>\n  static ExternalData ToExternalDataSubstr(T&& object,\n                                           absl::string_view substr) {\n    return ExternalData{\n        RiegeliToExternalStorage(ExternalRef::Pointer(std::forward<T>(object))),\n        substr};\n  }\n\n  template <typename T>\n  struct HasToExternalDataWhole\n      : std::disjunction<HasRiegeliToExternalDataWhole<T>,\n                         HasToExternalDataSubstr<T>> {};\n\n  template <typename T,\n            std::enable_if_t<HasRiegeliToExternalDataWhole<T>::value, int> = 0>\n  static ExternalData ToExternalDataWhole(T&& object) {\n    return RiegeliToExternalData(ExternalRef::Pointer(std::forward<T>(object)));\n  }\n  template <\n      typename T,\n      std::enable_if_t<\n          std::conjunction_v<std::negation<HasRiegeliToExternalDataWhole<T>>,\n                             HasToExternalDataSubstr<T>>,\n          int> = 0>\n  static ExternalData ToExternalDataWhole(T&& object) {\n    const absl::string_view data = BytesRef(object);\n    return ExternalRef::ToExternalDataSubstr(std::forward<T>(object), data);\n  }\n\n  template <typename T,\n            std::enable_if_t<HasRiegeliToExternalDataWhole<T>::value, int> = 0>\n  static ExternalData ToExternalDataWhole(\n      T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view data) {\n    return RiegeliToExternalData(ExternalRef::Pointer(std::forward<T>(object)));\n  }\n  template <\n      typename T,\n      std::enable_if_t<\n          std::conjunction_v<std::negation<HasRiegeliToExternalDataWhole<T>>,\n                             HasToExternalDataSubstr<T>>,\n          int> = 0>\n  static ExternalData ToExternalDataWhole(T&& object, absl::string_view data) {\n    return ExternalRef::ToExternalDataSubstr(std::forward<T>(object), data);\n  }\n\n  template <typename T>\n  class ConverterToExternalDataWhole {\n   public:\n    ConverterToExternalDataWhole(const ConverterToExternalDataWhole&) = delete;\n    ConverterToExternalDataWhole& operator=(\n        const ConverterToExternalDataWhole&) = delete;\n\n    template <\n        typename SubT,\n        std::enable_if_t<std::is_convertible_v<const SubT&, BytesRef>, int> = 0>\n    void operator()(SubT&& subobject) && {\n      // The constructor processes the subobject.\n      const absl::string_view data = BytesRef(subobject);\n      ConverterToExternalDataWhole<SubT> converter(\n          std::forward<SubT>(subobject), data, context_, use_external_data_);\n    }\n\n    template <typename SubT>\n    void operator()(SubT&& subobject, absl::string_view substr) && {\n      // The constructor processes the subobject.\n      ConverterToExternalDataSubstr<SubT> converter(\n          std::forward<SubT>(subobject), substr, context_, use_external_data_);\n    }\n\n   private:\n    friend class ExternalRef;\n\n    ABSL_ATTRIBUTE_ALWAYS_INLINE\n    explicit ConverterToExternalDataWhole(\n        T&& object, absl::string_view data, void* context,\n        UseExternalDataFunction use_external_data)\n        : context_(context), use_external_data_(use_external_data) {\n      if (RiegeliExternalCopy(&object)) {\n        use_external_data_(context_, ExternalDataCopy(data));\n        ExternalRef::CallOperatorWhole(std::forward<T>(object));\n        return;\n      }\n      std::move(*this).Callback(std::forward<T>(object), data);\n    }\n\n    template <\n        typename DependentT = T,\n        std::enable_if_t<HasToExternalDataWhole<DependentT>::value, int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object,\n                                               absl::string_view data) && {\n      use_external_data_(context_, ExternalRef::ToExternalDataWhole(\n                                       std::forward<T>(object), data));\n    }\n    template <typename DependentT = T,\n              std::enable_if_t<\n                  std::conjunction_v<\n                      std::negation<HasToExternalDataWhole<DependentT>>,\n                      HasToExternalDataWhole<absl::remove_cvref_t<DependentT>>>,\n                  int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(\n        T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view data) && {\n      use_external_data_(context_, ExternalRef::ToExternalDataWhole(\n                                       absl::remove_cvref_t<T>(object)));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<\n            std::conjunction_v<std::negation<HasToExternalDataWhole<\n                                   absl::remove_cvref_t<DependentT>>>,\n                               HasExternalDelegateWhole<\n                                   DependentT, ConverterToExternalDataWhole>>,\n            int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object,\n                                               absl::string_view data) && {\n      ExternalRef::ExternalDelegateWhole(std::forward<T>(object), data,\n                                         std::move(*this));\n    }\n    template <typename DependentT = T,\n              std::enable_if_t<\n                  std::conjunction_v<\n                      std::negation<HasToExternalDataWhole<\n                          absl::remove_cvref_t<DependentT>>>,\n                      std::negation<HasExternalDelegateWhole<\n                          DependentT, ConverterToExternalDataWhole>>,\n                      HasExternalDelegateWhole<absl::remove_cvref_t<DependentT>,\n                                               ConverterToExternalDataWhole>>,\n                  int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(\n        T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view data) && {\n      ExternalRef::ExternalDelegateWhole(absl::remove_cvref_t<T>(object),\n                                         std::move(*this));\n    }\n    template <typename DependentT = T,\n              std::enable_if_t<\n                  std::conjunction_v<std::negation<HasToExternalDataWhole<\n                                         absl::remove_cvref_t<DependentT>>>,\n                                     std::negation<HasExternalDelegateWhole<\n                                         absl::remove_cvref_t<DependentT>,\n                                         ConverterToExternalDataWhole>>>,\n                  int> = 0>\n    void Callback(T&& object, ABSL_ATTRIBUTE_UNUSED absl::string_view data) {\n      auto* const storage =\n          new ExternalObjectWhole<std::decay_t<T>>(std::forward<T>(object));\n      const absl::string_view moved_data = BytesRef(**storage);\n      use_external_data_(\n          context_,\n          ExternalData{\n              ExternalStorage(\n                  storage,\n                  [](void* ptr) {\n                    delete static_cast<ExternalObjectWhole<std::decay_t<T>>*>(\n                        ptr);\n                  }),\n              moved_data});\n    }\n\n    void* context_;\n    UseExternalDataFunction use_external_data_;\n  };\n\n  template <typename T>\n  class ConverterToExternalDataSubstr {\n   public:\n    ConverterToExternalDataSubstr(const ConverterToExternalDataSubstr&) =\n        delete;\n    ConverterToExternalDataSubstr& operator=(\n        const ConverterToExternalDataSubstr&) = delete;\n\n    template <typename SubT>\n    void operator()(SubT&& subobject) && {\n      std::move (*this)(std::forward<SubT>(subobject), substr_);\n    }\n\n    template <typename SubT>\n    void operator()(SubT&& subobject, absl::string_view substr) && {\n      RIEGELI_ASSERT_EQ(substr_.size(), substr.size())\n          << \"ExternalRef: size mismatch\";\n      // The constructor processes the subobject.\n      ConverterToExternalDataSubstr<SubT> converter(\n          std::forward<SubT>(subobject), substr, context_, use_external_data_);\n    }\n\n   private:\n    friend class ExternalRef;\n\n    ABSL_ATTRIBUTE_ALWAYS_INLINE\n    explicit ConverterToExternalDataSubstr(\n        T&& object, absl::string_view substr, void* context,\n        UseExternalDataFunction use_external_data)\n        : substr_(substr),\n          context_(context),\n          use_external_data_(use_external_data) {\n      AssertSubstr(object, substr_);\n      if (RiegeliExternalCopy(&object)) {\n        use_external_data_(context_, ExternalDataCopy(substr_));\n        ExternalRef::CallOperatorSubstr(std::forward<T>(object), substr_);\n        return;\n      }\n      std::move(*this).Callback(std::forward<T>(object));\n    }\n\n    template <\n        typename DependentT = T,\n        std::enable_if_t<HasToExternalDataSubstr<DependentT>::value, int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      use_external_data_(context_, ExternalRef::ToExternalDataSubstr(\n                                       std::forward<T>(object), substr_));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<\n            std::conjunction_v<\n                std::negation<HasToExternalDataSubstr<DependentT>>,\n                HasToExternalDataSubstr<absl::remove_cvref_t<DependentT>>>,\n            int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      use_external_data_(context_,\n                         ExternalRef::ToExternalDataSubstr(\n                             absl::remove_cvref_t<T>(object), substr_));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<\n            std::conjunction_v<std::negation<HasToExternalDataSubstr<\n                                   absl::remove_cvref_t<DependentT>>>,\n                               HasExternalDelegateSubstr<\n                                   DependentT, ConverterToExternalDataSubstr>>,\n            int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      ExternalRef::ExternalDelegateSubstr(std::forward<T>(object), substr_,\n                                          std::move(*this));\n    }\n    template <\n        typename DependentT = T,\n        std::enable_if_t<\n            std::conjunction_v<\n                std::negation<\n                    HasToExternalDataSubstr<absl::remove_cvref_t<DependentT>>>,\n                std::negation<HasExternalDelegateSubstr<\n                    DependentT, ConverterToExternalDataSubstr>>,\n                HasExternalDelegateSubstr<absl::remove_cvref_t<DependentT>,\n                                          ConverterToExternalDataSubstr>>,\n            int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      ExternalRef::ExternalDelegateSubstr(absl::remove_cvref_t<T>(object),\n                                          substr_, std::move(*this));\n    }\n    template <typename DependentT = T,\n              std::enable_if_t<\n                  std::conjunction_v<std::negation<HasToExternalDataSubstr<\n                                         absl::remove_cvref_t<DependentT>>>,\n                                     std::negation<HasExternalDelegateSubstr<\n                                         absl::remove_cvref_t<DependentT>,\n                                         ConverterToExternalDataSubstr>>>,\n                  int> = 0>\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Callback(T&& object) && {\n      use_external_data_(\n          context_,\n          ExternalData{\n              ExternalStorage(\n                  new ExternalObjectSubstr<std::decay_t<T>>(\n                      std::forward<T>(object), substr_),\n                  [](void* ptr) {\n                    delete static_cast<ExternalObjectSubstr<std::decay_t<T>>*>(\n                        ptr);\n                  }),\n              substr_});\n    }\n\n    absl::string_view substr_;\n    void* context_;\n    UseExternalDataFunction use_external_data_;\n  };\n\n  class StorageBase {\n   protected:\n    StorageBase() = default;\n\n    StorageBase(const StorageBase&) = delete;\n    StorageBase& operator=(const StorageBase&) = delete;\n\n    virtual ~StorageBase() = default;\n\n    void Initialize(absl::string_view substr) { substr_ = substr; }\n\n   private:\n    friend class ExternalRef;\n\n    // Converts the external object either to `absl::string_view` or\n    // `Chain::Block` by calling once either `use_string_view` or\n    // `use_chain_block`.\n    virtual void ToChainBlock(size_t max_bytes_to_copy, void* context,\n                              UseStringViewFunction use_string_view,\n                              UseChainBlockFunction use_chain_block) && = 0;\n\n    // Converts the external object either to `absl::string_view` or\n    // `absl::Cord` by calling once either `use_string_view` or `use_cord`.\n    virtual void ToCord(size_t max_bytes_to_copy, void* context,\n                        UseStringViewFunction use_string_view,\n                        UseCordFunction use_cord) && = 0;\n\n    // Converts the external object to `ExternalData` by calling once\n    // `use_external_data`.\n    virtual void ToExternalData(\n        void* context, UseExternalDataFunction use_external_data) && = 0;\n\n    bool empty() const { return substr_.empty(); }\n    const char* data() const { return substr_.data(); }\n    size_t size() const { return substr_.size(); }\n    absl::string_view substr() const { return substr_; }\n\n    absl::string_view substr_;\n  };\n\n  template <typename T>\n  class StorageWholeWithoutCallOperator final : public StorageBase {\n   public:\n    StorageWholeWithoutCallOperator() = default;\n\n    StorageWholeWithoutCallOperator(const StorageWholeWithoutCallOperator&) =\n        delete;\n    StorageWholeWithoutCallOperator& operator=(\n        const StorageWholeWithoutCallOperator&) = delete;\n\n   private:\n    friend class ExternalRef;\n\n    void Initialize(Initializer<T> object) {\n      object_.emplace(\n          std::move(object).Reference(std::move(temporary_storage_)));\n      StorageBase::Initialize(BytesRef(*object_));\n    }\n\n    void ToChainBlock(size_t max_bytes_to_copy, void* context,\n                      UseStringViewFunction use_string_view,\n                      UseChainBlockFunction use_chain_block) &&\n        override {\n      if (size() <= max_bytes_to_copy) {\n        use_string_view(context, substr());\n        return;\n      }\n      // The constructor processes the object.\n      ConverterToChainBlockWhole<T> converter(*std::move(object_), substr(),\n                                              context, use_string_view,\n                                              use_chain_block);\n    }\n\n    void ToCord(size_t max_bytes_to_copy, void* context,\n                UseStringViewFunction use_string_view,\n                UseCordFunction use_cord) &&\n        override {\n      if (size() <= max_bytes_to_copy) {\n        use_string_view(context, substr());\n        return;\n      }\n      // The constructor processes the object.\n      ConverterToCordWhole<T> converter(*std::move(object_), substr(), context,\n                                        use_string_view, use_cord);\n    }\n\n    void ToExternalData(void* context,\n                        UseExternalDataFunction use_external_data) &&\n        override {\n      // The constructor processes the object.\n      ConverterToExternalDataWhole<T> converter(*std::move(object_), substr(),\n                                                context, use_external_data);\n    }\n\n    TemporaryStorage<T&&> object_;\n    ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS TemporaryStorage<T> temporary_storage_;\n  };\n\n  template <typename T>\n  class StorageWholeWithCallOperator final : public StorageBase {\n   public:\n    StorageWholeWithCallOperator() = default;\n\n    StorageWholeWithCallOperator(const StorageWholeWithCallOperator&) = delete;\n    StorageWholeWithCallOperator& operator=(\n        const StorageWholeWithCallOperator&) = delete;\n\n    ~StorageWholeWithCallOperator() {\n      if (object_ != nullptr) {\n        ExternalRef::CallOperatorSubstr(std::forward<T>(*object_), substr());\n      }\n    }\n\n   private:\n    friend class ExternalRef;\n\n    void Initialize(Initializer<T> object) {\n      T&& reference =\n          std::move(object).Reference(std::move(temporary_storage_));\n      object_ = &reference;\n      StorageBase::Initialize(BytesRef(*object_));\n    }\n\n    void ToChainBlock(size_t max_bytes_to_copy, void* context,\n                      UseStringViewFunction use_string_view,\n                      UseChainBlockFunction use_chain_block) &&\n        override {\n      if (size() <= max_bytes_to_copy) {\n        use_string_view(context, substr());\n        return;\n      }\n      // The constructor processes the object.\n      ConverterToChainBlockWhole<T> converter(\n          ExtractObject(), substr(), context, use_string_view, use_chain_block);\n    }\n\n    void ToCord(size_t max_bytes_to_copy, void* context,\n                UseStringViewFunction use_string_view,\n                UseCordFunction use_cord) &&\n        override {\n      if (size() <= max_bytes_to_copy) {\n        use_string_view(context, substr());\n        return;\n      }\n      // The constructor processes the object.\n      ConverterToCordWhole<T> converter(ExtractObject(), substr(), context,\n                                        use_string_view, use_cord);\n    }\n\n    void ToExternalData(void* context,\n                        UseExternalDataFunction use_external_data) &&\n        override {\n      // The constructor processes the object.\n      ConverterToExternalDataWhole<T> converter(ExtractObject(), substr(),\n                                                context, use_external_data);\n    }\n\n    T&& ExtractObject() {\n      return std::forward<T>(*std::exchange(object_, nullptr));\n    }\n\n    std::remove_reference_t<T>* object_ = nullptr;\n    ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS TemporaryStorage<T> temporary_storage_;\n  };\n\n  template <typename Arg>\n  class StorageSubstrWithoutCallOperator final : public StorageBase {\n   private:\n    friend class ExternalRef;\n\n    using T = TargetRefT<Arg>;\n\n    void Initialize(Arg arg, absl::string_view substr) {\n      StorageBase::Initialize(substr);\n      arg_.emplace(std::forward<Arg>(arg));\n    }\n\n    void ToChainBlock(size_t max_bytes_to_copy, void* context,\n                      UseStringViewFunction use_string_view,\n                      UseChainBlockFunction use_chain_block) &&\n        override {\n      if (size() <= max_bytes_to_copy) {\n        use_string_view(context, substr());\n        return;\n      }\n      // The constructor processes the object.\n      ConverterToChainBlockSubstr<T> converter(\n          initializer().Reference(), substr(), context, use_string_view,\n          use_chain_block);\n    }\n\n    void ToCord(size_t max_bytes_to_copy, void* context,\n                UseStringViewFunction use_string_view,\n                UseCordFunction use_cord) &&\n        override {\n      if (size() <= max_bytes_to_copy) {\n        use_string_view(context, substr());\n        return;\n      }\n      // The constructor processes the object.\n      ConverterToCordSubstr<T> converter(initializer().Reference(), substr(),\n                                         context, use_string_view, use_cord);\n    }\n\n    void ToExternalData(void* context,\n                        UseExternalDataFunction use_external_data) &&\n        override {\n      // The constructor processes the object.\n      ConverterToExternalDataSubstr<T> converter(\n          initializer().Reference(), substr(), context, use_external_data);\n    }\n\n    Initializer<T> initializer() { return std::forward<Arg>(*arg_); }\n\n    TemporaryStorage<Arg&&> arg_;\n  };\n\n  template <typename T>\n  class StorageSubstrWithCallOperator final : public StorageBase {\n   public:\n    StorageSubstrWithCallOperator() = default;\n\n    StorageSubstrWithCallOperator(const StorageSubstrWithCallOperator&) =\n        delete;\n    StorageSubstrWithCallOperator& operator=(\n        const StorageSubstrWithCallOperator&) = delete;\n\n    ~StorageSubstrWithCallOperator() {\n      if (object_ != nullptr) {\n        ExternalRef::CallOperatorSubstr(std::forward<T>(*object_), substr());\n      }\n    }\n\n   private:\n    friend class ExternalRef;\n\n    void Initialize(Initializer<T> object, absl::string_view substr) {\n      StorageBase::Initialize(substr);\n      T&& reference =\n          std::move(object).Reference(std::move(temporary_storage_));\n      object_ = &reference;\n    }\n\n    void ToChainBlock(size_t max_bytes_to_copy, void* context,\n                      UseStringViewFunction use_string_view,\n                      UseChainBlockFunction use_chain_block) &&\n        override {\n      if (size() <= max_bytes_to_copy) {\n        use_string_view(context, substr());\n        return;\n      }\n      // The constructor processes the object.\n      ConverterToChainBlockSubstr<T> converter(\n          ExtractObject(), substr(), context, use_string_view, use_chain_block);\n    }\n\n    void ToCord(size_t max_bytes_to_copy, void* context,\n                UseStringViewFunction use_string_view,\n                UseCordFunction use_cord) &&\n        override {\n      if (size() <= max_bytes_to_copy) {\n        use_string_view(context, substr());\n        return;\n      }\n      // The constructor processes the object.\n      ConverterToCordSubstr<T> converter(ExtractObject(), substr(), context,\n                                         use_string_view, use_cord);\n    }\n\n    void ToExternalData(void* context,\n                        UseExternalDataFunction use_external_data) &&\n        override {\n      // The constructor processes the object.\n      ConverterToExternalDataSubstr<T> converter(ExtractObject(), substr(),\n                                                 context, use_external_data);\n    }\n\n    T&& ExtractObject() {\n      return std::forward<T>(*std::exchange(object_, nullptr));\n    }\n\n    std::remove_reference_t<T>* object_ = nullptr;\n    ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS TemporaryStorage<T> temporary_storage_;\n  };\n\n  template <typename T, typename Enable = void>\n  struct StorageWholeImpl;\n\n  template <typename T>\n  struct StorageWholeImpl<\n      T, std::enable_if_t<!HasCallOperator<absl::remove_cvref_t<T>>::value>> {\n    using type = StorageWholeWithoutCallOperator<T>;\n  };\n\n  template <typename T>\n  struct StorageWholeImpl<\n      T, std::enable_if_t<HasCallOperator<absl::remove_cvref_t<T>>::value>> {\n    using type = StorageWholeWithCallOperator<T>;\n  };\n\n  template <typename Arg, typename Enable = void>\n  struct StorageSubstrImpl;\n\n  template <typename Arg>\n  struct StorageSubstrImpl<Arg,\n                           std::enable_if_t<!HasCallOperator<\n                               absl::remove_cvref_t<TargetRefT<Arg>>>::value>> {\n    using type = StorageSubstrWithoutCallOperator<Arg>;\n  };\n\n  template <typename Arg>\n  struct StorageSubstrImpl<\n      Arg, std::enable_if_t<\n               HasCallOperator<absl::remove_cvref_t<TargetRefT<Arg>>>::value>> {\n    using type = StorageSubstrWithCallOperator<TargetRefT<Arg>>;\n  };\n\n public:\n  // The type of the `storage` parameter for the constructor and\n  // `ExternalRef::From()` which take an external object convertible\n  // to `BytesRef`.\n  template <typename T>\n  using StorageWhole = typename StorageWholeImpl<T>::type;\n\n  // The type of the `storage` parameter for the constructor and\n  // `ExternalRef::From()` which take an external object and its substring.\n  template <typename Arg>\n  using StorageSubstr = typename StorageSubstrImpl<Arg>::type;\n\n  // Constructs an `ExternalRef` from an external object or its `Initializer`.\n  // See class comments for expectations on the external object.\n  //\n  // The object must be convertible to `BytesRef`.\n  //\n  // `storage` must outlive usages of the returned `ExternalRef`.\n  template <typename Arg,\n            std::enable_if_t<SupportsExternalRefWhole<TargetRefT<Arg>>::value,\n                             int> = 0>\n  /*implicit*/ ExternalRef(Arg&& arg ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                           StorageWhole<TargetRefT<Arg>>&& storage\n                               ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : storage_(&storage) {\n    storage.Initialize(std::forward<Arg>(arg));\n  }\n\n  // Constructs an `ExternalRef` from an external object or its `Initializer`.\n  // See class comments for expectations on the external object.\n  //\n  // `substr` must be owned by the object if it gets created or moved.\n  //\n  // `storage` must outlive usages of the returned `ExternalRef`.\n  //\n  // The object is not created if an initializer is passed rather than an\n  // already constructed object, the object type does not use the call operator,\n  // and only `absl::string_view` turns out to be needed. Hence `StorageSubstr`\n  // is parameterized by `Arg&&` rather than `TargetRefT<Arg>`, so that it can\n  // keep the original initializer.\n  template <typename Arg,\n            std::enable_if_t<SupportsExternalRefSubstr<TargetRefT<Arg>>::value,\n                             int> = 0>\n  explicit ExternalRef(\n      Arg&& arg ABSL_ATTRIBUTE_LIFETIME_BOUND, absl::string_view substr,\n      StorageSubstr<Arg&&>&& storage ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : storage_(&storage) {\n    storage.Initialize(std::forward<Arg>(arg), substr);\n  }\n\n  ExternalRef(ExternalRef&& that) = default;\n  ExternalRef& operator=(ExternalRef&&) = delete;\n\n  // Like `ExternalRef` constructor, but `RiegeliSupportsExternalRef()` or\n  // `RiegeliSupportsExternalRefWhole()` is not needed. The caller is\n  // responsible for using an appropriate type of the external object.\n  template <typename Arg,\n            std::enable_if_t<std::is_convertible_v<TargetRefT<Arg>, BytesRef>,\n                             int> = 0>\n  static ExternalRef From(Arg&& arg ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                          StorageWhole<TargetRefT<Arg>>&& storage\n                              ABSL_ATTRIBUTE_LIFETIME_BOUND = {}) {\n    storage.Initialize(std::forward<Arg>(arg));\n    return ExternalRef(&storage);\n  }\n\n  // Like `ExternalRef` constructor, but `RiegeliSupportsExternalRef()` is not\n  // needed. The caller is responsible for using an appropriate type of the\n  // external object.\n  //\n  // The object is not created if an initializer is passed rather than an\n  // already constructed object, the object type does not use the call operator,\n  // and only `absl::string_view` turns out to be needed. Hence `StorageSubstr`\n  // is parameterized by `Arg&&` rather than `TargetRefT<Arg>`, so that it can\n  // keep the original initializer.\n  template <typename Arg>\n  static ExternalRef From(\n      Arg&& arg ABSL_ATTRIBUTE_LIFETIME_BOUND, absl::string_view substr,\n      StorageSubstr<Arg&&>&& storage ABSL_ATTRIBUTE_LIFETIME_BOUND = {}) {\n    storage.Initialize(std::forward<Arg>(arg), substr);\n    return ExternalRef(&storage);\n  }\n\n  // Returns `true` if the data size is 0.\n  bool empty() const { return storage_->empty(); }\n\n  // Returns the data pointer.\n  const char* data() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return storage_->data();\n  }\n\n  // Returns the data size.\n  size_t size() const { return storage_->size(); }\n\n  // Returns the data as `absl::string_view`.\n  //\n  // This `ExternalRef` must outlive usages of the returned `absl::string_view`.\n  /*implicit*/ operator absl::string_view() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return storage_->substr();\n  }\n\n  // The data can be converted to `Chain` using:\n  //  * `Chain::Chain(ExternalRef)`\n  //  * `Chain::Reset(ExternalRef)` or `riegeli::Reset(Chain&, ExternalRef)`\n  //  * `Chain::Append(ExternalRef)`\n  //  * `Chain::Prepend(ExternalRef)`\n\n  // Converts the data to `absl::Cord`.\n  explicit operator absl::Cord() && {\n    absl::Cord result;\n    // Destruction of a just default-constructed `absl::Cord` can be optimized\n    // out. Construction in place is more efficient than assignment.\n    result.~Cord();\n    std::move(*storage_).ToCord(\n        cord_internal::kMaxBytesToCopyToEmptyCord, &result,\n        [](void* context, absl::string_view data) {\n          new (context) absl::Cord(cord_internal::MakeBlockyCord(data));\n        },\n        [](void* context, absl::Cord data) {\n          new (context) absl::Cord(std::move(data));\n        });\n    return result;\n  }\n\n  // Supports `riegeli::Reset(absl::Cord&, ExternalRef)`.\n  friend void RiegeliReset(absl::Cord& dest, ExternalRef src) {\n    std::move(src).AssignTo(dest);\n  }\n\n  // Appends the data to `dest`.\n  void AppendTo(absl::Cord& dest) && {\n    std::move(*storage_).ToCord(\n        cord_internal::MaxBytesToCopyToCord(dest), &dest,\n        [](void* context, absl::string_view data) {\n          cord_internal::AppendToBlockyCord(data,\n                                            *static_cast<absl::Cord*>(context));\n        },\n        [](void* context, absl::Cord data) {\n          static_cast<absl::Cord*>(context)->Append(std::move(data));\n        });\n  }\n\n  // Prepends the data to `dest`.\n  void PrependTo(absl::Cord& dest) && {\n    std::move(*storage_).ToCord(\n        cord_internal::MaxBytesToCopyToCord(dest), &dest,\n        [](void* context, absl::string_view data) {\n          cord_internal::PrependToBlockyCord(\n              data, *static_cast<absl::Cord*>(context));\n        },\n        [](void* context, absl::Cord data) {\n          static_cast<absl::Cord*>(context)->Prepend(std::move(data));\n        });\n  }\n\n  // Returns a type-erased external object with its deleter and data.\n  explicit operator ExternalData() && {\n    ExternalData result{ExternalStorage(nullptr, nullptr), absl::string_view()};\n    // Destruction of just constructed `ExternalData` can be optimized out.\n    // Construction in place is more efficient than assignment.\n    result.~ExternalData();\n    std::move(*storage_).ToExternalData(\n        &result, [](void* context, ExternalData data) {\n          new (context) ExternalData(std::move(data));\n        });\n    return result;\n  }\n\n private:\n  // For `InitializeTo()`, `AssignTo()`, `AppendTo()`, and `PrependTo()`.\n  friend class Chain;\n\n  explicit ExternalRef(StorageBase* storage) : storage_(storage) {}\n\n  // Assigns the data to `dest` which is expected to be just\n  // default-constructed.\n  void InitializeTo(Chain& dest) && {\n    // Destruction of a just default-constructed `Chain` can be optimized out.\n    // Construction in place is more efficient than assignment.\n    dest.~Chain();\n    std::move(*storage_).ToChainBlock(\n        Chain::kMaxBytesToCopyToEmpty, &dest,\n        [](void* context, absl::string_view data) {\n          new (context) Chain(data);\n        },\n        [](void* context, Chain::Block data) {\n          new (context) Chain(std::move(data));\n        });\n  }\n\n  // Assigns the data to `dest`.\n  void AssignTo(Chain& dest) && {\n    std::move(*storage_).ToChainBlock(\n        Chain::kMaxBytesToCopyToEmpty, &dest,\n        [](void* context, absl::string_view data) {\n          static_cast<Chain*>(context)->Reset(data);\n        },\n        [](void* context, Chain::Block data) {\n          static_cast<Chain*>(context)->Reset(std::move(data));\n        });\n  }\n\n  // Assigns the data to `dest`.\n  void AssignTo(absl::Cord& dest) && {\n    std::move(*storage_).ToCord(\n        cord_internal::kMaxBytesToCopyToEmptyCord, &dest,\n        [](void* context, absl::string_view data) {\n          cord_internal::AssignToBlockyCord(data,\n                                            *static_cast<absl::Cord*>(context));\n        },\n        [](void* context, absl::Cord data) {\n          *static_cast<absl::Cord*>(context) = std::move(data);\n        });\n  }\n\n  // Appends the data to `dest`.\n  void AppendTo(Chain& dest) && {\n    std::move(*storage_).ToChainBlock(\n        dest.MaxBytesToCopy(), &dest,\n        [](void* context, absl::string_view data) {\n          static_cast<Chain*>(context)->Append(data);\n        },\n        [](void* context, Chain::Block data) {\n          static_cast<Chain*>(context)->Append(std::move(data));\n        });\n  }\n  void AppendTo(Chain& dest, Chain::Options options) && {\n    ChainWithOptions chain_with_options = {&dest, options};\n    std::move(*storage_).ToChainBlock(\n        dest.MaxBytesToCopy(options), &chain_with_options,\n        [](void* context, absl::string_view data) {\n          static_cast<ChainWithOptions*>(context)->dest->Append(\n              data, static_cast<ChainWithOptions*>(context)->options);\n        },\n        [](void* context, Chain::Block data) {\n          static_cast<ChainWithOptions*>(context)->dest->Append(\n              std::move(data),\n              static_cast<ChainWithOptions*>(context)->options);\n        });\n  }\n\n  // Prepends the data to `dest`.\n  void PrependTo(Chain& dest) && {\n    std::move(*storage_).ToChainBlock(\n        dest.MaxBytesToCopy(), &dest,\n        [](void* context, absl::string_view data) {\n          static_cast<Chain*>(context)->Prepend(data);\n        },\n        [](void* context, Chain::Block data) {\n          static_cast<Chain*>(context)->Prepend(std::move(data));\n        });\n  }\n  void PrependTo(Chain& dest, Chain::Options options) && {\n    ChainWithOptions chain_with_options = {&dest, options};\n    std::move(*storage_).ToChainBlock(\n        dest.MaxBytesToCopy(options), &chain_with_options,\n        [](void* context, absl::string_view data) {\n          static_cast<ChainWithOptions*>(context)->dest->Prepend(\n              data, static_cast<ChainWithOptions*>(context)->options);\n        },\n        [](void* context, Chain::Block data) {\n          static_cast<ChainWithOptions*>(context)->dest->Prepend(\n              std::move(data),\n              static_cast<ChainWithOptions*>(context)->options);\n        });\n  }\n\n  struct ChainWithOptions {\n    Chain* dest;\n    Chain::Options options;\n  };\n\n  StorageBase* storage_;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_EXTERNAL_REF_BASE_H_\n"
  },
  {
    "path": "riegeli/base/external_ref_support.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_EXTERNAL_REF_SUPPORT_H_\n#define RIEGELI_BASE_EXTERNAL_REF_SUPPORT_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <string>\n#include <type_traits>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/external_data.h\"\n\nnamespace riegeli {\n\n// Default implementation for `ExternalRef` support.\ninline bool RiegeliExternalCopy(ABSL_ATTRIBUTE_UNUSED const void* self) {\n  return false;\n}\n\n// Indicates support for `ExternalRef(std::string&&)`.\nvoid RiegeliSupportsExternalRefWhole(std::string*);\n\n// Indicates support for:\n//  * `ExternalRef(std::vector<char>&&)`\n//  * `ExternalRef(std::vector<T>&&, substr)`\ntemplate <typename T>\nvoid RiegeliSupportsExternalRef(std::vector<T>*);\n\n// Indicates support for `ExternalRef(std::unique_ptr<T, Deleter>&&, substr)`.\ntemplate <typename T, typename Deleter>\nvoid RiegeliSupportsExternalRef(std::unique_ptr<T, Deleter>*);\n\ntemplate <typename T>\ninline ExternalStorage RiegeliToExternalStorage(std::unique_ptr<T>* self) {\n  return ExternalStorage(const_cast<std::remove_cv_t<T>*>(self->release()),\n                         [](void* ptr) { delete static_cast<T*>(ptr); });\n}\n\ntemplate <typename T>\ninline ExternalStorage RiegeliToExternalStorage(std::unique_ptr<T[]>* self) {\n  return ExternalStorage(const_cast<std::remove_cv_t<T>*>(self->release()),\n                         [](void* ptr) { delete[] static_cast<T*>(ptr); });\n}\n\n// Indicates support for:\n//  * `ExternalRef(const std::shared_ptr<T>&, substr)`\n//  * `ExternalRef(std::shared_ptr<T>&&, substr)`\ntemplate <typename T>\nvoid RiegeliSupportsExternalRef(const std::shared_ptr<T>*);\n\nnamespace external_ref_internal {\n\ntemplate <typename T>\nstruct PointerType {\n  using type = T*;\n};\ntemplate <typename T>\nstruct PointerType<T&> {\n  using type = const T*;\n};\ntemplate <typename T>\nstruct PointerType<T&&> {\n  using type = T*;\n};\n\ntemplate <typename T>\nusing PointerTypeT = typename PointerType<T>::type;\n\ntemplate <typename T, typename Enable = void>\nstruct HasRiegeliSupportsExternalRefWhole : std::false_type {};\n\ntemplate <typename T>\nstruct HasRiegeliSupportsExternalRefWhole<\n    T, std::void_t<decltype(RiegeliSupportsExternalRefWhole(\n           std::declval<PointerTypeT<T>>()))>> : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct HasRiegeliSupportsExternalRef : std::false_type {};\n\ntemplate <typename T>\nstruct HasRiegeliSupportsExternalRef<\n    T, std::void_t<decltype(RiegeliSupportsExternalRef(\n           std::declval<PointerTypeT<T>>()))>> : std::true_type {};\n\n}  // namespace external_ref_internal\n\ntemplate <typename T>\nstruct SupportsExternalRefWhole\n    : std::conjunction<\n          std::disjunction<\n              external_ref_internal::HasRiegeliSupportsExternalRefWhole<T>,\n              external_ref_internal::HasRiegeliSupportsExternalRef<T>>,\n          std::is_convertible<const T&, BytesRef>> {};\n\ntemplate <typename T>\nstruct SupportsExternalRefSubstr\n    : external_ref_internal::HasRiegeliSupportsExternalRef<T> {};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_EXTERNAL_REF_SUPPORT_H_\n"
  },
  {
    "path": "riegeli/base/global.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_GLOBAL_H_\n#define RIEGELI_BASE_GLOBAL_H_\n\n#include <functional>\n#include <new>\n#include <type_traits>\n\n#include \"absl/base/nullability.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `Global<T>()` returns a const reference to a default-constructed object of\n// type `T`.\n//\n// All calls with the given `T` type return a reference to the same object.\n//\n// The object is created when `Global` is first called with the given `T` type,\n// and is never destroyed.\ntemplate <typename T,\n          std::enable_if_t<std::is_default_constructible_v<T>, int> = 0>\nconst T& Global();\n\n// `Global(construct)` returns a reference to an object returned by `construct`.\n//\n// The object is created when `Global` is first called with the given\n// `construct` type, and is never destroyed.\n//\n// If `T` is not const-qualified, this is recommended only when the object is\n// thread-safe, or when it will be accessed only in a thread-safe way despite\n// its non-const type.\n//\n// The `construct` type should be a lambda with no captures. This restriction is\n// a safeguard against making the object dependent on local state, which would\n// be misleadingly ignored for subsequent calls. Since distinct lambdas have\n// distinct types, distinct call sites with lambdas return references to\n// distinct objects.\ntemplate <typename Construct,\n          std::enable_if_t<std::conjunction_v<std::is_empty<Construct>,\n                                              std::is_invocable<Construct>>,\n                           int> = 0>\nstd::decay_t<std::invoke_result_t<Construct>>& Global(Construct construct);\n\n// `Global(construct, initialize)` returns a reference to an object returned by\n// `construct`. After construction, `initialize` is called on the reference.\n//\n// The object is created when `Global` is first called with the given\n// `construct` and `initialize` types, and is never destroyed.\n//\n// If `T` is not const-qualified, this is recommended only when the object is\n// thread-safe, or when it will be accessed only in a thread-safe way despite\n// its non-const type.\n//\n// The `construct` and `initialize` types should be lambdas with no captures.\n// This restriction is a safeguard against making the object dependent on local\n// state, which would be misleadingly ignored for subsequent calls. Since\n// distinct lambdas have distinct types, distinct call sites with lambdas return\n// references to distinct objects.\ntemplate <\n    typename Construct, typename Initialize,\n    std::enable_if_t<\n        std::conjunction_v<\n            std::is_empty<Construct>, std::is_empty<Initialize>,\n            std::is_invocable<Initialize,\n                              std::decay_t<std::invoke_result_t<Construct>>&>>,\n        int> = 0>\nstd::decay_t<std::invoke_result_t<Construct>>& Global(Construct construct,\n                                                      Initialize initialize);\n\n// Implementation details follow.\n\nnamespace global_internal {\n\ntemplate <typename T>\nclass NoDestructor {\n public:\n  NoDestructor() { new (storage_) T(); }\n\n  template <typename Construct>\n  explicit NoDestructor(Construct construct) {\n    new (storage_) T(std::invoke(construct));\n  }\n\n  template <typename Construct, typename Initialize>\n  explicit NoDestructor(Construct construct, Initialize initialize) {\n    new (storage_) T(std::invoke(construct));\n    std::invoke(initialize, object());\n  }\n\n  NoDestructor(const NoDestructor&) = delete;\n  NoDestructor& operator=(const NoDestructor&) = delete;\n\n  T& object() { return *std::launder(reinterpret_cast<T*>(storage_)); }\n\n private:\n  alignas(T) char storage_[sizeof(T)];\n};\n\n}  // namespace global_internal\n\ntemplate <typename T, std::enable_if_t<std::is_default_constructible_v<T>, int>>\ninline const T& Global() {\n  static global_internal::NoDestructor<const T> kStorage;\n  return kStorage.object();\n}\n\ntemplate <typename Construct,\n          std::enable_if_t<std::conjunction_v<std::is_empty<Construct>,\n                                              std::is_invocable<Construct>>,\n                           int>>\ninline std::decay_t<std::invoke_result_t<Construct>>& Global(\n    Construct construct) {\n  static global_internal::NoDestructor<\n      std::decay_t<std::invoke_result_t<Construct>>>\n      kStorage(construct);\n  return kStorage.object();\n}\n\ntemplate <\n    typename Construct, typename Initialize,\n    std::enable_if_t<\n        std::conjunction_v<\n            std::is_empty<Construct>, std::is_empty<Initialize>,\n            std::is_invocable<Initialize,\n                              std::decay_t<std::invoke_result_t<Construct>>&>>,\n        int>>\ninline std::decay_t<std::invoke_result_t<Construct>>& Global(\n    Construct construct, Initialize initialize) {\n  static global_internal::NoDestructor<\n      std::decay_t<std::invoke_result_t<Construct>>>\n      kStorage(construct, initialize);\n  return kStorage.object();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_GLOBAL_H_\n"
  },
  {
    "path": "riegeli/base/hybrid_direct_common.h",
    "content": "// Copyright 2026 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_HYBRID_DIRECT_COMMON_H_\n#define RIEGELI_BASE_HYBRID_DIRECT_COMMON_H_\n\n// IWYU pragma: private, include \"riegeli/base/hybrid_direct_map.h\"\n// IWYU pragma: private, include \"riegeli/base/hybrid_direct_set.h\"\n\n#include <stddef.h>\n\n#include <type_traits>\n\n#include \"absl/base/nullability.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nnamespace hybrid_direct_internal {\n\ntemplate <typename Key, typename Enable = void>\nstruct HasRiegeliHybridDirectToRawKey : std::false_type {};\n\ntemplate <typename Key>\nstruct HasRiegeliHybridDirectToRawKey<\n    Key, std::enable_if_t<std::is_unsigned_v<\n             decltype(RiegeliHybridDirectToRawKey(std::declval<Key>()))>>>\n    : std::true_type {};\n\ntemplate <typename Key, typename Enable = void>\nstruct HasRiegeliHybridDirectFromRawKey : std::false_type {};\n\ntemplate <typename Key>\nstruct HasRiegeliHybridDirectFromRawKey<\n    Key, std::enable_if_t<std::is_convertible_v<\n             decltype(RiegeliHybridDirectFromRawKey(\n                 RiegeliHybridDirectToRawKey(std::declval<Key>()),\n                 static_cast<Key*>(nullptr))),\n             Key>>> : std::true_type {};\n\n}  // namespace hybrid_direct_internal\n\n// The default `Traits` parameter for `HybridDirectMap` and `HybridDirectSet`,\n// which specifies a mapping of keys to an unsigned integer type.\n//\n// Key types supported by default are integral types, enum types, and types\n// supporting `RiegeliHybridDirectToRawKey()` as below. The latter takes\n// precedence.\n//\n// To override `HybridDirectTraits` for a type `Key`, define a free function\n// `friend RawKey RiegeliHybridDirectToRawKey(Key key)` as a friend of `Key`\n// inside class definition or in the same namespace as `Key`, so that it can be\n// found via ADL. Different `Key` values must yield different `RawKey` values.\n//\n// Optionally, define also a free function\n// `friend Key RiegeliHybridDirectFromRawKey(RawKey raw_key, Key*)`.\n// This is needed only for iterators.\n//\n// The second argument of `RiegeliHybridDirectFromRawKey()` is always a null\n// pointer, used to choose the right overload based on the type.\n//\n// `expected_min_key` is the expected lower bound of keys. Keys smaller than\n// that are never put in the array. `expected_min_key` has a type which supports\n// `static_cast<RawKey>(expected_min_key)`.\ntemplate <typename Key, auto expected_min_key = 0, typename Enable = void>\nstruct HybridDirectTraits;\n\ntemplate <typename Key, auto expected_min_key>\nstruct HybridDirectTraits<\n    Key, expected_min_key,\n    std::enable_if_t<\n        hybrid_direct_internal::HasRiegeliHybridDirectToRawKey<Key>::value>> {\n private:\n  using RawKey = decltype(RiegeliHybridDirectToRawKey(std::declval<Key>()));\n\n public:\n  static RawKey ToRawKey(Key key) {\n    // Wrap-around is not an error.\n    return static_cast<RawKey>(RiegeliHybridDirectToRawKey(key) -\n                               static_cast<RawKey>(expected_min_key));\n  }\n\n  template <\n      typename DependentKey = Key,\n      std::enable_if_t<hybrid_direct_internal::HasRiegeliHybridDirectFromRawKey<\n                           DependentKey>::value,\n                       int> = 0>\n  static Key FromRawKey(RawKey raw_key) {\n    // Wrap-around is not an error.\n    return RiegeliHybridDirectFromRawKey(\n        static_cast<RawKey>(raw_key + static_cast<RawKey>(expected_min_key)),\n        static_cast<Key*>(nullptr));\n  }\n};\n\ntemplate <typename Key, auto expected_min_key>\nstruct HybridDirectTraits<\n    Key, expected_min_key,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<\n            hybrid_direct_internal::HasRiegeliHybridDirectToRawKey<Key>>,\n        std::is_integral<Key>>>> {\n private:\n  using RawKey = std::make_unsigned_t<Key>;\n\n public:\n  static RawKey ToRawKey(Key key) {\n    // Wrap-around is not an error.\n    return static_cast<RawKey>(static_cast<RawKey>(key) -\n                               static_cast<RawKey>(expected_min_key));\n  }\n\n  static Key FromRawKey(RawKey raw_key) {\n    // Wrap-around is not an error.\n    return static_cast<Key>(\n        static_cast<RawKey>(raw_key + static_cast<RawKey>(expected_min_key)));\n  }\n};\n\ntemplate <typename Key, auto expected_min_key>\nstruct HybridDirectTraits<\n    Key, expected_min_key,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<\n            hybrid_direct_internal::HasRiegeliHybridDirectToRawKey<Key>>,\n        std::is_enum<Key>>>> {\n private:\n  using RawKey = std::make_unsigned_t<std::underlying_type_t<Key>>;\n\n public:\n  static RawKey ToRawKey(Key key) {\n    // Wrap-around is not an error.\n    return static_cast<RawKey>(static_cast<RawKey>(key) -\n                               static_cast<RawKey>(expected_min_key));\n  }\n\n  static Key FromRawKey(RawKey raw_key) {\n    // Wrap-around is not an error.\n    return static_cast<Key>(\n        static_cast<RawKey>(raw_key + static_cast<RawKey>(expected_min_key)));\n  }\n};\n\n// The default `direct_capacity` parameter for `HybridDirectMap` and\n// `HybridDirectSet` building.\nconstexpr size_t kHybridDirectDefaultDirectCapacity = 128;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_HYBRID_DIRECT_COMMON_H_\n"
  },
  {
    "path": "riegeli/base/hybrid_direct_internal.h",
    "content": "// Copyright 2026 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_HYBRID_DIRECT_INTERNAL_H_\n#define RIEGELI_BASE_HYBRID_DIRECT_INTERNAL_H_\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <new>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compare.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli::hybrid_direct_internal {\n\n// Wraps a `T` which is constructed explicitly later, rather than when\n// `DelayedConstructor<T>` is constructed.\n//\n// In contrast to `std::optional<T>`, this avoids the overhead of tracking\n// whether the object has been constructed, at the cost of passing this\n// responsibility to the caller.\n//\n// Either `emplace()` or `Abandon()` must be called exactly once.\n// If `emplace()` is called, the regular destructor should be called later.\n// If `Abandon()` is called, the regular destructor must not be called.\ntemplate <typename T>\nclass DelayedConstructor {\n public:\n  // Does not construct the wrapped object yet.\n  DelayedConstructor() noexcept {}\n\n  DelayedConstructor(const DelayedConstructor&) = delete;\n  DelayedConstructor& operator=(const DelayedConstructor&) = delete;\n\n  // Destroys the wrapped object. It must have been constructed.\n  ~DelayedConstructor() { value_.~T(); }\n\n  // Constructs the wrapped object. It must not have been constructed yet.\n  template <typename... Args,\n            std::enable_if_t<std::is_constructible_v<T, Args&&...>, int> = 0>\n  T& emplace(Args&&... args) ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    new (&value_) T(std::forward<Args>(args)...);\n    return value_;\n  }\n\n  // Destroys the `DelayedConstructor`. The wrapped object must not have been\n  // constructed. This is needed for `SizedArray`.\n  void Abandon() {}\n\n  // Returns the wrapped object. It must have been constructed.\n  T& operator*() ABSL_ATTRIBUTE_LIFETIME_BOUND { return value_; }\n  const T& operator*() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return value_; }\n\n private:\n  union {\n    T value_;\n  };\n};\n\n// A deleter for `SizedArray<T, supports_abandon>`.\n//\n// If `supports_abandon` is true, truncation with `AbandonAfter()` is supported,\n// at the cost of some overhead.\n//\n// A moved-from `SizedDeleter` reports a positive size. This helps to trigger\n// a null pointer dereference when a moved-from `SizedArray` is used.\ntemplate <typename T, bool supports_abandon = false>\nclass SizedDeleter {\n public:\n  static size_t max_size() { return kSizeMask / sizeof(T); }\n\n  SizedDeleter() = default;\n\n  explicit SizedDeleter(size_t size) : size_(size) {}\n\n  SizedDeleter(SizedDeleter&& that) noexcept\n      : size_(std::exchange(that.size_, kPoisonedSize)) {}\n\n  SizedDeleter& operator=(SizedDeleter&& that) noexcept {\n    size_ = std::exchange(that.size_, kPoisonedSize);\n    return *this;\n  }\n\n  void operator()(T* ptr) const {\n    for (T* iter = ptr + (size_ & kSizeMask); iter != ptr;) {\n      --iter;\n      iter->~T();\n    }\n    if (ABSL_PREDICT_FALSE((size_ & kOverallocated) != 0)) {\n      // The allocated size is not tracked and sized delete must not be used.\n      if constexpr (alignof(T) <= __STDCPP_DEFAULT_NEW_ALIGNMENT__) {\n        operator delete[](ptr);\n      } else {\n        operator delete[](ptr, std::align_val_t(alignof(T)));\n      }\n      return;\n    }\n    if constexpr (alignof(T) <= __STDCPP_DEFAULT_NEW_ALIGNMENT__) {\n      operator delete[](ptr, size_ * sizeof(T));\n    } else {\n      operator delete[](ptr, size_ * sizeof(T), std::align_val_t(alignof(T)));\n    }\n  }\n\n  size_t size() const { return size_ & kSizeMask; }\n\n  // If the pointer associated with this deleter is `nullptr`, returns `true`\n  // when this deleter is moved-from. Otherwise the result is meaningless.\n  bool IsMovedFromIfNull() const { return size_ == kPoisonedSize; }\n\n  // Reduces the size to `new_size`. Calls `Abandon()` on elements being\n  // abandoned. The regular destructor will not be called for them.\n  //\n  // `SizedDeleter` is optimized for the case when `AbandonAfter()` is never\n  // called with a changed size.\n  template <bool dependent_supports_abandon = supports_abandon,\n            std::enable_if_t<dependent_supports_abandon, int> = 0>\n  void AbandonAfter(T* ptr, size_t new_size) {\n    RIEGELI_ASSERT_LE(new_size, size_ & kSizeMask)\n        << \"Failed precondition of SizedDeleter::AbandonAfter(): \"\n           \"array size overflow\";\n    if (ABSL_PREDICT_TRUE(new_size == size_)) return;\n    T* const new_end = ptr + new_size;\n    for (T* iter = ptr + (size_ & kSizeMask); iter != new_end;) {\n      --iter;\n      iter->Abandon();\n    }\n    size_ = new_size | kOverallocated;\n  }\n\n private:\n  // A moved-from `SizedDeleter` has `size_ == kPoisonedSize`. In debug mode\n  // this asserts against using a moved-from object. In non-debug mode, if the\n  // key is not too large, then this triggers a null pointer dereference with an\n  // offset up to 1MB, which is assumed to reliably crash.\n  static constexpr size_t kPoisonedSize = (size_t{1} << 20) / sizeof(T);\n\n  // If `supports_abandon` is true, `size_` tracks the current size and whether\n  // the original size has been reduced with `AbandonAfter()`. In that case\n  // sized delete is not called because the allocated size is not tracked.\n  static constexpr size_t kSizeMask =\n      std::numeric_limits<size_t>::max() >> (supports_abandon ? 1 : 0);\n  static constexpr size_t kOverallocated = ~kSizeMask;\n\n  // The number of elements. If marked with `kOverallocated`, the allocated size\n  // is not tracked and sized delete must not be used.\n  size_t size_ = 0;\n};\n\n// Like `std::unique_ptr<T[]>`, but the size is stored in the deleter.\n// It is available as `get_deleter().size()` and used for sized delete.\n//\n// If `supports_abandon` is true, truncation with `get_deleter().AbandonAfter()`\n// is supported, at the cost of some overhead.\n//\n// A moved-from `SizedArray` is `nullptr` but reports a positive size. This\n// helps to trigger a null pointer dereference when a moved-from `SizedArray`\n// is used.\ntemplate <typename T, bool supports_abandon = false>\nusing SizedArray = std::unique_ptr<T[], SizedDeleter<T, supports_abandon>>;\n\n// Like `std::make_unique<T[]>(size)`.\n//\n// If `supports_abandon` is true, truncation with `get_deleter().AbandonAfter()`\n// is supported, at the cost of some overhead.\ntemplate <typename T, bool supports_abandon = false>\ninline SizedArray<T, supports_abandon> MakeSizedArray(size_t size) {\n  T* ptr;\n  if constexpr (alignof(T) <= __STDCPP_DEFAULT_NEW_ALIGNMENT__) {\n    ptr = static_cast<T*>(operator new[](size * sizeof(T)));\n  } else {\n    ptr = static_cast<T*>(operator new[](size * sizeof(T),\n                                         std::align_val_t(alignof(T))));\n  }\n  T* const end = ptr + size;\n  for (T* iter = ptr; iter != end; ++iter) {\n    new (iter) T();\n  }\n  return SizedArray<T, supports_abandon>(\n      ptr, SizedDeleter<T, supports_abandon>(size));\n}\n\n// Like `std::make_unique_for_overwrite<T[]>(size)`.\n//\n// If `supports_abandon` is true, truncation with `get_deleter().AbandonAfter()`\n// is supported, at the cost of some overhead.\ntemplate <typename T, bool supports_abandon = false>\ninline SizedArray<T, supports_abandon> MakeSizedArrayForOverwrite(size_t size) {\n  T* ptr;\n  if constexpr (alignof(T) <= __STDCPP_DEFAULT_NEW_ALIGNMENT__) {\n    ptr = static_cast<T*>(operator new[](size * sizeof(T)));\n  } else {\n    ptr = static_cast<T*>(operator new[](size * sizeof(T),\n                                         std::align_val_t(alignof(T))));\n  }\n  T* const end = ptr + size;\n  for (T* iter = ptr; iter != end; ++iter) {\n    new (iter) T;\n  }\n  return SizedArray<T, supports_abandon>(\n      ptr, SizedDeleter<T, supports_abandon>(size));\n}\n\n// Performs an assignment, but the behavior is undefined if the old value of the\n// destination is not null. This allows the compiler skip generating the code\n// which deletes the old value.\n//\n// This is meant for initializing member variables of smart pointer types in\n// functions where the compiler cannot determine itself that the old value is\n// always null.\ntemplate <typename Dest, typename Src>\ninline void AssignToAssumedNull(Dest& dest, Src&& src) {\n  RIEGELI_ASSUME_EQ(dest, nullptr)\n      << \"Failed precondition of AssignToAssumedNull(): \"\n         \"old value of destination is not null\";\n  dest = std::forward<Src>(src);\n}\n\n// An iterator over a sequence of consecutive indices. Does not support the full\n// iterator API, only what is needed by `HybridDirectMap` and `HybridDirectSet`.\ntemplate <typename Index>\nclass IndexIterator : public WithEqual<IndexIterator<Index>> {\n public:\n  explicit IndexIterator(Index index) : index_(index) {}\n\n  IndexIterator(const IndexIterator&) = default;\n  IndexIterator& operator=(const IndexIterator&) = default;\n\n  Index operator*() const { return index_; }\n  IndexIterator& operator++() {\n    ++index_;\n    return *this;\n  }\n\n  friend bool operator==(const IndexIterator& a, const IndexIterator& b) {\n    return a.index_ == b.index_;\n  }\n\n private:\n  Index index_;\n};\n\n}  // namespace riegeli::hybrid_direct_internal\n\n#endif  // RIEGELI_BASE_HYBRID_DIRECT_INTERNAL_H_\n"
  },
  {
    "path": "riegeli/base/hybrid_direct_map.h",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_HYBRID_DIRECT_MAP_H_\n#define RIEGELI_BASE_HYBRID_DIRECT_MAP_H_\n\n#include <stddef.h>\n\n#include <functional>\n#include <initializer_list>\n#include <iterator>\n#include <limits>\n#include <memory>\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/container/flat_hash_map.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/debug.h\"\n#include \"riegeli/base/hybrid_direct_common.h\"  // IWYU pragma: export\n#include \"riegeli/base/hybrid_direct_internal.h\"\n#include \"riegeli/base/invoker.h\"\n#include \"riegeli/base/iterable.h\"\n#include \"riegeli/base/type_traits.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nnamespace hybrid_direct_internal {\n\n// Part of `HybridDirectMap` excluding constructors and assignment. This is\n// separated to make copy and move constructors and assignment available\n// conditionally.\ntemplate <typename Key, typename Value, typename Traits>\nclass HybridDirectMapImpl {\n private:\n  template <bool is_const>\n  class IteratorImpl;\n\n public:\n  using key_type = Key;\n  using mapped_type = Value;\n  using value_type = std::pair<const Key, Value>;\n  using reference = ReferencePair<const Key, Value&>;\n  using const_reference = ReferencePair<const Key, const Value&>;\n  using pointer = ArrowProxy<reference>;\n  using const_pointer = ArrowProxy<const_reference>;\n  using iterator = IteratorImpl<false>;\n  using const_iterator = IteratorImpl<true>;\n  using size_type = size_t;\n  using difference_type = ptrdiff_t;\n\n  static size_t max_size();\n\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n\n  // Returns a pointer to the value associated with `key`, or `nullptr` if `key`\n  // is absent.\n  //\n  // This can be a bit faster than `find()`.\n  Value* absl_nullable FindOrNull(Key key) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const Value* absl_nullable FindOrNull(Key key) const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns a reference to the value associated with `key`, or a reference to\n  // `default_value` if `key` is absent.\n  const Value& FindOrDefault(\n      Key key, const Value& default_value ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  iterator find(Key key) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator find(Key key) const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  bool contains(Key key) const;\n\n  Value& at(Key key) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const Value& at(Key key) const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  bool empty() const {\n    return direct_values_.get_deleter().size() == 0 &&\n           ABSL_PREDICT_TRUE(slow_map_ == nullptr);\n  }\n  size_t size() const;\n\n  iterator begin() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator begin() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator cbegin() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return begin();\n  }\n  iterator end() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator end() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator cend() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return end(); }\n\n protected:\n  HybridDirectMapImpl() = default;\n\n  HybridDirectMapImpl(const HybridDirectMapImpl& that) noexcept;\n  HybridDirectMapImpl& operator=(const HybridDirectMapImpl& that) noexcept;\n\n  HybridDirectMapImpl(HybridDirectMapImpl&& that) = default;\n  HybridDirectMapImpl& operator=(HybridDirectMapImpl&& that) = default;\n\n  template <typename Src, typename KeyProjection, typename ValueProjection>\n  void Initialize(Src&& src, const KeyProjection& key_projection,\n                  const ValueProjection& value_projection,\n                  size_t direct_capacity);\n\n  template <typename Index, typename KeyProjection, typename ValueProjection>\n  void InitializeByIndex(Index size, const KeyProjection& key_projection,\n                         const ValueProjection& value_projection,\n                         size_t direct_capacity);\n\n  static bool Equal(const HybridDirectMapImpl& a, const HybridDirectMapImpl& b);\n\n private:\n  using RawKey = std::decay_t<decltype(Traits::ToRawKey(std::declval<Key>()))>;\n  static_assert(std::is_unsigned_v<RawKey>);\n\n  using DirectValues =\n      SizedArray<DelayedConstructor<Value>, /*supports_abandon=*/true>;\n  using DirectMap = SizedArray<Value* absl_nullable>;\n  using SlowMap = absl::flat_hash_map<RawKey, Value>;\n\n  static constexpr int kInverseMinLoadFactor = 4;  // 25%.\n\n  template <typename Src, typename Iterator, typename KeyProjection,\n            typename ValueProjection>\n  void Optimize(Iterator first, Iterator last, size_t size,\n                const KeyProjection& key_projection,\n                const ValueProjection& value_projection,\n                size_t direct_capacity);\n\n  absl_nullable DirectValues CopyDirectValues() const;\n  absl_nullable DirectMap\n  CopyDirectMap(DelayedConstructor<Value>* absl_nullable dest_values) const;\n  absl_nullable std::unique_ptr<SlowMap> CopySlowMap() const;\n\n  ABSL_ATTRIBUTE_NORETURN static void KeyNotFound(Key key);\n\n  size_t FirstRawKey() const;\n\n  size_t capacity() const {\n    return direct_map_.get_deleter().size() +\n           (slow_map_ == nullptr ? 0 : slow_map_->capacity());\n  }\n\n  // Stores values for `direct_map_`, in no particular order.\n  absl_nullable DirectValues direct_values_;\n  // Indexed by raw key below `direct_map_.get_deleter().size()`. Elements\n  // corresponding to present values point to elements of `direct_values_`.\n  // The remaining elements are `nullptr`.\n  absl_nullable DirectMap direct_map_;\n  // If not `nullptr`, stores the mapping for keys too large for `direct_map_`.\n  // Uses `std::unique_ptr` rather than `std::optional` to reduce memory usage\n  // in the common case when `slow_map_` is not used.\n  //\n  // Invariant: if `slow_map_ != nullptr` then `!slow_map_->empty()`.\n  absl_nullable std::unique_ptr<SlowMap> slow_map_;\n};\n\n}  // namespace hybrid_direct_internal\n\n// `HybridDirectMap` is a map optimized for keys being mostly small integers\n// or enums, especially if they are dense near zero. It supports only lookups\n// and iteration, but no incremental modification.\n//\n// It stores a part of the map covering some range of small keys in an array\n// of pointers to values, directly indexed by the key. The remaining keys are\n// stored in an `absl::flat_hash_map`.\n//\n// `Traits` specifies a mapping of keys to an unsigned integer type. It must\n// support at least the following static members:\n//\n// ```\n//   // Translates the key to a raw key, which is an unsigned integer type.\n//   // Different `Key` values must yield different `RawKey` values. Small raw\n//   // keys are put in the array.\n//   static RawKey ToRawKey(Key key);\n//\n//   // Translates the raw key back to a key.\n//   //\n//   // This is optional. Needed only for iterators.\n//   static Key FromRawKey(RawKey raw_key);\n// ```\n//\n// `direct_capacity`, if specified during building, is the intended capacity\n// of the array part. The actual capacity can be smaller if all keys fit\n// in the array, or larger if the array remains at least 25% full. Default:\n// `kHybridDirectDefaultDirectCapacity` (128).\n//\n// In the case of duplicate keys, the first value wins.\ntemplate <typename Key, typename Value,\n          typename Traits = HybridDirectTraits<Key>>\nclass HybridDirectMap\n    : public hybrid_direct_internal::HybridDirectMapImpl<Key, Value, Traits>,\n      public ConditionallyConstructible<std::is_copy_constructible_v<Value>,\n                                        true>,\n      public ConditionallyAssignable<std::is_copy_constructible_v<Value>, true>,\n      public WithEqual<HybridDirectMap<Key, Value, Traits>> {\n private:\n  template <typename Src, typename Enable = void>\n  struct HasCompatibleKeys : std::false_type {};\n  template <typename Src>\n  struct HasCompatibleKeys<\n      Src, std::enable_if_t<std::is_convertible_v<\n               decltype(std::declval<ElementTypeT<const Src&>>().first), Key>>>\n      : std::true_type {};\n\n  template <typename Src, typename Enable = void>\n  struct HasCompatibleValues : std::false_type {};\n  template <typename Src>\n  struct HasCompatibleValues<\n      Src, std::enable_if_t<std::is_convertible_v<\n               decltype(std::declval<ElementTypeT<Src>>().second), Value>>>\n      : std::true_type {};\n\n  template <typename Src, typename KeyProjection, typename Enable = void>\n  struct HasProjectableKeys : std::false_type {};\n  template <typename Src, typename KeyProjection>\n  struct HasProjectableKeys<\n      Src, KeyProjection,\n      std::enable_if_t<std::is_convertible_v<\n          std::invoke_result_t<const KeyProjection&, ElementTypeT<const Src&>>,\n          Key>>> : std::true_type {};\n\n  template <typename Src, typename ValueProjection, typename Enable = void>\n  struct HasProjectableValues : std::false_type {};\n  template <typename Src, typename ValueProjection>\n  struct HasProjectableValues<\n      Src, ValueProjection,\n      std::enable_if_t<std::is_convertible_v<\n          std::invoke_result_t<const ValueProjection&, ElementTypeT<Src>>,\n          Value>>> : std::true_type {};\n\n  template <typename Index, typename KeyProjection, typename Enable = void>\n  struct HasGeneratableKeys : std::false_type {};\n  template <typename Index, typename KeyProjection>\n  struct HasGeneratableKeys<\n      Index, KeyProjection,\n      std::enable_if_t<std::is_convertible_v<\n          std::invoke_result_t<const KeyProjection&, Index>, Key>>>\n      : std::true_type {};\n\n  template <typename Index, typename ValueProjection, typename Enable = void>\n  struct HasGeneratableValues : std::false_type {};\n  template <typename Index, typename ValueProjection>\n  struct HasGeneratableValues<\n      Index, ValueProjection,\n      std::enable_if_t<std::is_convertible_v<\n          std::invoke_result_t<const ValueProjection&, Index>, Value>>>\n      : std::true_type {};\n\n  template <typename Src>\n  struct DefaultKeyProjection {\n    Key operator()(ElementTypeT<const Src&> entry) const { return entry.first; }\n  };\n\n  template <typename Src>\n  struct DefaultValueProjection {\n    auto&& operator()(ElementTypeT<Src>&& entry) const {\n      return std::forward<ElementTypeT<Src>>(entry).second;\n    }\n  };\n\n public:\n  // Constructs an empty `HybridDirectMap`.\n  HybridDirectMap() = default;\n\n  // Builds `HybridDirectMap` from an iterable `src`. Moves values if `src` is\n  // an rvalue which owns its elements.\n  template <typename Src,\n            std::enable_if_t<\n                std::conjunction_v<\n                    NotSameRef<HybridDirectMap, Src>, IsForwardIterable<Src>,\n                    HasCompatibleKeys<Src>, HasCompatibleValues<Src>>,\n                int> = 0>\n  explicit HybridDirectMap(Src&& src) {\n    this->Initialize(std::forward<Src>(src), DefaultKeyProjection<Src>(),\n                     DefaultValueProjection<Src>(),\n                     kHybridDirectDefaultDirectCapacity);\n  }\n  template <typename Src,\n            std::enable_if_t<std::conjunction_v<IsForwardIterable<Src>,\n                                                HasCompatibleKeys<Src>,\n                                                HasCompatibleValues<Src>>,\n                             int> = 0>\n  explicit HybridDirectMap(Src&& src, size_t direct_capacity) {\n    this->Initialize(std::forward<Src>(src), DefaultKeyProjection<Src>(),\n                     DefaultValueProjection<Src>(), direct_capacity);\n  }\n\n  // Builds `HybridDirectMap` from an initializer list.\n  /*implicit*/ HybridDirectMap(\n      std::initializer_list<std::pair<Key, Value>> src,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    this->Initialize(src, DefaultKeyProjection<decltype(src)>(),\n                     DefaultValueProjection<decltype(src)>(), direct_capacity);\n  }\n\n  // Builds `HybridDirectMap` from an iterable `src`. Moves values if `src` is\n  // an rvalue which owns its elements.\n  //\n  // Keys and values are extracted using `key_projection()` and\n  // `value_projection()` rather than `.first` and `.second`. `key_projection()`\n  // may be called multiple times for each entry so it should be efficient.\n  // `value_projection()` is called once for each entry so it can be expensive.\n  template <\n      typename Src, typename KeyProjection = DefaultKeyProjection<Src>,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::negation<std::is_convertible<KeyProjection, size_t>>,\n              IsForwardIterable<Src>, HasProjectableKeys<Src, KeyProjection>,\n              HasCompatibleValues<Src>>,\n          int> = 0>\n  explicit HybridDirectMap(\n      Src&& src, const KeyProjection& key_projection,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    this->Initialize(std::forward<Src>(src), key_projection,\n                     DefaultValueProjection<Src>(), direct_capacity);\n  }\n  template <\n      typename Src, typename KeyProjection = DefaultKeyProjection<Src>,\n      typename ValueProjection = DefaultValueProjection<Src>,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::negation<std::is_convertible<KeyProjection, size_t>>,\n              std::negation<std::is_convertible<ValueProjection, size_t>>,\n              IsForwardIterable<Src>, HasProjectableKeys<Src, KeyProjection>,\n              HasProjectableValues<Src, ValueProjection>>,\n          int> = 0>\n  explicit HybridDirectMap(\n      Src&& src, const KeyProjection& key_projection,\n      const ValueProjection& value_projection,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    this->Initialize(std::forward<Src>(src), key_projection, value_projection,\n                     direct_capacity);\n  }\n\n  // Builds `HybridDirectMap` from keys and values computed by invoking\n  // `key_projection()` and `value_projection()` with indices from [0..`size`).\n  //\n  // `key_projection()` may be called multiple times for each index so it should\n  // be efficient. `value_projection()` is called once for each index so it can\n  // be expensive.\n  template <typename Index, typename KeyProjection, typename ValueProjection,\n            std::enable_if_t<\n                std::conjunction_v<\n                    std::is_integral<Index>,\n                    std::negation<std::is_convertible<KeyProjection, size_t>>,\n                    std::negation<std::is_convertible<ValueProjection, size_t>>,\n                    HasGeneratableKeys<Index, KeyProjection>,\n                    HasGeneratableValues<Index, ValueProjection>>,\n                int> = 0>\n  explicit HybridDirectMap(\n      Index size, const KeyProjection& key_projection,\n      const ValueProjection& value_projection,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    this->InitializeByIndex(size, key_projection, value_projection,\n                            direct_capacity);\n  }\n\n  HybridDirectMap(const HybridDirectMap& that) = default;\n  HybridDirectMap& operator=(const HybridDirectMap& that) = default;\n\n  HybridDirectMap(HybridDirectMap&& that) = default;\n  HybridDirectMap& operator=(HybridDirectMap&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `HybridDirectMap`.\n  using HybridDirectMap::HybridDirectMapImpl::Reset;\n  template <typename Src,\n            std::enable_if_t<std::conjunction_v<IsForwardIterable<Src>,\n                                                HasCompatibleKeys<Src>,\n                                                HasCompatibleValues<Src>>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      Src&& src, size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    this->Reset();\n    this->Initialize(std::forward<Src>(src), DefaultKeyProjection<Src>(),\n                     DefaultValueProjection<Src>(), direct_capacity);\n  }\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      std::initializer_list<std::pair<Key, Value>> src,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    this->Reset();\n    this->Initialize(src, DefaultKeyProjection<decltype(src)>(),\n                     DefaultValueProjection<decltype(src)>(), direct_capacity);\n  }\n  template <\n      typename Src, typename KeyProjection = DefaultKeyProjection<Src>,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::negation<std::is_convertible<KeyProjection, size_t>>,\n              IsForwardIterable<Src>, HasProjectableKeys<Src, KeyProjection>,\n              HasCompatibleValues<Src>>,\n          int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      Src&& src, const KeyProjection& key_projection,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    this->Reset();\n    this->Initialize(std::forward<Src>(src), key_projection,\n                     DefaultValueProjection<Src>(), direct_capacity);\n  }\n  template <\n      typename Src, typename KeyProjection = DefaultKeyProjection<Src>,\n      typename ValueProjection = DefaultValueProjection<Src>,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::negation<std::is_convertible<KeyProjection, size_t>>,\n              std::negation<std::is_convertible<ValueProjection, size_t>>,\n              IsForwardIterable<Src>, HasProjectableKeys<Src, KeyProjection>,\n              HasProjectableValues<Src, ValueProjection>>,\n          int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      Src&& src, const KeyProjection& key_projection,\n      const ValueProjection& value_projection,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    this->Reset();\n    this->Initialize(std::forward<Src>(src), key_projection, value_projection,\n                     direct_capacity);\n  }\n  template <typename Index, typename KeyProjection, typename ValueProjection,\n            std::enable_if_t<\n                std::conjunction_v<\n                    std::is_integral<Index>,\n                    std::negation<std::is_convertible<KeyProjection, size_t>>,\n                    std::negation<std::is_convertible<ValueProjection, size_t>>,\n                    HasGeneratableKeys<Index, KeyProjection>,\n                    HasGeneratableValues<Index, ValueProjection>>,\n                int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      Index size, const KeyProjection& key_projection,\n      const ValueProjection& value_projection,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    this->Reset();\n    this->InitializeByIndex(size, key_projection, value_projection,\n                            direct_capacity);\n  }\n\n  friend bool operator==(const HybridDirectMap& a, const HybridDirectMap& b) {\n    return HybridDirectMap::HybridDirectMapImpl::Equal(a, b);\n  }\n};\n\nnamespace hybrid_direct_internal {\n\ntemplate <typename Key, typename Value, typename Traits>\ntemplate <bool is_const>\nclass HybridDirectMapImpl<Key, Value, Traits>::IteratorImpl\n    : public WithEqual<IteratorImpl<is_const>> {\n public:\n  using iterator_concept = std::forward_iterator_tag;\n  // `iterator_category` is only `std::input_iterator_tag` because the\n  // `LegacyForwardIterator` requirement and above require `reference` to be\n  // a true reference type.\n  using iterator_category = std::input_iterator_tag;\n  using value_type = std::pair<const Key, Value>;\n  using reference =\n      ReferencePair<const Key,\n                    std::conditional_t<is_const, const Value&, Value&>>;\n  using pointer = ArrowProxy<reference>;\n  using difference_type = ptrdiff_t;\n\n  IteratorImpl() = default;\n\n  // Conversion from `iterator` to `const_iterator`.\n  template <bool that_is_const,\n            std::enable_if_t<is_const && !that_is_const, int> = 0>\n  /*implicit*/ IteratorImpl(IteratorImpl<that_is_const> that) noexcept\n      : direct_map_end_(that.direct_map_end_),\n        direct_map_size_(that.direct_map_size_),\n        raw_key_complement_(that.raw_key_complement_),\n        slow_map_iter_(that.slow_map_iter_) {}\n\n  IteratorImpl(const IteratorImpl& that) = default;\n  IteratorImpl& operator=(const IteratorImpl& that) = default;\n\n  reference operator*() const {\n    if (ABSL_PREDICT_TRUE(raw_key_complement_ > 0)) {\n      return reference{Traits::FromRawKey(IntCast<RawKey>(direct_map_size_ -\n                                                          raw_key_complement_)),\n                       **(direct_map_end_ - raw_key_complement_)};\n    }\n    const auto iter = *slow_map_iter_;\n    return reference{Traits::FromRawKey(iter->first), iter->second};\n  }\n  pointer operator->() const { return pointer(**this); }\n  IteratorImpl& operator++() {\n    if (ABSL_PREDICT_TRUE(raw_key_complement_ > 0)) {\n      do {\n        --raw_key_complement_;\n        if (ABSL_PREDICT_FALSE(raw_key_complement_ == 0)) break;\n      } while (*(direct_map_end_ - raw_key_complement_) == nullptr);\n    } else {\n      ++*slow_map_iter_;\n    }\n    return *this;\n  }\n  IteratorImpl operator++(int) {\n    IteratorImpl result = *this;\n    ++*this;\n    return result;\n  }\n\n  friend bool operator==(IteratorImpl a, IteratorImpl b) {\n    RIEGELI_ASSERT_EQ(a.direct_map_end_, b.direct_map_end_)\n        << \"Failed precondition of operator==(HybridDirectMap::iterator): \"\n           \"incomparable iterators\";\n    RIEGELI_ASSERT_EQ(a.direct_map_size_, b.direct_map_size_)\n        << \"Failed precondition of operator==(HybridDirectMap::iterator): \"\n           \"incomparable iterators\";\n    RIEGELI_ASSERT_EQ(a.slow_map_iter_ != std::nullopt,\n                      b.slow_map_iter_ != std::nullopt)\n        << \"Failed precondition of operator==(HybridDirectMap::iterator): \"\n           \"incomparable iterators\";\n    if (a.raw_key_complement_ != b.raw_key_complement_) return false;\n    if (ABSL_PREDICT_TRUE(a.slow_map_iter_ == std::nullopt)) return true;\n    return *a.slow_map_iter_ == *b.slow_map_iter_;\n  }\n\n private:\n  friend class HybridDirectMapImpl;\n\n  explicit IteratorImpl(std::conditional_t<is_const, const HybridDirectMapImpl*,\n                                           HybridDirectMapImpl*>\n                            map ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                        size_t raw_key_complement)\n      : direct_map_end_(map->direct_map_.get() +\n                        map->direct_map_.get_deleter().size()),\n        direct_map_size_(map->direct_map_.get_deleter().size()),\n        raw_key_complement_(raw_key_complement) {}\n\n  explicit IteratorImpl(\n      std::conditional_t<is_const, const HybridDirectMapImpl*,\n                         HybridDirectMapImpl*>\n          map ABSL_ATTRIBUTE_LIFETIME_BOUND,\n      size_t raw_key_complement,\n      std::conditional_t<is_const, typename SlowMap::const_iterator,\n                         typename SlowMap::iterator>\n          slow_map_iter)\n      : direct_map_end_(map->direct_map_.get() +\n                        map->direct_map_.get_deleter().size()),\n        direct_map_size_(map->direct_map_.get_deleter().size()),\n        raw_key_complement_(raw_key_complement),\n        slow_map_iter_(slow_map_iter) {}\n\n  // The end of the `direct_map_` array.\n  //\n  // Counting backwards simplifies computing `end()` and advancing the iterator.\n  absl_nullable const std::conditional_t<\n      is_const, const Value*, Value*>* absl_nullable direct_map_end_ = nullptr;\n  // `direct_map_.get_deleter().size()`.\n  size_t direct_map_size_ = 0;\n  // `direct_map_size_ - raw_key` when iterating over `direct_map_`,\n  // otherwise 0.\n  //\n  // Invariant: if `raw_key_complement_ > 0` then\n  // `*(direct_map_end_ - raw_key_complement_) != nullptr`.\n  //\n  // Counting backwards simplifies computing `end()` and advancing the iterator.\n  size_t raw_key_complement_ = 0;\n  // Iterator over `*slow_map_` when `slow_map_ != nullptr`, otherwise\n  // `std::nullopt`.\n  //\n  // Invariant: if `raw_key_complement_ > 0` and `slow_map_ != nullptr` then\n  // `slow_map_iter_ == slow_map_->begin()`.\n  //\n  // Distinguishing `std::nullopt` instead of using the default-constructed\n  // `SlowMap::iterator` makes the common case of `operator==` faster by\n  // reducing usage of `SlowMap` iterators.\n  std::optional<std::conditional_t<is_const, typename SlowMap::const_iterator,\n                                   typename SlowMap::iterator>>\n      slow_map_iter_;\n};\n\n}  // namespace hybrid_direct_internal\n\n// Implementation details follow.\n\nnamespace hybrid_direct_internal {\n\ntemplate <typename Key, typename Value, typename Traits>\ninline size_t HybridDirectMapImpl<Key, Value, Traits>::max_size() {\n  return UnsignedMin(SizedDeleter<Value* absl_nullable>::max_size(),\n                     SizedDeleter<DelayedConstructor<Value>,\n                                  /*supports_abandon=*/true>::max_size()) /\n         kInverseMinLoadFactor;\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nvoid HybridDirectMapImpl<Key, Value, Traits>::Reset() {\n  direct_values_ = DirectValues();\n  direct_map_ = DirectMap();\n  slow_map_.reset();\n}\n\ntemplate <typename Key, typename Value, typename Traits>\ntemplate <typename Src, typename KeyProjection, typename ValueProjection>\nvoid HybridDirectMapImpl<Key, Value, Traits>::Initialize(\n    Src&& src, const KeyProjection& key_projection,\n    const ValueProjection& value_projection, size_t direct_capacity) {\n  using std::begin;\n  using std::end;\n  if constexpr (IterableHasSize<Src>::value) {\n    using std::size;\n    const size_t src_size = size(src);\n    RIEGELI_ASSERT_EQ(src_size,\n                      IntCast<size_t>(std::distance(begin(src), end(src))))\n        << \"Failed precondition of HybridDirectMap initialization: \"\n           \"size does not match the distance between iterators\";\n    if (src_size > 0) {\n      Optimize<Src>(begin(src), end(src), src_size, key_projection,\n                    value_projection, direct_capacity);\n    }\n  } else {\n    auto first = begin(src);\n    auto last = end(src);\n    const size_t src_size = IntCast<size_t>(std::distance(first, last));\n    if (src_size > 0) {\n      Optimize<Src>(first, last, src_size, key_projection, value_projection,\n                    direct_capacity);\n    }\n  }\n#if RIEGELI_DEBUG\n  // Detect building `HybridDirectMap` from a moved-from `src` if possible.\n  if constexpr (std::conjunction_v<std::negation<std::is_reference<Src>>,\n                                   std::is_move_constructible<Src>>) {\n    ABSL_ATTRIBUTE_UNUSED Src moved = std::forward<Src>(src);\n  }\n#endif\n}\n\ntemplate <typename Key, typename Value, typename Traits>\ntemplate <typename Index, typename KeyProjection, typename ValueProjection>\nvoid HybridDirectMapImpl<Key, Value, Traits>::InitializeByIndex(\n    Index size, const KeyProjection& key_projection,\n    const ValueProjection& value_projection, size_t direct_capacity) {\n  if (size > 0) {\n    RIEGELI_CHECK_LE(UnsignedCast(size), std::numeric_limits<size_t>::max())\n        << \"Failed precondition of HybridDirectMap initialization: \"\n           \"size overflow\";\n    // The template parameter of `Optimize()` serves only to determine whether\n    // to apply `std::move_iterator`.\n    Optimize<const Index[1]>(hybrid_direct_internal::IndexIterator<Index>(0),\n                             hybrid_direct_internal::IndexIterator<Index>(size),\n                             IntCast<size_t>(size), key_projection,\n                             value_projection, direct_capacity);\n  }\n}\n\ntemplate <typename Key, typename Value, typename Traits>\ntemplate <typename Src, typename Iterator, typename KeyProjection,\n          typename ValueProjection>\nvoid HybridDirectMapImpl<Key, Value, Traits>::Optimize(\n    Iterator first, Iterator last, size_t size,\n    const KeyProjection& key_projection,\n    const ValueProjection& value_projection, size_t direct_capacity) {\n  RIEGELI_ASSERT_GE(size, 0u)\n      << \"Failed precondition of HybridDirectMapImpl::Optimize(): \"\n         \"an empty map must have been handled before\";\n  RIEGELI_CHECK_LE(size, max_size())\n      << \"Failed precondition of HybridDirectMap initialization: \"\n         \"size overflow\";\n  RawKey max_raw_key = 0;\n  for (auto iter = first; iter != last; ++iter) {\n    const RawKey raw_key = Traits::ToRawKey(std::invoke(key_projection, *iter));\n    max_raw_key = UnsignedMax(max_raw_key, raw_key);\n  }\n  const size_t max_num_direct_keys =\n      UnsignedMax(direct_capacity, size * kInverseMinLoadFactor);\n  size_t direct_values_index;\n  if (max_raw_key < max_num_direct_keys) {\n    // All keys are suitable for `direct_map_`. `slow_map_` is not used.\n    //\n    // There is no need for `direct_map_` to cover raw keys above `max_raw_key`\n    // because their lookup is fast if `slow_map_` is `nullptr`.\n    hybrid_direct_internal::AssignToAssumedNull(\n        direct_values_,\n        MakeSizedArray<DelayedConstructor<Value>, /*supports_abandon=*/true>(\n            size));\n    hybrid_direct_internal::AssignToAssumedNull(\n        direct_map_,\n        MakeSizedArray<Value* absl_nullable>(IntCast<size_t>(max_raw_key) + 1));\n    direct_values_index = 0;\n    for (auto iter = first; iter != last; ++iter) {\n      const RawKey raw_key =\n          Traits::ToRawKey(std::invoke(key_projection, *iter));\n      if (ABSL_PREDICT_FALSE(direct_map_[raw_key] != nullptr)) continue;\n      direct_map_[raw_key] =\n          &direct_values_[direct_values_index++].emplace(riegeli::Invoker(\n              value_projection, *MaybeMakeMoveIterator<Src>(iter)));\n    }\n  } else {\n    // Some keys are too large for `direct_map_`. `slow_map_` is used.\n    //\n    // `direct_map_` covers all raw keys below `max_num_direct_keys` rather than\n    // only up to `max_raw_key`, to reduce lookups in `slow_map_`.\n    size_t num_direct_values = 0;\n    for (auto iter = first; iter != last; ++iter) {\n      const RawKey raw_key =\n          Traits::ToRawKey(std::invoke(key_projection, *iter));\n      num_direct_values += raw_key < max_num_direct_keys ? 1 : 0;\n    }\n    RIEGELI_ASSERT_LT(num_direct_values, size)\n        << \"Some keys should have been too large for direct_map_\";\n    if (ABSL_PREDICT_FALSE(num_direct_values == 0)) {\n      // The distribution is unfortunate: all keys are too large for\n      // `direct_map_`. No lookup hits can be optimized. Do not allocate\n      // `direct_map_` full of absent keys to save memory, at the cost of\n      // not optimizing any lookup misses.\n    } else {\n      hybrid_direct_internal::AssignToAssumedNull(\n          direct_values_,\n          MakeSizedArray<DelayedConstructor<Value>, /*supports_abandon=*/true>(\n              num_direct_values));\n      hybrid_direct_internal::AssignToAssumedNull(\n          direct_map_,\n          MakeSizedArray<Value* absl_nullable>(max_num_direct_keys));\n    }\n    hybrid_direct_internal::AssignToAssumedNull(slow_map_,\n                                                std::make_unique<SlowMap>());\n    slow_map_->reserve(size - num_direct_values);\n    direct_values_index = 0;\n    for (auto iter = first; iter != last; ++iter) {\n      const RawKey raw_key =\n          Traits::ToRawKey(std::invoke(key_projection, *iter));\n      if (raw_key < max_num_direct_keys) {\n        if (ABSL_PREDICT_FALSE(direct_map_[raw_key] != nullptr)) continue;\n        direct_map_[raw_key] =\n            &direct_values_[direct_values_index++].emplace(riegeli::Invoker(\n                value_projection, *MaybeMakeMoveIterator<Src>(iter)));\n      } else {\n        slow_map_->try_emplace(\n            raw_key, riegeli::Invoker(value_projection,\n                                      *MaybeMakeMoveIterator<Src>(iter)));\n      }\n    }\n  }\n  direct_values_.get_deleter().AbandonAfter(direct_values_.get(),\n                                            direct_values_index);\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nHybridDirectMapImpl<Key, Value, Traits>::HybridDirectMapImpl(\n    const HybridDirectMapImpl& that) noexcept\n    : direct_values_(that.CopyDirectValues()),\n      direct_map_(that.CopyDirectMap(direct_values_.get())),\n      slow_map_(that.CopySlowMap()) {}\n\ntemplate <typename Key, typename Value, typename Traits>\nHybridDirectMapImpl<Key, Value, Traits>&\nHybridDirectMapImpl<Key, Value, Traits>::operator=(\n    const HybridDirectMapImpl& that) noexcept {\n  absl_nullable DirectValues new_direct_values = that.CopyDirectValues();\n  direct_map_ = that.CopyDirectMap(new_direct_values.get());\n  direct_values_ = std::move(new_direct_values);\n  slow_map_ = that.CopySlowMap();\n  return *this;\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nauto HybridDirectMapImpl<Key, Value, Traits>::CopyDirectValues() const ->\n    absl_nullable DirectValues {\n  if (direct_values_ == nullptr) return nullptr;\n  DirectValues dest_ptr =\n      MakeSizedArray<DelayedConstructor<Value>, /*supports_abandon=*/true>(\n          direct_values_.get_deleter().size());\n  DelayedConstructor<Value>* src_iter = direct_values_.get();\n  DelayedConstructor<Value>* const end =\n      dest_ptr.get() + dest_ptr.get_deleter().size();\n  for (DelayedConstructor<Value>* dest_iter = dest_ptr.get(); dest_iter != end;\n       ++dest_iter) {\n    dest_iter->emplace(**src_iter);\n    ++src_iter;\n  }\n  return dest_ptr;\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nauto HybridDirectMapImpl<Key, Value, Traits>::CopyDirectMap(\n    DelayedConstructor<Value>* absl_nullable dest_values) const ->\n    absl_nullable DirectMap {\n  if (direct_map_ == nullptr) return nullptr;\n  DelayedConstructor<Value>* const absl_nullable src_values =\n      direct_values_.get();\n  DirectMap dest_ptr = MakeSizedArrayForOverwrite<Value* absl_nullable>(\n      direct_map_.get_deleter().size());\n  Value* absl_nullable* src_iter = direct_map_.get();\n  Value* absl_nullable* const end =\n      dest_ptr.get() + dest_ptr.get_deleter().size();\n  for (Value* absl_nullable* dest_iter = dest_ptr.get(); dest_iter != end;\n       ++dest_iter) {\n    *dest_iter =\n        *src_iter == nullptr\n            ? nullptr\n            : reinterpret_cast<Value*>(reinterpret_cast<char*>(dest_values) +\n                                       ((reinterpret_cast<char*>(*src_iter) -\n                                         reinterpret_cast<char*>(src_values))));\n    ++src_iter;\n  }\n  return dest_ptr;\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nauto HybridDirectMapImpl<Key, Value, Traits>::CopySlowMap() const ->\n    absl_nullable std::unique_ptr<SlowMap> {\n  if (slow_map_ == nullptr) return nullptr;\n  return std::make_unique<SlowMap>(*slow_map_);\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline Value* absl_nullable\nHybridDirectMapImpl<Key, Value, Traits>::FindOrNull(Key key)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return const_cast<Value*>(std::as_const(*this).FindOrNull(key));\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline const Value* absl_nullable\nHybridDirectMapImpl<Key, Value, Traits>::FindOrNull(Key key) const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT(!direct_map_.get_deleter().IsMovedFromIfNull() ||\n                 direct_map_ != nullptr)\n      << \"Moved-from HybridDirectMap\";\n  const RawKey raw_key = Traits::ToRawKey(key);\n  if (raw_key < direct_map_.get_deleter().size()) return direct_map_[raw_key];\n  if (ABSL_PREDICT_TRUE(slow_map_ == nullptr)) return nullptr;\n  const auto iter = slow_map_->find(raw_key);\n  if (iter == slow_map_->end()) return nullptr;\n  return &iter->second;\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline const Value&\nHybridDirectMapImpl<Key, Value, Traits>::FindOrDefault(\n    Key key, const Value& default_value ABSL_ATTRIBUTE_LIFETIME_BOUND) const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT(!direct_map_.get_deleter().IsMovedFromIfNull() ||\n                 direct_map_ != nullptr)\n      << \"Moved-from HybridDirectMap\";\n  const RawKey raw_key = Traits::ToRawKey(key);\n  if (raw_key < direct_map_.get_deleter().size()) {\n    const Value* const absl_nullable value = direct_map_[raw_key];\n    if (value == nullptr) return default_value;\n    return *value;\n  }\n  if (ABSL_PREDICT_TRUE(slow_map_ == nullptr)) return default_value;\n  const auto iter = slow_map_->find(raw_key);\n  if (iter == slow_map_->end()) return default_value;\n  return iter->second;\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline auto\nHybridDirectMapImpl<Key, Value, Traits>::find(Key key)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND -> iterator {\n  RIEGELI_ASSERT(!direct_map_.get_deleter().IsMovedFromIfNull() ||\n                 direct_map_ != nullptr)\n      << \"Moved-from HybridDirectMap\";\n  const RawKey raw_key = Traits::ToRawKey(key);\n  if (raw_key < direct_map_.get_deleter().size()) {\n    if (ABSL_PREDICT_TRUE(slow_map_ == nullptr)) {\n      return iterator(this, direct_map_[raw_key] == nullptr\n                                ? 0\n                                : direct_map_.get_deleter().size() - raw_key);\n    }\n    if (direct_map_[raw_key] == nullptr) {\n      return iterator(this, 0, slow_map_->end());\n    }\n    return iterator(this, direct_map_.get_deleter().size() - raw_key,\n                    slow_map_->begin());\n  }\n  if (ABSL_PREDICT_TRUE(slow_map_ == nullptr)) return iterator(this, 0);\n  return iterator(this, 0, slow_map_->find(raw_key));\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline auto\nHybridDirectMapImpl<Key, Value, Traits>::find(Key key) const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND -> const_iterator {\n  RIEGELI_ASSERT(!direct_map_.get_deleter().IsMovedFromIfNull() ||\n                 direct_map_ != nullptr)\n      << \"Moved-from HybridDirectMap\";\n  const RawKey raw_key = Traits::ToRawKey(key);\n  if (raw_key < direct_map_.get_deleter().size()) {\n    if (ABSL_PREDICT_TRUE(slow_map_ == nullptr)) {\n      return const_iterator(this,\n                            direct_map_[raw_key] == nullptr\n                                ? 0\n                                : direct_map_.get_deleter().size() - raw_key);\n    }\n    if (direct_map_[raw_key] == nullptr) {\n      return const_iterator(this, 0, slow_map_->cend());\n    }\n    return const_iterator(this, direct_map_.get_deleter().size() - raw_key,\n                          slow_map_->cbegin());\n  }\n  if (ABSL_PREDICT_TRUE(slow_map_ == nullptr)) return const_iterator(this, 0);\n  return const_iterator(this, 0, std::as_const(*slow_map_).find(raw_key));\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool\nHybridDirectMapImpl<Key, Value, Traits>::contains(Key key) const {\n  RIEGELI_ASSERT(!direct_map_.get_deleter().IsMovedFromIfNull() ||\n                 direct_map_ != nullptr)\n      << \"Moved-from HybridDirectMap\";\n  const RawKey raw_key = Traits::ToRawKey(key);\n  if (raw_key < direct_map_.get_deleter().size()) {\n    return direct_map_[raw_key] != nullptr;\n  }\n  if (ABSL_PREDICT_TRUE(slow_map_ == nullptr)) return false;\n  return slow_map_->contains(raw_key);\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline Value&\nHybridDirectMapImpl<Key, Value, Traits>::at(Key key)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return const_cast<Value&>(std::as_const(*this).at(key));\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline const Value&\nHybridDirectMapImpl<Key, Value, Traits>::at(Key key) const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT(!direct_map_.get_deleter().IsMovedFromIfNull() ||\n                 direct_map_ != nullptr)\n      << \"Moved-from HybridDirectMap\";\n  const RawKey raw_key = Traits::ToRawKey(key);\n  if (raw_key < direct_map_.get_deleter().size()) {\n    const Value* const absl_nullable value = direct_map_[raw_key];\n    if (ABSL_PREDICT_FALSE(value == nullptr)) KeyNotFound(key);\n    return *value;\n  }\n  if (ABSL_PREDICT_FALSE(slow_map_ == nullptr)) KeyNotFound(key);\n  const auto iter = slow_map_->find(raw_key);\n  if (ABSL_PREDICT_FALSE(iter == slow_map_->end())) KeyNotFound(key);\n  return iter->second;\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nABSL_ATTRIBUTE_NORETURN void\nHybridDirectMapImpl<Key, Value, Traits>::KeyNotFound(Key key) {\n  RIEGELI_CHECK_UNREACHABLE()\n      << \"HybridDirectMap key not found: \" << riegeli::Debug(key);\n}\n\ntemplate <typename Key, typename Value, typename Traits>\ninline size_t HybridDirectMapImpl<Key, Value, Traits>::FirstRawKey() const {\n  const size_t direct_map_size = direct_map_.get_deleter().size();\n  for (size_t raw_key = 0; raw_key < direct_map_size; ++raw_key) {\n    if (direct_map_[raw_key] != nullptr) return raw_key;\n  }\n  return direct_map_size;\n}\n\ntemplate <typename Key, typename Value, typename Traits>\ninline size_t HybridDirectMapImpl<Key, Value, Traits>::size() const {\n  return direct_values_.get_deleter().size() +\n         (ABSL_PREDICT_TRUE(slow_map_ == nullptr) ? 0 : slow_map_->size());\n}\n\ntemplate <typename Key, typename Value, typename Traits>\ninline auto HybridDirectMapImpl<Key, Value, Traits>::begin()\n    ABSL_ATTRIBUTE_LIFETIME_BOUND -> iterator {\n  const size_t raw_key_complement =\n      direct_map_.get_deleter().size() - FirstRawKey();\n  if (ABSL_PREDICT_TRUE(slow_map_ == nullptr)) {\n    return iterator(this, raw_key_complement);\n  }\n  return iterator(this, raw_key_complement, slow_map_->begin());\n}\n\ntemplate <typename Key, typename Value, typename Traits>\ninline auto HybridDirectMapImpl<Key, Value, Traits>::begin() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND -> const_iterator {\n  const size_t raw_key_complement =\n      direct_map_.get_deleter().size() - FirstRawKey();\n  if (ABSL_PREDICT_TRUE(slow_map_ == nullptr)) {\n    return const_iterator(this, raw_key_complement);\n  }\n  return const_iterator(this, raw_key_complement, slow_map_->cbegin());\n}\n\ntemplate <typename Key, typename Value, typename Traits>\ninline auto HybridDirectMapImpl<Key, Value, Traits>::end()\n    ABSL_ATTRIBUTE_LIFETIME_BOUND -> iterator {\n  if (ABSL_PREDICT_TRUE(slow_map_ == nullptr)) return iterator(this, 0);\n  return iterator(this, 0, slow_map_->end());\n}\n\ntemplate <typename Key, typename Value, typename Traits>\ninline auto HybridDirectMapImpl<Key, Value, Traits>::end() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND -> const_iterator {\n  if (ABSL_PREDICT_TRUE(slow_map_ == nullptr)) return const_iterator(this, 0);\n  return const_iterator(this, 0, slow_map_->cend());\n}\n\ntemplate <typename Key, typename Value, typename Traits>\nbool HybridDirectMapImpl<Key, Value, Traits>::Equal(\n    const HybridDirectMapImpl& a, const HybridDirectMapImpl& b) {\n  if (a.size() != b.size()) return false;\n  const HybridDirectMapImpl* outer;\n  const HybridDirectMapImpl* inner;\n  if (a.capacity() <= b.capacity()) {\n    outer = &a;\n    inner = &b;\n  } else {\n    outer = &b;\n    inner = &a;\n  }\n  for (const_reference entry : *outer) {\n    const auto* const found = inner->FindOrNull(entry.first);\n    if (found == nullptr || *found != entry.second) return false;\n  }\n  return true;\n}\n\n}  // namespace hybrid_direct_internal\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_HYBRID_DIRECT_MAP_H_\n"
  },
  {
    "path": "riegeli/base/hybrid_direct_set.h",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_HYBRID_DIRECT_SET_H_\n#define RIEGELI_BASE_HYBRID_DIRECT_SET_H_\n\n#include <stddef.h>\n\n#include <cstring>\n#include <functional>\n#include <initializer_list>\n#include <iterator>\n#include <limits>\n#include <memory>\n#include <optional>\n#include <type_traits>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/container/flat_hash_set.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/hybrid_direct_common.h\"  // IWYU pragma: export\n#include \"riegeli/base/hybrid_direct_internal.h\"\n#include \"riegeli/base/iterable.h\"\n#include \"riegeli/base/type_traits.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `HybridDirectSet` is a set optimized for keys being mostly small integers\n// or enums, especially if they are dense near zero. It supports only lookups\n// and iteration, but no incremental modification.\n//\n// It stores a part of the set covering some range of small keys in an array\n// of booleans, directly indexed by the key. The remaining keys are stored in\n// an `absl::flat_hash_set`.\n//\n// `Traits` specifies a mapping of keys to an unsigned integer type. It must\n// support at least the following static members:\n//\n// ```\n//   // Translates the key to a raw key, which is an unsigned integer type.\n//   // Small raw keys are put in the array.\n//   static RawKey ToRawKey(Key key);\n//\n//   // Translates the raw key back to a key.\n//   //\n//   // This is optional. Needed only for iterators.\n//   static Key FromRawKey(RawKey raw_key);\n// ```\n//\n// `direct_capacity`, if specified during building, is the intended capacity\n// of the array part. The actual capacity can be smaller if all keys fit\n// in the array, or larger if the array remains at least 25% full. Default:\n// `kHybridDirectDefaultDirectCapacity` (128).\n//\n// In the case of duplicate keys, one is retained.\ntemplate <typename Key, typename Traits = HybridDirectTraits<Key>>\nclass HybridDirectSet : public WithEqual<HybridDirectSet<Key, Traits>> {\n private:\n  template <typename Src, typename Enable = void>\n  struct HasCompatibleKeys : std::false_type {};\n  template <typename Src>\n  struct HasCompatibleKeys<\n      Src,\n      std::enable_if_t<std::is_convertible_v<ElementTypeT<const Src&>, Key>>>\n      : std::true_type {};\n\n  template <typename Src, typename KeyProjection, typename Enable = void>\n  struct HasProjectableKeys : std::false_type {};\n  template <typename Src, typename KeyProjection>\n  struct HasProjectableKeys<\n      Src, KeyProjection,\n      std::enable_if_t<std::is_convertible_v<\n          std::invoke_result_t<const KeyProjection&, ElementTypeT<const Src&>>,\n          Key>>> : std::true_type {};\n\n  template <typename Index, typename KeyProjection, typename Enable = void>\n  struct HasGeneratableKeys : std::false_type {};\n  template <typename Index, typename KeyProjection>\n  struct HasGeneratableKeys<\n      Index, KeyProjection,\n      std::enable_if_t<std::is_convertible_v<\n          std::invoke_result_t<const KeyProjection&, Index>, Key>>>\n      : std::true_type {};\n\n  template <typename Src>\n  struct DefaultKeyProjection {\n    Key operator()(ElementTypeT<const Src&> key) const { return key; }\n  };\n\n public:\n  using value_type = Key;\n  using reference = Key;\n  using const_reference = Key;\n  using pointer = void;\n  using const_pointer = void;\n  class iterator;\n  using const_iterator = iterator;\n  using size_type = size_t;\n  using difference_type = ptrdiff_t;\n\n  static size_t max_size();\n\n  // Constructs an empty `HybridDirectSet`.\n  HybridDirectSet() = default;\n\n  // Builds `HybridDirectSet` from an iterable `src`.\n  template <\n      typename Src,\n      std::enable_if_t<\n          std::conjunction_v<NotSameRef<HybridDirectSet, Src>,\n                             IsForwardIterable<Src>, HasCompatibleKeys<Src>>,\n          int> = 0>\n  explicit HybridDirectSet(const Src& src) {\n    Initialize(src, DefaultKeyProjection<Src>(),\n               kHybridDirectDefaultDirectCapacity);\n  }\n  template <typename Src,\n            std::enable_if_t<std::conjunction_v<IsForwardIterable<Src>,\n                                                HasCompatibleKeys<Src>>,\n                             int> = 0>\n  explicit HybridDirectSet(const Src& src, size_t direct_capacity) {\n    Initialize(src, DefaultKeyProjection<Src>(), direct_capacity);\n  }\n\n  // Builds `HybridDirectSet` from an initializer list.\n  /*implicit*/ HybridDirectSet(\n      std::initializer_list<Key> src,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    Initialize(src, DefaultKeyProjection<decltype(src)>(), direct_capacity);\n  }\n\n  // Builds `HybridDirectSet` from an iterable `src`.\n  //\n  // Keys are extracted using `key_projection()`. `key_projection()` may be\n  // called multiple times for each key so it should be efficient.\n  template <\n      typename Src, typename KeyProjection = DefaultKeyProjection<Src>,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::negation<std::is_convertible<KeyProjection, size_t>>,\n              IsForwardIterable<Src>, HasProjectableKeys<Src, KeyProjection>>,\n          int> = 0>\n  explicit HybridDirectSet(\n      const Src& src, const KeyProjection& key_projection,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    Initialize(src, key_projection, direct_capacity);\n  }\n\n  // Builds `HybridDirectSet` from keys computed by invoking `key_projection()`\n  // with indices from [0..`size`).\n  //\n  // `key_projection()` may be called multiple times for each index so it should\n  // be efficient.\n  template <typename Index, typename KeyProjection,\n            std::enable_if_t<\n                std::conjunction_v<\n                    std::is_integral<Index>,\n                    std::negation<std::is_convertible<KeyProjection, size_t>>,\n                    HasGeneratableKeys<Index, KeyProjection>>,\n                int> = 0>\n  explicit HybridDirectSet(\n      Index size, const KeyProjection& key_projection,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    InitializeByIndex(size, key_projection, direct_capacity);\n  }\n\n  HybridDirectSet(const HybridDirectSet& that) noexcept;\n  HybridDirectSet& operator=(const HybridDirectSet& that) noexcept;\n\n  HybridDirectSet(HybridDirectSet&& that) = default;\n  HybridDirectSet& operator=(HybridDirectSet&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `HybridDirectSet`.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n  template <typename Src,\n            std::enable_if_t<std::conjunction_v<IsForwardIterable<Src>,\n                                                HasCompatibleKeys<Src>>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      const Src& src,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    Reset();\n    Initialize(src, DefaultKeyProjection<Src>(), direct_capacity);\n  }\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      std::initializer_list<Key> src,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    Reset();\n    Initialize(src, DefaultKeyProjection<decltype(src)>(), direct_capacity);\n  }\n  template <\n      typename Src, typename KeyProjection = DefaultKeyProjection<Src>,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::negation<std::is_convertible<KeyProjection, size_t>>,\n              IsForwardIterable<Src>, HasProjectableKeys<Src, KeyProjection>>,\n          int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      const Src& src, const KeyProjection& key_projection,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    Reset();\n    Initialize(src, key_projection, direct_capacity);\n  }\n  template <typename Index, typename KeyProjection,\n            std::enable_if_t<\n                std::conjunction_v<\n                    std::is_integral<Index>,\n                    std::negation<std::is_convertible<KeyProjection, size_t>>,\n                    HasGeneratableKeys<Index, KeyProjection>>,\n                int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      Index size, const KeyProjection& key_projection,\n      size_t direct_capacity = kHybridDirectDefaultDirectCapacity) {\n    Reset();\n    InitializeByIndex(size, key_projection, direct_capacity);\n  }\n\n  bool contains(Key key) const;\n\n  bool empty() const { return size_ == 0; }\n  size_t size() const { return size_; }\n\n  iterator begin() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator begin() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator cbegin() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return begin();\n  }\n  iterator end() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator end() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator cend() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return end(); }\n\n  friend bool operator==(const HybridDirectSet& a, const HybridDirectSet& b) {\n    return Equal(a, b);\n  }\n\n private:\n  using RawKey = std::decay_t<decltype(Traits::ToRawKey(std::declval<Key>()))>;\n  static_assert(std::is_unsigned_v<RawKey>);\n\n  using DirectSet = hybrid_direct_internal::SizedArray<bool>;\n  using SlowSet = absl::flat_hash_set<RawKey>;\n\n  static constexpr int kInverseMinLoadFactor = 4;  // 25%.\n\n  template <typename Src, typename KeyProjection>\n  void Initialize(const Src& src, const KeyProjection& key_projection,\n                  size_t direct_capacity);\n\n  template <typename Index, typename KeyProjection>\n  void InitializeByIndex(Index size, const KeyProjection& key_projection,\n                         size_t direct_capacity);\n\n  template <typename Iterator, typename KeyProjection>\n  void Optimize(Iterator first, Iterator last, size_t size,\n                const KeyProjection& key_projection, size_t direct_capacity);\n\n  absl_nullable DirectSet CopyDirectSet() const;\n  absl_nullable std::unique_ptr<SlowSet> CopySlowSet() const;\n\n  size_t FirstRawKey() const;\n\n  size_t capacity() const {\n    return direct_set_.get_deleter().size() +\n           (slow_set_ == nullptr ? 0 : slow_set_->capacity());\n  }\n\n  static bool Equal(const HybridDirectSet& a, const HybridDirectSet& b);\n\n  // Indexed by raw key below `direct_set_.get_deleter().size()`.\n  absl_nullable DirectSet direct_set_;\n  // If not `nullptr`, stores the set of keys too large for `direct_set_`.\n  // Uses `std::unique_ptr` rather than `std::optional` to reduce memory usage\n  // in the common case when `slow_set_` is not used.\n  //\n  // Invariant: if `slow_set_ != nullptr` then `!slow_set_->empty()`.\n  absl_nullable std::unique_ptr<SlowSet> slow_set_;\n  size_t size_ = 0;\n};\n\ntemplate <typename Key, typename Traits>\nclass HybridDirectSet<Key, Traits>::iterator : public WithEqual<iterator> {\n public:\n  using iterator_concept = std::forward_iterator_tag;\n  // `iterator_category` is only `std::input_iterator_tag` because the\n  // `LegacyForwardIterator` requirement and above require `reference` to be\n  // a true reference type.\n  using iterator_category = std::input_iterator_tag;\n  using value_type = Key;\n  using reference = Key;\n  using pointer = void;\n  using difference_type = ptrdiff_t;\n\n  iterator() = default;\n\n  iterator(const iterator& that) = default;\n  iterator& operator=(const iterator& that) = default;\n\n  reference operator*() const {\n    if (ABSL_PREDICT_TRUE(raw_key_complement_ > 0)) {\n      return Traits::FromRawKey(\n          IntCast<RawKey>(direct_set_size_ - raw_key_complement_));\n    }\n    return Traits::FromRawKey(**slow_set_iter_);\n  }\n  iterator& operator++() {\n    if (ABSL_PREDICT_TRUE(raw_key_complement_ > 0)) {\n      do {\n        --raw_key_complement_;\n        if (ABSL_PREDICT_FALSE(raw_key_complement_ == 0)) break;\n      } while (!*(direct_set_end_ - raw_key_complement_));\n    } else {\n      ++*slow_set_iter_;\n    }\n    return *this;\n  }\n  iterator operator++(int) {\n    iterator result = *this;\n    ++*this;\n    return result;\n  }\n\n  friend bool operator==(iterator a, iterator b) {\n    RIEGELI_ASSERT_EQ(a.direct_set_end_, b.direct_set_end_)\n        << \"Failed precondition of operator==(HybridDirectSet::iterator): \"\n           \"incomparable iterators\";\n    RIEGELI_ASSERT_EQ(a.direct_set_size_, b.direct_set_size_)\n        << \"Failed precondition of operator==(HybridDirectSet::iterator): \"\n           \"incomparable iterators\";\n    RIEGELI_ASSERT_EQ(a.slow_set_iter_ != std::nullopt,\n                      b.slow_set_iter_ != std::nullopt)\n        << \"Failed precondition of operator==(HybridDirectSet::iterator): \"\n           \"incomparable iterators\";\n    if (a.raw_key_complement_ != b.raw_key_complement_) return false;\n    if (ABSL_PREDICT_TRUE(a.slow_set_iter_ == std::nullopt)) return true;\n    return *a.slow_set_iter_ == *b.slow_set_iter_;\n  }\n\n private:\n  friend class HybridDirectSet;\n\n  explicit iterator(const HybridDirectSet* set ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                    size_t raw_key_complement)\n      : direct_set_end_(set->direct_set_.get() +\n                        set->direct_set_.get_deleter().size()),\n        direct_set_size_(set->direct_set_.get_deleter().size()),\n        raw_key_complement_(raw_key_complement) {}\n\n  explicit iterator(const HybridDirectSet* set ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                    size_t raw_key_complement,\n                    typename SlowSet::const_iterator slow_set_iter)\n      : direct_set_end_(set->direct_set_.get() +\n                        set->direct_set_.get_deleter().size()),\n        direct_set_size_(set->direct_set_.get_deleter().size()),\n        raw_key_complement_(raw_key_complement),\n        slow_set_iter_(slow_set_iter) {}\n\n  // The end of the `direct_set_` array.\n  //\n  // Counting backwards simplifies checking for iteration over `direct_set_`.\n  const bool* absl_nullable direct_set_end_ = nullptr;\n  // `direct_set_.get_deleter().size()`.\n  size_t direct_set_size_ = 0;\n  // `direct_set_size_ - raw_key` when iterating over `direct_set_`,\n  // otherwise 0.\n  //\n  // Invariant: if `raw_key_complement_ > 0` then\n  // `*(direct_set_end_ - raw_key_complement_) != nullptr`.\n  //\n  // Counting backwards simplifies computing `end()` and advancing the iterator.\n  size_t raw_key_complement_ = 0;\n  // Iterator over `*slow_set_` when `slow_set_ != nullptr`, otherwise\n  // `std::nullopt`.\n  //\n  // Invariant: if `raw_key_complement_ > 0` and `slow_set_ != nullptr` then\n  // `slow_set_iter_ == slow_set_->begin()`.\n  //\n  // Distinguishing `std::nullopt` instead of using the default-constructed\n  // `SlowSet::iterator` makes the common case of `operator==` faster by\n  // reducing usage of `SlowSet` iterators.\n  std::optional<typename SlowSet::const_iterator> slow_set_iter_;\n};\n\n// Implementation details follow.\n\ntemplate <typename Key, typename Traits>\ninline size_t HybridDirectSet<Key, Traits>::max_size() {\n  return hybrid_direct_internal::SizedDeleter<bool>::max_size() /\n         kInverseMinLoadFactor;\n}\n\ntemplate <typename Key, typename Traits>\nHybridDirectSet<Key, Traits>::HybridDirectSet(\n    const HybridDirectSet& that) noexcept\n    : direct_set_(that.CopyDirectSet()),\n      slow_set_(that.CopySlowSet()),\n      size_(that.size_) {}\n\ntemplate <typename Key, typename Traits>\nHybridDirectSet<Key, Traits>& HybridDirectSet<Key, Traits>::operator=(\n    const HybridDirectSet& that) noexcept {\n  direct_set_ = that.CopyDirectSet();\n  slow_set_ = that.CopySlowSet();\n  size_ = that.size_;\n  return *this;\n}\n\ntemplate <typename Key, typename Traits>\nvoid HybridDirectSet<Key, Traits>::Reset() {\n  direct_set_ = DirectSet();\n  slow_set_.reset();\n  size_ = 0;\n}\n\ntemplate <typename Key, typename Traits>\ntemplate <typename Src, typename KeyProjection>\nvoid HybridDirectSet<Key, Traits>::Initialize(\n    const Src& src, const KeyProjection& key_projection,\n    size_t direct_capacity) {\n  using std::begin;\n  using std::end;\n  if constexpr (IterableHasSize<Src>::value) {\n    using std::size;\n    const size_t src_size = size(src);\n    RIEGELI_ASSERT_EQ(src_size,\n                      IntCast<size_t>(std::distance(begin(src), end(src))))\n        << \"Failed precondition of HybridDirectSet initialization: \"\n           \"size does not match the distance between iterators\";\n    if (src_size > 0) {\n      Optimize(begin(src), end(src), src_size, key_projection, direct_capacity);\n    }\n  } else {\n    auto first = begin(src);\n    auto last = end(src);\n    const size_t src_size = IntCast<size_t>(std::distance(first, last));\n    if (src_size > 0)\n      Optimize(first, last, src_size, key_projection, direct_capacity);\n  }\n}\n\ntemplate <typename Key, typename Traits>\ntemplate <typename Index, typename KeyProjection>\nvoid HybridDirectSet<Key, Traits>::InitializeByIndex(\n    Index size, const KeyProjection& key_projection, size_t direct_capacity) {\n  if (size > 0) {\n    RIEGELI_CHECK_LE(UnsignedCast(size), std::numeric_limits<size_t>::max())\n        << \"Failed precondition of HybridDirectSet initialization: \"\n           \"size overflow\";\n    Optimize(hybrid_direct_internal::IndexIterator<Index>(0),\n             hybrid_direct_internal::IndexIterator<Index>(size),\n             IntCast<size_t>(size), key_projection, direct_capacity);\n  }\n}\n\ntemplate <typename Key, typename Traits>\ntemplate <typename Iterator, typename KeyProjection>\nvoid HybridDirectSet<Key, Traits>::Optimize(Iterator first, Iterator last,\n                                            size_t size,\n                                            const KeyProjection& key_projection,\n                                            size_t direct_capacity) {\n  RIEGELI_ASSERT_GE(size, 0u)\n      << \"Failed precondition of HybridDirectSet::Optimize(): \"\n         \"an empty map must have been handled before\";\n  RIEGELI_CHECK_LE(size, max_size())\n      << \"Failed precondition of HybridDirectSet initialization: \"\n         \"size overflow\";\n  RawKey max_raw_key = 0;\n  for (auto iter = first; iter != last; ++iter) {\n    const RawKey raw_key = Traits::ToRawKey(std::invoke(key_projection, *iter));\n    max_raw_key = UnsignedMax(max_raw_key, raw_key);\n  }\n  const size_t max_num_direct_keys =\n      UnsignedMax(direct_capacity, size * kInverseMinLoadFactor);\n  size_ = size;\n  if (max_raw_key < max_num_direct_keys) {\n    // All keys are suitable for `direct_set_`. `slow_set_` is not used.\n    //\n    // There is no need for `direct_set_` to cover raw keys above `max_raw_key`\n    // because their lookup is fast if `slow_set_` is `nullptr`.\n    hybrid_direct_internal::AssignToAssumedNull(\n        direct_set_, hybrid_direct_internal::MakeSizedArray<bool>(\n                         IntCast<size_t>(max_raw_key) + 1));\n    for (auto iter = first; iter != last; ++iter) {\n      const RawKey raw_key =\n          Traits::ToRawKey(std::invoke(key_projection, *iter));\n      if (ABSL_PREDICT_FALSE(direct_set_[raw_key])) --size_;\n      direct_set_[raw_key] = true;\n    }\n  } else {\n    // Some keys are too large for `direct_set_`. `slow_set_` is used.\n    //\n    // `direct_set_` covers all raw keys below `max_num_direct_keys` rather than\n    // only up to `max_raw_key`, to reduce lookups in `slow_set_`.\n    size_t num_direct_elements = 0;\n    for (auto iter = first; iter != last; ++iter) {\n      const RawKey raw_key =\n          Traits::ToRawKey(std::invoke(key_projection, *iter));\n      num_direct_elements += raw_key < max_num_direct_keys ? 1 : 0;\n    }\n    RIEGELI_ASSERT_LT(num_direct_elements, size)\n        << \"Some keys should have been too large for direct_set_\";\n    if (ABSL_PREDICT_FALSE(num_direct_elements == 0)) {\n      // The distribution is unfortunate: all keys are too large for\n      // `direct_set_`. No lookup hits can be optimized. Do not allocate\n      // `direct_set_` full of absent keys to save memory, at the cost of\n      // not optimizing any lookup misses.\n    } else {\n      hybrid_direct_internal::AssignToAssumedNull(\n          direct_set_,\n          hybrid_direct_internal::MakeSizedArray<bool>(max_num_direct_keys));\n    }\n    hybrid_direct_internal::AssignToAssumedNull(slow_set_,\n                                                std::make_unique<SlowSet>());\n    slow_set_->reserve(size - num_direct_elements);\n    for (auto iter = first; iter != last; ++iter) {\n      const RawKey raw_key =\n          Traits::ToRawKey(std::invoke(key_projection, *iter));\n      if (raw_key < max_num_direct_keys) {\n        if (ABSL_PREDICT_FALSE(direct_set_[raw_key])) --size_;\n        direct_set_[raw_key] = true;\n      } else {\n        const auto inserted = slow_set_->insert(raw_key);\n        if (ABSL_PREDICT_FALSE(!inserted.second)) --size_;\n      }\n    }\n  }\n}\n\ntemplate <typename Key, typename Traits>\nauto HybridDirectSet<Key, Traits>::CopyDirectSet() const ->\n    absl_nullable DirectSet {\n  if (direct_set_ == nullptr) return nullptr;\n  DirectSet dest_ptr = hybrid_direct_internal::MakeSizedArrayForOverwrite<bool>(\n      direct_set_.get_deleter().size());\n  std::memcpy(dest_ptr.get(), direct_set_.get(),\n              dest_ptr.get_deleter().size() * sizeof(bool));\n  return dest_ptr;\n}\n\ntemplate <typename Key, typename Traits>\nauto HybridDirectSet<Key, Traits>::CopySlowSet() const ->\n    absl_nullable std::unique_ptr<SlowSet> {\n  if (slow_set_ == nullptr) return nullptr;\n  return std::make_unique<SlowSet>(*slow_set_);\n}\n\ntemplate <typename Key, typename Traits>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool HybridDirectSet<Key, Traits>::contains(\n    Key key) const {\n  RIEGELI_ASSERT(!direct_set_.get_deleter().IsMovedFromIfNull() ||\n                 direct_set_ != nullptr)\n      << \"Moved-from HybridDirectSet\";\n  const RawKey raw_key = Traits::ToRawKey(key);\n  if (raw_key < direct_set_.get_deleter().size()) return direct_set_[raw_key];\n  if (ABSL_PREDICT_TRUE(slow_set_ == nullptr)) return false;\n  return slow_set_->contains(raw_key);\n}\n\ntemplate <typename Key, typename Traits>\ninline size_t HybridDirectSet<Key, Traits>::FirstRawKey() const {\n  const size_t direct_set_size = direct_set_.get_deleter().size();\n  for (size_t raw_key = 0; raw_key < direct_set_size; ++raw_key) {\n    if (direct_set_[raw_key]) return raw_key;\n  }\n  return direct_set_size;\n}\n\ntemplate <typename Key, typename Traits>\ninline auto HybridDirectSet<Key, Traits>::begin() ABSL_ATTRIBUTE_LIFETIME_BOUND\n    -> iterator {\n  const size_t raw_key_complement =\n      direct_set_.get_deleter().size() - FirstRawKey();\n  if (ABSL_PREDICT_TRUE(slow_set_ == nullptr)) {\n    return iterator(this, raw_key_complement);\n  }\n  return iterator(this, raw_key_complement, slow_set_->begin());\n}\n\ntemplate <typename Key, typename Traits>\ninline auto HybridDirectSet<Key, Traits>::begin() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND -> const_iterator {\n  const size_t raw_key_complement =\n      direct_set_.get_deleter().size() - FirstRawKey();\n  if (ABSL_PREDICT_TRUE(slow_set_ == nullptr)) {\n    return const_iterator(this, raw_key_complement);\n  }\n  return const_iterator(this, raw_key_complement, slow_set_->cbegin());\n}\n\ntemplate <typename Key, typename Traits>\ninline auto HybridDirectSet<Key, Traits>::end() ABSL_ATTRIBUTE_LIFETIME_BOUND\n    -> iterator {\n  if (ABSL_PREDICT_TRUE(slow_set_ == nullptr)) return iterator(this, 0);\n  return iterator(this, 0, slow_set_->end());\n}\n\ntemplate <typename Key, typename Traits>\ninline auto HybridDirectSet<Key, Traits>::end() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND -> const_iterator {\n  if (ABSL_PREDICT_TRUE(slow_set_ == nullptr)) return const_iterator(this, 0);\n  return const_iterator(this, 0, slow_set_->cend());\n}\n\ntemplate <typename Key, typename Traits>\nbool HybridDirectSet<Key, Traits>::Equal(const HybridDirectSet& a,\n                                         const HybridDirectSet& b) {\n  if (a.size() != b.size()) return false;\n  const HybridDirectSet* outer;\n  const HybridDirectSet* inner;\n  if (a.capacity() <= b.capacity()) {\n    outer = &a;\n    inner = &b;\n  } else {\n    outer = &b;\n    inner = &a;\n  }\n  for (Key key : *outer) {\n    if (!inner->contains(key)) return false;\n  }\n  return true;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_HYBRID_DIRECT_SET_H_\n"
  },
  {
    "path": "riegeli/base/initializer.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_INITIALIZER_H_\n#define RIEGELI_BASE_INITIALIZER_H_\n\n#include <stddef.h>\n\n#include <functional>\n#include <memory>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"riegeli/base/initializer_internal.h\"\n#include \"riegeli/base/invoker.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/temporary_storage.h\"\n#include \"riegeli/base/type_erased_ref.h\"\n#include \"riegeli/base/type_traits.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\ntemplate <typename T>\nclass Initializer;\n\nnamespace initializer_internal {\n\n// `IsInitializer` detects `Initializer` types with the given target type.\n\ntemplate <typename T, typename Arg>\nstruct IsInitializer : std::false_type {};\n\ntemplate <typename T>\nstruct IsInitializer<T, Initializer<T>> : std::true_type {};\n\n// Part of `Initializer<T>` for `T` being a non-reference type.\ntemplate <typename T>\nclass InitializerBase {\n public:\n  // Constructs the `T`.\n  /*implicit*/ operator T() && { return std::move(*this).Construct(); }\n\n  // Constructs the `T`.\n  //\n  // Usually conversion to `T` is preferred because it can avoid creating a\n  // temporary if the context accepts an arbitrary type convertible to `T` and\n  // it leads to simpler source code. An explicit `Construct()` call can force\n  // construction right away while avoiding specifying the full target type.\n  T Construct() && { return methods()->construct(context()); }\n\n  // Constructs the `std::decay_t<T>` on the heap.\n  //\n  // In contrast to `std::make_unique()`, this supports custom deleters.\n  //\n  // For a non-default-constructed deleter, use `UniquePtr(deleter)`.\n  template <typename Target, typename Deleter,\n            std::enable_if_t<std::is_convertible_v<std::decay_t<T>*, Target*>,\n                             int> = 0>\n  /*implicit*/ operator std::unique_ptr<Target, Deleter>() && {\n    return std::move(*this).template UniquePtr<Deleter>();\n  }\n  template <typename Target, typename Deleter,\n            std::enable_if_t<std::is_convertible_v<std::decay_t<T>*, Target*>,\n                             int> = 0>\n  /*implicit*/ operator std::unique_ptr<Target, Deleter>() const& {\n    return UniquePtr<Deleter>();\n  }\n\n  // Constructs the `std::decay_t<T>` on the heap.\n  //\n  // In contrast to `std::make_unique()`, this supports custom deleters.\n  //\n  // Usually conversion to `std::unique_ptr` is preferred because it leads to\n  // simpler source code. An explicit `UniquePtr()` call can force construction\n  // right away while avoiding writing the full target type, and it allows to\n  // use a non-default-constructed deleter.\n  template <typename Deleter = std::default_delete<std::decay_t<T>>>\n  std::unique_ptr<std::decay_t<T>, Deleter> UniquePtr() && {\n    return std::unique_ptr<std::decay_t<T>, Deleter>(\n        new std::decay_t<T>(std::move(*this)));\n  }\n  template <typename Deleter>\n  std::unique_ptr<std::decay_t<T>, Deleter> UniquePtr(Deleter&& deleter) && {\n    return std::unique_ptr<std::decay_t<T>, Deleter>(\n        new std::decay_t<T>(std::move(*this)), std::forward<Deleter>(deleter));\n  }\n\n  // Constructs the `T` in `storage` which must outlive the returned reference,\n  // or returns a reference to an already constructed object if a compatible\n  // object was passed to `Initializer` constructor.\n  //\n  // `Reference()` instead of conversion to `T` or `Construct()` can avoid\n  // moving the object if the caller does not need to store the object, or if it\n  // will be moved later because the target location for the object is not ready\n  // yet.\n  //\n  // `storage` must outlive usages of the returned reference.\n  T&& Reference(\n      TemporaryStorage<T>&& storage ABSL_ATTRIBUTE_LIFETIME_BOUND = {}) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return methods()->reference(context(), std::move(storage));\n  }\n\n private:\n  static T ConstructMethodDefault(TypeErasedRef context);\n\n  template <typename Arg>\n  static T ConstructMethodFromObject(TypeErasedRef context);\n\n  template <typename... Args>\n  static T ConstructMethodFromMaker(TypeErasedRef context);\n\n  template <typename... Args>\n  static T ConstructMethodFromConstMaker(TypeErasedRef context);\n\n  template <typename Arg>\n  static T ConstructMethodFromConvertedReference(TypeErasedRef context);\n\n  static T&& ReferenceMethodDefault(TypeErasedRef context,\n                                    TemporaryStorage<T>&& storage);\n\n  template <typename Arg>\n  static T&& ReferenceMethodFromObject(TypeErasedRef context,\n                                       TemporaryStorage<T>&& storage);\n\n  template <typename... Args>\n  static T&& ReferenceMethodFromMaker(TypeErasedRef context,\n                                      TemporaryStorage<T>&& storage);\n\n  template <typename... Args>\n  static T&& ReferenceMethodFromConstMaker(TypeErasedRef context,\n                                           TemporaryStorage<T>&& storage);\n\n  template <typename Arg>\n  static T&& ReferenceMethodFromConvertedReference(\n      TypeErasedRef context, TemporaryStorage<T>&& storage);\n\n protected:\n  struct Methods {\n    T (*construct)(TypeErasedRef context);\n    T && (*reference)(TypeErasedRef context, TemporaryStorage<T>&& storage);\n  };\n\n  explicit InitializerBase(const Methods* methods);\n\n  template <typename Arg>\n  explicit InitializerBase(const Methods* methods, Arg&& arg);\n\n  InitializerBase(InitializerBase&& that) = default;\n  InitializerBase& operator=(InitializerBase&&) = delete;\n\n  template <typename Dummy = void>\n  static constexpr Methods kMethodsDefault = {ConstructMethodDefault,\n                                              ReferenceMethodDefault};\n\n  template <typename Arg>\n  static constexpr Methods kMethodsFromObject = {\n      ConstructMethodFromObject<Arg>, ReferenceMethodFromObject<Arg>};\n\n  template <typename... Args>\n  static constexpr Methods kMethodsFromMaker = {\n      ConstructMethodFromMaker<Args...>, ReferenceMethodFromMaker<Args...>};\n\n  template <typename... Args>\n  static constexpr Methods kMethodsFromConstMaker = {\n      ConstructMethodFromConstMaker<Args...>,\n      ReferenceMethodFromConstMaker<Args...>};\n\n  template <typename Arg>\n  static constexpr Methods kMethodsFromConvertedReference = {\n      ConstructMethodFromConvertedReference<Arg>,\n      ReferenceMethodFromConvertedReference<Arg>};\n\n  const Methods* methods() const { return methods_; }\n  TypeErasedRef context() const { return context_; }\n\n private:\n  const Methods* methods_;\n  TypeErasedRef context_;\n};\n\n// Part of `Initializer<T>` for `T` being a move-assignable non-reference type.\ntemplate <typename T>\nclass InitializerAssignableBase : public InitializerBase<T> {\n public:\n  // `riegeli::Reset(dest, Initializer)` makes `dest` equivalent to the\n  // constructed `T`. This avoids constructing a temporary `T` and moving from\n  // it.\n  friend void RiegeliReset(T& dest, InitializerAssignableBase&& src) {\n    src.methods()->reset(src.context(), dest);\n  }\n\n private:\n  static void ResetMethodDefault(TypeErasedRef context, T& dest);\n\n  template <typename Arg>\n  static void ResetMethodFromObject(TypeErasedRef context, T& dest);\n\n  template <typename... Args>\n  static void ResetMethodFromMaker(TypeErasedRef context, T& dest);\n\n  template <typename... Args>\n  static void ResetMethodFromConstMaker(TypeErasedRef context, T& dest);\n\n  template <typename Arg>\n  static void ResetMethodFromConvertedReference(TypeErasedRef context, T& dest);\n\n protected:\n  struct Methods : InitializerAssignableBase::InitializerBase::Methods {\n    void (*reset)(TypeErasedRef context, T& dest);\n  };\n\n  template <typename Dummy = void>\n  static constexpr Methods kMethodsDefault = {\n      InitializerAssignableBase::InitializerBase::template kMethodsDefault<>,\n      ResetMethodDefault};\n\n  template <typename Arg>\n  static constexpr Methods kMethodsFromObject = {\n      InitializerAssignableBase::InitializerBase::template kMethodsFromObject<\n          Arg>,\n      ResetMethodFromObject<Arg>};\n\n  template <typename... Args>\n  static constexpr Methods kMethodsFromMaker = {\n      InitializerAssignableBase::InitializerBase::template kMethodsFromMaker<\n          Args...>,\n      ResetMethodFromMaker<Args...>};\n\n  template <typename... Args>\n  static constexpr Methods kMethodsFromConstMaker = {\n      InitializerAssignableBase::InitializerBase::\n          template kMethodsFromConstMaker<Args...>,\n      ResetMethodFromConstMaker<Args...>};\n\n  template <typename Arg>\n  static constexpr Methods kMethodsFromConvertedReference = {\n      InitializerAssignableBase::InitializerBase::\n          template kMethodsFromConvertedReference<Arg>,\n      ResetMethodFromConvertedReference<Arg>};\n\n  explicit InitializerAssignableBase(const Methods* methods)\n      : InitializerAssignableBase::InitializerBase(methods) {}\n\n  template <typename Arg>\n  explicit InitializerAssignableBase(const Methods* methods, Arg&& arg)\n      : InitializerAssignableBase::InitializerBase(methods,\n                                                   std::forward<Arg>(arg)) {}\n\n  InitializerAssignableBase(InitializerAssignableBase&& that) = default;\n  InitializerAssignableBase& operator=(InitializerAssignableBase&&) = delete;\n\n  const Methods* methods() const {\n    return static_cast<const Methods*>(\n        InitializerAssignableBase::InitializerBase::methods());\n  }\n};\n\n// Part of `Initializer<T>` for `T` being a reference type.\ntemplate <typename T>\nclass InitializerReference {\n public:\n  // Constructs the `T`.\n  /*implicit*/ operator T() && { return std::move(*this).Construct(); }\n\n  // Constructs the `T`.\n  //\n  // Usually conversion to `T` is preferred because it leads to simpler source\n  // code. An explicit `Construct()` call can force construction right away\n  // while avoiding specifying the full target type.\n  T Construct() && { return methods()->construct(context()); }\n\n  // Constructs the `std::decay_t<T>` on the heap.\n  //\n  // In contrast to `std::make_unique()`, this supports custom deleters.\n  //\n  // For a non-default-constructed deleter, use `UniquePtr(deleter)`.\n  template <typename Target, typename Deleter,\n            std::enable_if_t<std::is_convertible_v<std::decay_t<T>*, Target*>,\n                             int> = 0>\n  /*implicit*/ operator std::unique_ptr<Target, Deleter>() && {\n    return std::move(*this).template UniquePtr<Deleter>();\n  }\n  template <typename Target, typename Deleter,\n            std::enable_if_t<std::is_convertible_v<std::decay_t<T>*, Target*>,\n                             int> = 0>\n  /*implicit*/ operator std::unique_ptr<Target, Deleter>() const& {\n    return UniquePtr<Deleter>();\n  }\n\n  // Constructs the `std::decay_t<T>` on the heap.\n  //\n  // In contrast to `std::make_unique()`, this supports custom deleters.\n  //\n  // Usually conversion to `std::unique_ptr` is preferred because it leads to\n  // simpler source code. An explicit `UniquePtr()` call can force construction\n  // right away while avoiding writing the full target type, and it allows to\n  // use a non-default-constructed deleter.\n  template <typename Deleter = std::default_delete<std::decay_t<T>>,\n            typename DependentT = T,\n            std::enable_if_t<\n                std::is_constructible_v<std::decay_t<DependentT>, DependentT>,\n                int> = 0>\n  std::unique_ptr<std::decay_t<T>, Deleter> UniquePtr() && {\n    return std::unique_ptr<std::decay_t<T>, Deleter>(\n        new std::decay_t<T>(std::move(*this)));\n  }\n  template <typename Deleter, typename DependentT = T,\n            std::enable_if_t<\n                std::is_constructible_v<std::decay_t<DependentT>, DependentT>,\n                int> = 0>\n  std::unique_ptr<std::decay_t<T>, Deleter> UniquePtr(Deleter&& deleter) && {\n    return std::unique_ptr<std::decay_t<T>, Deleter>(\n        new std::decay_t<T>(std::move(*this)), std::forward<Deleter>(deleter));\n  }\n\n  // `Reference()` can be defined in terms of conversion to `T` because\n  // reference storage is never used for reference types.\n  //\n  // Unused `storage` parameter makes the signature compatible with the\n  // non-reference specialization.\n  T&& Reference() && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    // `T` is a reference type here, so `T&&` is the same as `T`.\n    return std::move(*this).Construct();\n  }\n  T&& Reference(ABSL_ATTRIBUTE_UNUSED TemporaryStorage<T>&& storage) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(*this).Reference();\n  }\n\n private:\n  template <typename Arg>\n  static T ConstructMethodFromObject(TypeErasedRef context);\n\n  template <typename... Args>\n  static T ConstructMethodFromMaker(TypeErasedRef context);\n\n  template <typename... Args>\n  static T ConstructMethodFromConstMaker(TypeErasedRef context);\n\n  template <typename Arg>\n  static T ConstructMethodFromConvertedReference(TypeErasedRef context);\n\n protected:\n  struct Methods {\n    T (*construct)(TypeErasedRef context);\n  };\n\n  explicit InitializerReference(const Methods* methods);\n\n  template <typename Arg>\n  explicit InitializerReference(const Methods* methods, Arg&& arg);\n\n  InitializerReference(InitializerReference&& that) = default;\n  InitializerReference& operator=(InitializerReference&&) = delete;\n\n  template <typename Arg>\n  static constexpr Methods kMethodsFromObject = {\n      ConstructMethodFromObject<Arg>};\n\n  template <typename... Args>\n  static constexpr Methods kMethodsFromMaker = {\n      ConstructMethodFromMaker<Args...>};\n\n  template <typename... Args>\n  static constexpr Methods kMethodsFromConstMaker = {\n      ConstructMethodFromConstMaker<Args...>};\n\n  template <typename Arg>\n  static constexpr Methods kMethodsFromConvertedReference = {\n      ConstructMethodFromConvertedReference<Arg>};\n\n  const Methods* methods() const { return methods_; }\n  TypeErasedRef context() const { return context_; }\n\n private:\n  const Methods* methods_;\n  TypeErasedRef context_;\n};\n\ntemplate <typename T, typename Enable = void>\nstruct InitializerImpl;\n\ntemplate <typename T>\nstruct InitializerImpl<T, std::enable_if_t<!std::is_convertible_v<T&&, T>>> {\n  using type = InitializerBase<T>;\n};\n\ntemplate <typename T>\nstruct InitializerImpl<\n    T, std::enable_if_t<std::conjunction_v<\n           std::negation<std::is_reference<T>>, std::is_convertible<T&&, T>,\n           std::negation<std::is_move_assignable<T>>>>> {\n  using type = InitializerBase<T>;\n};\n\ntemplate <typename T>\nstruct InitializerImpl<\n    T, std::enable_if_t<std::conjunction_v<std::negation<std::is_reference<T>>,\n                                           std::is_convertible<T&&, T>,\n                                           std::is_move_assignable<T>>>> {\n  using type = InitializerAssignableBase<T>;\n};\n\ntemplate <typename T>\nstruct InitializerImpl<T, std::enable_if_t<std::is_reference_v<T>>> {\n  using type = InitializerReference<T>;\n};\n\n}  // namespace initializer_internal\n\n// A parameter of type `Initializer<T>` allows the caller to specify a `T` by\n// passing a value convertible to `T`, or constructor arguments for `T` packed\n// in `riegeli::Maker(args...)` or `riegeli::Maker<T>(args...)`.\n//\n// In contrast to accepting `T` directly, this allows to construct the object\n// in-place, avoiding constructing a temporary and moving from it. This also\n// avoids separate overloads for `const T&` and `T&&` or a template.\n//\n// `Initializer<T>(arg)` does not own `arg`, even if it involves temporaries,\n// hence it should be used only as a parameter of a function or constructor,\n// so that the temporaries outlive its usage. Instead of storing an\n// `Initializer<T>` in a variable or returning it from a function, consider\n// `riegeli::OwningMaker<T>(args...)`, `MakerTypeFor<T, Args...>`, or `T`.\ntemplate <typename T>\nclass ABSL_NULLABILITY_COMPATIBLE Initializer\n    : public initializer_internal::InitializerImpl<T>::type {\n private:\n  using Base = typename initializer_internal::InitializerImpl<T>::type;\n\n public:\n  // Constructs `Initializer<T>` which specifies `T()`.\n  template <\n      typename DependentT = T,\n      std::enable_if_t<std::is_default_constructible_v<DependentT>, int> = 0>\n  Initializer() : Base(&Base::template kMethodsDefault<>) {}\n\n  // Constructs `Initializer<T>` from a value convertible to `T`.\n  template <\n      typename Arg,\n      std::enable_if_t<\n          std::conjunction_v<std::negation<initializer_internal::IsInitializer<\n                                 T, std::decay_t<Arg>>>,\n                             std::is_convertible<Arg&&, T>>,\n          int> = 0>\n  /*implicit*/ Initializer(Arg&& arg ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : Base(&Base::template kMethodsFromObject<Arg>, std::forward<Arg>(arg)) {}\n\n  // Constructs `Initializer<T&>` from `std::reference_wrapper<Arg>` with a\n  // compatible `Arg`.\n  template <typename Arg,\n            std::enable_if_t<\n                std::conjunction_v<\n                    std::is_reference<T>,\n                    std::is_convertible<Arg*, std::remove_reference_t<T>*>>,\n                int> = 0>\n  /*implicit*/ Initializer(std::reference_wrapper<Arg> arg)\n      : Base(&Base::template kMethodsFromObject<Arg&>, arg.get()) {}\n\n  // Constructs `Initializer<T>` from constructor arguments for `T` packed in\n  // `riegeli::Maker(args...)`.\n  //\n  // Prefer `Template(riegeli::Maker<T>(args...))` over\n  // `Template<T>(riegeli::Maker(args...))` if CTAD for `Template` can be used.\n  template <typename... Args,\n            std::enable_if_t<std::is_constructible_v<T, Args&&...>, int> = 0>\n  /*implicit*/ Initializer(\n      MakerType<Args...>&& args ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : Base(&Base::template kMethodsFromMaker<Args...>, std::move(args)) {}\n  template <\n      typename... Args,\n      std::enable_if_t<std::is_constructible_v<T, const Args&...>, int> = 0>\n  /*implicit*/ Initializer(\n      const MakerType<Args...>& args ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : Base(&Base::template kMethodsFromConstMaker<Args...>, args) {}\n\n  // Constructs `Initializer<T>` from constructor arguments for `T` packed in\n  // `riegeli::Maker<T>(args...)`.\n  template <\n      typename... Args,\n      std::enable_if_t<std::conjunction_v<std::negation<std::is_convertible<\n                                              MakerTypeFor<T, Args...>&&, T>>,\n                                          std::is_constructible<T, Args&&...>>,\n                       int> = 0>\n  /*implicit*/ Initializer(\n      MakerTypeFor<T, Args...>&& args ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : Base(&Base::template kMethodsFromMaker<Args...>,\n             std::move(args).maker()) {}\n  template <typename... Args,\n            std::enable_if_t<\n                std::conjunction_v<std::negation<std::is_convertible<\n                                       const MakerTypeFor<T, Args...>&, T>>,\n                                   std::is_constructible<T, const Args&...>>,\n                int> = 0>\n  /*implicit*/ Initializer(\n      const MakerTypeFor<T, Args...>& args ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : Base(&Base::template kMethodsFromConstMaker<Args...>, args.maker()) {}\n\n  // Constructs `Initializer<T>` from constructor arguments for `T` packed in\n  // `riegeli::Maker<Target>(args...)` with a different but compatible `Target`.\n  template <typename Target, typename... Args,\n            std::enable_if_t<\n                std::conjunction_v<std::negation<std::is_same<Target, T>>,\n                                   std::negation<std::is_convertible<\n                                       MakerTypeFor<Target, Args...>&&, T>>,\n                                   std::is_constructible<Target, Args&&...>,\n                                   IsConvertibleFromResult<T, Target&&>>,\n                int> = 0>\n  /*implicit*/ Initializer(\n      MakerTypeFor<Target, Args...>&& args ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : Base(&Base::template kMethodsFromConvertedReference<\n                 MakerTypeFor<Target, Args...>>,\n             std::move(args)) {}\n  template <typename Target, typename... Args,\n            std::enable_if_t<std::conjunction_v<\n                                 std::negation<std::is_same<Target, T>>,\n                                 std::negation<std::is_convertible<\n                                     const MakerTypeFor<Target, Args...>&, T>>,\n                                 std::is_constructible<Target, const Args&...>,\n                                 IsConvertibleFromResult<T, Target&&>>,\n                             int> = 0>\n  /*implicit*/ Initializer(\n      const MakerTypeFor<Target, Args...>& args ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : Base(&Base::template kMethodsFromConvertedReference<\n                 const MakerTypeFor<Target, Args...>&>,\n             args) {}\n\n  // Constructs `Initializer<T>` from a factory function for `T` packed in\n  // `riegeli::Invoker(function, args...)` with a possibly different but\n  // compatible function result.\n  template <\n      typename Function, typename... Args,\n      std::enable_if_t<std::conjunction_v<\n                           std::negation<std::is_convertible<\n                               InvokerType<Function, Args...>&&, T>>,\n                           IsConvertibleFromResult<\n                               T, std::invoke_result_t<Function&&, Args&&...>>>,\n                       int> = 0>\n  /*implicit*/ Initializer(\n      InvokerType<Function, Args...>&& invoker ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : Base(&Base::template kMethodsFromObject<InvokerType<Function, Args...>>,\n             std::move(invoker)) {}\n  template <\n      typename Function, typename... Args,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::negation<std::is_convertible<\n                  const InvokerType<Function, Args...>&, T>>,\n              IsConvertibleFromResult<\n                  T, std::invoke_result_t<const Function&, const Args&...>>>,\n          int> = 0>\n  /*implicit*/ Initializer(const InvokerType<Function, Args...>& invoker\n                               ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : Base(&Base::template kMethodsFromObject<\n                 const InvokerType<Function, Args...>&>,\n             invoker) {}\n\n  // Constructs `Initializer<T>` from `Initializer<Target>` with a different but\n  // compatible `Target`.\n  template <\n      typename Target,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::negation<std::is_same<Target, T>>,\n              std::negation<std::is_convertible<Initializer<Target>&&, T>>,\n              IsConvertibleFromResult<T, Target&&>>,\n          int> = 0>\n  /*implicit*/ Initializer(Initializer<Target>&& initializer)\n      : Base(\n            &Base::template kMethodsFromConvertedReference<Initializer<Target>>,\n            std::move(initializer)) {}\n\n  Initializer(Initializer&& that) = default;\n  Initializer& operator=(Initializer&&) = delete;\n\n  // For `ABSL_NULLABILITY_COMPATIBLE`.\n  using pointer = std::conditional_t<std::is_pointer_v<T>, T, void*>;\n};\n\n// `Target<T>::type` and `TargetT<T>` deduce the appropriate target type such\n// that `T` is convertible to `Initializer<TargetT<T>>`.\n//\n// This allows a single template to uniformly handle a `Target` passed directly,\n// as `riegeli::Maker<Target>(args...)`, as\n// `riegeli::Invoker(function, args...)`, or as `Initializer<Target>`. This is\n// also useful for CTAD guides to deduce a template argument as `TargetT<T>`.\n//\n// They are undefined in the case of `riegeli::Maker(args...)` which requires\n// the target type to be specified by the caller, or when the object is not\n// usable in the given const and reference context.\n\nnamespace initializer_internal {\n\ntemplate <typename Value, typename Reference>\nstruct TargetImpl {\n  using type = Value;\n};\n\ntemplate <typename T, typename Reference>\nstruct TargetImpl<std::reference_wrapper<T>, Reference> {\n  using type = T&;\n};\n\ntemplate <typename... Args, typename Reference>\nstruct TargetImpl<MakerType<Args...>, Reference> {\n  // No `type` member when the target type is unspecified.\n};\n\ntemplate <typename Target, typename... Args, typename Reference>\nstruct TargetImpl<MakerTypeFor<Target, Args...>, Reference>\n    : MakerTarget<Reference> {};\n\ntemplate <typename Function, typename... Args, typename Reference>\nstruct TargetImpl<InvokerType<Function, Args...>, Reference>\n    : InvokerTarget<Reference> {};\n\ntemplate <typename T, typename Reference>\nstruct TargetImpl<Initializer<T>, Reference> {\n  using type = T;\n};\n\n};  // namespace initializer_internal\n\ntemplate <typename T>\nstruct Target : initializer_internal::TargetImpl<std::decay_t<T>, T&&> {};\n\ntemplate <typename T>\nusing TargetT = typename Target<T>::type;\n\n// `TargetRef<T>::type` and `TargetRefT<T>` are like `TargetT<T>`, but if the\n// object is already constructed, then they are the corresponding reference type\n// instead of the value type. It is still true that `T` is convertible to\n// `Initializer<TargetRefT<T>>`.\n//\n// This allows to avoid moving or copying the object if a reference to it is\n// sufficient.\n\nnamespace initializer_internal {\n\ntemplate <typename Value, typename Reference>\nstruct TargetRefImpl {\n  using type = Reference;\n};\n\ntemplate <typename T, typename Reference>\nstruct TargetRefImpl<std::reference_wrapper<T>, Reference> {\n  using type = T&;\n};\n\ntemplate <typename... Args, typename Reference>\nstruct TargetRefImpl<MakerType<Args...>, Reference> {\n  // No `type` member when the target type is unspecified.\n};\n\ntemplate <typename Target, typename... Args, typename Reference>\nstruct TargetRefImpl<MakerTypeFor<Target, Args...>, Reference>\n    : MakerTarget<Reference> {};\n\ntemplate <typename Function, typename... Args, typename Reference>\nstruct TargetRefImpl<InvokerType<Function, Args...>, Reference>\n    : InvokerTargetRef<Reference> {};\n\ntemplate <typename T, typename Reference>\nstruct TargetRefImpl<Initializer<T>, Reference> {\n  using type = T;\n};\n\n};  // namespace initializer_internal\n\ntemplate <typename T>\nstruct TargetRef : initializer_internal::TargetRefImpl<std::decay_t<T>, T&&> {};\n\ntemplate <typename T>\nusing TargetRefT = typename TargetRef<T>::type;\n\n// Implementation details follow.\n\nnamespace initializer_internal {\n\ntemplate <typename T>\ninline InitializerBase<T>::InitializerBase(const Methods* methods)\n    : methods_(methods) {}\n\ntemplate <typename T>\ntemplate <typename Arg>\ninline InitializerBase<T>::InitializerBase(const Methods* methods, Arg&& arg)\n    : methods_(methods), context_(std::forward<Arg>(arg)) {}\n\ntemplate <typename T>\nT InitializerBase<T>::ConstructMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef context) {\n  return T();\n}\n\ntemplate <typename T>\ntemplate <typename Arg>\nT InitializerBase<T>::ConstructMethodFromObject(TypeErasedRef context) {\n  return T(context.Cast<Arg>());\n}\n\ntemplate <typename T>\ntemplate <typename... Args>\nT InitializerBase<T>::ConstructMethodFromMaker(TypeErasedRef context) {\n  return context.Cast<MakerType<Args...>>().template Construct<T>();\n}\n\ntemplate <typename T>\ntemplate <typename... Args>\nT InitializerBase<T>::ConstructMethodFromConstMaker(TypeErasedRef context) {\n  return context.Cast<const MakerType<Args...>&>().template Construct<T>();\n}\n\ntemplate <typename T>\ntemplate <typename Arg>\nT InitializerBase<T>::ConstructMethodFromConvertedReference(\n    TypeErasedRef context) {\n  return T(context.Cast<Arg>().Reference());\n}\n\ntemplate <typename T>\nT&& InitializerBase<T>::ReferenceMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef context,\n    TemporaryStorage<T>&& storage) {\n  return std::move(storage).emplace();\n}\n\ntemplate <typename T>\ntemplate <typename Arg>\nT&& InitializerBase<T>::ReferenceMethodFromObject(\n    TypeErasedRef context, TemporaryStorage<T>&& storage) {\n  if constexpr (CanBindReference<T&&, Arg&&>::value) {\n    return BindReference<T&&>(context.Cast<Arg>());\n  } else {\n    return std::move(storage).emplace(context.Cast<Arg>());\n  }\n}\n\ntemplate <typename T>\ntemplate <typename... Args>\nT&& InitializerBase<T>::ReferenceMethodFromMaker(\n    TypeErasedRef context, TemporaryStorage<T>&& storage) {\n  return context.Cast<MakerType<Args...>>().template Reference<T>(\n      std::move(storage));\n}\n\ntemplate <typename T>\ntemplate <typename... Args>\nT&& InitializerBase<T>::ReferenceMethodFromConstMaker(\n    TypeErasedRef context, TemporaryStorage<T>&& storage) {\n  return context.Cast<const MakerType<Args...>&>().template Reference<T>(\n      std::move(storage));\n}\n\ntemplate <typename T>\ntemplate <typename Arg>\nT&& InitializerBase<T>::ReferenceMethodFromConvertedReference(\n    TypeErasedRef context, TemporaryStorage<T>&& storage) {\n  return std::move(storage).emplace(context.Cast<Arg>().Reference());\n}\n\ntemplate <typename T>\nvoid InitializerAssignableBase<T>::ResetMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef context, T& dest) {\n  riegeli::Reset(dest);\n}\n\ntemplate <typename T>\ntemplate <typename Arg>\nvoid InitializerAssignableBase<T>::ResetMethodFromObject(TypeErasedRef context,\n                                                         T& dest) {\n  riegeli::Reset(dest, context.Cast<Arg>());\n}\n\ntemplate <typename T>\ntemplate <typename... Args>\nvoid InitializerAssignableBase<T>::ResetMethodFromMaker(TypeErasedRef context,\n                                                        T& dest) {\n  riegeli::Reset(dest, context.Cast<MakerType<Args...>>());\n}\n\ntemplate <typename T>\ntemplate <typename... Args>\nvoid InitializerAssignableBase<T>::ResetMethodFromConstMaker(\n    TypeErasedRef context, T& dest) {\n  riegeli::Reset(dest, context.Cast<const MakerType<Args...>&>());\n}\n\ntemplate <typename T>\ntemplate <typename Arg>\nvoid InitializerAssignableBase<T>::ResetMethodFromConvertedReference(\n    TypeErasedRef context, T& dest) {\n  riegeli::Reset(dest, context.Cast<Arg>().Reference());\n}\n\ntemplate <typename T>\ninline InitializerReference<T>::InitializerReference(const Methods* methods)\n    : methods_(methods) {}\n\ntemplate <typename T>\ntemplate <typename Arg>\ninline InitializerReference<T>::InitializerReference(const Methods* methods,\n                                                     Arg&& arg)\n    : methods_(methods), context_(std::forward<Arg>(arg)) {}\n\ntemplate <typename T>\ntemplate <typename Arg>\nT InitializerReference<T>::ConstructMethodFromObject(TypeErasedRef context) {\n  return T(context.Cast<Arg>());\n}\n\ntemplate <typename T>\ntemplate <typename... Args>\nT InitializerReference<T>::ConstructMethodFromMaker(TypeErasedRef context) {\n  return context.Cast<MakerType<Args...>>().template Construct<T>();\n}\n\ntemplate <typename T>\ntemplate <typename... Args>\nT InitializerReference<T>::ConstructMethodFromConstMaker(\n    TypeErasedRef context) {\n  return context.Cast<const MakerType<Args...>&>().template Construct<T>();\n}\n\ntemplate <typename T>\ntemplate <typename Arg>\nT InitializerReference<T>::ConstructMethodFromConvertedReference(\n    TypeErasedRef context) {\n  return T(context.Cast<Arg>().Reference());\n}\n\n}  // namespace initializer_internal\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_INITIALIZER_H_\n"
  },
  {
    "path": "riegeli/base/initializer_internal.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_INITIALIZER_INTERNAL_H_\n#define RIEGELI_BASE_INITIALIZER_INTERNAL_H_\n\n#include <stddef.h>\n\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/casts.h\"\n#include \"absl/base/nullability.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli::initializer_internal {\n\n// `CanBindReference<T&&, Arg&&>::value` is `true` if `Arg&&` can be implicitly\n// converted to `T&&` without creating a temporary.\n//\n// Due to not all compilers implementing http://wg21.link/cwg2352 (converting\n// `T*&` to `const T* const&` could have bound the result to a temporary),\n// this covers also the case when the corresponding pointers can be converted.\n// `BindReference()` should be used for the actual conversion.\n\ntemplate <typename T, typename Arg>\nstruct CanBindReference : std::false_type {};\n\ntemplate <typename T, typename Arg>\nstruct CanBindReference<T&, Arg&> : std::is_convertible<Arg*, T*> {};\n\ntemplate <typename T, typename Arg>\nstruct CanBindReference<T&, Arg&&> : std::false_type {};\n\ntemplate <typename T, typename Arg>\nstruct CanBindReference<const T&, Arg&&> : std::is_convertible<Arg*, const T*> {\n};\n\ntemplate <typename T, typename Arg>\nstruct CanBindReference<T&&, Arg&> : std::false_type {};\n\ntemplate <typename T, typename Arg>\nstruct CanBindReference<T&&, Arg&&> : std::is_convertible<Arg*, T*> {};\n\n// `BindReference<T&&>(arg)` returns `arg` implicitly converted to `T&&`.\n//\n// Due to not all compilers implementing http://wg21.link/cwg2352 (converting\n// `T*&` to `const T* const&` could have bound the result to a temporary),\n// this is not implemented as a simple implicit conversion, but by converting\n// the reference to a pointer, implicitly converting the pointer, and\n// dereferencing back.\ntemplate <typename T, typename Arg,\n          std::enable_if_t<CanBindReference<T&&, Arg&&>::value, int> = 0>\ninline T&& BindReference(Arg&& arg) {\n  return std::forward<T>(\n      *absl::implicit_cast<std::remove_reference_t<T>*>(&arg));\n}\n\n}  // namespace riegeli::initializer_internal\n\n#endif  // RIEGELI_BASE_INITIALIZER_INTERNAL_H_\n"
  },
  {
    "path": "riegeli/base/intrusive_shared_ptr.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_INTRUSIVE_SHARED_PTR_H_\n#define RIEGELI_BASE_INTRUSIVE_SHARED_PTR_H_\n\n#include <stddef.h>\n\n#include <cstddef>\n#include <memory>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/external_data.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/ownership.h\"\n\nnamespace riegeli {\n\nnamespace intrusive_shared_ptr_internal {\n\ntemplate <typename T, typename Enable = void>\nstruct HasHasUniqueOwner : std::false_type {};\n\ntemplate <typename T>\nstruct HasHasUniqueOwner<\n    T, std::enable_if_t<std::is_convertible_v<\n           decltype(std::declval<const T&>().HasUniqueOwner()), bool>>>\n    : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct HasGetCount : std::false_type {};\n\ntemplate <typename T>\nstruct HasGetCount<T,\n                   std::enable_if_t<std::is_convertible_v<\n                       decltype(std::declval<const T&>().GetCount()), size_t>>>\n    : std::true_type {};\n\n}  // namespace intrusive_shared_ptr_internal\n\n// `IntrusiveSharedPtr<T>` implements shared ownership of an object of type `T`.\n// It can also be empty, with the pointer being `nullptr`.\n//\n// The actual object can be of a subtype of `T`, as long as `T::Unref()`\n// correctly deletes the object in such a case, which typically requires that\n// `T` has a virtual destructor.\n//\n// `T` maintains its own reference count, e.g. as a member of type `RefCount`.\n// `T` should support:\n//\n// ```\n//   // Increments the reference count of `*this`.\n//   void Ref() const;\n//\n//   // Decrements the reference count of `*this`. Deletes `this` when the\n//   // reference count reaches 0.\n//   void Unref() const;\n//\n//   // Returns `true` if there is only one owner of the object.\n//   //\n//   // This can be used to check if the object may be modified.\n//   //\n//   // Optional. Needed for `IntrusiveSharedPtr::IsUnique()`.\n//   bool HasUniqueOwner() const;\n// ```\n//\n// Compared to `std::shared_ptr`, `IntrusiveSharedPtr` supports `IsUnique()`,\n// and has a smaller memory overhead (the pointer has 1 word instead of 2, the\n// object typically has 1 word of overhead instead of 3). OTOH it requires\n// cooperation from `T`, and has fewer features, e.g. no aliasing constructor,\n// no weak pointers.\n//\n// Compared to `SharedPtr`, `IntrusiveSharedPtr` is harder to use, because\n// it requires the object to maintain its own reference count. OTOH\n// `IntrusiveSharedPtr` supports custom allocation and deallocation, and\n// conversion to an `IntrusiveSharedPtr` to a non-leftmost or virtual base\n// class. Prefer `SharedPtr` unless `IntrusiveSharedPtr` is needed.\ntemplate <typename T>\nclass ABSL_ATTRIBUTE_TRIVIAL_ABI ABSL_NULLABILITY_COMPATIBLE IntrusiveSharedPtr\n    : public WithEqual<IntrusiveSharedPtr<T>> {\n public:\n  // Creates an empty `IntrusiveSharedPtr`.\n  constexpr IntrusiveSharedPtr() = default;\n  /*implicit*/ constexpr IntrusiveSharedPtr(std::nullptr_t) noexcept {}\n  IntrusiveSharedPtr& operator=(std::nullptr_t) {\n    Reset();\n    return *this;\n  }\n\n  // Creates an `IntrusiveSharedPtr` holding `ptr`.\n  //\n  // Takes ownership of `ptr` unless the second parameter is `kShareOwnership`.\n  explicit IntrusiveSharedPtr(T* ptr ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                              PassOwnership = kPassOwnership) noexcept\n      : ptr_(ptr) {}\n  explicit IntrusiveSharedPtr(T* ptr ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                              ShareOwnership) noexcept\n      : ptr_(Ref(ptr)) {}\n\n  // Creates an `IntrusiveSharedPtr` holding a constructed value.\n  //\n  // The object is constructed with `new`, which means that `T::Unref()` should\n  // delete the object with `delete this`.\n  explicit IntrusiveSharedPtr(Initializer<T> value) : ptr_(std::move(value)) {}\n\n  // Creates an `IntrusiveSharedPtr` holding a constructed value of a compatible\n  // type.\n  //\n  // The object is constructed with `new`, which means that `T::Unref()` should\n  // delete the object with `delete this`.\n  template <typename SubInitializer,\n            std::enable_if_t<\n                std::is_convertible_v<TargetT<SubInitializer>*, T*>, int> = 0>\n  explicit IntrusiveSharedPtr(SubInitializer&& value)\n      : ptr_(Initializer<TargetT<SubInitializer>>(\n            std::forward<SubInitializer>(value))) {}\n\n  // Converts from an `IntrusiveSharedPtr` with a compatible type.\n  template <typename SubT,\n            std::enable_if_t<std::is_convertible_v<SubT*, T*>, int> = 0>\n  /*implicit*/ IntrusiveSharedPtr(const IntrusiveSharedPtr<SubT>& that) noexcept\n      : ptr_(Ref(that.ptr_.get())) {}\n  template <typename SubT,\n            std::enable_if_t<std::is_convertible_v<SubT*, T*>, int> = 0>\n  IntrusiveSharedPtr& operator=(const IntrusiveSharedPtr<SubT>& that) noexcept {\n    ptr_.reset(Ref(that.ptr_.get()));\n    return *this;\n  }\n\n  // Converts from an `IntrusiveSharedPtr` with a compatible type.\n  //\n  // The source `IntrusiveSharedPtr` is left empty.\n  template <typename SubT,\n            std::enable_if_t<std::is_convertible_v<SubT*, T*>, int> = 0>\n  /*implicit*/ IntrusiveSharedPtr(IntrusiveSharedPtr<SubT>&& that) noexcept\n      : ptr_(std::move(that).ptr_) {}\n  template <typename SubT,\n            std::enable_if_t<std::is_convertible_v<SubT*, T*>, int> = 0>\n  IntrusiveSharedPtr& operator=(IntrusiveSharedPtr<SubT>&& that) noexcept {\n    ptr_.reset(std::move(that).ptr_);\n    return *this;\n  }\n\n  IntrusiveSharedPtr(const IntrusiveSharedPtr& that) noexcept\n      : ptr_(Ref(that.ptr_.get())) {}\n  IntrusiveSharedPtr& operator=(const IntrusiveSharedPtr& that) noexcept {\n    ptr_.reset(Ref(that.ptr_.get()));\n    return *this;\n  }\n\n  // The source `IntrusiveSharedPtr` is left empty.\n  IntrusiveSharedPtr(IntrusiveSharedPtr&& that) = default;\n  IntrusiveSharedPtr& operator=(IntrusiveSharedPtr&& that) = default;\n\n  // Replaces the object, or makes `*this` empty if `ptr == nullptr`.\n  //\n  // Takes ownership of `ptr` unless the second parameter is `kShareOwnership`.\n  //\n  // The old object, if any, is destroyed afterwards.\n  ABSL_ATTRIBUTE_REINITIALIZES\n  void Reset(T* ptr = nullptr, PassOwnership = kPassOwnership) {\n    ptr_.reset(ptr);\n  }\n  ABSL_ATTRIBUTE_REINITIALIZES\n  void Reset(T* ptr, ShareOwnership) { ptr_.reset(Ref(ptr)); }\n\n  // Replaces the object with a constructed value.\n  //\n  // The old object, if any, is destroyed afterwards.\n  //\n  // The object is constructed with `new`, which means that `T::Unref()` should\n  // delete the object with `delete this`.\n  //\n  // If `T` supports `HasUniqueOwner()` and `*this` is the only owner of an\n  // object known to have the same move-assignable type, the existing object is\n  // assigned or reset instead of allocating and constructing a new object.\n  ABSL_ATTRIBUTE_REINITIALIZES\n  void Reset(Initializer<T> value) { ResetImpl(std::move(value)); }\n\n  // Replaces the object with a constructed value of a compatible type.\n  //\n  // The old object, if any, is destroyed afterwards.\n  //\n  // The object is constructed with `new`, which means that `T::Unref()` should\n  // delete the object with `delete this`.\n  template <typename SubInitializer,\n            std::enable_if_t<\n                std::is_convertible_v<TargetT<SubInitializer>*, T*>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(SubInitializer&& value) {\n    ptr_ = Initializer<TargetT<SubInitializer>>(\n        std::forward<SubInitializer>(value));\n  }\n\n  // Returns `true` if `*this` is the only owner of the object.\n  //\n  // This can be used to check if the object may be modified (in contrast to\n  // `std::shared_ptr::unique()`).\n  //\n  // If `*this` is empty, returns `false`.\n  //\n  // Supported if `T` supports `HasUniqueOwner()`.\n  template <typename DependentT = T,\n            std::enable_if_t<intrusive_shared_ptr_internal::HasHasUniqueOwner<\n                                 DependentT>::value,\n                             int> = 0>\n  bool IsUnique() const {\n    return ptr_ != nullptr && ptr_->HasUniqueOwner();\n  }\n\n  // Returns the current reference count.\n  //\n  // If the `IntrusiveSharedPtr` is accessed by multiple threads, this is a\n  // snapshot of the count which may change asynchronously, hence usage of\n  // `GetRefCount()` should be limited to cases not important for correctness,\n  // like producing debugging output.\n  //\n  // The reference count can be reliably compared against 1 with `IsUnique()`.\n  //\n  // Supported if `T` supports `GetCount()`.\n  template <typename DependentT = T,\n            std::enable_if_t<\n                intrusive_shared_ptr_internal::HasGetCount<DependentT>::value,\n                int> = 0>\n  size_t GetRefCount() const {\n    if (ptr_ == nullptr) return 0;\n    return ptr_->GetRefCount();\n  }\n\n  // Returns the pointer.\n  T* get() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return ptr_.get(); }\n\n  // Dereferences the pointer.\n  T& operator*() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_NE(ptr_, nullptr)\n        << \"Failed precondition of IntrusiveSharedPtr::operator*: null pointer\";\n    return *ptr_;\n  }\n  T* operator->() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_NE(ptr_, nullptr)\n        << \"Failed precondition of IntrusiveSharedPtr::operator->: null \"\n           \"pointer\";\n    return ptr_.get();\n  }\n\n  // Returns the pointer. This `IntrusiveSharedPtr` is left empty.\n  T* Release() { return ptr_.release(); }\n\n  template <typename OtherT>\n  friend bool operator==(const IntrusiveSharedPtr& a,\n                         const IntrusiveSharedPtr<OtherT>& b) {\n    return a.get() == b.get();\n  }\n  friend bool operator==(const IntrusiveSharedPtr& a, std::nullptr_t) {\n    return a.get() == nullptr;\n  }\n\n  // Indicates support for:\n  //  * `ExternalRef(const IntrusiveSharedPtr&, substr)`\n  //  * `ExternalRef(IntrusiveSharedPtr&&, substr)`\n  friend void RiegeliSupportsExternalRef(const IntrusiveSharedPtr*) {}\n\n  // Supports `ExternalRef`.\n  friend ExternalStorage RiegeliToExternalStorage(IntrusiveSharedPtr* self) {\n    return ExternalStorage(const_cast<std::remove_cv_t<T>*>(self->Release()),\n                           [](void* ptr) {\n                             if (ptr != nullptr) static_cast<T*>(ptr)->Unref();\n                           });\n  }\n\n  // Supports `riegeli::Debug()`.\n  template <typename DebugStream>\n  friend void RiegeliDebug(const IntrusiveSharedPtr& src, DebugStream& dest) {\n    dest.Debug(src.get());\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const IntrusiveSharedPtr* self,\n                                        MemoryEstimator& memory_estimator) {\n    if (memory_estimator.RegisterNode(self->get())) {\n      memory_estimator.RegisterDynamicObject(self->get());\n    }\n  }\n\n private:\n  // For converting from a `SharedPtr` with a compatible type.\n  template <typename SubT>\n  friend class IntrusiveSharedPtr;\n\n  using pointer = T*;  // For `ABSL_NULLABILITY_COMPATIBLE`.\n\n  struct Unrefer {\n    void operator()(T* ptr) const { ptr->Unref(); }\n  };\n\n  template <typename SubT>\n  static SubT* Ref(SubT* ptr) {\n    if (ptr != nullptr) ptr->Ref();\n    return ptr;\n  }\n\n  template <typename DependentT>\n  struct IsAssignable\n      : std::conjunction<\n            intrusive_shared_ptr_internal::HasHasUniqueOwner<DependentT>,\n            std::disjunction<\n                std::negation<std::has_virtual_destructor<DependentT>>,\n                std::is_final<DependentT>>,\n            std::is_move_assignable<DependentT>> {};\n\n  void ResetImpl(Initializer<T> value) {\n    if constexpr (IsAssignable<T>::value) {\n      if (IsUnique()) {\n        *ptr_ = std::move(value);\n        return;\n      }\n    }\n    ptr_ = std::move(value);\n  }\n\n  std::unique_ptr<T, Unrefer> ptr_;\n};\n\ntemplate <typename T>\nexplicit IntrusiveSharedPtr(T* ptr, PassOwnership = kPassOwnership)\n    -> IntrusiveSharedPtr<T>;\ntemplate <typename T>\nexplicit IntrusiveSharedPtr(T* ptr, ShareOwnership) -> IntrusiveSharedPtr<T>;\ntemplate <typename T, std::enable_if_t<!std::is_pointer_v<T>, int> = 0>\nexplicit IntrusiveSharedPtr(T&& value) -> IntrusiveSharedPtr<TargetT<T>>;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_INTRUSIVE_SHARED_PTR_H_\n"
  },
  {
    "path": "riegeli/base/invoker.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_INVOKER_H_\n#define RIEGELI_BASE_INVOKER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <tuple>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"riegeli/base/type_traits.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\ntemplate <typename Function, typename... Args>\nclass InvokerType;\n\nnamespace invoker_internal {\n\ntemplate <typename Function, typename... Args>\nclass InvokerBase\n    : public ConditionallyAssignable<\n          std::conjunction_v<std::negation<std::is_reference<Args>>...>> {\n protected:\n  template <typename DependentFunction = Function>\n  using Result = std::invoke_result_t<DependentFunction&&, Args&&...>;\n  template <typename DependentFunction = Function>\n  using ConstResult =\n      std::invoke_result_t<const DependentFunction&, const Args&...>;\n\n public:\n  // Constructs `InvokerType` from `function` convertible to `Function` and\n  // `args...` convertible to `Args...`.\n  template <\n      typename SrcFunction, typename... SrcArgs,\n      std::enable_if_t<\n          std::conjunction_v<NotSameRef<InvokerBase, SrcFunction, SrcArgs...>,\n                             std::is_invocable<Function&&, Args&&...>,\n                             std::is_convertible<SrcFunction&&, Function>,\n                             std::is_convertible<SrcArgs&&, Args>...>,\n          int> = 0>\n  /*implicit*/ InvokerBase(SrcFunction&& function, SrcArgs&&... args)\n      : function_(std::forward<SrcFunction>(function)),\n        args_(std::forward<SrcArgs>(args)...) {}\n\n  InvokerBase(InvokerBase&& that) = default;\n  InvokerBase& operator=(InvokerBase&& that) = default;\n\n  InvokerBase(const InvokerBase& that) = default;\n  InvokerBase& operator=(const InvokerBase& that) = default;\n\n  // Invokes the function.\n  //\n  // Usually conversion to the result of invocation is preferred because it can\n  // avoid creating a temporary if the context accepts an arbitrary type\n  // convertible to the result of invocation. An explicit `Invoke()` call can\n  // force construction right away while avoiding specifying the full result\n  // type.\n  template <typename DependentFunction = Function>\n  Result<DependentFunction> Invoke() && {\n    return std::apply(std::forward<Function>(function_), std::move(args_));\n  }\n  template <typename DependentFunction = Function>\n  ConstResult<DependentFunction> Invoke() const& {\n    return std::apply(function_, args_);\n  }\n\n  // Extracts the function.\n  Function& function() & { return function_; }\n  const Function& function() const& { return function_; }\n  Function&& function() && { return std::move(function_); }\n  const Function&& function() const&& { return std::move(function_); }\n\n  // Extracts the given argument.\n  template <size_t index, std::enable_if_t<(index < sizeof...(Args)), int> = 0>\n  std::tuple_element_t<index, std::tuple<Args...>>& arg() & {\n    return std::get<index>(args_);\n  }\n  template <size_t index, std::enable_if_t<(index < sizeof...(Args)), int> = 0>\n  const std::tuple_element_t<index, std::tuple<Args...>>& arg() const& {\n    return std::get<index>(args_);\n  }\n  template <size_t index, std::enable_if_t<(index < sizeof...(Args)), int> = 0>\n  std::tuple_element_t<index, std::tuple<Args...>>& arg() && {\n    return std::get<index>(std::move(args_));\n  }\n  template <size_t index, std::enable_if_t<(index < sizeof...(Args)), int> = 0>\n  const std::tuple_element_t<index, std::tuple<Args...>>& arg() const&& {\n    return std::get<index>(std::move(args_));\n  }\n\n private:\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS Function function_;\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS std::tuple<Args...> args_;\n};\n\ntemplate <typename Enable, typename Function, typename... Args>\nclass InvokerConditionalConversion : public InvokerBase<Function, Args...> {\n private:\n  using Result =\n      typename InvokerConditionalConversion::InvokerBase::template Result<>;\n  using ConstResult = typename InvokerConditionalConversion::InvokerBase::\n      template ConstResult<>;\n\n public:\n  using InvokerConditionalConversion::InvokerBase::InvokerBase;\n\n  InvokerConditionalConversion(InvokerConditionalConversion&& that) = default;\n  InvokerConditionalConversion& operator=(InvokerConditionalConversion&& that) =\n      default;\n\n  InvokerConditionalConversion(const InvokerConditionalConversion& that) =\n      default;\n  InvokerConditionalConversion& operator=(\n      const InvokerConditionalConversion& that) = default;\n\n  // Invokes the function.\n  /*implicit*/ operator Result() && { return std::move(*this).Invoke(); }\n  // Invokes the function.\n  /*implicit*/ operator ConstResult() const& { return this->Invoke(); }\n};\n\n// Disable const functionality when the const function is not invocable with the\n// const arguments.\ntemplate <typename Function, typename... Args>\nclass InvokerConditionalConversion<\n    std::enable_if_t<std::conjunction_v<\n        std::is_invocable<Function&&, Args&&...>,\n        std::negation<std::is_invocable<const Function&, const Args&...>>>>,\n    Function, Args...> : public InvokerBase<Function, Args...> {\n private:\n  using Result =\n      typename InvokerConditionalConversion::InvokerBase::template Result<>;\n\n public:\n  using InvokerConditionalConversion::InvokerBase::InvokerBase;\n\n  InvokerConditionalConversion(InvokerConditionalConversion&& that) = default;\n  InvokerConditionalConversion& operator=(InvokerConditionalConversion&& that) =\n      default;\n\n  InvokerConditionalConversion(const InvokerConditionalConversion& that) =\n      default;\n  InvokerConditionalConversion& operator=(\n      const InvokerConditionalConversion& that) = default;\n\n  // Invokes the function.\n  /*implicit*/ operator Result() && { return std::move(*this).Invoke(); }\n};\n\n// Disable functionality when the function is not invocable with the arguments.\ntemplate <typename Function, typename... Args>\nclass InvokerConditionalConversion<\n    std::enable_if_t<!std::is_invocable_v<Function&&, Args&&...>>, Function,\n    Args...> : public InvokerBase<Function, Args...> {\n public:\n  using InvokerConditionalConversion::InvokerBase::InvokerBase;\n\n  InvokerConditionalConversion(InvokerConditionalConversion&& that) = default;\n  InvokerConditionalConversion& operator=(InvokerConditionalConversion&& that) =\n      default;\n\n  InvokerConditionalConversion(const InvokerConditionalConversion& that) =\n      default;\n  InvokerConditionalConversion& operator=(\n      const InvokerConditionalConversion& that) = default;\n};\n\n}  // namespace invoker_internal\n\n// `InvokerType<Function, Args...>`, usually made with\n// `riegeli::Invoker(function, args...)`, packs a function together with its\n// arguments. `InvokerType<Function, Args...>` is convertible to\n// `Initializer<T>` when the result of `Function` is convertible to `T`.\n//\n// This allows the function taking `Initializer<T>` to construct the object\n// in-place, avoiding constructing a temporary and moving from it.\n//\n// `InvokerType` complements `MakerType` by extending constructors with factory\n// functions.\n//\n// The function and arguments are interpreted as by `std::invoke()`: the\n// function can also be a member pointer, in which case the first argument is\n// the target reference, reference wrapper, or pointer.\ntemplate <typename Function, typename... Args>\nclass InvokerType\n    : public invoker_internal::InvokerConditionalConversion<void, Function,\n                                                            Args...> {\n private:\n  template <typename DependentFunction = Function>\n  using Result =\n      typename InvokerType::InvokerBase::template Result<DependentFunction>;\n  template <typename DependentFunction = Function>\n  using ConstResult = typename InvokerType::InvokerBase::template ConstResult<\n      DependentFunction>;\n\n public:\n  using InvokerType::InvokerConditionalConversion::InvokerConditionalConversion;\n\n  InvokerType(InvokerType&& that) = default;\n  InvokerType& operator=(InvokerType&& that) = default;\n\n  InvokerType(const InvokerType& that) = default;\n  InvokerType& operator=(const InvokerType& that) = default;\n\n  // Invokes the function and stores `std::decay_t` of the result of invocation\n  // on the heap.\n  //\n  // In contrast to `std::make_unique()`, this supports custom deleters.\n  //\n  // For a non-default-constructed deleter, use `UniquePtr(deleter)`.\n  template <\n      typename Target, typename Deleter, typename DependentFunction = Function,\n      std::enable_if_t<\n          std::conjunction_v<\n              IsConstructibleFromResult<std::decay_t<Result<DependentFunction>>,\n                                        Result<DependentFunction>>,\n              std::is_convertible<std::decay_t<Result<DependentFunction>>*,\n                                  Target*>>,\n          int> = 0>\n  /*implicit*/ operator std::unique_ptr<Target, Deleter>() && {\n    return std::move(*this).template UniquePtr<Deleter>();\n  }\n  template <\n      typename Target, typename Deleter, typename DependentFunction = Function,\n      std::enable_if_t<\n          std::conjunction_v<\n              IsConstructibleFromResult<\n                  std::decay_t<ConstResult<DependentFunction>>,\n                  ConstResult<DependentFunction>>,\n              std::is_convertible<std::decay_t<ConstResult<DependentFunction>>*,\n                                  Target*>>,\n          int> = 0>\n  /*implicit*/ operator std::unique_ptr<Target, Deleter>() const& {\n    return UniquePtr<Deleter>();\n  }\n\n  // Invokes the function and stores `std::decay_t` of the result of invocation\n  // on the heap.\n  //\n  // In contrast to `std::make_unique()`, this supports custom deleters.\n  //\n  // Usually conversion to `std::unique_ptr` is preferred because it leads to\n  // simpler source code. An explicit `UniquePtr()` call can force construction\n  // right away while avoiding writing the full target type, and it allows to\n  // use a non-default-constructed deleter.\n  //\n  // The `default_deleter` template parameter lets `UniquePtr<T>()` with an\n  // explicit template argument unambiguously call another overload of\n  // `UniquePtr()`.\n\n  template <int default_deleter = 0, typename DependentFunction = Function,\n            std::enable_if_t<IsConstructibleFromResult<\n                                 std::decay_t<Result<DependentFunction>>,\n                                 Result<DependentFunction>>::value,\n                             int> = 0>\n  std::unique_ptr<std::decay_t<Result<DependentFunction>>> UniquePtr() && {\n    return std::unique_ptr<std::decay_t<Result<>>>(\n        new std::decay_t<Result<>>(std::move(*this)));\n  }\n  template <int default_deleter = 0, typename DependentFunction = Function,\n            std::enable_if_t<IsConstructibleFromResult<\n                                 std::decay_t<ConstResult<DependentFunction>>,\n                                 ConstResult<DependentFunction>>::value,\n                             int> = 0>\n  std::unique_ptr<std::decay_t<ConstResult<DependentFunction>>> UniquePtr()\n      const& {\n    return std::unique_ptr<std::decay_t<ConstResult<>>>(\n        new std::decay_t<ConstResult<>>(*this));\n  }\n\n  template <typename Deleter, typename DependentFunction = Function,\n            std::enable_if_t<IsConstructibleFromResult<\n                                 std::decay_t<Result<DependentFunction>>,\n                                 Result<DependentFunction>>::value,\n                             int> = 0>\n  std::unique_ptr<std::decay_t<Result<DependentFunction>>, Deleter>\n  UniquePtr() && {\n    return std::unique_ptr<std::decay_t<Result<>>, Deleter>(\n        new std::decay_t<Result<>>(std::move(*this)));\n  }\n  template <typename Deleter, typename DependentFunction = Function,\n            std::enable_if_t<IsConstructibleFromResult<\n                                 std::decay_t<ConstResult<DependentFunction>>,\n                                 ConstResult<DependentFunction>>::value,\n                             int> = 0>\n  std::unique_ptr<std::decay_t<ConstResult<DependentFunction>>, Deleter>\n  UniquePtr() const& {\n    return std::unique_ptr<std::decay_t<ConstResult<>>, Deleter>(\n        new std::decay_t<ConstResult<>>(*this));\n  }\n\n  template <typename Deleter, typename DependentFunction = Function,\n            std::enable_if_t<IsConstructibleFromResult<\n                                 std::decay_t<Result<DependentFunction>>,\n                                 Result<DependentFunction>>::value,\n                             int> = 0>\n  std::unique_ptr<std::decay_t<Result<DependentFunction>>, Deleter> UniquePtr(\n      Deleter&& deleter) && {\n    return std::unique_ptr<std::decay_t<Result<>>, Deleter>(\n        new std::decay_t<Result<>>(std::move(*this)),\n        std::forward<Deleter>(deleter));\n  }\n  template <typename Deleter, typename DependentFunction = Function,\n            std::enable_if_t<IsConstructibleFromResult<\n                                 std::decay_t<ConstResult<DependentFunction>>,\n                                 ConstResult<DependentFunction>>::value,\n                             int> = 0>\n  std::unique_ptr<std::decay_t<ConstResult<DependentFunction>>, Deleter>\n  UniquePtr(Deleter&& deleter) const& {\n    return std::unique_ptr<std::decay_t<ConstResult<>>, Deleter>(\n        new std::decay_t<ConstResult<>>(*this), std::forward<Deleter>(deleter));\n  }\n};\n\ntemplate <typename Function, typename... Args>\nexplicit InvokerType(Function&&, Args&&...)\n    -> InvokerType<std::decay_t<Function>, std::decay_t<Args>...>;\n\n// `InvokerTargetRef<T>::type` and `InvokerTargetRefT<T>` deduce the appropriate\n// target type of a possibly const-qualified `InvokerType<Function, Args...>`\n// or its reference, such that `T` is convertible to `InvokerTargetRefT<T>`,\n// and `T::Invoke()` returns `InvokerTargetRefT<T>`.\n//\n// They are undefined when the invoker is not usable in the given const and\n// reference context.\n\ntemplate <typename T>\nstruct InvokerTargetRef;\n\ntemplate <typename Function, typename... Args>\nstruct InvokerTargetRef<InvokerType<Function, Args...>>\n    : std::invoke_result<Function&&, Args&&...> {};\n\ntemplate <typename Function, typename... Args>\nstruct InvokerTargetRef<const InvokerType<Function, Args...>>\n    : std::invoke_result<const Function&, const Args&...> {};\n\ntemplate <typename T>\nstruct InvokerTargetRef<T&> : InvokerTargetRef<const T> {};\n\ntemplate <typename T>\nstruct InvokerTargetRef<T&&> : InvokerTargetRef<T> {};\n\ntemplate <typename T>\nusing InvokerTargetRefT = typename InvokerTargetRef<T>::type;\n\n// `InvokerTarget<T>::type` and `InvokerTargetT<T>` deduce the appropriate\n// target type of a possibly const-qualified `InvokerType<Function, Args...>`\n// or its reference, decayed to its value type, such that `T` is convertible to\n// `InvokerTargetT<T>`.\n//\n// This makes the result independent from whether the function returns a value\n// or a reference, if the result needs to be stored for later.\n//\n// They are undefined when the invoker is not usable in the given const and\n// reference context.\n\nnamespace invoker_internal {\n\ntemplate <typename T, typename Enable = void>\nstruct InvokerTargetImpl {\n  // No `type` member when the invoker is not usable in the given const and\n  // reference context.\n};\n\ntemplate <typename T>\nstruct InvokerTargetImpl<T, std::void_t<InvokerTargetRefT<T>>>\n    : std::decay<InvokerTargetRefT<T>> {};\n\n}  // namespace invoker_internal\n\ntemplate <typename T>\nstruct InvokerTarget : invoker_internal::InvokerTargetImpl<T> {};\n\ntemplate <typename T>\nusing InvokerTargetT = typename InvokerTarget<T>::type;\n\n// `riegeli::Invoker(function, args...)` returns\n// `InvokerType<Function, Args...>` which packs a function together with its\n// arguments. `InvokerType<Function, Args...>` is convertible to\n// `Initializer<T>` when the result of `Function` is convertible to `T`.\n//\n// This allows the function taking `Initializer<T>` to construct the object\n// in-place, avoiding constructing a temporary and moving from it.\n//\n// `riegeli::Invoker()` complements `riegeli::Maker()` by extending constructors\n// with factory functions.\n//\n// The function and arguments are interpreted as by `std::invoke()`: the\n// function can also be a member pointer, in which case the first argument is\n// the target reference, reference wrapper, or pointer.\n//\n// `riegeli::Invoker(function, args...)` does not own `function` or `args`, even\n// if they involve temporaries, hence it should be used only as a parameter of a\n// function or constructor, so that the temporaries outlive its usage. For\n// storing a `InvokerType` in a variable or returning it from a function, use\n// `riegeli::OwningInvoker(function, args...)` or construct `InvokerType`\n// directly.\ntemplate <typename Function, typename... Args,\n          std::enable_if_t<std::is_invocable_v<Function&&, Args&&...>, int> = 0>\ninline InvokerType<Function&&, Args&&...> Invoker(\n    Function&& function ABSL_ATTRIBUTE_LIFETIME_BOUND,\n    Args&&... args ABSL_ATTRIBUTE_LIFETIME_BOUND) {\n  return {std::forward<Function>(function), std::forward<Args>(args)...};\n}\n\n// `riegeli::OwningInvoker()` is like `riegeli::Invoker()`, but the arguments\n// are stored by value instead of by reference. This is useful for storing the\n// `InvokerType` in a variable or returning it from a function.\n//\n// If a particular argument is heavy and its lifetime is sufficient for storing\n// it by reference, wrap it in `std::ref()` or `std::cref()`.\ntemplate <typename Function, typename... Args,\n          std::enable_if_t<std::is_invocable_v<unwrap_ref_decay_t<Function>,\n                                               unwrap_ref_decay_t<Args>...>,\n                           int> = 0>\ninline InvokerType<unwrap_ref_decay_t<Function>, unwrap_ref_decay_t<Args>...>\nOwningInvoker(Function&& function, Args&&... args) {\n  return {std::forward<Function>(function), std::forward<Args>(args)...};\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_INVOKER_H_\n"
  },
  {
    "path": "riegeli/base/iterable.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_ITERABLE_H_\n#define RIEGELI_BASE_ITERABLE_H_\n\n#include <stddef.h>\n\n#include <iterator>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"riegeli/base/type_traits.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nnamespace iterable_internal {\n\n// Let unqualified `begin()` below refer either to a function named `begin()`\n// found via ADL or to `std::begin()`, as appropriate for the given iterable.\n// This is done in a separate namespace to avoid defining `riegeli::begin`.\n// Same for `end()` and `size()`.\n\nusing std::begin;\nusing std::end;\nusing std::size;\n\ntemplate <typename T, typename Enable = void>\nstruct IsIterable : std::false_type {};\n\ntemplate <typename T>\nstruct IsIterable<T, std::void_t<decltype(*begin(std::declval<T&>()))>>\n    : std::true_type {};\n\ntemplate <typename Iterable, typename Enable = void>\nstruct IteratorType {};\n\ntemplate <typename Iterable>\nstruct IteratorType<Iterable, std::enable_if_t<IsIterable<Iterable>::value>>\n    : type_identity<decltype(begin(std::declval<Iterable&>()))> {};\n\ntemplate <typename Iterable, bool move = false, typename Enable = void>\nstruct ElementTypeInternal {};\n\ntemplate <typename Iterable>\nstruct ElementTypeInternal<Iterable, false,\n                           std::enable_if_t<IsIterable<Iterable>::value>>\n    : type_identity<decltype(*begin(std::declval<Iterable&>()))> {};\n\ntemplate <typename Iterable>\nstruct ElementTypeInternal<Iterable, true,\n                           std::enable_if_t<IsIterable<Iterable>::value>>\n    : type_identity<decltype(*std::make_move_iterator(\n          begin(std::declval<Iterable&>())))> {};\n\ntemplate <typename Iterable, typename Enable = void>\nstruct IterableHasSize : std::false_type {};\n\ntemplate <typename Iterable>\nstruct IterableHasSize<\n    Iterable, std::enable_if_t<std::is_convertible_v<\n                  decltype(size(std::declval<const Iterable&>())), size_t>>>\n    : std::true_type {};\n\n}  // namespace iterable_internal\n\n// `IsIterable<T>::value` is `true` when `T` is iterable, supporting\n// `begin(iterable)` after `using std::begin;` (not all details are verified).\nusing iterable_internal::IsIterable;\n\n// `IteratorTypeT<Iterable>::type` and `IteratorTypeT<Iterable>` is the type of\n// iterators  over `Iterable`.\nusing iterable_internal::IteratorType;\n\ntemplate <typename Iterable>\nusing IteratorTypeT = typename IteratorType<Iterable>::type;\n\n// `HasMovableElements<Iterable>::value` is `true` when moving (rather than\n// copying) out of elements of `Iterable` is safe. This is the case when\n// `Iterable` owns its elements, i.e. it is not a view container like\n// `absl::Span<T>`, and it is not an lvalue reference.\n//\n// By default an iterable is detected as owning its elements when iterating over\n// `Iterable` and `const Iterable` yields elements of different types. This\n// also catches cases where `Iterable` always yields const elements or is const\n// itself. In these cases moving would be equivalent to copying, and trying to\n// move would just yield unnecessarily separate template instantiations.\n//\n// To customize that for a class `Iterable`, define a free function\n// `friend constexpr bool RiegeliHasMovableElements(Iterable*)` as a friend of\n// `Iterable` inside class definition or in the same namespace as `Iterable`,\n// so that it can be found via ADL.\n//\n// The argument of `RiegeliHasMovableElements(Iterable*)` is always a null\n// pointer, used to choose the right overload based on the type.\n\ntemplate <typename Iterable, typename Enable = void>\nstruct HasMovableElements\n    : std::negation<std::is_same<\n          typename iterable_internal::ElementTypeInternal<Iterable>::type,\n          typename iterable_internal::ElementTypeInternal<\n              const Iterable>::type>> {};\n\ntemplate <typename Iterable>\nstruct HasMovableElements<\n    Iterable,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<std::is_reference<Iterable>>,\n        std::is_convertible<decltype(RiegeliHasMovableElements(\n                                static_cast<Iterable* absl_nullable>(nullptr))),\n                            bool>>>>\n    : std::bool_constant<RiegeliHasMovableElements(\n          static_cast<Iterable* absl_nullable>(nullptr))> {};\n\ntemplate <typename Iterable>\nstruct HasMovableElements<Iterable&> : std::false_type {};\n\ntemplate <typename Iterable>\nstruct HasMovableElements<Iterable&&> : HasMovableElements<Iterable> {};\n\n// `MaybeMakeMoveIterator<Iterable>(iterator)` is\n// `std::make_move_iterator(iterator)` or `iterator`, depending on whether\n// moving out of elements of `Iterable` is safe.\ntemplate <typename Iterable, typename Iterator>\ninline auto MaybeMakeMoveIterator(Iterator iterator) {\n  if constexpr (HasMovableElements<Iterable>::value) {\n    return std::move_iterator<Iterator>(std::move(iterator));\n  } else {\n    return iterator;\n  }\n}\n\n// `ElementType<Iterable>::type` and `ElementTypeT<Iterable>` is the type of\n// elements yielded by iterating over `Iterable`.\n//\n// The result is a reference, except when iteration yields temporary objects.\n// If moving out of elements of `Iterable` is safe, this is an rvalue reference.\n\ntemplate <typename Iterable, typename Enable = void>\nstruct ElementType {};\n\ntemplate <typename Iterable>\nstruct ElementType<Iterable, std::enable_if_t<IsIterable<Iterable>::value>>\n    : iterable_internal::ElementTypeInternal<\n          Iterable, HasMovableElements<Iterable>::value> {};\n\ntemplate <typename Iterable>\nusing ElementTypeT = typename ElementType<Iterable>::type;\n\n// `IsIterableOf<Iterable, Element>::value` is `true` when iterating over\n// `Iterable` yields elements convertible to `Element`.\n\ntemplate <typename Iterable, typename Element, typename Enable = void>\nstruct IsIterableOf : std::false_type {};\n\ntemplate <typename Iterable, typename Element>\nstruct IsIterableOf<Iterable, Element,\n                    std::enable_if_t<IsIterable<Iterable>::value>>\n    : std::is_convertible<ElementTypeT<Iterable>, Element> {};\n\n// `IsIterableOfPairs<Iterable, Key, Value>::value` is `true` when iterating\n// over `Iterable` yields pairs or pair proxies with keys convertible to `Key`\n// and values convertible to `Value`.\n\ntemplate <typename Iterable, typename Key, typename Value,\n          typename Enable = void>\nstruct IsIterableOfPairs : std::false_type {};\n\ntemplate <typename Iterable, typename Key, typename Value>\nstruct IsIterableOfPairs<\n    Iterable, Key, Value,\n    std::enable_if_t<std::conjunction_v<\n        IsIterable<Iterable>,\n        std::is_convertible<\n            decltype(std::declval<ElementTypeT<Iterable>>().first), Key>,\n        std::is_convertible<\n            decltype(std::declval<ElementTypeT<Iterable>>().second), Value>>>>\n    : std::true_type {};\n\n// `IsIterableOfPairsWithAssignableValues<Iterable, Key, Value>::value`\n// is `true` when iterating over `Iterable` yields pair proxies with keys\n// convertible to `Key` and values assignable from `Value`.\n\ntemplate <typename Iterable, typename Key, typename Value,\n          typename Enable = void>\nstruct IsIterableOfPairsWithAssignableValues : std::false_type {};\n\ntemplate <typename Iterable, typename Key, typename Value>\nstruct IsIterableOfPairsWithAssignableValues<\n    Iterable, Key, Value,\n    std::enable_if_t<std::conjunction_v<\n        IsIterable<Iterable>,\n        std::is_convertible<\n            decltype(std::declval<ElementTypeT<Iterable>>().first), Key>,\n        std::is_assignable<\n            decltype(std::declval<ElementTypeT<Iterable>>().second), Value>>>>\n    : std::true_type {};\n\n// TODO: Use `typename std::iterator_traits<Iterator>::iterator_concept`\n// instead when C++20 is unconditionally available.\n\nnamespace iterable_internal {\n\ntemplate <typename Iterator, typename Enable = void>\nstruct IteratorConcept\n    : type_identity<\n          typename std::iterator_traits<Iterator>::iterator_category> {};\n\ntemplate <typename Iterator>\nstruct IteratorConcept<\n    Iterator,\n    std::void_t<typename std::iterator_traits<Iterator>::iterator_concept>>\n    : type_identity<typename std::iterator_traits<Iterator>::iterator_concept> {\n};\n\n}  // namespace iterable_internal\n\n// `IsForwardIterable<Iterable>::value` is `true` when the iterator over\n// `Iterable` is a forward iterator, in particular when it can be iterated\n// over multiple times.\n\ntemplate <typename Iterable, typename Enable = void>\nstruct IsForwardIterable : std::false_type {};\n\ntemplate <typename Iterable>\nstruct IsForwardIterable<\n    Iterable,\n    std::enable_if_t<std::conjunction_v<\n        IsIterable<Iterable>,\n        std::is_convertible<typename iterable_internal::IteratorConcept<\n                                IteratorTypeT<Iterable>>::type,\n                            std::forward_iterator_tag>>>> : std::true_type {};\n\n// `IsRandomAccessIterable<Iterable>::value` is `true` when the iterator over\n// `Iterable` is a random access iterator.\n\ntemplate <typename Iterable, typename Enable = void>\nstruct IsRandomAccessIterable : std::false_type {};\n\ntemplate <typename Iterable>\nstruct IsRandomAccessIterable<\n    Iterable,\n    std::enable_if_t<std::conjunction_v<\n        IsIterable<Iterable>,\n        std::is_convertible<typename iterable_internal::IteratorConcept<\n                                IteratorTypeT<Iterable>>::type,\n                            std::random_access_iterator_tag>>>>\n    : std::true_type {};\n\n// `IterableHasSize<Iterable>::value` is `true` when `Iterable` supports\n// `size(iterable)` after `using std::size;`.\nusing iterable_internal::IterableHasSize;\n\n// Represents the result of `operator->` if `operator*` returns a proxy\n// object rather than a true reference. In particular this can be used as\n// `iterator::pointer` if `iterator::reference` is not a true reference.\ntemplate <typename Reference>\nclass ArrowProxy {\n public:\n  explicit ArrowProxy(Reference ref) : ref_(std::move(ref)) {}\n\n  ArrowProxy(const ArrowProxy& that) = default;\n  ArrowProxy& operator=(const ArrowProxy& that) = default;\n\n  ArrowProxy(ArrowProxy&& that) noexcept = default;\n  ArrowProxy& operator=(ArrowProxy&& that) noexcept = default;\n\n  const Reference* operator->() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return &ref_;\n  }\n\n private:\n  Reference ref_;\n};\n\n// A pair-like type to be used as `iterator::reference` for iterators over a map\n// with a separate storage for keys and values. In C++20 this lets the iterator\n// satisfy `std::indirectly_readable`.\n//\n// It extends `std::pair<T1, T2>` with conversions from `std::pair<U1, U2>&`\n// and with `std::basic_common_reference` specializations.\n//\n// Since C++23, `std::pair<T1, T2>` can be used directly instead.\ntemplate <typename T1, typename T2>\nclass ReferencePair : public std::pair<T1, T2> {\n public:\n  using ReferencePair::pair::pair;\n\n  template <\n      class U1, class U2,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::is_constructible<T1, U1&>, std::is_constructible<T2, U2&>,\n              std::negation<std::conjunction<std::is_convertible<U1&, T1>,\n                                             std::is_convertible<U2&, T2>>>>,\n          int> = 0>\n  explicit constexpr ReferencePair(std::pair<U1, U2>& p)\n      : ReferencePair::pair(p.first, p.second) {}\n\n  template <class U1, class U2,\n            std::enable_if_t<std::conjunction_v<std::is_convertible<U1&, T1>,\n                                                std::is_convertible<U2&, T2>>,\n                             int> = 0>\n  /*implicit*/ constexpr ReferencePair(std::pair<U1, U2>& p)\n      : ReferencePair::pair(p.first, p.second) {}\n};\n\n}  // namespace riegeli\n\n#if __cplusplus >= 202002L\n\ntemplate <typename T1, typename T2, typename U1, typename U2,\n          template <typename> class TQual, template <typename> class UQual>\nstruct std::basic_common_reference<riegeli::ReferencePair<T1, T2>,\n                                   std::pair<U1, U2>, TQual, UQual> {\n  using type =\n      riegeli::ReferencePair<std::common_reference_t<TQual<T1>, UQual<U1>>,\n                             std::common_reference_t<TQual<T2>, UQual<U2>>>;\n};\n\ntemplate <typename T1, typename T2, typename U1, typename U2,\n          template <typename> class TQual, template <typename> class UQual>\nstruct std::basic_common_reference<\n    std::pair<T1, T2>, riegeli::ReferencePair<U1, U2>, TQual, UQual> {\n  using type =\n      riegeli::ReferencePair<std::common_reference_t<TQual<T1>, UQual<U1>>,\n                             std::common_reference_t<TQual<T2>, UQual<U2>>>;\n};\n\ntemplate <typename T1, typename T2, typename U1, typename U2,\n          template <typename> class TQual, template <typename> class UQual>\nstruct std::basic_common_reference<riegeli::ReferencePair<T1, T2>,\n                                   riegeli::ReferencePair<U1, U2>, TQual,\n                                   UQual> {\n  using type =\n      riegeli::ReferencePair<std::common_reference_t<TQual<T1>, UQual<U1>>,\n                             std::common_reference_t<TQual<T2>, UQual<U2>>>;\n};\n\n#endif\n\n#endif  // RIEGELI_BASE_ITERABLE_H_\n"
  },
  {
    "path": "riegeli/base/maker.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_MAKER_H_\n#define RIEGELI_BASE_MAKER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <tuple>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/temporary_storage.h\"\n#include \"riegeli/base/type_traits.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `MakerType<Args...>`, usually made with `riegeli::Maker(args...)`, packs\n// constructor arguments for a yet unspecified type, which will be specified by\n// the caller. `MakerType<Args...>` is convertible to `Initializer<T>` for any\n// `T` which can be constructed from `Args...`.\n//\n// This allows the function taking `Initializer<T>` to construct the object\n// in-place, avoiding constructing a temporary and moving from it.\n//\n// In contrast to `MakerTypeFor<T, Args...>`, `MakerType<Args...>` requires the\n// caller to know `T`.\n//\n// `InvokerType` complements `MakerType` by extending constructors with factory\n// functions.\ntemplate <typename... Args>\nclass MakerType\n    : public ConditionallyAssignable<\n          std::conjunction_v<std::negation<std::is_reference<Args>>...>> {\n public:\n  // Constructs `MakerType` from `args...` convertible to `Args...`.\n  template <typename... SrcArgs,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<MakerType, SrcArgs...>,\n                                   std::is_convertible<SrcArgs&&, Args>...>,\n                int> = 0>\n  /*implicit*/ MakerType(SrcArgs&&... args)\n      : args_(std::forward<SrcArgs>(args)...) {}\n\n  MakerType(MakerType&& that) = default;\n  MakerType& operator=(MakerType&& that) = default;\n\n  MakerType(const MakerType& that) = default;\n  MakerType& operator=(const MakerType& that) = default;\n\n  // Constructs the `T`.\n  template <typename T,\n            std::enable_if_t<std::is_constructible_v<T, Args&&...>, int> = 0>\n  T Construct() && {\n    return std::make_from_tuple<T>(std::move(args_));\n  }\n  template <\n      typename T,\n      std::enable_if_t<std::is_constructible_v<T, const Args&...>, int> = 0>\n  T Construct() const& {\n    return std::make_from_tuple<T>(args_);\n  }\n\n  // Constructs the `std::decay_t<T>` on the heap.\n  //\n  // In contrast to `std::make_unique()`, this supports custom deleters.\n  template <typename T, typename Deleter = std::default_delete<std::decay_t<T>>,\n            std::enable_if_t<\n                std::is_constructible_v<std::decay_t<T>, Args&&...>, int> = 0>\n  std::unique_ptr<std::decay_t<T>, Deleter> UniquePtr() && {\n    return std::unique_ptr<std::decay_t<T>, Deleter>(\n        new std::decay_t<T>(std::move(*this).template Construct<T>()));\n  }\n  template <typename T, typename Deleter,\n            std::enable_if_t<\n                std::is_constructible_v<std::decay_t<T>, Args&&...>, int> = 0>\n  std::unique_ptr<std::decay_t<T>, Deleter> UniquePtr(Deleter&& deleter) && {\n    return std::unique_ptr<std::decay_t<T>, Deleter>(\n        new std::decay_t<T>(std::move(*this).template Construct<T>()),\n        std::forward<Deleter>(deleter));\n  }\n  template <\n      typename T, typename Deleter = std::default_delete<std::decay_t<T>>,\n      std::enable_if_t<std::is_constructible_v<std::decay_t<T>, const Args&...>,\n                       int> = 0>\n  std::unique_ptr<std::decay_t<T>, Deleter> UniquePtr() const& {\n    return std::unique_ptr<std::decay_t<T>, Deleter>(\n        new std::decay_t<T>(Construct<T>()));\n  }\n  template <\n      typename T, typename Deleter,\n      std::enable_if_t<std::is_constructible_v<std::decay_t<T>, const Args&...>,\n                       int> = 0>\n  std::unique_ptr<std::decay_t<T>, Deleter> UniquePtr(\n      Deleter&& deleter) const& {\n    return std::unique_ptr<std::decay_t<T>, Deleter>(\n        new std::decay_t<T>(Construct<T>()), std::forward<Deleter>(deleter));\n  }\n\n  // Constructs the `T` in `storage` which must outlive the returned reference.\n  //\n  // `Reference()` instead of `Construct()` supports `Initializer::Reference()`.\n  //\n  // If the `storage` argument is omitted, the result is returned by value\n  // instead of by reference, which is a more efficient way to construct the\n  // temporary.\n  template <typename T,\n            std::enable_if_t<std::is_constructible_v<T, Args&&...>, int> = 0>\n  T Reference() && {\n    return std::move(*this).template Construct<T>();\n  }\n  template <typename T,\n            std::enable_if_t<std::is_constructible_v<T, Args&&...>, int> = 0>\n  T&& Reference(\n      TemporaryStorage<T>&& storage ABSL_ATTRIBUTE_LIFETIME_BOUND) && {\n    return std::apply(\n        [&](Args&&... args) -> T&& {\n          return std::move(storage).emplace(std::forward<Args>(args)...);\n        },\n        std::move(args_));\n  }\n  template <\n      typename T,\n      std::enable_if_t<std::is_constructible_v<T, const Args&...>, int> = 0>\n  T Reference() const& {\n    return Construct<T>();\n  }\n  template <\n      typename T,\n      std::enable_if_t<std::is_constructible_v<T, const Args&...>, int> = 0>\n  T&& Reference(\n      TemporaryStorage<T>&& storage ABSL_ATTRIBUTE_LIFETIME_BOUND) const& {\n    return std::apply(\n        [&](const Args&... args) -> T&& {\n          return std::move(storage).emplace(args...);\n        },\n        args_);\n  }\n\n  // `riegeli::Reset(dest, MakerType)` makes `dest` equivalent to the\n  // constructed `T`. This avoids constructing a temporary `T` and moving from\n  // it.\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<std::negation<std::is_reference<T>>,\n                                          SupportsReset<T, Args&&...>>,\n                       int> = 0>\n  friend void RiegeliReset(T& dest, MakerType&& src) {\n    std::apply(\n        [&](Args&&... args) {\n          riegeli::Reset(dest, std::forward<Args>(args)...);\n        },\n        std::move(src.args_));\n  }\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<std::negation<std::is_reference<T>>,\n                                          SupportsReset<T, const Args&...>>,\n                       int> = 0>\n  friend void RiegeliReset(T& dest, const MakerType& src) {\n    std::apply([&](const Args&... args) { riegeli::Reset(dest, args...); },\n               src.args_);\n  }\n\n  // Extracts the given argument.\n  template <size_t index, std::enable_if_t<(index < sizeof...(Args)), int> = 0>\n  std::tuple_element_t<index, std::tuple<Args...>>& arg() & {\n    return std::get<index>(args_);\n  }\n  template <size_t index, std::enable_if_t<(index < sizeof...(Args)), int> = 0>\n  const std::tuple_element_t<index, std::tuple<Args...>>& arg() const& {\n    return std::get<index>(args_);\n  }\n  template <size_t index, std::enable_if_t<(index < sizeof...(Args)), int> = 0>\n  std::tuple_element_t<index, std::tuple<Args...>>& arg() && {\n    return std::get<index>(std::move(args_));\n  }\n  template <size_t index, std::enable_if_t<(index < sizeof...(Args)), int> = 0>\n  const std::tuple_element_t<index, std::tuple<Args...>>& arg() const&& {\n    return std::get<index>(std::move(args_));\n  }\n\n private:\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS std::tuple<Args...> args_;\n};\n\n// `MakerTypeFor<T, Args...>, usually made with `riegeli::Maker<T>(args...)`,\n// packs constructor arguments for `T`. `MakerTypeFor<T, Args...>` is\n// convertible to `Initializer<T>`.\n//\n// This allows the function taking `Initializer<T>` to construct the object\n// in-place, avoiding constructing a temporary and moving from it.\n//\n// In contrast to `MakerType<Args...>`, `MakerTypeFor<T, Args...>` allows the\n// caller to deduce `T`, e.g. using `TargetT`.\ntemplate <typename T, typename... Args>\nclass MakerTypeFor\n    : public ConditionallyAssignable<\n          std::conjunction_v<std::negation<std::is_reference<Args>>...>> {\n public:\n  // Constructs `MakerTypeFor` from `args...` convertible to `Args...`.\n  template <typename... SrcArgs,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<MakerTypeFor, SrcArgs...>,\n                                   std::is_constructible<T, Args&&...>,\n                                   std::is_convertible<SrcArgs&&, Args>...>,\n                int> = 0>\n  /*implicit*/ MakerTypeFor(SrcArgs&&... args)\n      : maker_(std::forward<SrcArgs>(args)...) {}\n\n  MakerTypeFor(MakerTypeFor&& that) = default;\n  MakerTypeFor& operator=(MakerTypeFor&& that) = default;\n\n  MakerTypeFor(const MakerTypeFor& that) = default;\n  MakerTypeFor& operator=(const MakerTypeFor& that) = default;\n\n  // Constructs the `T`.\n  template <\n      typename DependentT = T,\n      std::enable_if_t<std::is_constructible_v<DependentT, Args&&...>, int> = 0>\n  /*implicit*/ operator T() && {\n    return std::move(*this).Construct();\n  }\n  template <typename DependentT = T,\n            std::enable_if_t<\n                std::is_constructible_v<DependentT, const Args&...>, int> = 0>\n  /*implicit*/ operator T() const& {\n    return Construct();\n  }\n\n  // Constructs the `T`.\n  //\n  // Usually conversion to `T` is preferred because it can avoid creating a\n  // temporary if the context accepts an arbitrary type convertible to `T`.\n  // An explicit `Construct()` call can force construction right away while\n  // avoiding specifying the full target type.\n  template <\n      typename DependentT = T,\n      std::enable_if_t<std::is_constructible_v<DependentT, Args&&...>, int> = 0>\n  T Construct() && {\n    return std::move(*this).maker().template Construct<T>();\n  }\n  template <typename DependentT = T,\n            std::enable_if_t<\n                std::is_constructible_v<DependentT, const Args&...>, int> = 0>\n  T Construct() const& {\n    return this->maker().template Construct<T>();\n  }\n\n  // Constructs the `std::decay_t<T>` on the heap.\n  //\n  // In contrast to `std::make_unique()`, this supports deducing class template\n  // arguments and custom deleters.\n  //\n  // For a non-default-constructed deleter, use `UniquePtr(deleter)`.\n  template <\n      typename Target, typename Deleter,\n      std::enable_if_t<\n          std::conjunction_v<std::is_constructible<std::decay_t<T>, Args&&...>,\n                             std::is_convertible<std::decay_t<T>*, Target*>>,\n          int> = 0>\n  /*implicit*/ operator std::unique_ptr<Target, Deleter>() && {\n    return std::move(*this).template UniquePtr<Deleter>();\n  }\n  template <typename Target, typename Deleter,\n            std::enable_if_t<\n                std::conjunction_v<\n                    std::is_constructible<std::decay_t<T>, const Args&...>,\n                    std::is_convertible<std::decay_t<T>*, Target*>>,\n                int> = 0>\n  /*implicit*/ operator std::unique_ptr<Target, Deleter>() const& {\n    return UniquePtr<Deleter>();\n  }\n\n  // Constructs the `std::decay_t<T>` on the heap.\n  //\n  // In contrast to `std::make_unique()`, this supports deducing class template\n  // arguments and custom deleters.\n  //\n  // Usually conversion to `std::unique_ptr` is preferred because it leads to\n  // simpler source code. An explicit `UniquePtr()` call can force construction\n  // right away while avoiding writing the full target type, and it allows to\n  // use a non-default-constructed deleter.\n  template <typename Deleter = std::default_delete<std::decay_t<T>>,\n            typename DependentT = T,\n            std::enable_if_t<\n                std::is_constructible_v<std::decay_t<DependentT>, Args&&...>,\n                int> = 0>\n  std::unique_ptr<std::decay_t<T>, Deleter> UniquePtr() && {\n    return std::move(*this).maker().template UniquePtr<T, Deleter>();\n  }\n  template <typename Deleter, typename DependentT = T,\n            std::enable_if_t<\n                std::is_constructible_v<std::decay_t<DependentT>, Args&&...>,\n                int> = 0>\n  std::unique_ptr<std::decay_t<T>, Deleter> UniquePtr(Deleter&& deleter) && {\n    return std::move(*this).maker().template UniquePtr<T, Deleter>(\n        std::forward<Deleter>(deleter));\n  }\n  template <typename Deleter = std::default_delete<std::decay_t<T>>,\n            typename DependentT = T,\n            std::enable_if_t<std::is_constructible_v<std::decay_t<DependentT>,\n                                                     const Args&...>,\n                             int> = 0>\n  std::unique_ptr<std::decay_t<T>, Deleter> UniquePtr() const& {\n    return this->maker().template UniquePtr<T, Deleter>();\n  }\n  template <typename Deleter, typename DependentT = T,\n            std::enable_if_t<std::is_constructible_v<std::decay_t<DependentT>,\n                                                     const Args&...>,\n                             int> = 0>\n  std::unique_ptr<std::decay_t<T>, Deleter> UniquePtr(\n      Deleter&& deleter) const& {\n    return this->maker().template UniquePtr<T, Deleter>(\n        std::forward<Deleter>(deleter));\n  }\n\n  // Constructs the `T` in `storage` which must outlive the returned reference.\n  //\n  // `Reference()` instead of conversion to `T` or `Construct()` supports\n  // `Initializer::Reference()`.\n  //\n  // If the `storage` argument is omitted, the result is returned by value\n  // instead of by reference, which is a more efficient way to construct the\n  // temporary.\n  template <\n      typename DependentT = T,\n      std::enable_if_t<std::is_constructible_v<DependentT, Args&&...>, int> = 0>\n  T Reference() && {\n    return std::move(*this).maker().template Reference<T>();\n  }\n  template <\n      typename DependentT = T,\n      std::enable_if_t<std::is_constructible_v<DependentT, Args&&...>, int> = 0>\n  T&& Reference(\n      TemporaryStorage<T>&& storage ABSL_ATTRIBUTE_LIFETIME_BOUND) && {\n    return std::move(*this).maker().template Reference<T>(std::move(storage));\n  }\n  template <typename DependentT = T,\n            std::enable_if_t<\n                std::is_constructible_v<DependentT, const Args&...>, int> = 0>\n  T Reference() const& {\n    return this->maker().template Reference<T>();\n  }\n  template <typename DependentT = T,\n            std::enable_if_t<\n                std::is_constructible_v<DependentT, const Args&...>, int> = 0>\n  T&& Reference(\n      TemporaryStorage<T>&& storage ABSL_ATTRIBUTE_LIFETIME_BOUND) const& {\n    return this->maker().template Reference<T>(std::move(storage));\n  }\n\n  // `riegeli::Reset(dest, MakerTypeFor)` makes `dest` equivalent to the\n  // constructed `T`. This avoids constructing a temporary `T` and moving from\n  // it.\n  template <typename DependentT = T,\n            std::enable_if_t<\n                std::conjunction_v<std::negation<std::is_reference<DependentT>>,\n                                   SupportsReset<DependentT, Args&&...>>,\n                int> = 0>\n  friend void RiegeliReset(T& dest, MakerTypeFor&& src) {\n    riegeli::Reset(dest, std::move(src).maker());\n  }\n  template <typename DependentT = T,\n            std::enable_if_t<\n                std::conjunction_v<std::negation<std::is_reference<DependentT>>,\n                                   SupportsReset<DependentT, const Args&...>>,\n                int> = 0>\n  friend void RiegeliReset(T& dest, const MakerTypeFor& src) {\n    riegeli::Reset(dest, src.maker());\n  }\n\n  // Extracts the given argument.\n  template <size_t index, std::enable_if_t<(index < sizeof...(Args)), int> = 0>\n  std::tuple_element_t<index, std::tuple<Args...>>& arg() & {\n    return maker().template arg<index>();\n  }\n  template <size_t index, std::enable_if_t<(index < sizeof...(Args)), int> = 0>\n  const std::tuple_element_t<index, std::tuple<Args...>>& arg() const& {\n    return maker().template arg<index>();\n  }\n  template <size_t index, std::enable_if_t<(index < sizeof...(Args)), int> = 0>\n  std::tuple_element_t<index, std::tuple<Args...>>& arg() && {\n    return std::move(*this).maker().template arg<index>();\n  }\n  template <size_t index, std::enable_if_t<(index < sizeof...(Args)), int> = 0>\n  const std::tuple_element_t<index, std::tuple<Args...>>& arg() const&& {\n    return std::move(*this).maker().template arg<index>();\n  }\n\n  // Extracts the corresponding `MakerType` which does not specify `T`.\n  //\n  // This is useful for handling `MakerType` and `MakerTypeFor` generically.\n  MakerType<Args...>& maker() & { return maker_; }\n  const MakerType<Args...>& maker() const& { return maker_; }\n  MakerType<Args...>&& maker() && { return std::move(maker_); }\n  const MakerType<Args...>&& maker() const&& { return std::move(maker_); }\n\n private:\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS MakerType<Args...> maker_;\n};\n\n// `MakerTarget<T>::type` and `MakerTargetT<T>` deduce the appropriate target\n// type of a possibly const-qualified `MakerTypeFor<Target, Args...>` or its\n// reference, such that `T` is convertible to `MakerTargetT<T>`, and\n// `T::Construct()` returns `MakerTargetT<T>`.\n//\n// They are undefined when the maker is not usable in the given const and\n// reference context.\n\nnamespace maker_internal {\n\ntemplate <typename T, typename Enable = void>\nstruct MakerTargetImpl {\n  // No `type` member when the maker is not usable in the given const and\n  // reference context.\n};\n\ntemplate <typename Target, typename... Args>\nstruct MakerTargetImpl<\n    MakerTypeFor<Target, Args...>,\n    std::enable_if_t<std::is_constructible_v<Target, Args&&...>>> {\n  using type = Target;\n};\n\ntemplate <typename Target, typename... Args>\nstruct MakerTargetImpl<\n    const MakerTypeFor<Target, Args...>,\n    std::enable_if_t<std::is_constructible_v<Target, const Args&...>>> {\n  using type = Target;\n};\n\n}  // namespace maker_internal\n\ntemplate <typename T>\nstruct MakerTarget : maker_internal::MakerTargetImpl<T> {};\n\ntemplate <typename T>\nstruct MakerTarget<T&> : maker_internal::MakerTargetImpl<const T> {};\n\ntemplate <typename T>\nstruct MakerTarget<T&&> : maker_internal::MakerTargetImpl<T> {};\n\ntemplate <typename T>\nusing MakerTargetT = typename MakerTarget<T>::type;\n\n// `riegeli::Maker(args...)` returns `MakerType<Args&&...>` which packs\n// constructor arguments for a yet unspecified type, which will be specified by\n// the caller. `riegeli::Maker(args...)` is convertible to `Initializer<T>` for\n// any `T` which can be constructed from `Args...`.\n//\n// This allows the function taking `Initializer<T>` to construct the object\n// in-place, avoiding constructing a temporary and moving from it.\n//\n// In contrast to `riegeli::Maker<T>(args...)`, `riegeli::Maker(args...)`\n// requires the caller to know `T`. Prefer\n// `Template(riegeli::Maker<T>(args...))` over\n// `Template<T>(riegeli::Maker(args...))` if CTAD of `Template` can be used.\n//\n// `riegeli::Invoker()` complements `riegeli::Maker()` by extending constructors\n// with factory functions.\n//\n// `riegeli::Maker(args...)` does not own `args`, even if they involve\n// temporaries, hence it should be used only as a parameter of a function or\n// constructor, so that the temporaries outlive its usage. For storing a\n// `MakerType` in a variable or returning it from a function, use\n// `riegeli::OwningMaker(args...)` or construct `MakerType` directly.\n//\n// The `generic` template parameter lets `riegeli::Maker<T>()` with an explicit\n// template argument unambiguously call another overload of `riegeli::Maker()`.\ntemplate <int generic = 0, typename... Args>\nMakerType<Args&&...> Maker(Args&&... args ABSL_ATTRIBUTE_LIFETIME_BOUND) {\n  return {std::forward<Args>(args)...};\n}\n\n// `riegeli::Maker<T>(args...)` returns `MakerTypeFor<T, Args&&...>` which packs\n// constructor arguments for `T`. `riegeli::Maker<T>(args...)` is convertible to\n// `Initializer<T>`.\n//\n// This allows the function taking `Initializer<T>` to construct the object\n// in-place, avoiding constructing a temporary and moving from it.\n//\n// `riegeli::Invoker()` complements `riegeli::Maker<T>()` by extending\n// constructors with factory functions.\n//\n// In contrast to `riegeli::Maker(args...)`, `riegeli::Maker<T>(args...)` allows\n// the caller to deduce `T`, e.g. using `TargetT`.\n//\n// `riegeli::Maker<T>(args...)` does not own `args`, even if they involve\n// temporaries, hence it should be used only as a parameter of a function or\n// constructor, so that the temporaries outlive its usage. For storing a\n// `MakerTypeFor` in a variable or returning it from a function, use\n// `riegeli::OwningMaker<T>(args...)` or construct `MakerTypeFor` directly.\ntemplate <typename T, typename... Args,\n          std::enable_if_t<std::is_constructible_v<T, Args&&...>, int> = 0>\nMakerTypeFor<T, Args&&...> Maker(Args&&... args ABSL_ATTRIBUTE_LIFETIME_BOUND) {\n  return {std::forward<Args>(args)...};\n}\n\n// `riegeli::Maker<Template>()` is like `riegeli::Maker<T>()`, but the exact\n// target type is deduced using CTAD from the class template and the constructor\n// arguments.\n//\n// Only class templates with solely type template parameters are supported.\ntemplate <template <typename...> class Template, typename... Args,\n          std::enable_if_t<\n              std::is_constructible_v<\n                  DeduceClassTemplateArgumentsT<Template, Args...>, Args&&...>,\n              int> = 0>\nMakerTypeFor<DeduceClassTemplateArgumentsT<Template, Args...>, Args&&...> Maker(\n    Args&&... args ABSL_ATTRIBUTE_LIFETIME_BOUND) {\n  return {std::forward<Args>(args)...};\n}\n\n// `riegeli::OwningMaker()` is like `riegeli::Maker()`, but the arguments are\n// stored by value instead of by reference. This is useful for storing the\n// `MakerType` in a variable or returning it from a function.\n//\n// If a particular argument is heavy and its lifetime is sufficient for storing\n// it by reference, wrap it in `std::ref()` or `std::cref()`.\ntemplate <int generic = 0, typename... Args>\nMakerType<unwrap_ref_decay_t<Args>...> OwningMaker(Args&&... args) {\n  return {std::forward<Args>(args)...};\n}\n\n// `riegeli::OwningMaker<T>()` is like `riegeli::Maker<T>()`, but the arguments\n// are stored by value instead of by reference. This is useful for storing the\n// `MakerTypeFor` in a variable or returning it from a function.\n//\n// If a particular argument is heavy and its lifetime is sufficient for storing\n// it by reference, wrap it in `std::ref()` or `std::cref()`.\ntemplate <\n    typename T, typename... Args,\n    std::enable_if_t<std::is_constructible_v<T, unwrap_ref_decay_t<Args>&&...>,\n                     int> = 0>\nMakerTypeFor<T, unwrap_ref_decay_t<Args>...> OwningMaker(Args&&... args) {\n  return {std::forward<Args>(args)...};\n}\n\n// `riegeli::OwningMaker<Template>()` is like `riegeli::OwningMaker<T>()`, but\n// the exact target type is deduced using CTAD from the class template and the\n// constructor arguments.\n//\n// Only class templates with solely type template parameters are supported.\n//\n// If a particular argument is heavy and its lifetime is sufficient for storing\n// it by reference, wrap it in `std::ref()` or `std::cref()`.\ntemplate <template <typename...> class Template, typename... Args,\n          std::enable_if_t<std::is_constructible_v<\n                               DeduceClassTemplateArgumentsT<\n                                   Template, unwrap_ref_decay_t<Args>...>,\n                               unwrap_ref_decay_t<Args>...>,\n                           int> = 0>\nMakerTypeFor<\n    DeduceClassTemplateArgumentsT<Template, unwrap_ref_decay_t<Args>&&...>,\n    unwrap_ref_decay_t<Args>...>\nOwningMaker(Args&&... args) {\n  return {std::forward<Args>(args)...};\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_MAKER_H_\n"
  },
  {
    "path": "riegeli/base/memory_estimator.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/memory_estimator.h\"\n\n#ifdef __GXX_RTTI\n#include <cxxabi.h>  // IWYU pragma: keep\n#endif\n#include <stddef.h>\n\n#include <algorithm>\n#include <cstdlib>\n#include <string>\n#include <typeindex>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/container/flat_hash_set.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nbool MemoryEstimatorDefault::RegisterNodeImpl(const void* absl_nullable ptr) {\n  return ptr != nullptr && objects_seen_.insert(ptr).second;\n}\n\nvoid MemoryEstimatorReportingUnknownTypes::RegisterUnknownTypeImpl() {\n  unknown_types_no_rtti_ = true;\n}\n\nvoid MemoryEstimatorReportingUnknownTypes::RegisterUnknownTypeImpl(\n    std::type_index index) {\n  unknown_types_.insert(index);\n}\n\nstd::vector<std::string> MemoryEstimatorReportingUnknownTypes::UnknownTypes()\n    const {\n  std::vector<std::string> result;\n  result.reserve((unknown_types_no_rtti_ ? 1 : 0) + unknown_types_.size());\n  if (unknown_types_no_rtti_) result.emplace_back(\"<no rtti>\");\n  for (const std::type_index index : unknown_types_) {\n#ifdef __GXX_RTTI\n    int status = 0;\n    char* const absl_nullable demangled =\n        abi::__cxa_demangle(index.name(), nullptr, nullptr, &status);\n    if (status == 0 && demangled != nullptr) {\n      result.emplace_back(demangled);\n      std::free(demangled);\n      continue;\n    }\n#endif\n    result.emplace_back(index.name());\n  }\n  std::sort(result.begin(), result.end());\n  return result;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/memory_estimator.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_MEMORY_ESTIMATOR_H_\n#define RIEGELI_BASE_MEMORY_ESTIMATOR_H_\n\n#include <stddef.h>\n\n#include <array>\n#include <iterator>\n#include <memory>\n#include <optional>\n#include <string>\n#include <tuple>\n#include <type_traits>\n#include <typeindex>\n#include <typeinfo>  // IWYU pragma: keep\n#include <utility>\n#include <variant>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/container/flat_hash_map.h\"\n#include \"absl/container/flat_hash_set.h\"\n#include \"absl/container/inlined_vector.h\"\n#include \"absl/container/node_hash_map.h\"\n#include \"absl/container/node_hash_set.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/estimated_allocated_size.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace google::protobuf {\nclass Message;\n}  // namespace google::protobuf\n\nnamespace riegeli {\n\nclass MemoryEstimator;\n\nnamespace memory_estimator_internal {\n\ntemplate <typename T, typename Enable = void>\nstruct HasRiegeliRegisterSubobjects : std::false_type {};\n\ntemplate <typename T>\nstruct HasRiegeliRegisterSubobjects<\n    T, std::void_t<decltype(RiegeliRegisterSubobjects(\n           std::declval<const T*>(), std::declval<MemoryEstimator&>()))>>\n    : std::true_type {};\n\n}  // namespace memory_estimator_internal\n\n// Estimates the amount of memory owned by multiple objects.\nclass MemoryEstimator {\n public:\n  // Determines whether `RegisterSubobjects(const T*)` might be considered good:\n  // either the corresponding `RiegeliRegisterSubobjects()` is defined, or `T`\n  // is trivially destructible so the default definition which does nothing is\n  // likely appropriate.\n  //\n  // This is not necessarily accurate, in particular traversing a `T` might\n  // encounter types which do not have a good estimation, but if this yields\n  // `false`, then `RegisterSubobjects()` is most likely an underestimation and\n  // the caller might need some different way to estimate memory.\n  template <typename T>\n  struct RegisterSubobjectsIsGood\n      : std::disjunction<\n            memory_estimator_internal::HasRiegeliRegisterSubobjects<T>,\n            std::is_trivially_destructible<T>> {};\n\n  // Determines whether `RegisterSubobjects(const T*)` definitely does nothing:\n  // the corresponding `RiegeliRegisterSubobjects()` is not defined and `T` is\n  // trivially destructible.\n  //\n  // This can be used to skip a loop over elements of type `T`.\n  template <typename T>\n  struct RegisterSubobjectsIsTrivial\n      : std::conjunction<\n            std::negation<\n                memory_estimator_internal::HasRiegeliRegisterSubobjects<T>>,\n            std::is_trivially_destructible<T>> {};\n\n  MemoryEstimator() = default;\n\n  MemoryEstimator(const MemoryEstimator&) = delete;\n  MemoryEstimator& operator=(const MemoryEstimator&) = delete;\n\n  virtual ~MemoryEstimator() = default;\n\n  // Registers the given amount of memory as used.\n  void RegisterMemory(size_t memory) {\n    total_memory_ = SaturatingAdd(total_memory_, memory);\n  }\n\n  // Registers the given length of a block of dynamically allocated memory as\n  // used. The length should correspond to a single allocation. The actual\n  // registered amount includes estimated overhead of the memory allocator.\n  //\n  // If the address of the allocated memory is provided, it might be used for a\n  // better estimation.\n  void RegisterDynamicMemory(const void* ptr, size_t memory) {\n    RegisterDynamicMemoryImpl(ptr, memory);\n  }\n  void RegisterDynamicMemory(size_t memory) {\n    RegisterDynamicMemoryImpl(memory);\n  }\n\n  // Begins registering an object which might be shared by other objects.\n  //\n  // The argument should be a pointer which uniquely identifies the object.\n  // This is usually the pointer to the object itself, but that might not be\n  // possible if the object is not public and is registered by code external\n  // to it, in which case some proxy is needed.\n  //\n  // Returns `true` if this object was not seen yet. Only in this case the\n  // caller should register its memory and subobjects.\n  //\n  // If `ptr == nullptr` then always returns `false`.\n  bool RegisterNode(const void* absl_nullable ptr) {\n    return RegisterNodeImpl(ptr);\n  }\n\n  // Adds `T` to the stored set of unknown types, to be returned by\n  // `UnknownTypes()`.\n  //\n  // This indicates that traversal encountered a type for which\n  // `RegisterSubobjects()` is not customized and the type is not trivially\n  // destructible, which likely indicates that an interesting customization is\n  // missing and results are underestimated.\n  template <typename T>\n  void RegisterUnknownType();\n\n  // Returns `sizeof` the most derived object of which `object` is the base\n  // object.\n  //\n  // To customize `DynamicSizeOf()` for a class `T`, define a free function:\n  //   `friend size_t RiegeliDynamicSizeOf(const T* self)`\n  // as a friend of `T` inside class definition or in the same namespace as `T`,\n  // so that it can be found via ADL. That function typically calls a protected\n  // or private virtual function whose overrides return `sizeof(Derived)`.\n  //\n  // By default returns `sizeof(T)`.\n  template <typename T>\n  static size_t DynamicSizeOf(const T* object);\n\n  // Registers subobjects of `object`. Does not include memory corresponding to\n  // `sizeof(T)`.\n  //\n  // To customize `RegisterSubobjects()` for a class `T`, define a free\n  // function:\n  //   `friend void RiegeliRegisterSubobjects(\n  //        const T* self, MemoryEstimator& memory_estimator)`\n  // as a friend of `T` inside class definition or in the same namespace as `T`,\n  // so that it can be found via ADL. `MemoryEstimator` in the parameter type\n  // can also be a template parameter to reduce library dependencies.\n  //\n  // By default does nothing if `T` is trivially destructible, otherwise calls\n  // `RegisterUnknownType<T>()`, which likely indicates that an interesting\n  // customization is missing and results are underestimated.\n  //\n  // If `object` might be a member variable stored as a reference (e.g. in\n  // `std::tuple`), the type argument `T` must be specified explicitly, because\n  // `T` deduced from the argument of `RegisterSubobjects()` would be the\n  // corresponding value type. A member variable which is always a reference\n  // is trivially destructible and does not need to be registered.\n  //\n  // Predefined customizations include:\n  //  * `T[size]`\n  //  * `std::unique_ptr<T, Deleter>`\n  //  * `std::shared_ptr<T>`\n  //  * `std::basic_string<Char, Traits, Alloc>` (`Alloc` is ignored)\n  //  * `absl::Cord`\n  //  * `std::optional<T>`\n  //  * `std::variant<T...>`\n  //  * `std::pair<T, U>`\n  //  * `std::tuple<T...>`\n  //  * `std::array<T, size>`\n  //  * `std::vector<T, Alloc>` (`Alloc` is ignored)\n  //  * `absl::InlinedVector<T, N, Alloc>` (`Alloc` is ignored)\n  //  * `absl::flat_hash_set<T, Eq, Hash, Alloc>` (`Alloc` is ignored)\n  //  * `absl::flat_hash_map<K, V, Eq, Hash, Alloc>` (`Alloc` is ignored)\n  //  * `absl::node_hash_set<T, Eq, Hash, Alloc>` (`Alloc` is ignored)\n  //  * `absl::node_hash_map<K, V, Eq, Hash, Alloc>` (`Alloc` is ignored)\n  //  * `google::protobuf::Message`\n  template <typename T>\n  void RegisterSubobjects(const T* object);\n  template <typename T, std::enable_if_t<std::is_reference_v<T>, int> = 0>\n  void RegisterSubobjects(const std::remove_reference_t<T>* object);\n\n  // Registers each element of a range.\n  template <typename Iterator>\n  void RegisterSubobjects(Iterator begin, Iterator end);\n\n  // A shortcut for `RegisterDynamicMemory(object, DynamicSizeOf(object))`\n  // followed by `RegisterSubobjects(object)`.\n  template <typename T>\n  void RegisterDynamicObject(const T* object);\n\n  // Returns the total amount of memory added.\n  size_t TotalMemory() const { return total_memory_; }\n\n protected:\n  virtual void RegisterDynamicMemoryImpl(const void* ptr, size_t memory) = 0;\n  virtual void RegisterDynamicMemoryImpl(size_t memory) = 0;\n  virtual bool RegisterNodeImpl(const void* absl_nullable ptr) = 0;\n  virtual void RegisterUnknownTypeImpl() = 0;\n  virtual void RegisterUnknownTypeImpl(std::type_index index) = 0;\n\n private:\n  size_t total_memory_ = 0;\n};\n\n// A `MemoryEstimator` which gives a pretty good estimate for known types.\n//\n//  * Includes memory allocator overhead.\n//  * Takes object sharing into account.\n//  * Does not report unknown types.\nclass MemoryEstimatorDefault : public MemoryEstimator {\n public:\n  MemoryEstimatorDefault() = default;\n\n  MemoryEstimatorDefault(const MemoryEstimatorDefault& that) = delete;\n  MemoryEstimatorDefault& operator=(const MemoryEstimatorDefault& that) =\n      delete;\n\n protected:\n  void RegisterDynamicMemoryImpl(const void* ptr, size_t memory) override {\n    RegisterMemory(EstimatedAllocatedSize(ptr, memory));\n  }\n  void RegisterDynamicMemoryImpl(size_t memory) override {\n    RegisterMemory(EstimatedAllocatedSize(memory));\n  }\n  bool RegisterNodeImpl(const void* absl_nullable ptr) override;\n  void RegisterUnknownTypeImpl() override {}\n  void RegisterUnknownTypeImpl(\n      ABSL_ATTRIBUTE_UNUSED std::type_index index) override {}\n\n private:\n  absl::flat_hash_set<const void*> objects_seen_;\n};\n\n// A faster but less accurate `MemoryEstimator`.\n//\n//  * Does not include memory allocator overhead.\n//  * Does not take object sharing into account (if an object is shared then\n//    it is counted multiple times).\n//  * Does not report unknown types.\nclass MemoryEstimatorSimplified : public MemoryEstimator {\n public:\n  MemoryEstimatorSimplified() = default;\n\n  MemoryEstimatorSimplified(const MemoryEstimatorSimplified&) = delete;\n  MemoryEstimatorSimplified& operator=(const MemoryEstimatorSimplified&) =\n      delete;\n\n protected:\n  void RegisterDynamicMemoryImpl(ABSL_ATTRIBUTE_UNUSED const void* ptr,\n                                 size_t memory) override {\n    RegisterMemory(memory);\n  }\n  void RegisterDynamicMemoryImpl(size_t memory) override {\n    RegisterMemory(memory);\n  }\n  bool RegisterNodeImpl(const void* absl_nullable ptr) override {\n    return ptr != nullptr;\n  }\n  void RegisterUnknownTypeImpl() override {}\n  void RegisterUnknownTypeImpl(\n      ABSL_ATTRIBUTE_UNUSED std::type_index index) override {}\n};\n\n// A `MemoryEstimator` which can report encountered types for which\n// `RegisterSubobjects()` is not customized and which are not trivially\n// destructible, indicating whether interesting customizations are missing and\n// results are underestimated. Otherwise behaves like `MemoryEstimatorDefault`.\n//\n//  * Includes memory allocator overhead.\n//  * Takes object sharing into account.\n//  * Reports unknown types.\nclass MemoryEstimatorReportingUnknownTypes : public MemoryEstimatorDefault {\n public:\n  MemoryEstimatorReportingUnknownTypes() = default;\n\n  MemoryEstimatorReportingUnknownTypes(\n      const MemoryEstimatorReportingUnknownTypes&) = delete;\n  MemoryEstimatorReportingUnknownTypes& operator=(\n      const MemoryEstimatorReportingUnknownTypes&) = delete;\n\n  // Returns names of encountered types for which `RegisterSubobjects()` is not\n  // customized and which are not trivially destructible. If the result is not\n  // empty, this likely indicates that interesting customizations are missing\n  // and results are underestimated.\n  //\n  // If RTTI is not available, \"<no rtti>\" is returned as a placeholder.\n  std::vector<std::string> UnknownTypes() const;\n\n protected:\n  void RegisterUnknownTypeImpl() override;\n  void RegisterUnknownTypeImpl(std::type_index index) override;\n\n private:\n  bool unknown_types_no_rtti_ = false;\n  absl::flat_hash_set<std::type_index> unknown_types_;\n};\n\n// Uses `MemoryEstimatorDefault` to estimate memory owned by a single object\n// and its subobjects, including `sizeof` the original object.\ntemplate <typename T>\ninline size_t EstimateMemory(const T& object) {\n  MemoryEstimatorDefault memory_estimator;\n  memory_estimator.RegisterMemory(MemoryEstimator::DynamicSizeOf(&object));\n  memory_estimator.RegisterSubobjects(&object);\n  return memory_estimator.TotalMemory();\n}\n\n// Uses `MemoryEstimatorSimplified` to estimate memory owned by a single object\n// and its subobjects, including `sizeof` the original object.\ntemplate <typename T>\ninline size_t EstimateMemorySimplified(const T& object) {\n  MemoryEstimatorSimplified memory_estimator;\n  memory_estimator.RegisterMemory(MemoryEstimator::DynamicSizeOf(&object));\n  memory_estimator.RegisterSubobjects(&object);\n  return memory_estimator.TotalMemory();\n}\n\n// Uses `MemoryEstimatorReportingUnknownTypes` to estimate memory owned by a\n// single object and its subobjects, including `sizeof` the original object, and\n// unknown types.\n\nstruct TotalMemoryWithUnknownTypes {\n  size_t total_memory;\n  std::vector<std::string> unknown_types;\n};\n\ntemplate <typename T>\ninline TotalMemoryWithUnknownTypes EstimateMemoryReportingUnknownTypes(\n    const T& object) {\n  MemoryEstimatorReportingUnknownTypes memory_estimator;\n  memory_estimator.RegisterMemory(MemoryEstimator::DynamicSizeOf(&object));\n  memory_estimator.RegisterSubobjects(&object);\n  return TotalMemoryWithUnknownTypes{memory_estimator.TotalMemory(),\n                                     memory_estimator.UnknownTypes()};\n}\n\n// Implementation details follow.\n\ntemplate <typename T>\ninline void MemoryEstimator::RegisterUnknownType() {\n  RegisterUnknownTypeImpl(\n#if __cpp_rtti\n      std::type_index(typeid(T))\n#endif\n  );\n}\n\nnamespace memory_estimator_internal {\n\ntemplate <typename T, typename Enable = void>\nstruct HasRiegeliDynamicSizeOf : std::false_type {};\n\ntemplate <typename T>\nstruct HasRiegeliDynamicSizeOf<\n    T, std::enable_if_t<std::is_convertible_v<\n           decltype(RiegeliDynamicSizeOf(std::declval<const T*>())), size_t>>>\n    : std::true_type {};\n\ntemplate <typename T>\ninline size_t DynamicSizeOf(const T* object) {\n  if constexpr (HasRiegeliDynamicSizeOf<T>::value) {\n    return RiegeliDynamicSizeOf(object);\n  } else {\n    return sizeof(T);\n  }\n}\n\ntemplate <typename T>\ninline void RegisterSubobjects(const T* object,\n                               MemoryEstimator& memory_estimator) {\n  if constexpr (HasRiegeliRegisterSubobjects<T>::value) {\n    RiegeliRegisterSubobjects(object, memory_estimator);\n  } else if constexpr (!std::is_trivially_destructible<T>::value) {\n    memory_estimator.RegisterUnknownType<T>();\n  }\n}\n\n}  // namespace memory_estimator_internal\n\ntemplate <typename T>\ninline size_t MemoryEstimator::DynamicSizeOf(const T* object) {\n  return memory_estimator_internal::DynamicSizeOf(object);\n}\n\ntemplate <typename T>\ninline void MemoryEstimator::RegisterSubobjects(const T* object) {\n  memory_estimator_internal::RegisterSubobjects(object, *this);\n}\n\ntemplate <typename T, std::enable_if_t<std::is_reference_v<T>, int>>\ninline void MemoryEstimator::RegisterSubobjects(\n    ABSL_ATTRIBUTE_UNUSED const std::remove_reference_t<T>* object) {}\n\ntemplate <typename Iterator>\ninline void MemoryEstimator::RegisterSubobjects(Iterator begin, Iterator end) {\n  if (!RegisterSubobjectsIsTrivial<\n          typename std::iterator_traits<Iterator>::value_type>::value) {\n    for (; begin != end; ++begin) RegisterSubobjects(&*begin);\n  }\n}\n\ntemplate <typename T>\ninline void MemoryEstimator::RegisterDynamicObject(const T* object) {\n  RegisterDynamicMemory(object, MemoryEstimator::DynamicSizeOf(object));\n  RegisterSubobjects(object);\n}\n\ntemplate <typename T, size_t size>\ninline void RiegeliRegisterSubobjects(const T (*self)[size],\n                                      MemoryEstimator& memory_estimator) {\n  memory_estimator.RegisterSubobjects(*self + 0, *self + size);\n}\n\ntemplate <typename T, typename Deleter,\n          std::enable_if_t<std::conjunction_v<std::negation<std::is_void<T>>,\n                                              std::negation<std::is_array<T>>>,\n                           int> = 0>\ninline void RiegeliRegisterSubobjects(\n    const absl_nullable std::unique_ptr<T, Deleter>* self,\n    MemoryEstimator& memory_estimator) {\n  memory_estimator.RegisterSubobjects<Deleter>(&self->get_deleter());\n  if (*self != nullptr) memory_estimator.RegisterDynamicObject(self->get());\n}\n\nnamespace memory_estimator_internal {\n\n// Reflects the layout of a control block of `std::shared_ptr` from libc++.\nstruct SharedPtrControlBlock {\n  virtual ~SharedPtrControlBlock() = default;\n  long shared_count;\n  long weak_count;\n};\n\n}  // namespace memory_estimator_internal\n\ntemplate <typename T,\n          std::enable_if_t<std::conjunction_v<std::negation<std::is_void<T>>,\n                                              std::negation<std::is_array<T>>>,\n                           int> = 0>\ninline void RiegeliRegisterSubobjects(const\n                                      absl_nullable std::shared_ptr<T>* self,\n                                      MemoryEstimator& memory_estimator) {\n  if (memory_estimator.RegisterNode(self->get())) {\n    memory_estimator.RegisterDynamicMemory(\n        sizeof(memory_estimator_internal::SharedPtrControlBlock) +\n        MemoryEstimator::DynamicSizeOf(&**self));\n    memory_estimator.RegisterSubobjects(&**self);\n  }\n}\n\ntemplate <typename T, size_t size>\ninline void RiegeliRegisterSubobjects(\n    const absl_nullable std::shared_ptr<T[size]>* self,\n    MemoryEstimator& memory_estimator) {\n  if (memory_estimator.RegisterNode(self->get())) {\n    memory_estimator.RegisterDynamicMemory(\n        sizeof(memory_estimator_internal::SharedPtrControlBlock) +\n        sizeof(T[size]));\n    memory_estimator.RegisterSubobjects(self->get(), self->get() + size);\n  }\n}\n\ntemplate <typename Char, typename Traits, typename Alloc>\ninline void RiegeliRegisterSubobjects(\n    const std::basic_string<Char, Traits, Alloc>* self,\n    MemoryEstimator& memory_estimator) {\n  if (self->capacity() > std::basic_string<Char, Traits, Alloc>().capacity()) {\n    memory_estimator.RegisterDynamicMemory((self->capacity() + 1) *\n                                           sizeof(Char));\n  } else {\n    // Assume short string optimization.\n  }\n}\n\ninline void RiegeliRegisterSubobjects(const absl::Cord* self,\n                                      MemoryEstimator& memory_estimator) {\n  // Scale `self->EstimatedMemoryUsage()` by a fraction corresponding to how\n  // much of its memory is newly seen.\n  size_t new_bytes = 0;\n  size_t total_bytes = 0;\n  for (const absl::string_view fragment : self->Chunks()) {\n    if (memory_estimator.RegisterNode(fragment.data())) {\n      new_bytes += fragment.size();\n    }\n    total_bytes += fragment.size();\n  }\n  memory_estimator.RegisterMemory(static_cast<size_t>(\n      static_cast<double>(self->EstimatedMemoryUsage() - sizeof(absl::Cord)) *\n      (static_cast<double>(new_bytes) / static_cast<double>(total_bytes))));\n}\n\ntemplate <typename T>\ninline void RiegeliRegisterSubobjects(const std::optional<T>* self,\n                                      MemoryEstimator& memory_estimator) {\n  if (*self != std::nullopt) memory_estimator.RegisterSubobjects(&**self);\n}\n\nnamespace memory_estimator_internal {\n\nstruct RegisterSubobjectsVisitor {\n  template <typename T>\n  void operator()(const T& object) const {\n    memory_estimator.RegisterSubobjects(&object);\n  }\n\n  MemoryEstimator& memory_estimator;\n};\n\n}  // namespace memory_estimator_internal\n\ntemplate <typename... T>\ninline void RiegeliRegisterSubobjects(const std::variant<T...>* self,\n                                      MemoryEstimator& memory_estimator) {\n  std::visit(\n      memory_estimator_internal::RegisterSubobjectsVisitor{memory_estimator},\n      *self);\n}\n\ntemplate <typename T, typename U>\ninline void RiegeliRegisterSubobjects(const std::pair<T, U>* self,\n                                      MemoryEstimator& memory_estimator) {\n  memory_estimator.RegisterSubobjects<T>(&self->first);\n  memory_estimator.RegisterSubobjects<U>(&self->second);\n}\n\nnamespace memory_estimator_internal {\n\ntemplate <size_t index, typename... T>\ninline void RegisterTupleElements(const std::tuple<T...>* self,\n                                  MemoryEstimator& memory_estimator) {\n  if constexpr (index < sizeof...(T)) {\n    memory_estimator\n        .RegisterSubobjects<std::tuple_element_t<index, std::tuple<T...>>>(\n            &std::get<index>(*self));\n    RegisterTupleElements<index + 1>(self, memory_estimator);\n  }\n}\n\n}  // namespace memory_estimator_internal\n\ntemplate <typename... T>\ninline void RiegeliRegisterSubobjects(const std::tuple<T...>* self,\n                                      MemoryEstimator& memory_estimator) {\n  memory_estimator_internal::RegisterTupleElements<0>(self, memory_estimator);\n}\n\ntemplate <typename T, size_t size>\ninline void RiegeliRegisterSubobjects(const std::array<T, size>* self,\n                                      MemoryEstimator& memory_estimator) {\n  memory_estimator.RegisterSubobjects(self->cbegin(), self->cend());\n}\n\ntemplate <typename T, typename Alloc>\ninline void RiegeliRegisterSubobjects(const std::vector<T, Alloc>* self,\n                                      MemoryEstimator& memory_estimator) {\n  if (self->capacity() > 0) {\n    memory_estimator.RegisterDynamicMemory(self->capacity() * sizeof(T));\n    memory_estimator.RegisterSubobjects(self->cbegin(), self->cend());\n  }\n}\n\ntemplate <typename Alloc>\ninline void RiegeliRegisterSubobjects(const std::vector<bool, Alloc>* self,\n                                      MemoryEstimator& memory_estimator) {\n  if (self->capacity() > 0) {\n    memory_estimator.RegisterDynamicMemory(self->capacity() / 8);\n  }\n}\n\ntemplate <typename T, size_t N, typename Alloc>\ninline void RiegeliRegisterSubobjects(\n    const absl::InlinedVector<T, N, Alloc>* self,\n    MemoryEstimator& memory_estimator) {\n  if (self->capacity() > N) {\n    memory_estimator.RegisterDynamicMemory(self->capacity() * sizeof(T));\n  }\n  memory_estimator.RegisterSubobjects(self->cbegin(), self->cend());\n}\n\ntemplate <typename T, typename Eq, typename Hash, typename Alloc>\ninline void RiegeliRegisterSubobjects(\n    const absl::flat_hash_set<T, Eq, Hash, Alloc>* self,\n    MemoryEstimator& memory_estimator) {\n  memory_estimator.RegisterMemory(\n      absl::container_internal::hashtable_debug_internal::HashtableDebugAccess<\n          absl::flat_hash_set<T, Eq, Hash, Alloc>>::AllocatedByteSize(*self));\n  memory_estimator.RegisterSubobjects(self->cbegin(), self->cend());\n}\n\ntemplate <typename K, typename V, typename Eq, typename Hash, typename Alloc>\ninline void RiegeliRegisterSubobjects(\n    const absl::flat_hash_map<K, V, Eq, Hash, Alloc>* self,\n    MemoryEstimator& memory_estimator) {\n  memory_estimator.RegisterMemory(\n      absl::container_internal::hashtable_debug_internal::HashtableDebugAccess<\n          absl::flat_hash_map<K, V, Eq, Hash,\n                              Alloc>>::AllocatedByteSize(*self));\n  memory_estimator.RegisterSubobjects(self->cbegin(), self->cend());\n}\n\ntemplate <typename T, typename Eq, typename Hash, typename Alloc>\ninline void RiegeliRegisterSubobjects(\n    const absl::node_hash_set<T, Eq, Hash, Alloc>* self,\n    MemoryEstimator& memory_estimator) {\n  memory_estimator.RegisterMemory(\n      absl::container_internal::hashtable_debug_internal::HashtableDebugAccess<\n          absl::node_hash_set<T, Eq, Hash, Alloc>>::AllocatedByteSize(*self));\n  memory_estimator.RegisterSubobjects(self->cbegin(), self->cend());\n}\n\ntemplate <typename K, typename V, typename Eq, typename Hash, typename Alloc>\ninline void RiegeliRegisterSubobjects(\n    const absl::node_hash_map<K, V, Eq, Hash, Alloc>* self,\n    MemoryEstimator& memory_estimator) {\n  memory_estimator.RegisterMemory(\n      absl::container_internal::hashtable_debug_internal::HashtableDebugAccess<\n          absl::node_hash_map<K, V, Eq, Hash,\n                              Alloc>>::AllocatedByteSize(*self));\n  memory_estimator.RegisterSubobjects(self->cbegin(), self->cend());\n}\n\ntemplate <typename T,\n          std::enable_if_t<\n              std::is_convertible_v<T*, google::protobuf::Message*>, int> = 0>\ninline void RiegeliRegisterSubobjects(const T* self,\n                                      MemoryEstimator& memory_estimator) {\n  memory_estimator.RegisterMemory(self->SpaceUsedLong() - sizeof(T));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_MEMORY_ESTIMATOR_H_\n"
  },
  {
    "path": "riegeli/base/moving_dependency.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_MOVING_DEPENDENCY_H_\n#define RIEGELI_BASE_MOVING_DEPENDENCY_H_\n\n#include <stddef.h>\n\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/type_traits.h\"\n\nnamespace riegeli {\n\n// `MovingDependency<Handle, Manager, Mover>` extends\n// `Dependency<Handle, Manager>` with assistance in moving the host object\n// (containing the dependency), adjusting the host state while moving the\n// dependency, e.g. if moving the dependency can invalidate pointers stored\n// in the host object.\n//\n// The host object should have a defaulted move constructor and assignment,\n// so that it is move constructible and/or move assignable whenever the\n// dependency is. Customizations of moving belong to its base classes, members,\n// and the `Mover` class passed to `MovingDependency`.\n//\n// If `Dependency<Handle, Manager>::kIsStable`, then it is assumed that\n// no assistance is needed, and `MovingDependency<Handle, Manager, Mover>` is\n// equivalent to `Dependency<Handle, Manager>`. Otherwise the `Mover` class\n// specifies adjustment of the host state. It should have the following members:\n//\n// ```\n//   // Returns a member pointer of the host class to the `MovingDependency`.\n//   // This is used to get the type of the host class, and to find the host\n//   // object from the `MovingDependency` by following the member pointer in\n//   // the reverse direction.\n//   //\n//   // Skip this for `Host` using virtual inheritance.\n//   static auto member() { return &Host::dep_; }\n//\n//   // Constructor, called when base classes of the host object are already\n//   // moved, but the dependency is not moved yet. The host object is being\n//   // moved from `that` to `self`.\n//   //\n//   // Parameters are optional (possibly only the second one).\n//   explicit Mover(Host& self, Host& that);\n//\n//   // Called when the dependency is already moved. The host object is being\n//   // moved to `self`.\n//   //\n//   // Actions can also be performed in the destructor, but `Done()` receoves\n//   // `self`, so that it does not have to be stored in `Mover`.\n//   //\n//   // This method is optional. The parameter is optional.\n//   void Done(Host& self);\n// ```\n//\n// If `Host` uses virtual inheritance, even indirectly, then the leaf class\n// is responsible for moving virtual base classes. `Host` should have move\n// constructor and assignment defined explicitly. Their availability can be\n// conditional on movability of the `Dependency` only starting from C++20,\n// using the `requires` clause. `Mover::member()` should not be defined;\n// this way of finding the host object does not work on Windows when virtual\n// inheritance is used. The `MovingDependency` should be move-constructed by\n// passing `*this, that` as additional parameters to its constructor, and it\n// should be move-assigned by calling `Reset()` instead of `operator=` and\n// passing `*this, that` as additional parameters.\n//\n// This template is specialized but does not have a primary definition.\ntemplate <typename Handle, typename Manager, typename Mover,\n          typename Enable = void>\nclass MovingDependency;\n\nnamespace moving_dependency_internal {\n\nclass SimpleClass {\n public:\n  int member;\n};\n\ntemplate <\n    typename Mover, typename Host,\n    std::enable_if_t<std::is_constructible_v<Mover, Host&, Host&>, int> = 0>\nMover MakeMover(Host& self, Host& that) {\n  return Mover(self, that);\n}\n\ntemplate <typename Mover, typename Host,\n          std::enable_if_t<\n              std::conjunction_v<\n                  std::negation<std::is_constructible<Mover, Host&, Host&>>,\n                  std::is_constructible<Mover, Host&>>,\n              int> = 0>\nMover MakeMover(Host& self, ABSL_ATTRIBUTE_UNUSED Host& that) {\n  return Mover(self);\n}\n\ntemplate <typename Mover, typename Host,\n          std::enable_if_t<\n              std::conjunction_v<\n                  std::negation<std::is_constructible<Mover, Host&, Host&>>,\n                  std::negation<std::is_constructible<Mover, Host&>>,\n                  std::is_default_constructible<Mover>>,\n              int> = 0>\nMover MakeMover(ABSL_ATTRIBUTE_UNUSED Host& self,\n                ABSL_ATTRIBUTE_UNUSED Host& that) {\n  return Mover();\n}\n\ntemplate <typename Mover, typename Host, typename Enable = void>\nstruct HasDoneWithSelf : std::false_type {};\n\ntemplate <typename Mover, typename Host>\nstruct HasDoneWithSelf<\n    Mover, Host,\n    std::void_t<decltype(std::declval<Mover&>().Done(std::declval<Host&>()))>>\n    : std::true_type {};\n\ntemplate <typename Mover, typename Enable = void>\nstruct HasDoneWithoutSelf : std::false_type {};\n\ntemplate <typename Mover>\nstruct HasDoneWithoutSelf<Mover,\n                          std::void_t<decltype(std::declval<Mover&>().Done())>>\n    : std::true_type {};\n\ntemplate <typename Mover, typename Host,\n          std::enable_if_t<HasDoneWithSelf<Mover, Host>::value, int> = 0>\ninline void Done(Mover& mover, Host& self) {\n  mover.Done(self);\n}\n\ntemplate <typename Mover, typename Host,\n          std::enable_if_t<\n              std::conjunction_v<std::negation<HasDoneWithSelf<Mover, Host>>,\n                                 HasDoneWithoutSelf<Mover>>,\n              int> = 0>\ninline void Done(Mover& mover, ABSL_ATTRIBUTE_UNUSED Host& self) {\n  mover.Done();\n}\n\ntemplate <typename Mover, typename Host,\n          std::enable_if_t<\n              std::conjunction_v<std::negation<HasDoneWithSelf<Mover, Host>>,\n                                 std::negation<HasDoneWithoutSelf<Mover>>>,\n              int> = 0>\ninline void Done(ABSL_ATTRIBUTE_UNUSED Mover& mover,\n                 ABSL_ATTRIBUTE_UNUSED Host& self) {}\n\ntemplate <typename Enable, typename T, typename... Args>\nstruct HasResetImpl : std::false_type {};\n\ntemplate <typename T, typename... Args>\nstruct HasResetImpl<\n    std::void_t<decltype(std::declval<T&>().Reset(std::declval<Args&&>()...))>,\n    T, Args...> : std::true_type {};\n\ntemplate <typename T, typename... Args>\nstruct HasReset : HasResetImpl<void, T, Args...> {};\n\ntemplate <typename Handle, typename Manager, typename Mover>\nclass MovingDependencyImpl : public Dependency<Handle, Manager> {\n public:\n  using MovingDependencyImpl::Dependency::Dependency;\n\n  // Not supported when `Host` uses virtual inheritance.\n  MovingDependencyImpl(MovingDependencyImpl&& that) noexcept\n      : MovingDependencyImpl(static_cast<MovingDependencyImpl&&>(that),\n                             moving_dependency_internal::MakeMover<Mover>(\n                                 this->get_host(Mover::member()),\n                                 that.get_host(Mover::member()))) {}\n\n  // Not supported when `Host` uses virtual inheritance.\n  MovingDependencyImpl& operator=(MovingDependencyImpl&& that) noexcept {\n    Mover mover = moving_dependency_internal::MakeMover<Mover>(\n        this->get_host(Mover::member()), that.get_host(Mover::member()));\n    MovingDependencyImpl::Dependency::operator=(\n        static_cast<typename MovingDependencyImpl::Dependency&&>(that));\n    moving_dependency_internal::Done(mover, this->get_host(Mover::member()));\n    return *this;\n  }\n\n  // Required when `Host` uses virtual inheritance.\n  template <typename Host>\n  MovingDependencyImpl(MovingDependencyImpl&& that, Host& this_host,\n                       Host& that_host) noexcept\n      : MovingDependencyImpl(\n            static_cast<MovingDependencyImpl&&>(that),\n            moving_dependency_internal::MakeMover<Mover>(this_host, that_host),\n            this_host) {}\n\n  // Required when `Host` uses virtual inheritance.\n  template <typename Host>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(MovingDependencyImpl&& that,\n                                          Host& this_host,\n                                          Host& that_host) noexcept {\n    Mover mover =\n        moving_dependency_internal::MakeMover<Mover>(this_host, that_host);\n    MovingDependencyImpl::Dependency::operator=(\n        static_cast<typename MovingDependencyImpl::Dependency&&>(that));\n    moving_dependency_internal::Done(mover, this_host);\n  }\n\n  // Not `using MovingDependencyImpl::Dependency::Reset` because it might have\n  // no overloads.\n  template <typename... Args,\n            std::enable_if_t<\n                moving_dependency_internal::HasReset<\n                    typename MovingDependencyImpl::Dependency, Args...>::value,\n                int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Args&&... args) {\n    MovingDependencyImpl::Dependency::Reset(std::forward<Args>(args)...);\n  }\n\n private:\n  explicit MovingDependencyImpl(MovingDependencyImpl&& that, Mover mover)\n      : MovingDependencyImpl::Dependency(\n            static_cast<typename MovingDependencyImpl::Dependency&&>(that)) {\n    moving_dependency_internal::Done(mover, this->get_host(Mover::member()));\n  }\n\n  template <typename Host>\n  explicit MovingDependencyImpl(MovingDependencyImpl&& that, Mover mover,\n                                Host& this_host)\n      : MovingDependencyImpl::Dependency(\n            static_cast<typename MovingDependencyImpl::Dependency&&>(that)) {\n    moving_dependency_internal::Done(mover, this_host);\n  }\n\n  template <typename Host>\n  Host& get_host(MovingDependency<Handle, Manager, Mover> Host::* member) {\n    // This assertion detects virtual inheritance on Windows.\n    static_assert(sizeof(member) == sizeof(&SimpleClass::member),\n                  \"For a host class using virtual inheritance, \"\n                  \"MovingDependency must have this_host and that_host \"\n                  \"passed explicitly.\");\n    alignas(alignof(Host)) char storage[sizeof(Host)];\n    const size_t offset =\n        reinterpret_cast<char*>(&(reinterpret_cast<Host*>(storage)->*member)) -\n        storage;\n    return *reinterpret_cast<Host*>(reinterpret_cast<char*>(this) - offset);\n  }\n};\n\n}  // namespace moving_dependency_internal\n\n// Specialization when `Dependency<Handle, Manager>` is stable: delegate to it.\ntemplate <typename Handle, typename Manager, typename Mover>\nclass MovingDependency<Handle, Manager, Mover,\n                       std::enable_if_t<Dependency<Handle, Manager>::kIsStable>>\n    : public Dependency<Handle, Manager> {\n public:\n  using MovingDependency::Dependency::Dependency;\n\n  // Not supported when `Host` uses virtual inheritance.\n  MovingDependency(MovingDependency&& that) = default;\n  // Not supported when `Host` uses virtual inheritance.\n  MovingDependency& operator=(MovingDependency&& that) = default;\n\n  // Required when `Host` uses virtual inheritance.\n  template <typename Host>\n  MovingDependency(MovingDependency&& that,\n                   ABSL_ATTRIBUTE_UNUSED Host& this_host,\n                   ABSL_ATTRIBUTE_UNUSED Host& that_host) noexcept\n      : MovingDependency::Dependency(\n            static_cast<typename MovingDependency::Dependency&&>(that)) {}\n\n  // Required when `Host` uses virtual inheritance.\n  template <typename Host>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      MovingDependency&& that, ABSL_ATTRIBUTE_UNUSED Host& this_host,\n      ABSL_ATTRIBUTE_UNUSED Host& that_host) noexcept {\n    MovingDependency::Dependency::operator=(\n        static_cast<typename MovingDependency::Dependency&&>(that));\n  }\n\n  // Not `using MovingDependency::Dependency::Reset` because it might have\n  // no overloads.\n  template <typename... Args,\n            std::enable_if_t<\n                moving_dependency_internal::HasReset<\n                    typename MovingDependency::Dependency, Args...>::value,\n                int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Args&&... args) {\n    MovingDependency::Dependency::Reset(std::forward<Args>(args)...);\n  }\n};\n\n// Specialization when `Dependency<Handle, Manager>` is not stable.\ntemplate <typename Handle, typename Manager, typename Mover>\nclass MovingDependency<\n    Handle, Manager, Mover,\n    std::enable_if_t<!Dependency<Handle, Manager>::kIsStable>>\n    : public moving_dependency_internal::MovingDependencyImpl<Handle, Manager,\n                                                              Mover>,\n      public ConditionallyConstructible<\n          false, std::is_move_constructible_v<Dependency<Handle, Manager>>>,\n      public ConditionallyAssignable<\n          false, std::is_move_assignable_v<Dependency<Handle, Manager>>> {\n public:\n  using MovingDependency::MovingDependencyImpl::MovingDependencyImpl;\n\n  MovingDependency(MovingDependency&& that) = default;\n  MovingDependency& operator=(MovingDependency&& that) = default;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_MOVING_DEPENDENCY_H_\n"
  },
  {
    "path": "riegeli/base/new_aligned.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_NEW_ALIGNED_H_\n#define RIEGELI_BASE_NEW_ALIGNED_H_\n\n#include <stddef.h>\n\n#include <limits>  // IWYU pragma: keep\n#include <new>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/numeric/bits.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"                    // IWYU pragma: keep\n#include \"riegeli/base/estimated_allocated_size.h\"  // IWYU pragma: keep\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nnamespace new_aligned_internal {\n\ntemplate <typename T>\ninline void EnsureSpaceForObject(size_t& num_bytes) {\n  // Allocate enough space to construct the object, even if the caller does not\n  // need the whole tail part of the object.\n  num_bytes = UnsignedMax(num_bytes, sizeof(T));\n}\n\ntemplate <>\ninline void EnsureSpaceForObject<void>(\n    ABSL_ATTRIBUTE_UNUSED size_t& num_bytes) {}\n\ntemplate <typename T, typename... Args>\ninline void ConstructObject(T* ptr, Args&&... args) {\n  new (ptr) T(std::forward<Args>(args)...);\n}\n\ntemplate <>\ninline void ConstructObject(ABSL_ATTRIBUTE_UNUSED void* ptr) {}\n\ntemplate <typename T>\ninline void DestroyObject(T* ptr) {\n  ptr->~T();\n}\n\ntemplate <>\ninline void DestroyObject(ABSL_ATTRIBUTE_UNUSED void* ptr) {}\n\n}  // namespace new_aligned_internal\n\n// `NewAligned()`/`DeleteAligned()` provide memory allocation with the specified\n// alignment known at compile time, with the size specified in bytes, and which\n// allow deallocation to be faster by knowing the size.\n//\n// The alignment and size passed to `DeleteAligned()` must be the same as in the\n// corresponding `NewAligned()`. Pointer types must be compatible as with new\n// and delete expressions.\n//\n// If `T` is `void`, raw memory is allocated but no object is constructed or\n// destroyed.\n//\n// If the allocated size is given in terms of objects rather than bytes\n// and the type is not over-aligned (i.e. its alignment is not larger than\n// `alignof(max_align_t))`, it is simpler to use `std::allocator<T>()` instead.\n// If the type is over-aligned, `std::allocator<T>()` works correctly only when\n// `operator new(size_t, std::align_val_t)` from C++17 is available.\n\n// TODO: Test this with overaligned types.\n\ntemplate <typename T, size_t alignment = alignof(T), typename... Args>\ninline T* NewAligned(size_t num_bytes, Args&&... args) {\n  static_assert(absl::has_single_bit(alignment),\n                \"alignment must be a power of 2\");\n  new_aligned_internal::EnsureSpaceForObject<T>(num_bytes);\n  T* ptr;\n  if constexpr (alignment <= __STDCPP_DEFAULT_NEW_ALIGNMENT__) {\n    ptr = static_cast<T*>(operator new(num_bytes));\n  } else {\n    ptr = static_cast<T*>(operator new(num_bytes, std::align_val_t(alignment)));\n  }\n  new_aligned_internal::ConstructObject(ptr, std::forward<Args>(args)...);\n  return ptr;\n}\n\ntemplate <typename T, size_t alignment = alignof(T)>\ninline void DeleteAligned(T* ptr, size_t num_bytes) {\n  static_assert(absl::has_single_bit(alignment),\n                \"alignment must be a power of 2\");\n  new_aligned_internal::EnsureSpaceForObject<T>(num_bytes);\n  new_aligned_internal::DestroyObject(ptr);\n  if constexpr (alignment <= __STDCPP_DEFAULT_NEW_ALIGNMENT__) {\n    operator delete(ptr, num_bytes);\n  } else {\n    operator delete(ptr, num_bytes, std::align_val_t(alignment));\n  }\n}\n\n// `SizeReturningNewAligned()` is like `NewAligned()`, but it returns the number\n// of bytes actually allocated, which can be greater than the requested number\n// of bytes.\n//\n// The object can be freed with `DeleteAligned()`, passing either\n// `min_num_bytes` or `*actual_num_bytes`, or anything between.\n//\n// `*actual_num_bytes` is already set during the constructor call.\ntemplate <typename T, size_t alignment = alignof(T), typename... Args>\ninline T* SizeReturningNewAligned(size_t min_num_bytes,\n                                  size_t* actual_num_bytes, Args&&... args) {\n  static_assert(absl::has_single_bit(alignment),\n                \"alignment must be a power of 2\");\n  new_aligned_internal::EnsureSpaceForObject<T>(min_num_bytes);\n  T* ptr;\n  const size_t capacity = EstimatedAllocatedSize(min_num_bytes);\n  if constexpr (alignment <= __STDCPP_DEFAULT_NEW_ALIGNMENT__) {\n    ptr = static_cast<T*>(operator new(capacity));\n  } else {\n    ptr = static_cast<T*>(operator new(capacity, std::align_val_t(alignment)));\n  }\n  *actual_num_bytes = capacity;\n  new_aligned_internal::ConstructObject(ptr, std::forward<Args>(args)...);\n  return ptr;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_NEW_ALIGNED_H_\n"
  },
  {
    "path": "riegeli/base/null_safe_memcpy.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_NULL_SAFE_MEMCPY_H_\n#define RIEGELI_BASE_NULL_SAFE_MEMCPY_H_\n\n#include <stddef.h>\n\n#include <cstring>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"  // IWYU pragma: keep\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `riegeli::null_safe_memcpy()` is like `std::memcpy()` but accepts null\n// pointers, as long as the size is zero.\n//\n// Everything here applies also to `memmove()`, `memset()`, and some other\n// functions.\n//\n// Background:\n//\n// `std::memcpy(nullptr, _, 0)` and `std::memcpy(_, nullptr, 0)` have undefined\n// behavior, both in C and in C++. This is consistent with C rules that pointer\n// arithmetic on `NULL` is undefined, even with zero offsets.\n//\n// In C++ pointer arithmetic on `nullptr` is defined, and an empty range\n// starting from `nullptr` is valid, such as a default-constructed\n// `absl::string_view`. Unfortunately `memcpy()` works like in C.\n//\n// https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3322.pdf is going to change\n// `memcpy()` in C2y. There is no proposal so far for C++ though.\n//\n// Some compilers make use of this undefined behavior. In particular GCC\n// optimizes explicit comparisons against `nullptr` after a pointer has been\n// passed to `memcpy()`. In GCC and Clang, ubsan checks that parameters of\n// `memcpy()` are not null, based on annotations in headers.\n//\n// A portable implementation of null-safe `memcpy()` uses an explicit length\n// check against zero. Unfortunately compilers do not optimize out that\n// conditional, even if the runtime library works with null pointers.\n//\n// According to\n// https://github.com/llvm/llvm-project/issues/49459#issuecomment-2579439921 and\n// https://github.com/llvm/llvm-project/issues/146484#issuecomment-3022856421,\n// `memcpy()` with null pointers has always worked in Clang. Hence for Clang\n// `riegeli::null_safe_memcpy()` just calls `std::memcpy()`, while disabling\n// the ubsan check which would otherwise complain.\n\n#ifdef __clang__\n__attribute__((no_sanitize(\"nonnull-attribute\")))\n#endif\ninline void null_safe_memcpy(void* absl_nullable dest,\n                             const void* absl_nullable src, size_t length) {\n#ifndef __clang__\n  if (ABSL_PREDICT_FALSE(length == 0)) return;\n#endif\n  std::memcpy(dest, src, length);\n}\n\n#ifdef __clang__\n__attribute__((no_sanitize(\"nonnull-attribute\")))\n#endif\ninline void null_safe_memmove(void* absl_nullable dest,\n                              const void* absl_nullable src, size_t length) {\n#ifndef __clang__\n  if (ABSL_PREDICT_FALSE(length == 0)) return;\n#endif\n  std::memmove(dest, src, length);\n}\n\n#ifdef __clang__\n__attribute__((no_sanitize(\"nonnull-attribute\")))\n#endif\ninline void null_safe_memset(void* absl_nullable dest, int c, size_t length) {\n#ifndef __clang__\n  if (ABSL_PREDICT_FALSE(length == 0)) return;\n#endif\n  std::memset(dest, c, length);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_NULL_SAFE_MEMCPY_H_\n"
  },
  {
    "path": "riegeli/base/object.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/object.h\"\n\n#include <stdint.h>\n\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/type_id.h\"\n\nnamespace riegeli {\n\nObjectState& ObjectState::operator=(ObjectState&& that) noexcept {\n  DeleteStatus(std::exchange(\n      status_ptr_, std::exchange(that.status_ptr_, kClosedSuccessfully)));\n  return *this;\n}\n\nObjectState::~ObjectState() { DeleteStatus(status_ptr_); }\n\nvoid ObjectState::Reset(Closed) {\n  DeleteStatus(std::exchange(status_ptr_, kClosedSuccessfully));\n}\n\nvoid ObjectState::Reset() { DeleteStatus(std::exchange(status_ptr_, kOk)); }\n\nbool ObjectState::MarkClosed() {\n  if (ABSL_PREDICT_TRUE(not_failed())) {\n    status_ptr_ = kClosedSuccessfully;\n    return true;\n  }\n  reinterpret_cast<FailedStatus*>(status_ptr_)->closed = true;\n  return false;\n}\n\nvoid ObjectState::MarkNotFailed() {\n  DeleteStatus(\n      std::exchange(status_ptr_, is_open() ? kOk : kClosedSuccessfully));\n}\n\ninline void ObjectState::DeleteStatus(uintptr_t status_ptr) {\n  if (ABSL_PREDICT_FALSE(status_ptr != kOk &&\n                         status_ptr != kClosedSuccessfully)) {\n    delete reinterpret_cast<FailedStatus*>(status_ptr);\n  }\n}\n\nabsl::Status ObjectState::status() const {\n  if (status_ptr_ == kOk) return absl::OkStatus();\n  if (status_ptr_ == kClosedSuccessfully) {\n    return absl::FailedPreconditionError(\"Object closed\");\n  }\n  return reinterpret_cast<const FailedStatus*>(status_ptr_)->status;\n}\n\nbool ObjectState::Fail(absl::Status status) {\n  RIEGELI_ASSERT(!status.ok())\n      << \"Failed precondition of ObjectState::Fail(): status not failed\";\n  if (status_ptr_ == kOk || status_ptr_ == kClosedSuccessfully) {\n    status_ptr_ = reinterpret_cast<uintptr_t>(new FailedStatus{\n        status_ptr_ == kClosedSuccessfully, std::move(status)});\n  }\n  return false;\n}\n\nvoid ObjectState::SetStatus(absl::Status status) {\n  RIEGELI_ASSERT(!status.ok())\n      << \"Failed precondition of ObjectState::SetStatus(): status not failed\";\n  RIEGELI_ASSERT(!not_failed())\n      << \"Failed precondition of ObjectState::SetStatus(): \"\n         \"ObjectState not failed\";\n  reinterpret_cast<FailedStatus*>(status_ptr_)->status = std::move(status);\n}\n\nbool Object::Close() {\n  if (ABSL_PREDICT_FALSE(!state_.is_open())) return state_.not_failed();\n  Done();\n  return state_.MarkClosed();\n}\n\nvoid Object::Done() {}\n\nbool Object::Fail(absl::Status status) {\n  RIEGELI_ASSERT(!status.ok())\n      << \"Failed precondition of Object::Fail(): status not failed\";\n  if (ABSL_PREDICT_FALSE(!not_failed())) return false;\n  state_.Fail(std::move(status));\n  state_.SetStatus(AnnotateStatus(state_.status()));\n  OnFail();\n  return false;\n}\n\nvoid Object::SetStatus(absl::Status status) {\n  RIEGELI_ASSERT(!status.ok())\n      << \"Failed precondition of Object::SetStatus(): status not failed\";\n  RIEGELI_ASSERT(!not_failed())\n      << \"Failed precondition of Object::SetStatus(): Object not failed\";\n  state_.SetStatus(std::move(status));\n}\n\nvoid Object::OnFail() {}\n\nabsl::Status Object::AnnotateStatusImpl(absl::Status status) { return status; }\n\nbool Object::FailWithoutAnnotation(absl::Status status) {\n  RIEGELI_ASSERT(!status.ok())\n      << \"Failed precondition of Object::FailWithoutAnnotation(): \"\n         \"status not failed\";\n  if (ABSL_PREDICT_FALSE(!not_failed())) return false;\n  state_.Fail(std::move(status));\n  OnFail();\n  return false;\n}\n\nabsl::Status Object::StatusOrAnnotate(absl::Status other_status) {\n  if (ABSL_PREDICT_FALSE(!ok())) return status();\n  return AnnotateStatus(std::move(other_status));\n}\n\nTypeId Object::GetTypeId() const { return TypeId(); }\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/object.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_OBJECT_H_\n#define RIEGELI_BASE_OBJECT_H_\n\n#include <stdint.h>\n\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/type_id.h\"\n\nnamespace riegeli {\n\n// By convention, a constructor with a single parameter of type `Closed`\n// constructs the object as closed.\nstruct Closed {};\ninline constexpr Closed kClosed = {};\n\n// Internal representation of the basic state of class `Object` and similar\n// classes: whether the object is open or closed, and whether it is not failed\n// or failed (with an associated `absl::Status` for failure details).\nclass ABSL_ATTRIBUTE_TRIVIAL_ABI ObjectState {\n public:\n  // Creates a closed `ObjectState`.\n  explicit ObjectState(Closed) noexcept : status_ptr_(kClosedSuccessfully) {}\n\n  // Creates an open `ObjectState`.\n  ObjectState() = default;\n\n  ObjectState(const ObjectState& that) = delete;\n  ObjectState& operator=(const ObjectState& that) = delete;\n\n  ObjectState(ObjectState&& that) noexcept;\n  ObjectState& operator=(ObjectState&& that) noexcept;\n\n  ~ObjectState();\n\n  // Makes `*this` equivalent to a newly constructed `ObjectState`. This avoids\n  // constructing a temporary `ObjectState` and moving from it.\n  void Reset(Closed);\n  void Reset();\n\n  // Returns `true` if the `ObjectState` is OK, i.e. open and not failed.\n  bool ok() const;\n\n  // Returns `true` if the `ObjectState` is open, i.e. not closed.\n  bool is_open() const;\n\n  // Returns `true` if the `ObjectState` is not failed.\n  bool not_failed() const;\n\n  // Returns an `absl::Status` describing the failure if the `ObjectState` is\n  // failed, or an `absl::FailedPreconditionError()` if the `ObjectState` is\n  // successfully closed, or `absl::OkStatus()` if the `ObjectState` is OK.\n  absl::Status status() const;\n\n  // Marks the `ObjectState` as closed, keeping its `not_failed()` state\n  // unchanged.\n  //\n  // Returns `not_failed()`.\n  bool MarkClosed();\n\n  // Marks the `ObjectState` as failed with the given `status`, keeping its\n  // `is_open()` state unchanged.\n  //\n  // Always returns `false`.\n  //\n  // Precondition: `!status.ok()`\n  ABSL_ATTRIBUTE_COLD bool Fail(absl::Status status);\n\n  // Marks the `ObjectState` as not failed, keeping its `is_open()` state\n  // unchanged.\n  void MarkNotFailed();\n\n  // Replaces the status of an already failed `ObjectState`.\n  //\n  // Preconditions:\n  //   `!status.ok()`\n  //   `!not_failed()`\n  void SetStatus(absl::Status status);\n\n private:\n  struct FailedStatus {\n    // The `closed` flag may be changed from `false` to `true` by `Close()`.\n    bool closed;\n    // The actual `absl::Status`, never `absl::OkStatus()`.\n    absl::Status status;\n  };\n\n  static constexpr uintptr_t kOk = 0;\n  static constexpr uintptr_t kClosedSuccessfully = 1;\n\n  static void DeleteStatus(uintptr_t status_ptr);\n\n  // `status_ptr_` is `kOk`, or `kClosedSuccessfully`, or a `FailedStatus*`\n  // `reinterpret_cast` to `uintptr_t`.\n  uintptr_t status_ptr_ = kOk;\n};\n\n// `Object` is an abstract base class for data readers and writers, managing\n// their state: whether they are open or closed, and whether they are not failed\n// or failed (with an associated `absl::Status` for failure details). An\n// `Object` is OK when it is open and not failed.\n//\n// An `Object` becomes closed when `Close()` finishes, when constructed as\n// closed (usually with no parameters), or when moved from.\n//\n// An `Object` fails when it could not perform an operation due to an unexpected\n// reason.\n//\n// Derived `Object` classes can be movable but not copyable. After a move the\n// source `Object` is left closed.\nclass Object {\n public:\n  Object(const Object&) = delete;\n  Object& operator=(const Object&) = delete;\n\n  virtual ~Object() {}\n\n  // Indicates that the `Object` is no longer needed, but in the case of a\n  // writer that its destination is needed. `Close()` may also report new\n  // failures.\n  //\n  // If `!is_open()`, does nothing. Otherwise:\n  //  * In the case of a writer, pushes buffered data to the destination.\n  //  * In the case of a reader, verifies that the source is not truncated at\n  //    the current position, i.e. that it either has more data or ends cleanly\n  //    (for sources where truncation can be distinguished from a clean end).\n  //  * Synchronizes the current position to the source or destination (if\n  //    applicable).\n  //  * Closes owned dependencies.\n  //\n  // Returns `true` if the `Object` did not fail, i.e. if it was OK just before\n  // becoming closed.\n  //\n  // It is necessary to call `Close()` at the end of a successful writing\n  // session, and it is recommended to call `Close()` at the end of a successful\n  // reading session. It is not needed to call `Close()` on early returns,\n  // assuming that contents of the destination do not matter after all, e.g.\n  // because a failure is being reported instead; the destructor releases\n  // resources in any case.\n  bool Close();\n\n  // Returns `true` if the `Object` is OK, i.e. open and not failed.\n  bool ok() const { return state_.ok(); }\n\n  // Returns `true` if the `Object` is open, i.e. not closed.\n  bool is_open() const { return state_.is_open(); }\n\n  // Returns `true` if the `Object` is not failed.\n  bool not_failed() const { return state_.not_failed(); }\n\n  // Returns an `absl::Status` describing the failure if the `Object` is failed,\n  // or an `absl::FailedPreconditionError()` if the `Object` is successfully\n  // closed, or `absl::OkStatus()` if the `Object` is OK.\n  absl::Status status() const { return state_.status(); }\n\n  // Marks the `Object` as failed with the given `status`, keeping its\n  // `is_open()` state unchanged.\n  //\n  // In derived classes `Fail()` may have additional effects. In particular the\n  // status can be annotated with some details using `AnnotateStatus()`, and\n  // `OnFail()` may update other state.\n  //\n  // If `Fail()` is called multiple times, the first `absl::Status` wins.\n  //\n  // `Fail()` always returns `false`, for convenience of reporting the failure\n  // as a `false` result of a failing function.\n  //\n  // `Fail()` is normally called by other methods of the same `Object`, but it\n  // is public to allow injecting a failure related to the `Object` (such as\n  // unexpected data returned by it) if that failure does not have to be\n  // distinguished from failures of the `Object` itself.\n  //\n  // Precondition: `!status.ok()`\n  ABSL_ATTRIBUTE_COLD bool Fail(absl::Status status);\n\n  // Can annotate `status` with some context, appropriately for the derived\n  // class.\n  //\n  // This is called by `Fail()`, and can also be called externally to annotate a\n  // failure related to this `Object`.\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatus(absl::Status status);\n\n  // Returns `status()` if `!ok()`, otherwise returns\n  // `AnnotateStatus(std::move(other_status))`.\n  //\n  // This is typically used after a failed call which does not necessarily\n  // make the object not OK, e.g. after reading functions, to return an\n  // explanation if the source ends prematurely or has unexpected contents.\n  ABSL_ATTRIBUTE_COLD absl::Status StatusOrAnnotate(absl::Status other_status);\n\n  // Returns a token which allows to detect the class of the `Object` at\n  // runtime.\n  //\n  // By default returns `TypeId()`. In order for a class to participate in class\n  // detection at runtime, it must override `GetTypeId()`:\n  // ```\n  //   riegeli::TypeId A::GetTypeId() const override {\n  //     return riegeli::TypeId::For<A>();\n  //   }\n  // ```\n  //\n  // Then, to actually cast:\n  // ```\n  //   if (A* const a = object->GetIf<A>()) {\n  //     ...\n  //   }\n  // ```\n  //\n  // This solution is more limited but faster than `typeid` or `dynamic_cast`.\n  virtual TypeId GetTypeId() const;\n\n  // Casts the runtime type of `this`, as determined by `GetTypeId()`, down to\n  // `Target*`. Returns `nullptr` if the type does not match.\n  template <typename Target>\n  Target* GetIf();\n  template <typename Target>\n  const Target* GetIf() const;\n\n protected:\n  // Creates a closed `Object`.\n  explicit Object(Closed) noexcept : state_(kClosed) {}\n\n  // Creates an open `Object`.\n  Object() = default;\n\n  // Moves the part of the object defined in the `Object` class.\n  Object(Object&& that) = default;\n  Object& operator=(Object&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `Object`. This avoids\n  // constructing a temporary `Object` and moving from it. Derived classes which\n  // redefine `Reset()` should include a call to `Object::Reset()`.\n  void Reset(Closed) { state_.Reset(kClosed); }\n  void Reset() { state_.Reset(); }\n\n  // Marks the `Object` as not failed, keeping its `is_open()` state unchanged.\n  // This can be used if the `Object` supports recovery after some failures.\n  void MarkNotFailed() { state_.MarkNotFailed(); }\n\n  // Replaces the status of an already failed `Object`.\n  //\n  // Preconditions:\n  //   `!status.ok()`\n  //   `!not_failed()`\n  void SetStatus(absl::Status status);\n\n  // Implementation of `Close()`, called if the `Object` is not closed yet.\n  //\n  // `Close()` returns early if `!is_open()`, otherwise calls `Done()` and marks\n  // the `Object` as closed. See `Close()` for details of the responsibility of\n  // `Done()`.\n  //\n  // The default implementation in `Object::Done()` does nothing.\n  //\n  // Precondition: `is_open()`\n  virtual void Done();\n\n  // Called by `Fail()` and `FailWithoutAnnotation()`. Can update some other\n  // state, appropriately for the derived class.\n  //\n  // The default implementation in `Object::OnFail()` does nothing.\n  ABSL_ATTRIBUTE_COLD virtual void OnFail();\n\n  // Implementation of `AnnotateStatus()`.\n  //\n  // The default implementation in `Object::AnnotateStatusImpl()` returns\n  // `status` unchanged.\n  ABSL_ATTRIBUTE_COLD virtual absl::Status AnnotateStatusImpl(\n      absl::Status status);\n\n  // Exposes a variant of `Fail()` which does not call `AnnotateStatus()`.\n  //\n  // This can be called instead of `Fail()` if the annotation supplied by\n  // `AnnotateStatus()` would be irrelevant or duplicated in a particular case.\n  ABSL_ATTRIBUTE_COLD bool FailWithoutAnnotation(absl::Status status);\n\n  // Supports `Dependency`.\n  friend MakerType<Closed> RiegeliDependencySentinel(Object*) {\n    return {kClosed};\n  }\n\n private:\n  ObjectState state_;\n};\n\n// Implementation details follow.\n\ninline ObjectState::ObjectState(ObjectState&& that) noexcept\n    : status_ptr_(std::exchange(that.status_ptr_, kClosedSuccessfully)) {}\n\ninline bool ObjectState::ok() const { return status_ptr_ == kOk; }\n\ninline bool ObjectState::is_open() const {\n  if (ABSL_PREDICT_TRUE(status_ptr_ == kOk)) return true;\n  if (ABSL_PREDICT_TRUE(status_ptr_ == kClosedSuccessfully)) return false;\n  return !reinterpret_cast<const FailedStatus*>(status_ptr_)->closed;\n}\n\ninline bool ObjectState::not_failed() const {\n  return status_ptr_ == kOk || status_ptr_ == kClosedSuccessfully;\n}\n\ninline absl::Status Object::AnnotateStatus(absl::Status status) {\n  return AnnotateStatusImpl(std::move(status));\n}\n\ntemplate <typename Target>\ninline Target* Object::GetIf() {\n  static_assert(std::is_base_of_v<Object, Target>,\n                \"GetIf() supports only downcasts\");\n  if (GetTypeId() != TypeId::For<Target>()) return nullptr;\n  return static_cast<Target*>(this);\n}\n\ntemplate <typename Target>\ninline const Target* Object::GetIf() const {\n  static_assert(std::is_base_of_v<Object, Target>,\n                \"GetIf() supports only downcasts\");\n  if (GetTypeId() != TypeId::For<Target>()) return nullptr;\n  return static_cast<const Target*>(this);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_OBJECT_H_\n"
  },
  {
    "path": "riegeli/base/optional_compact_string.h",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_OPTIONAL_COMPACT_STRING_H_\n#define RIEGELI_BASE_OPTIONAL_COMPACT_STRING_H_\n\n#include <stdint.h>\n\n#include <cstddef>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/compact_string.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/iterable.h\"\n#include \"riegeli/base/type_traits.h\"\n\nnamespace riegeli {\n\n// Similar to `std::optional<CompactString>`, but takes up the same amount of\n// space as `CompactString`.\n//\n// `OptionalCompactString` is either null or stores data equivalent to a\n// `CompactString`. It allows examining the contents as `absl::string_view` or\n// `const char*`, but not as `CompactString`, except by copying or moving from.\nclass ABSL_ATTRIBUTE_TRIVIAL_ABI OptionalCompactString\n    : public WithCompare<OptionalCompactString> {\n public:\n  // Creates a null `OptionalCompactString`.\n  OptionalCompactString() = default;\n  /*implicit*/ OptionalCompactString(std::nullptr_t) {}\n  OptionalCompactString& operator=(std::nullptr_t) {\n    if (repr_ != kNullRepr) {\n      CompactString::MoveFromRaw(repr_);\n      repr_ = kNullRepr;\n    }\n    return *this;\n  }\n\n  // Creates an `OptionalCompactString` which holds a copy of `src`.\n  explicit OptionalCompactString(BytesRef src)\n      : repr_(CompactString(src).RawMove()) {}\n  OptionalCompactString& operator=(BytesRef src) {\n    const uintptr_t old_repr =\n        std::exchange(repr_, CompactString(src).RawMove());\n    if (old_repr != kNullRepr) CompactString::MoveFromRaw(old_repr);\n    return *this;\n  }\n\n  // Creates an `OptionalCompactString` which holds a copy of `src`.\n  explicit OptionalCompactString(const CompactString& src)\n      : repr_(CompactString(src).RawMove()) {}\n  OptionalCompactString& operator=(const CompactString& src) {\n    const uintptr_t old_repr =\n        std::exchange(repr_, CompactString(src).RawMove());\n    if (old_repr != kNullRepr) CompactString::MoveFromRaw(old_repr);\n    return *this;\n  }\n\n  // Creates an `OptionalCompactString` which holds `src`. The source\n  // `CompactString` is left empty.\n  explicit OptionalCompactString(CompactString&& src)\n      : repr_(std::move(src).RawMove()) {}\n  OptionalCompactString& operator=(CompactString&& src) {\n    const uintptr_t old_repr = std::exchange(repr_, std::move(src).RawMove());\n    if (old_repr != kNullRepr) CompactString::MoveFromRaw(old_repr);\n    return *this;\n  }\n\n  OptionalCompactString(const OptionalCompactString& that) noexcept\n      : repr_(that.repr_ == kNullRepr ? kNullRepr\n                                      : CompactString::CopyRaw(that.repr_)) {}\n  OptionalCompactString& operator=(const OptionalCompactString& that) noexcept {\n    const uintptr_t old_repr = std::exchange(\n        repr_, that.repr_ == kNullRepr ? kNullRepr\n                                       : CompactString::CopyRaw(that.repr_));\n    if (old_repr != kNullRepr) CompactString::MoveFromRaw(old_repr);\n    return *this;\n  }\n\n  // The source `OptionalCompactString` is left null.\n  OptionalCompactString(OptionalCompactString&& that) noexcept\n      : repr_(std::exchange(that.repr_, kNullRepr)) {}\n  OptionalCompactString& operator=(OptionalCompactString&& that) noexcept {\n    const uintptr_t old_repr =\n        std::exchange(repr_, std::exchange(that.repr_, kNullRepr));\n    if (old_repr != kNullRepr) CompactString::MoveFromRaw(old_repr);\n    return *this;\n  }\n\n  ~OptionalCompactString() {\n    if (repr_ != kNullRepr) CompactString::MoveFromRaw(repr_);\n  }\n\n  // Extracts the value as a `CompactString`.\n  //\n  // Precondition: `*this != nullptr`.\n  CompactString Release() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT(*this != nullptr)\n        << \"Failed precondition of OptionalCompactString::Release(): \"\n           \"OptionalCompactString is nullptr\";\n    return CompactString::MoveFromRaw(std::exchange(repr_, kNullRepr));\n  }\n\n  // Views the value as an `absl::string_view`.\n  //\n  // Precondition: `*this != nullptr`.\n  absl::string_view operator*() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT(*this != nullptr)\n        << \"Failed precondition of OptionalCompactString::operator*: \"\n           \"OptionalCompactString is nullptr\";\n    return CompactString::ViewFromRaw(&repr_);\n  }\n\n  ArrowProxy<absl::string_view> operator->() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT(*this != nullptr)\n        << \"Failed precondition of OptionalCompactString::operator->: \"\n           \"OptionalCompactString is nullptr\";\n    return ArrowProxy<absl::string_view>(**this);\n  }\n\n  // Ensures that the value is NUL-terminated after its size and returns\n  // a pointer to it. Returns `nullptr` for a null `OptionalCompactString`.\n  //\n  // In contrast to `std::string::c_str()`, this is a non-const operation.\n  // It may reallocate the string and it writes the NUL each time.\n  const char* c_str() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    if (repr_ == kNullRepr) return nullptr;\n    return CompactString::CStrFromRaw(&repr_);\n  }\n\n  friend bool operator==(const OptionalCompactString& a,\n                         const OptionalCompactString& b) {\n    if (a.repr_ == b.repr_) return true;\n    if (a.repr_ == kNullRepr || b.repr_ == kNullRepr) return false;\n    return *a == *b;\n  }\n  friend StrongOrdering RIEGELI_COMPARE(const OptionalCompactString& a,\n                                        const OptionalCompactString& b) {\n    if (a.repr_ == b.repr_) return StrongOrdering::equal;\n    if (a.repr_ == kNullRepr) return StrongOrdering::less;\n    if (b.repr_ == kNullRepr) return StrongOrdering::greater;\n    return riegeli::Compare(*a, *b);\n  }\n\n  friend bool operator==(const OptionalCompactString& a, std::nullptr_t) {\n    return a.repr_ == kNullRepr;\n  }\n  friend StrongOrdering RIEGELI_COMPARE(const OptionalCompactString& a,\n                                        std::nullptr_t) {\n    if (a.repr_ == kNullRepr) return StrongOrdering::equal;\n    return StrongOrdering::greater;\n  }\n\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<NotSameRef<OptionalCompactString, T>,\n                                          NotSameRef<std::nullptr_t, T>,\n                                          std::is_convertible<T&&, BytesRef>>,\n                       int> = 0>\n  friend bool operator==(const OptionalCompactString& a, T&& b) {\n    if (a.repr_ == kNullRepr) return false;\n    return *a == absl::string_view(b);\n  }\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<NotSameRef<OptionalCompactString, T>,\n                                          NotSameRef<std::nullptr_t, T>,\n                                          std::is_convertible<T&&, BytesRef>>,\n                       int> = 0>\n  friend StrongOrdering RIEGELI_COMPARE(const OptionalCompactString& a, T&& b) {\n    if (a.repr_ == kNullRepr) return StrongOrdering::less;\n    return riegeli::Compare(*a, absl::string_view(b));\n  }\n\n private:\n  static constexpr uintptr_t kNullRepr = 0;\n\n  uintptr_t repr_ = kNullRepr;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_OPTIONAL_COMPACT_STRING_H_\n"
  },
  {
    "path": "riegeli/base/options_parser.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/options_parser.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <algorithm>\n#include <cmath>\n#include <limits>\n#include <string>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/numbers.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/string_ref.h\"\n\nnamespace riegeli {\n\nValueParser::ValueParser(OptionsParser* options_parser, absl::string_view key,\n                         absl::string_view value)\n    : options_parser_(RIEGELI_EVAL_ASSERT_NOTNULL(options_parser)),\n      key_(key),\n      value_(value) {}\n\nValueParser::Function ValueParser::Int(int min_value, int max_value, int* out) {\n  RIEGELI_ASSERT_LE(min_value, max_value)\n      << \"Failed precondition of OptionsParser::IntOption(): \"\n         \"bounds in the wrong order\";\n  return [min_value, max_value, out](ValueParser& value_parser) {\n    int int_value;\n    if (ABSL_PREDICT_TRUE(absl::SimpleAtoi(value_parser.value(), &int_value) &&\n                          int_value >= min_value && int_value <= max_value)) {\n      *out = int_value;\n      return true;\n    }\n    return value_parser.InvalidValue(absl::StrCat(\n        \"integers in the range [\", min_value, \"..\", max_value, \"]\"));\n  };\n}\n\nValueParser::Function ValueParser::Bytes(uint64_t min_value, uint64_t max_value,\n                                         uint64_t* out) {\n  RIEGELI_ASSERT_LE(min_value, max_value)\n      << \"Failed precondition of BytesOption(): bounds in the wrong order\";\n  return [min_value, max_value, out](ValueParser& value_parser) {\n    absl::string_view value = value_parser.value();\n    double scale = 1.0;\n    if (ABSL_PREDICT_TRUE(!value.empty())) {\n      switch (value.back()) {\n        case 'B':\n          break;\n        case 'k':\n        case 'K':\n          scale = static_cast<double>(uint64_t{1} << 10);\n          break;\n        case 'M':\n          scale = static_cast<double>(uint64_t{1} << 20);\n          break;\n        case 'G':\n          scale = static_cast<double>(uint64_t{1} << 30);\n          break;\n        case 'T':\n          scale = static_cast<double>(uint64_t{1} << 40);\n          break;\n        case 'P':\n          scale = static_cast<double>(uint64_t{1} << 50);\n          break;\n        case 'E':\n          scale = static_cast<double>(uint64_t{1} << 60);\n          break;\n        default:\n          goto no_scale;\n      }\n      value.remove_suffix(1);\n    }\n  no_scale:\n    double double_value;\n    if (ABSL_PREDICT_TRUE(absl::SimpleAtod(value, &double_value) &&\n                          double_value >= 0.0)) {\n      double_value = std::round(double_value * scale);\n      const uint64_t uint64_value =\n          ABSL_PREDICT_FALSE(\n              double_value >=\n              static_cast<double>(std::numeric_limits<uint64_t>::max()))\n              ? std::numeric_limits<uint64_t>::max()\n              : static_cast<uint64_t>(double_value);\n      if (ABSL_PREDICT_TRUE(uint64_value >= min_value &&\n                            uint64_value <= max_value)) {\n        *out = uint64_value;\n        return true;\n      }\n    }\n    return value_parser.InvalidValue(\n        absl::StrCat(\"integers expressed as reals with \"\n                     \"optional suffix [BkKMGTPE], in the range [\",\n                     min_value, \"..\", max_value, \"]\"));\n  };\n}\n\nValueParser::Function ValueParser::Real(double min_value, double max_value,\n                                        double* out) {\n  RIEGELI_ASSERT_LE(min_value, max_value)\n      << \"Failed precondition of IntOption(): bounds in the wrong order\";\n  return [min_value, max_value, out](ValueParser& value_parser) {\n    double double_value;\n    if (ABSL_PREDICT_TRUE(\n            absl::SimpleAtod(value_parser.value(), &double_value) &&\n            double_value >= min_value && double_value <= max_value)) {\n      *out = double_value;\n      return true;\n    }\n    return value_parser.InvalidValue(\n        absl::StrCat(\"reals in the range [\", min_value, \"..\", max_value, \"]\"));\n  };\n}\n\nValueParser::Function ValueParser::Or(Initializer<Function> function1,\n                                      Initializer<Function> function2) {\n  return [function1 = std::move(function1).Construct(),\n          function2 =\n              std::move(function2).Construct()](ValueParser& value_parser) {\n    return function1(value_parser) || function2(value_parser);\n  };\n}\n\nValueParser::Function ValueParser::And(Initializer<Function> function1,\n                                       Initializer<Function> function2) {\n  return [function1 = std::move(function1).Construct(),\n          function2 =\n              std::move(function2).Construct()](ValueParser& value_parser) {\n    return function1(value_parser) && function2(value_parser);\n  };\n}\n\nValueParser::Function ValueParser::CopyTo(std::string* text) {\n  return [text](ValueParser& value_parser) {\n    absl::StrAppend(text, text->empty() ? \"\" : \",\", value_parser.key(),\n                    value_parser.value().empty() ? \"\" : \":\",\n                    value_parser.value());\n    return true;\n  };\n}\n\nValueParser::Function ValueParser::FailIfSeen(StringInitializer key) {\n  return [key = std::move(key).Construct()](ValueParser& value_parser) {\n    for (const OptionsParser::Option& option :\n         value_parser.options_parser_->options_) {\n      if (option.key == key) {\n        if (ABSL_PREDICT_FALSE(option.seen)) {\n          return value_parser.Fail(absl::InvalidArgumentError(absl::StrCat(\n              \"Option \", value_parser.key(), \" conflicts with option \", key)));\n        }\n        return true;\n      }\n    }\n    RIEGELI_ASSUME_UNREACHABLE() << \"Unknown option \" << key;\n  };\n}\n\nValueParser::Function ValueParser::FailIfAnySeen() {\n  return [](ValueParser& value_parser) {\n    for (const OptionsParser::Option& option :\n         value_parser.options_parser_->options_) {\n      if (ABSL_PREDICT_FALSE(option.seen)) {\n        return value_parser.Fail(absl::InvalidArgumentError(\n            absl::StrCat(\"Option \", value_parser.key(), \" must be first\")));\n      }\n    }\n    return true;\n  };\n}\n\nbool ValueParser::InvalidValue(absl::string_view valid_values) {\n  RIEGELI_ASSERT(!valid_values.empty())\n      << \"Failed precondition of OptionsParser::InvalidValue(): \"\n         \"empty valid values\";\n  absl::StrAppend(&valid_values_, valid_values_.empty() ? \"\" : \", \",\n                  valid_values);\n  return false;\n}\n\nbool OptionsParser::FromString(absl::string_view text) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  size_t option_begin = 0;\n  for (;;) {\n    size_t option_end = text.find(',', option_begin);\n    if (option_end == absl::string_view::npos) option_end = text.size();\n    if (option_begin != option_end) {\n      const absl::string_view key_value =\n          text.substr(option_begin, option_end - option_begin);\n      absl::string_view key;\n      absl::string_view value;\n      const size_t colon = key_value.find(':');\n      if (colon == absl::string_view::npos) {\n        key = key_value;\n      } else {\n        key = key_value.substr(0, colon);\n        value = key_value.substr(colon + 1);\n      }\n      const std::vector<Option>::iterator option = std::find_if(\n          options_.begin(), options_.end(),\n          [key](const Option& option) { return option.key == key; });\n      if (ABSL_PREDICT_FALSE(option == options_.end())) {\n        std::string message =\n            absl::StrCat(\"Unknown option \", key, \", valid options: \");\n        std::vector<Option>::const_iterator iter = options_.cbegin();\n        if (iter != options_.cend()) {\n          absl::StrAppend(&message, iter->key);\n          for (++iter; iter != options_.cend(); ++iter) {\n            absl::StrAppend(&message, \", \", iter->key);\n          }\n        }\n        return Fail(absl::InvalidArgumentError(message));\n      }\n      if (ABSL_PREDICT_FALSE(option->seen)) {\n        return Fail(absl::InvalidArgumentError(\n            absl::StrCat(\"Option \", key, \" is present more than once\")));\n      }\n      ValueParser value_parser(this, key, value);\n      if (ABSL_PREDICT_FALSE(!option->function(value_parser))) {\n        if (!value_parser.ok()) return Fail(value_parser.status());\n        return Fail(absl::InvalidArgumentError(absl::StrCat(\n            \"Option \", key, \": \",\n            \"invalid value: \", value.empty() ? \"(empty)\" : value,\n            value_parser.valid_values_.empty() ? \"\" : \", valid values: \",\n            value_parser.valid_values_)));\n      }\n      RIEGELI_ASSERT_OK(value_parser)\n          << \"Value parser of option \" << key\n          << \" returned true but failed the ValueParser\";\n      option->seen = true;\n    }\n    if (option_end == text.size()) break;\n    option_begin = option_end + 1;\n  }\n  return true;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/options_parser.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_OPTIONS_PARSER_H_\n#define RIEGELI_BASE_OPTIONS_PARSER_H_\n\n#include <stdint.h>\n\n#include <algorithm>\n#include <string>\n#include <type_traits>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/functional/any_invocable.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/string_ref.h\"\n\nnamespace riegeli {\n\nclass OptionsParser;\n\nclass ValueParser : public Object {\n public:\n  // Parser of an option value.\n  //\n  // Return values:\n  //  * `true`  - success (`FailIfSeen()` nor `FailIfAnySeen()` must not have\n  //              been called)\n  //  * `false` - failure (`InvalidValue()`, `FailIfSeen()`, or\n  //              `FailIfAnySeen()` may have been called)\n  using Function = absl::AnyInvocable<bool(ValueParser&) const>;\n\n  ValueParser(const ValueParser&) = delete;\n  ValueParser& operator=(const ValueParser&) = delete;\n\n  // Value parser for absent or empty argument.\n  template <typename T>\n  static Function Empty(T value, T* out);\n\n  // Value parser for explicitly enumerated valid values.\n  //\n  // An empty possible value matches also the case when \":\" with value is\n  // absent.\n  template <typename T>\n  static Function Enum(std::vector<std::pair<std::string, T>> possible_values,\n                       T* out);\n\n  // Value parser for integers in the range [`min_value`..`max_value`].\n  static Function Int(int min_value, int max_value, int* out);\n\n  // Value parser for integers expressed as reals with optional suffix\n  // `[BkKMGTPE]`, in the range [`min_value`..`max_value`].\n  static Function Bytes(uint64_t min_value, uint64_t max_value, uint64_t* out);\n\n  // Value parser for reals in the range [`min_value`..`max_value`].\n  static Function Real(double min_value, double max_value, double* out);\n\n  // Value parser which tries multiple parsers and returns the result of the\n  // first one which succeeds.\n  //\n  // The parsers must not include `FailIfSeen()` nor `FailIfAnySeen()`.\n  // Conflicts with other options should be checked outside the `Or()`.\n  static Function Or(Initializer<Function> function1,\n                     Initializer<Function> function2);\n  template <typename... Functions,\n            std::enable_if_t<(sizeof...(Functions) > 0), int> = 0>\n  static Function Or(Initializer<Function> function1,\n                     Initializer<Function> function2, Functions&&... functions);\n\n  // Value parser which runs multiple parsers and expects all of them to\n  // succeed.\n  static Function And(Initializer<Function> function1,\n                      Initializer<Function> function2);\n  template <typename... Functions,\n            std::enable_if_t<(sizeof...(Functions) > 0), int> = 0>\n  static Function And(Initializer<Function> function1,\n                      Initializer<Function> function2,\n                      Functions&&... functions);\n\n  // Value parser which appends the option to a separate options string\n  // (as comma-separated key:value pairs), to be parsed with a separate\n  // `OptionsParser`.\n  static Function CopyTo(std::string* text);\n\n  // Value parser which reports a conflict if an option with any of the given\n  // keys was seen before this option.\n  //\n  // Multiple occurrences of the same option are always invalid and do not have\n  // to be explicitly checked with `FailIfSeen()`.\n  static Function FailIfSeen(StringInitializer key);\n  template <typename... Keys, std::enable_if_t<(sizeof...(Keys) > 0), int> = 0>\n  static Function FailIfSeen(StringInitializer key, Keys&&... keys);\n\n  // Value parser which reports a conflict if an any option was seen before this\n  // option.\n  static Function FailIfAnySeen();\n\n  // Returns the key of the option being parsed.\n  absl::string_view key() const { return key_; }\n\n  // Returns the value of the option being parsed.\n  absl::string_view value() const { return value_; }\n\n  // Reports that the value is invalid, given a human-readable description of\n  // values which would have been valid.\n  //\n  // Multiple descriptions from several `InvalidValue()` calls are joined with\n  // commas. This normally happens if all parsers from `Or()` fail.\n  //\n  // Precondition: `!valid_values.empty()`\n  //\n  // Always returns `false`.\n  bool InvalidValue(absl::string_view valid_values);\n\n private:\n  friend class OptionsParser;\n\n  explicit ValueParser(OptionsParser* options_parser, absl::string_view key,\n                       absl::string_view value);\n\n  OptionsParser* options_parser_;\n  absl::string_view key_;\n  absl::string_view value_;\n  // When `InvalidValue()` was called, a human-readable description of valid\n  // values, otherwise empty.\n  std::string valid_values_;\n};\n\nclass OptionsParser : public Object {\n public:\n  OptionsParser() noexcept {}\n\n  OptionsParser(const OptionsParser&) = delete;\n  OptionsParser& operator=(const OptionsParser&) = delete;\n\n  // Registers an option with the given key. Its value must be accepted by the\n  // value parser.\n  //\n  // The value parser may be implemented explicitly (e.g. as a lambda)\n  // or returned by one of functions below (called on this `OptionsParser`).\n  void AddOption(StringInitializer key,\n                 Initializer<ValueParser::Function> function);\n\n  // Parses options from text. Valid options must have been registered with\n  // `AddOptions()`.\n  // ```\n  //   options ::= option? (\",\" option?)*\n  //   option ::= key (\":\" value)?\n  //   key ::= (char except ',' and ':')*\n  //   value ::= (char except ',')*\n  // ```\n  //\n  // For each recognized option key, calls the corresponding value parser.\n  // If \":\" with value is absent, `absl::string_view()` is passed as the value.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool FromString(absl::string_view text);\n\n private:\n  friend class ValueParser;\n\n  struct Option {\n    explicit Option(StringInitializer key,\n                    Initializer<ValueParser::Function> function)\n        : key(std::move(key)), function(std::move(function)) {}\n\n    std::string key;\n    ValueParser::Function function;\n    bool seen = false;\n  };\n\n  std::vector<Option> options_;\n};\n\n// Implementation details follow.\n\ntemplate <typename T>\nValueParser::Function ValueParser::Empty(T value, T* out) {\n  return [value = std::move(value), out](ValueParser& value_parser) {\n    if (ABSL_PREDICT_TRUE(value_parser.value().empty())) {\n      *out = value;\n      return true;\n    }\n    return value_parser.InvalidValue(\"(empty)\");\n  };\n}\n\ntemplate <typename T>\nValueParser::Function ValueParser::Enum(\n    std::vector<std::pair<std::string, T>> possible_values, T* out) {\n  return [possible_values = std::move(possible_values),\n          out](ValueParser& value_parser) {\n    for (const std::pair<std::string, T>& possible_value : possible_values) {\n      if (value_parser.value() == possible_value.first) {\n        *out = possible_value.second;\n        return true;\n      }\n    }\n    for (const std::pair<std::string, T>& possible_value : possible_values) {\n      value_parser.InvalidValue(possible_value.first.empty()\n                                    ? absl::string_view(\"(empty)\")\n                                    : absl::string_view(possible_value.first));\n    }\n    return false;\n  };\n}\n\ntemplate <typename... Functions,\n          std::enable_if_t<(sizeof...(Functions) > 0), int>>\nValueParser::Function ValueParser::Or(Initializer<Function> function1,\n                                      Initializer<Function> function2,\n                                      Functions&&... functions) {\n  return Or(function1, Or(function2, std::forward<Functions>(functions)...));\n}\n\ntemplate <typename... Functions,\n          std::enable_if_t<(sizeof...(Functions) > 0), int>>\nValueParser::Function ValueParser::And(Initializer<Function> function1,\n                                       Initializer<Function> function2,\n                                       Functions&&... functions) {\n  return And(function1, And(function2, std::forward<Functions>(functions)...));\n}\n\ntemplate <typename... Keys, std::enable_if_t<(sizeof...(Keys) > 0), int>>\nValueParser::Function ValueParser::FailIfSeen(StringInitializer key,\n                                              Keys&&... keys) {\n  return And(FailIfSeen(std::move(key)),\n             FailIfSeen(std::forward<Keys>(keys)...));\n}\n\ninline void OptionsParser::AddOption(\n    StringInitializer key, Initializer<ValueParser::Function> function) {\n  options_.emplace_back(std::move(key), std::move(function));\n  RIEGELI_ASSERT(std::none_of(\n      options_.cbegin(), options_.cend() - 1,\n      [&](const Option& option) { return option.key == options_.back().key; }))\n      << \"Failed precondition of OptionsParser::AddOption(): option \"\n      << options_.back().key << \"already registered\";\n}\n\n};  // namespace riegeli\n\n#endif  // RIEGELI_BASE_OPTIONS_PARSER_H_\n"
  },
  {
    "path": "riegeli/base/ownership.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_OWNERSHIP_H_\n#define RIEGELI_BASE_OWNERSHIP_H_\n\n#include <type_traits>\n\nnamespace riegeli {\n\n// `PassOwnership` and `ShareOwnership` type tags specify how ownership of a\n// potentially shared object is transferred, for cases when this is not implied\n// by parameter types.\n//\n//  * `PassOwnership`: the original owner drops its reference. The reference\n//    count is decreased unless the new owner gets a reference instead.\n//\n//  * `ShareOwnership`: The original owner keeps its reference. The reference\n//    count is increased if the new owner also gets a reference.\n\nstruct PassOwnership {};\ninline constexpr PassOwnership kPassOwnership = {};\n\nstruct ShareOwnership {};\ninline constexpr ShareOwnership kShareOwnership = {};\n\n// `IsOwnership<T>::value` is `true` if `T` is `PassOwnership` or\n// `ShareOwnership`.\n\ntemplate <typename T>\nstruct IsOwnership : std::false_type {};\n\ntemplate <>\nstruct IsOwnership<PassOwnership> : std::true_type {};\n\ntemplate <>\nstruct IsOwnership<ShareOwnership> : std::true_type {};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_OWNERSHIP_H_\n"
  },
  {
    "path": "riegeli/base/parallelism.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/parallelism.h\"\n\n#include <stddef.h>\n\n#include <deque>\n#include <thread>\n#include <utility>\n\n#include \"absl/base/thread_annotations.h\"\n#include \"absl/functional/any_invocable.h\"\n#include \"absl/synchronization/mutex.h\"\n#include \"absl/time/time.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/global.h\"\n\nnamespace riegeli::internal {\n\nThreadPool::~ThreadPool() {\n  absl::MutexLock lock(mutex_);\n  exiting_ = true;\n  mutex_.Await(absl::Condition(\n      +[](size_t* num_threads) { return *num_threads == 0; }, &num_threads_));\n}\n\nvoid ThreadPool::Schedule(absl::AnyInvocable<void() &&> task) {\n  {\n    absl::MutexLock lock(mutex_);\n    RIEGELI_ASSERT(!exiting_)\n        << \"Failed precondition of ThreadPool::Schedule(): no new threads may \"\n           \"be scheduled while the thread pool is exiting\";\n    tasks_.push_back(std::move(task));\n    if (num_idle_threads_ >= tasks_.size()) return;\n    ++num_threads_;\n  }\n  std::thread([this] {\n    for (;;) {\n      absl::ReleasableMutexLock lock(mutex_);\n      ++num_idle_threads_;\n      mutex_.AwaitWithTimeout(\n          absl::Condition(\n              +[](ThreadPool* self)\n                   ABSL_EXCLUSIVE_LOCKS_REQUIRED(self->mutex_) {\n                     return !self->tasks_.empty() || self->exiting_;\n                   },\n              this),\n          absl::Seconds(1));\n      --num_idle_threads_;\n      if (tasks_.empty() || exiting_) {\n        --num_threads_;\n        return;\n      }\n      absl::AnyInvocable<void() &&> task = std::move(tasks_.front());\n      tasks_.pop_front();\n      lock.Release();\n      std::move(task)();\n    }\n  }).detach();\n}\n\nThreadPool& ThreadPool::global() {\n  return Global([] { return ThreadPool(); });\n}\n\n}  // namespace riegeli::internal\n"
  },
  {
    "path": "riegeli/base/parallelism.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_PARALLELISM_H_\n#define RIEGELI_BASE_PARALLELISM_H_\n\n#include <stddef.h>\n\n#include <deque>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/thread_annotations.h\"\n#include \"absl/functional/any_invocable.h\"\n#include \"absl/synchronization/mutex.h\"\n\nnamespace riegeli::internal {\n\n// A thread pool with lazily created worker threads, without a thread count\n// limit. Worker threads exit after being idle for one minute.\nclass ThreadPool {\n public:\n  ThreadPool() {}\n\n  ThreadPool(const ThreadPool&) = delete;\n  ThreadPool& operator=(const ThreadPool&) = delete;\n\n  ~ThreadPool();\n\n  static ThreadPool& global();\n\n  void Schedule(absl::AnyInvocable<void() &&> task);\n\n private:\n  absl::Mutex mutex_;\n  bool exiting_ ABSL_GUARDED_BY(mutex_) = false;\n  size_t num_threads_ ABSL_GUARDED_BY(mutex_) = 0;\n  size_t num_idle_threads_ ABSL_GUARDED_BY(mutex_) = 0;\n  std::deque<absl::AnyInvocable<void() &&>> tasks_ ABSL_GUARDED_BY(mutex_);\n};\n\n}  // namespace riegeli::internal\n\n#endif  // RIEGELI_BASE_PARALLELISM_H_\n"
  },
  {
    "path": "riegeli/base/port.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_PORT_H_\n#define RIEGELI_BASE_PORT_H_\n\n#include \"absl/base/nullability.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\n// Clang has `__has_builtin()`. Other compilers need other means to detect\n// availability of builtins.\n#ifdef __has_builtin\n#define RIEGELI_INTERNAL_HAS_BUILTIN(x) __has_builtin(x)\n#else\n#define RIEGELI_INTERNAL_HAS_BUILTIN(x) 0\n#endif\n\n#define RIEGELI_INTERNAL_IS_GCC_VERSION(major, minor) \\\n  (__GNUC__ > (major) || (__GNUC__ == (major) && __GNUC_MINOR__ >= (minor)))\n\n#endif  // RIEGELI_BASE_PORT_H_\n"
  },
  {
    "path": "riegeli/base/recycling_pool.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_RECYCLING_POOL_H_\n#define RIEGELI_BASE_RECYCLING_POOL_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <atomic>\n#include <limits>\n#include <list>\n#include <memory>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/base/thread_annotations.h\"\n#include \"absl/container/flat_hash_map.h\"\n#include \"absl/container/inlined_vector.h\"\n#include \"absl/container/node_hash_map.h\"\n#include \"absl/synchronization/mutex.h\"\n#include \"absl/time/time.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/background_cleaning.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/global.h\"\n\nnamespace riegeli {\n\n// Options for `RecyclingPool` and `KeyedRecyclingPool`.\nclass RecyclingPoolOptions {\n public:\n  RecyclingPoolOptions() = default;\n\n  // Maximum number of objects to keep in a pool.\n  //\n  // 0 effectively disables the pool: objects are destroyed immediately.\n  //\n  // Default: `kDefaultMaxSize` (16).\n  static constexpr size_t kDefaultMaxSize = 16;\n  RecyclingPoolOptions& set_max_size(size_t max_size) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    max_size_ = SaturatingIntCast<uint32_t>(max_size);\n    return *this;\n  }\n  RecyclingPoolOptions&& set_max_size(size_t max_size) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_max_size(max_size));\n  }\n  size_t max_size() const { return max_size_; }\n\n  // Maximum time for keeping an object in a pool. Objects idling for more than\n  // this will be evicted in a background thread.\n  //\n  // `absl::InfiniteDuration()` disables time-based eviction.\n  //\n  // `absl::ZeroDuration()` lets objects be destroyed right after they are no\n  // longer needed, but asynchronously, i.e. without blocking the calling\n  // thread.\n  //\n  // Default: `kDefaultMaxAge` (`absl::Minutes(1)`).\n  static constexpr absl::Duration kDefaultMaxAge = absl::Minutes(1);\n  RecyclingPoolOptions& set_max_age(absl::Duration max_age) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    max_age_seconds_ = AgeToSeconds(max_age);\n    return *this;\n  }\n  RecyclingPoolOptions&& set_max_age(absl::Duration max_age) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_max_age(max_age));\n  }\n  absl::Duration max_age() const {\n    return max_age_seconds_ == std::numeric_limits<uint32_t>::max()\n               ? absl::InfiniteDuration()\n               : absl::Seconds(max_age_seconds_);\n  }\n\n private:\n  // For `max_size_` and `max_age_seconds_`.\n  template <typename T, typename Deleter>\n  friend class RecyclingPool;\n  // For `max_size_` and `max_age_seconds_`.\n  template <typename T, typename Key, typename Deleter>\n  friend class KeyedRecyclingPool;\n\n  static uint32_t AgeToSeconds(absl::Duration age);  // Round up.\n\n  // Use `uint32_t` instead of `size_t` to reduce the object size.\n  uint32_t max_size_ = IntCast<uint32_t>(kDefaultMaxSize);\n  // Use `uint32_t` instead of `absl::Duration` to reduce the object size.\n  // `std::numeric_limits<uint32_t>::max()` means `absl::InfiniteDuration()`.\n  uint32_t max_age_seconds_ = 60;  // `AgeToSeconds(kDefaultMaxAge)`\n};\n\n// `RecyclingPool<T, Deleter>` keeps a pool of idle objects of type `T`, so that\n// instead of creating a new object of type `T`, an existing object can be\n// recycled. This is helpful if constructing a new object is more expensive than\n// resetting an existing object to the desired state.\n//\n// Deleter specifies how an object should be eventually deleted, like in\n// `std::unique_ptr<T, Deleter>`.\n//\n// `RecyclingPool` is thread-safe.\ntemplate <typename T, typename Deleter = std::default_delete<T>>\nclass RecyclingPool : public BackgroundCleanee {\n public:\n  // A deleter which puts the object back into the pool.\n  class Recycler;\n\n  // A `std::unique_ptr` which puts the object back into the pool instead of\n  // deleting it. If a particular object is not suitable for recycling,\n  // `DoNotRecycle()` can be used.\n  using Handle = std::unique_ptr<T, Recycler>;\n\n  // A `std::unique_ptr` which deletes the object. If a particular object is\n  // suitable for recycling, it can be put back into the pool using `RawPut()`.\n  using RawHandle = std::unique_ptr<T, Deleter>;\n\n  // A refurbisher which does nothing; see `Get()`.\n  struct DefaultRefurbisher {\n    void operator()(ABSL_ATTRIBUTE_UNUSED T* ptr) const {}\n  };\n\n  explicit RecyclingPool(RecyclingPoolOptions options = RecyclingPoolOptions())\n      : max_age_seconds_(options.max_age_seconds_),\n        ring_buffer_by_age_(options.max_size()) {}\n\n  RecyclingPool(const RecyclingPool&) = delete;\n  RecyclingPool& operator=(const RecyclingPool&) = delete;\n\n  ~RecyclingPool();\n\n  // Returns a default global pool specific to template parameters of\n  // `RecyclingPool` and `options`.\n  static RecyclingPool& global(\n      RecyclingPoolOptions options = RecyclingPoolOptions());\n\n  // Uses a different `BackgroundCleaner` than `BackgroundCleaner::global()` for\n  // scheduling background cleaning. This is useful for testing.\n  //\n  // Precondition: `BackgroundCleaner` was not used yet.\n  void SetBackgroundCleaner(BackgroundCleaner* cleaner);\n\n  // Deletes all objects stored in the pool.\n  void Clear();\n\n  // Creates an object, or returns an existing object from the pool if possible.\n  //\n  // `factory` takes no arguments and returns `RawHandle`. It is called to\n  // create a new object.\n  //\n  // If `refurbisher` is specified, it takes a `T*` argument and its result is\n  // ignored. It is called before returning an existing object.\n  template <typename Factory, typename Refurbisher = DefaultRefurbisher>\n  Handle Get(Factory&& factory,\n             Refurbisher&& refurbisher = DefaultRefurbisher())\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Deletes the object immediately, does not return it into the pool.\n  //\n  // This can be called on the result of `Get()` if the object turned out to be\n  // not suitable for recycling.\n  //\n  // Equivalent to calling the original `Deleter` if `object != nullptr`.\n  static void DoNotRecycle(Handle object);\n\n  // Like `Get()`, but the object is not returned into the pool by the\n  // destructor of its handle. If the object is suitable for recycling, it can\n  // be put back into the pool using `RawPut()`.\n  template <typename Factory, typename Refurbisher = DefaultRefurbisher>\n  RawHandle RawGet(Factory&& factory,\n                   Refurbisher&& refurbisher = DefaultRefurbisher());\n\n  // Puts an idle object into the pool for recycling.\n  void RawPut(RawHandle object);\n\n protected:\n  void Clean(absl::Time now) override;\n\n private:\n  struct Entry {\n    RawHandle object;\n    absl::Time deadline;\n  };\n\n  uint32_t max_age_seconds_;\n  // If not `nullptr` then `this` has been registered at `*cleaner_`.\n  BackgroundCleaner* cleaner_ = nullptr;\n  BackgroundCleaner::Token cleaner_token_;\n  absl::Mutex mutex_;\n  // All objects, ordered by age (older to newer).\n  uint32_t ring_buffer_end_ ABSL_GUARDED_BY(mutex_) = 0;\n  uint32_t ring_buffer_size_ ABSL_GUARDED_BY(mutex_) = 0;\n  // `ABSL_GUARDED_BY(mutex_)` for elements but not for `size()`.\n  std::vector<Entry> ring_buffer_by_age_;\n};\n\n// `KeyedRecyclingPool<T, Key, Deleter>` keeps a pool of idle objects of type\n// `T`, so that instead of creating a new object of type `T`, an existing object\n// can be recycled. This is helpful if constructing a new object is more\n// expensive than resetting an existing object to the desired state.\n//\n// Deleter specifies how an object should be eventually deleted, like in\n// `std::unique_ptr<T, Deleter>`.\n//\n// The `Key` parameter allows to find an object to reuse only among compatible\n// objects, which should be assigned the same key. The `Key` type must be\n// equality comparable, hashable (by `absl::Hash`), default constructible, and\n// copyable.\n//\n// `KeyedRecyclingPool` is thread-safe.\ntemplate <typename T, typename Key, typename Deleter = std::default_delete<T>>\nclass KeyedRecyclingPool : public BackgroundCleanee {\n public:\n  // A deleter which puts the object back into the pool.\n  class Recycler;\n\n  // A `std::unique_ptr` which puts the object back into the pool instead of\n  // deleting it. If a particular object is not suitable for recycling,\n  // `DoNotRecycle()` can be used.\n  using Handle = std::unique_ptr<T, Recycler>;\n\n  // A `std::unique_ptr` which deletes the object. If a particular object is\n  // suitable for recycling, it can be put back into the pool using `RawPut()`.\n  using RawHandle = std::unique_ptr<T, Deleter>;\n\n  // A refurbisher which does nothing; see `Get()`.\n  struct DefaultRefurbisher {\n    void operator()(ABSL_ATTRIBUTE_UNUSED T* ptr) const {}\n  };\n\n  explicit KeyedRecyclingPool(\n      RecyclingPoolOptions options = RecyclingPoolOptions())\n      : max_size_(options.max_size_),\n        max_age_seconds_(options.max_age_seconds_) {}\n\n  KeyedRecyclingPool(const KeyedRecyclingPool&) = delete;\n  KeyedRecyclingPool& operator=(const KeyedRecyclingPool&) = delete;\n\n  ~KeyedRecyclingPool();\n\n  // Returns a default global pool specific to template parameters of\n  // `KeyedRecyclingPool` and `options`.\n  static KeyedRecyclingPool& global(\n      RecyclingPoolOptions options = RecyclingPoolOptions());\n\n  // Uses a different `BackgroundCleaner` than `BackgroundCleaner::global()` for\n  // scheduling background cleaning. This is useful for testing.\n  //\n  // Precondition: `BackgroundCleaner` was not used yet.\n  void SetBackgroundCleaner(BackgroundCleaner* cleaner);\n\n  // Deletes all objects stored in the pool.\n  void Clear();\n\n  // Creates an object, or returns an existing object from the pool if possible.\n  //\n  // `factory` takes no arguments and returns `RawHandle`. It is called to\n  // create a new object.\n  //\n  // If `refurbisher` is specified, it takes a `T*` argument and its result is\n  // ignored. It is called before returning an existing object.\n  template <typename Factory, typename Refurbisher = DefaultRefurbisher>\n  Handle Get(Key key, Factory&& factory,\n             Refurbisher&& refurbisher = DefaultRefurbisher())\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Deletes the object immediately, does not return it into the pool.\n  //\n  // This can be called on the result of `Get()` if the object turned out to be\n  // not suitable for recycling.\n  //\n  // Equivalent to calling the original `Deleter` if `object != nullptr`.\n  static void DoNotRecycle(Handle object);\n\n  // Like `Get()`, but the object is not returned into the pool by the\n  // destructor of its handle. If the object is suitable for recycling, it can\n  // be put back into the pool using `RawPut()`.\n  template <typename Factory, typename Refurbisher = DefaultRefurbisher>\n  RawHandle RawGet(const Key& key, Factory&& factory,\n                   Refurbisher&& refurbisher = DefaultRefurbisher());\n\n  // Puts an idle object into the pool for recycling, possibly under a different\n  // key.\n  void RawPut(const Key& key, RawHandle object);\n\n protected:\n  void Clean(absl::Time now) override;\n\n private:\n  struct ByAgeEntry {\n    explicit ByAgeEntry(const Key& key, absl::Time deadline)\n        : key(key), deadline(deadline) {}\n\n    Key key;\n    absl::Time deadline;\n  };\n\n  // Adding or removing elements in `ByAge` must not invalidate other iterators.\n  using ByAge = std::list<ByAgeEntry>;\n\n  struct ByKeyEntry {\n    ByKeyEntry(RawHandle object, typename ByAge::iterator by_age_iter)\n        : object(std::move(object)), by_age_iter(by_age_iter) {}\n\n    RawHandle object;\n    typename ByAge::iterator by_age_iter;\n  };\n\n  // `std::list` has a smaller overhead than `std::deque` for short sequences.\n  using ByKeyEntries = std::list<ByKeyEntry>;\n\n  using ByKey = absl::flat_hash_map<Key, ByKeyEntries>;\n\n  uint32_t max_size_;\n  uint32_t max_age_seconds_;\n  // If not `nullptr` then `this` has been registered at `*cleaner_`.\n  BackgroundCleaner* cleaner_ = nullptr;\n  BackgroundCleaner::Token cleaner_token_;\n  absl::Mutex mutex_;\n  // The key of each object, ordered by the age of the object (older to\n  // newer).\n  ByAge by_age_ ABSL_GUARDED_BY(mutex_);\n  // Objects grouped by their keys. Within each map value the list of objects is\n  // non-empty and is ordered by their age (older to newer). Each object is\n  // associated with the matching `by_age_` iterator.\n  ByKey by_key_ ABSL_GUARDED_BY(mutex_);\n  // Optimization for `Get()` followed by `Put()` with a matching key.\n  // If `cache_ != by_key_.end()`, then `cache_->second.back().object` was\n  // replaced with `nullptr` instead of erasing the corresponding entries,\n  // to avoid allocating them again if a matching objects is put again.\n  typename ByKey::iterator cache_ ABSL_GUARDED_BY(mutex_) = by_key_.end();\n};\n\n// Implementation details follow.\n\ninline uint32_t RecyclingPoolOptions::AgeToSeconds(absl::Duration age) {\n  if (age >= absl::Seconds(std::numeric_limits<uint32_t>::max())) {\n    return std::numeric_limits<uint32_t>::max();\n  }\n  if (age <= absl::ZeroDuration()) return 0;\n  int64_t seconds = absl::ToInt64Seconds(age);\n  if (age != absl::Seconds(seconds)) ++seconds;  // Round up.\n  return IntCast<uint32_t>(seconds);\n}\n\nnamespace recycling_pool_internal {\n\ntemplate <typename RecyclingPool, typename Deleter>\nclass RecyclerRepr {\n public:\n  RecyclerRepr() = default;\n\n  explicit RecyclerRepr(RecyclingPool* pool, Deleter&& deleter)\n      : deleter_(std::move(deleter)), pool_(pool) {}\n\n  RecyclingPool* pool() const { return pool_; }\n\n  Deleter& deleter() { return deleter_; }\n  const Deleter& deleter() const { return deleter_; }\n\n private:\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS Deleter deleter_;\n  RecyclingPool* pool_ = nullptr;\n};\n\nstruct RecyclingPoolOptionsKey : WithEqual<RecyclingPoolOptionsKey> {\n  explicit RecyclingPoolOptionsKey(uint32_t max_size, uint32_t max_age_seconds)\n      : max_size(max_size), max_age_seconds(max_age_seconds) {}\n\n  friend bool operator==(const RecyclingPoolOptionsKey& a,\n                         const RecyclingPoolOptionsKey& b) {\n    return a.max_size == b.max_size && a.max_age_seconds == b.max_age_seconds;\n  }\n\n  template <typename HashState>\n  friend HashState AbslHashValue(HashState hash_state,\n                                 const RecyclingPoolOptionsKey& self) {\n    return HashState::combine(std::move(hash_state), self.max_size,\n                              self.max_age_seconds);\n  }\n\n  uint32_t max_size;\n  uint32_t max_age_seconds;\n};\n\n}  // namespace recycling_pool_internal\n\ntemplate <typename T, typename Deleter>\nclass RecyclingPool<T, Deleter>::Recycler {\n public:\n  Recycler() = default;\n\n  explicit Recycler(RecyclingPool* pool, Deleter&& deleter)\n      : repr_(pool, std::move(deleter)) {\n    RIEGELI_ASSERT_NE(repr_.pool(), nullptr)\n        << \"Failed precondition of Recycler: null RecyclingPool pointer\";\n  }\n\n  void operator()(T* ptr) const {\n    RIEGELI_ASSERT_NE(repr_.pool(), nullptr)\n        << \"Failed precondition of RecyclingPool::Recycler: \"\n           \"default-constructed recycler used with an object\";\n    repr_.pool()->RawPut(RawHandle(ptr, original_deleter()));\n  }\n\n  Deleter& original_deleter() { return repr_.deleter(); }\n  const Deleter& original_deleter() const { return repr_.deleter(); }\n\n private:\n  recycling_pool_internal::RecyclerRepr<RecyclingPool, Deleter> repr_;\n};\n\ntemplate <typename T, typename Deleter>\nRecyclingPool<T, Deleter>::~RecyclingPool() {\n  if (cleaner_ != nullptr) cleaner_->Unregister(cleaner_token_);\n}\n\ntemplate <typename T, typename Deleter>\nRecyclingPool<T, Deleter>& RecyclingPool<T, Deleter>::global(\n    RecyclingPoolOptions options) {\n  class ABSL_CACHELINE_ALIGNED Pools {\n   public:\n    RecyclingPool& GetPool(RecyclingPoolOptions options) {\n      std::pair<const recycling_pool_internal::RecyclingPoolOptionsKey,\n                RecyclingPool>* cached = cache_.load(std::memory_order_acquire);\n      const recycling_pool_internal::RecyclingPoolOptionsKey options_key(\n          options.max_size_, options.max_age_seconds_);\n      if (ABSL_PREDICT_FALSE(cached == nullptr ||\n                             cached->first != options_key)) {\n        absl::MutexLock lock(mutex_);\n        const auto iter = pools_.try_emplace(options_key, options).first;\n        cached = &*iter;\n        cache_.store(cached, std::memory_order_release);\n      }\n      return cached->second;\n    }\n\n   private:\n    // If not `nullptr`, points to the most recently returned node from\n    // `pools_`.\n    std::atomic<std::pair<\n        const recycling_pool_internal::RecyclingPoolOptionsKey, RecyclingPool>*>\n        cache_{nullptr};\n    absl::Mutex mutex_;\n    // Pointer stability required for `GetPool()`, node stability required for\n    // `cache_`.\n    absl::node_hash_map<recycling_pool_internal::RecyclingPoolOptionsKey,\n                        RecyclingPool>\n        pools_ ABSL_GUARDED_BY(mutex_);\n  };\n\n  return Global([] { return Pools(); }).GetPool(options);\n}\n\ntemplate <typename T, typename Deleter>\nvoid RecyclingPool<T, Deleter>::SetBackgroundCleaner(\n    BackgroundCleaner* cleaner) {\n  RIEGELI_ASSERT_EQ(cleaner_, nullptr)\n      << \"Failed precondition of RecyclingPool::SetBackgroundCleaner(): \"\n         \"BackgroundCleaner was already used\";\n  cleaner_ = cleaner;\n  cleaner_token_ = cleaner_->Register(this);\n}\n\ntemplate <typename T, typename Deleter>\nvoid RecyclingPool<T, Deleter>::Clear() {\n  if (cleaner_ != nullptr) cleaner_->CancelCleaning(cleaner_token_);\n  absl::InlinedVector<RawHandle, 16> evicted;\n  absl::MutexLock lock(mutex_);\n  evicted.reserve(ring_buffer_size_);\n  while (ring_buffer_size_ > 0) {\n    if (ring_buffer_end_ == 0) {\n      ring_buffer_end_ = IntCast<uint32_t>(ring_buffer_by_age_.size());\n    }\n    --ring_buffer_end_;\n    --ring_buffer_size_;\n    evicted.push_back(std::move(ring_buffer_by_age_[ring_buffer_end_].object));\n  }\n  // Destroy `evicted` after releasing `mutex_`.\n}\n\ntemplate <typename T, typename Deleter>\ntemplate <typename Factory, typename Refurbisher>\ntypename RecyclingPool<T, Deleter>::Handle RecyclingPool<T, Deleter>::Get(\n    Factory&& factory,\n    Refurbisher&& refurbisher) ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RawHandle returned = RawGet(std::forward<Factory>(factory),\n                              std::forward<Refurbisher>(refurbisher));\n  return Handle(returned.release(),\n                Recycler(this, std::move(returned.get_deleter())));\n}\n\ntemplate <typename T, typename Deleter>\nvoid RecyclingPool<T, Deleter>::DoNotRecycle(Handle object) {\n  T* const ptr = object.release();\n  if (ptr != nullptr) object.get_deleter().original_deleter()(ptr);\n}\n\ntemplate <typename T, typename Deleter>\ntemplate <typename Factory, typename Refurbisher>\ntypename RecyclingPool<T, Deleter>::RawHandle RecyclingPool<T, Deleter>::RawGet(\n    Factory&& factory, Refurbisher&& refurbisher) {\n  RawHandle returned;\n  {\n    absl::MutexLock lock(mutex_);\n    if (ABSL_PREDICT_TRUE(ring_buffer_size_ > 0)) {\n      if (ring_buffer_end_ == 0) {\n        ring_buffer_end_ = IntCast<uint32_t>(ring_buffer_by_age_.size());\n      }\n      --ring_buffer_end_;\n      --ring_buffer_size_;\n      // Return the newest entry.\n      returned = std::move(ring_buffer_by_age_[ring_buffer_end_].object);\n    }\n  }\n  if (ABSL_PREDICT_TRUE(returned != nullptr)) {\n    std::forward<Refurbisher>(refurbisher)(returned.get());\n  } else {\n    returned = std::forward<Factory>(factory)();\n  }\n  return returned;\n}\n\ntemplate <typename T, typename Deleter>\nvoid RecyclingPool<T, Deleter>::RawPut(RawHandle object) {\n  if (ABSL_PREDICT_FALSE(ring_buffer_by_age_.empty())) return;\n  RawHandle evicted;\n  absl::Time deadline = absl::InfiniteFuture();\n  {\n    absl::MutexLock lock(mutex_);\n    // Add a newest entry. Evict the oldest entry if the pool is full.\n    if (max_age_seconds_ != std::numeric_limits<uint32_t>::max()) {\n      if (ABSL_PREDICT_FALSE(cleaner_ == nullptr)) {\n        cleaner_ = &BackgroundCleaner::global();\n        cleaner_token_ = cleaner_->Register(this);\n      }\n      deadline = cleaner_->TimeNow() + absl::Seconds(max_age_seconds_);\n    }\n    Entry& entry = ring_buffer_by_age_[ring_buffer_end_];\n    evicted = std::exchange(entry.object, std::move(object));\n    entry.deadline = deadline;\n    ++ring_buffer_end_;\n    if (ring_buffer_end_ == ring_buffer_by_age_.size()) ring_buffer_end_ = 0;\n    if (ABSL_PREDICT_TRUE(ring_buffer_size_ < ring_buffer_by_age_.size())) {\n      ++ring_buffer_size_;\n    }\n    // If `deadline == absl::InfiniteFuture()` then `cleaner_` might be\n    // `nullptr`.\n    if (ring_buffer_size_ > 1 || deadline == absl::InfiniteFuture()) {\n      // No need to schedule cleaning.\n      return;\n    }\n  }\n  // Schedule cleaning and destroy `evicted` after releasing `mutex_`.\n  cleaner_->ScheduleCleaning(cleaner_token_, deadline);\n}\n\ntemplate <typename T, typename Deleter>\nvoid RecyclingPool<T, Deleter>::Clean(absl::Time now) {\n  absl::InlinedVector<RawHandle, 16> evicted;\n  absl::Time deadline;\n  {\n    absl::MutexLock lock(mutex_);\n    size_t index = ring_buffer_end_;\n    if (index < ring_buffer_size_) index += ring_buffer_by_age_.size();\n    index -= ring_buffer_size_;\n    for (;;) {\n      if (ring_buffer_size_ == 0) {\n        // Everything evicted, no need to schedule cleaning.\n        return;\n      }\n      Entry& entry = ring_buffer_by_age_[index];\n      if (entry.deadline > now) {\n        // Schedule cleaning for the remaining entries.\n        deadline = entry.deadline;\n        break;\n      }\n      // Evict the oldest entry.\n      evicted.push_back(std::move(entry.object));\n      ++index;\n      if (index == ring_buffer_by_age_.size()) index = 0;\n      --ring_buffer_size_;\n    }\n  }\n  // Schedule cleaning and destroy `evicted` after releasing `mutex_`.\n  cleaner_->ScheduleCleaning(cleaner_token_, deadline);\n}\n\ntemplate <typename T, typename Key, typename Deleter>\nclass KeyedRecyclingPool<T, Key, Deleter>::Recycler {\n public:\n  Recycler() = default;\n\n  explicit Recycler(KeyedRecyclingPool* pool, Key&& key, Deleter&& deleter)\n      : repr_(pool, std::move(deleter)), key_(std::move(key)) {\n    RIEGELI_ASSERT_NE(repr_.pool(), nullptr)\n        << \"Failed precondition of Recycler: null KeyedRecyclingPool pointer\";\n  }\n\n  void operator()(T* ptr) const {\n    RIEGELI_ASSERT_NE(repr_.pool(), nullptr)\n        << \"Failed precondition of KeyedRecyclingPool::Recycler: \"\n           \"default-constructed recycler used with an object\";\n    repr_.pool()->RawPut(key_, RawHandle(ptr, original_deleter()));\n  }\n\n  Deleter& original_deleter() { return repr_.deleter(); }\n  const Deleter& original_deleter() const { return repr_.deleter(); }\n\n private:\n  recycling_pool_internal::RecyclerRepr<KeyedRecyclingPool, Deleter> repr_;\n  Key key_;\n};\n\ntemplate <typename T, typename Key, typename Deleter>\nKeyedRecyclingPool<T, Key, Deleter>::~KeyedRecyclingPool() {\n  if (cleaner_ != nullptr) cleaner_->Unregister(cleaner_token_);\n}\n\ntemplate <typename T, typename Key, typename Deleter>\nKeyedRecyclingPool<T, Key, Deleter>&\nKeyedRecyclingPool<T, Key, Deleter>::global(RecyclingPoolOptions options) {\n  class Pools {\n   public:\n    KeyedRecyclingPool& GetPool(RecyclingPoolOptions options) {\n      std::pair<const recycling_pool_internal::RecyclingPoolOptionsKey,\n                KeyedRecyclingPool>* cached =\n          cache_.load(std::memory_order_acquire);\n      const recycling_pool_internal::RecyclingPoolOptionsKey options_key(\n          options.max_size_, options.max_age_seconds_);\n      if (ABSL_PREDICT_FALSE(cached == nullptr ||\n                             cached->first != options_key)) {\n        absl::MutexLock lock(mutex_);\n        const auto iter = pools_.try_emplace(options_key, options).first;\n        cached = &*iter;\n        cache_.store(cached, std::memory_order_release);\n      }\n      return cached->second;\n    }\n\n   private:\n    // If not `nullptr`, points to the most recently returned node from\n    // `pools_`.\n    std::atomic<\n        std::pair<const recycling_pool_internal::RecyclingPoolOptionsKey,\n                  KeyedRecyclingPool>*>\n        cache_{nullptr};\n    absl::Mutex mutex_;\n    // Pointer stability required for `GetPool()`, node stability required for\n    // `cache_`.\n    absl::node_hash_map<recycling_pool_internal::RecyclingPoolOptionsKey,\n                        KeyedRecyclingPool>\n        pools_ ABSL_GUARDED_BY(mutex_);\n  };\n\n  return Global([] { return Pools(); }).GetPool(options);\n}\n\ntemplate <typename T, typename Key, typename Deleter>\nvoid KeyedRecyclingPool<T, Key, Deleter>::SetBackgroundCleaner(\n    BackgroundCleaner* cleaner) {\n  RIEGELI_ASSERT_EQ(cleaner_, nullptr)\n      << \"Failed precondition of KeyedRecyclingPool::SetBackgroundCleaner(): \"\n         \"BackgroundCleaner was already used\";\n  cleaner_ = cleaner;\n  cleaner_token_ = cleaner_->Register(this);\n}\n\ntemplate <typename T, typename Key, typename Deleter>\nvoid KeyedRecyclingPool<T, Key, Deleter>::Clear() {\n  if (cleaner_ != nullptr) cleaner_->CancelCleaning(cleaner_token_);\n  ByAge evicted_by_age;\n  ByKey evicted_by_key;\n  absl::MutexLock lock(mutex_);\n  evicted_by_age = std::exchange(by_age_, ByAge());\n  evicted_by_key = std::exchange(by_key_, ByKey());\n  cache_ = by_key_.end();\n  // Destroy `evicted_by_age` and `evicted_by_key` after releasing `mutex_`.\n}\n\ntemplate <typename T, typename Key, typename Deleter>\ntemplate <typename Factory, typename Refurbisher>\ntypename KeyedRecyclingPool<T, Key, Deleter>::Handle\nKeyedRecyclingPool<T, Key, Deleter>::Get(Key key, Factory&& factory,\n                                         Refurbisher&& refurbisher)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RawHandle returned = RawGet(key, std::forward<Factory>(factory),\n                              std::forward<Refurbisher>(refurbisher));\n  return Handle(\n      returned.release(),\n      Recycler(this, std::move(key), std::move(returned.get_deleter())));\n}\n\ntemplate <typename T, typename Key, typename Deleter>\nvoid KeyedRecyclingPool<T, Key, Deleter>::DoNotRecycle(Handle object) {\n  T* const ptr = object.release();\n  if (ptr != nullptr) object.get_deleter().original_deleter()(ptr);\n}\n\ntemplate <typename T, typename Key, typename Deleter>\ntemplate <typename Factory, typename Refurbisher>\ntypename KeyedRecyclingPool<T, Key, Deleter>::RawHandle\nKeyedRecyclingPool<T, Key, Deleter>::RawGet(const Key& key, Factory&& factory,\n                                            Refurbisher&& refurbisher) {\n  RawHandle returned;\n  {\n    absl::MutexLock lock(mutex_);\n    if (cache_ != by_key_.end()) {\n      // Finish erasing the cached entry.\n      ByKeyEntries& by_key_entries = cache_->second;\n      RIEGELI_ASSERT(!by_key_entries.empty())\n          << \"Failed invariant of KeyedRecyclingPool: \"\n             \"empty by_key_ value\";\n      RIEGELI_ASSERT_EQ(by_key_entries.back().object, nullptr)\n          << \"Failed invariant of KeyedRecyclingPool: \"\n             \"non-nullptr object pointed to by cache_\";\n      by_age_.erase(by_key_entries.back().by_age_iter);\n      by_key_entries.pop_back();\n      if (by_key_entries.empty()) by_key_.erase(cache_);\n    }\n    const typename ByKey::iterator by_key_iter = by_key_.find(key);\n    if (ABSL_PREDICT_TRUE(by_key_iter != by_key_.end())) {\n      // Return the newest entry with this key.\n      ByKeyEntries& by_key_entries = by_key_iter->second;\n      RIEGELI_ASSERT(!by_key_entries.empty())\n          << \"Failed invariant of KeyedRecyclingPool: \"\n             \"empty by_key_ value\";\n      RIEGELI_ASSERT_NE(by_key_entries.back().object, nullptr)\n          << \"Failed invariant of KeyedRecyclingPool: \"\n             \"nullptr object not pointed to by cache_\";\n      returned = std::move(by_key_entries.back().object);\n    }\n    cache_ = by_key_iter;\n  }\n  if (ABSL_PREDICT_TRUE(returned != nullptr)) {\n    std::forward<Refurbisher>(refurbisher)(returned.get());\n  } else {\n    returned = std::forward<Factory>(factory)();\n  }\n  return returned;\n}\n\ntemplate <typename T, typename Key, typename Deleter>\nvoid KeyedRecyclingPool<T, Key, Deleter>::RawPut(const Key& key,\n                                                 RawHandle object) {\n  if (ABSL_PREDICT_FALSE(max_size_ == 0)) return;\n  RawHandle evicted;\n  absl::Time deadline = absl::InfiniteFuture();\n  {\n    absl::MutexLock lock(mutex_);\n    // Add a newest entry with this key.\n    if (max_age_seconds_ != std::numeric_limits<uint32_t>::max()) {\n      if (ABSL_PREDICT_FALSE(cleaner_ == nullptr)) {\n        cleaner_ = &BackgroundCleaner::global();\n        cleaner_token_ = cleaner_->Register(this);\n      }\n      deadline = cleaner_->TimeNow() + absl::Seconds(max_age_seconds_);\n    }\n    if (ABSL_PREDICT_TRUE(cache_ != by_key_.end())) {\n      ByKeyEntries& by_key_entries = cache_->second;\n      RIEGELI_ASSERT(!by_key_entries.empty())\n          << \"Failed invariant of KeyedRecyclingPool: \"\n             \"empty by_key_ value\";\n      RIEGELI_ASSERT_EQ(by_key_entries.back().object, nullptr)\n          << \"Failed invariant of KeyedRecyclingPool: \"\n             \"non-nullptr object pointed to by cache_\";\n      if (ABSL_PREDICT_TRUE(cache_->first == key)) {\n        // `cache_` hit. Set the object pointer again, move the entry to the end\n        // of `by_age_`, and update its deadline.\n        ByKeyEntry& by_key_entry = by_key_entries.back();\n        by_key_entry.object = std::move(object);\n        by_age_.splice(by_age_.end(), by_age_, by_key_entry.by_age_iter);\n        by_key_entry.by_age_iter->deadline = deadline;\n        goto done;\n      }\n      // `cache_` miss. Finish erasing the cached entry.\n      by_age_.erase(by_key_entries.back().by_age_iter);\n      by_key_entries.pop_back();\n      if (by_key_entries.empty()) by_key_.erase(cache_);\n    }\n    by_age_.emplace_back(key, deadline);\n    // Local scope so that `goto done` does not jump into the scope of\n    // `by_age_iter`.\n    {\n      typename ByAge::iterator by_age_iter = by_age_.end();\n      --by_age_iter;\n      // This invalidates `by_key_` iterators, including `cache_`.\n      by_key_[key].emplace_back(std::move(object), by_age_iter);\n    }\n    if (ABSL_PREDICT_FALSE(by_age_.size() > max_size_)) {\n      // Evict the oldest entry.\n      const typename ByKey::iterator by_key_iter =\n          by_key_.find(by_age_.front().key);\n      RIEGELI_ASSERT(by_key_iter != by_key_.end())\n          << \"Failed invariant of KeyedRecyclingPool: \"\n             \"a key from by_age_ absent in by_key_\";\n      ByKeyEntries& by_key_entries = by_key_iter->second;\n      RIEGELI_ASSERT(!by_key_entries.empty())\n          << \"Failed invariant of KeyedRecyclingPool: \"\n             \"empty by_key_ value\";\n      RIEGELI_ASSERT_NE(by_key_entries.front().object, nullptr)\n          << \"Failed invariant of KeyedRecyclingPool: \"\n             \"nullptr object not pointed to by cache_\";\n      evicted = std::move(by_key_entries.front().object);\n      by_key_entries.pop_front();\n      if (by_key_entries.empty()) by_key_.erase(by_key_iter);\n      by_age_.pop_front();\n    }\n  done:\n    cache_ = by_key_.end();\n    // If `deadline == absl::InfiniteFuture()` then `cleaner_` might be\n    // `nullptr`.\n    if (by_age_.size() > 1 || deadline == absl::InfiniteFuture()) {\n      // No need to schedule cleaning.\n      return;\n    }\n  }\n  // Schedule cleaning and destroy `evicted` after releasing `mutex_`.\n  cleaner_->ScheduleCleaning(cleaner_token_, deadline);\n}\n\ntemplate <typename T, typename Key, typename Deleter>\nvoid KeyedRecyclingPool<T, Key, Deleter>::Clean(absl::Time now) {\n  absl::InlinedVector<RawHandle, 16> evicted;\n  absl::Time deadline;\n  {\n    absl::MutexLock lock(mutex_);\n    for (;; by_age_.pop_front()) {\n      if (by_age_.empty()) {\n        // Everything evicted, no need to schedule cleaning.\n        return;\n      }\n      const ByAgeEntry& by_age_entry = by_age_.front();\n      if (by_age_entry.deadline > now) {\n        if (cache_ != by_key_.end() && cache_->first == by_age_entry.key) {\n          const ByKeyEntries& by_key_entries = cache_->second;\n          RIEGELI_ASSERT(!by_key_entries.empty())\n              << \"Failed invariant of KeyedRecyclingPool: \"\n                 \"empty by_key_ value\";\n          if (by_key_entries.front().object == nullptr) {\n            // Finish erasing the cached entry.\n            RIEGELI_ASSERT_EQ(by_key_entries.size(), 1u)\n                << \"Failed invariant of KeyedRecyclingPool: \"\n                   \"nullptr object not at the end of by_key_ value\";\n            by_key_.erase(cache_);\n            cache_ = by_key_.end();\n            continue;\n          }\n        }\n        // Schedule cleaning for the remaining entries.\n        deadline = by_age_entry.deadline;\n        break;\n      }\n      // Evict the oldest entry.\n      const typename ByKey::iterator by_key_iter =\n          by_key_.find(by_age_entry.key);\n      RIEGELI_ASSERT(by_key_iter != by_key_.end())\n          << \"Failed invariant of KeyedRecyclingPool: \"\n             \"a key from by_age_ absent in by_key_\";\n      ByKeyEntries& by_key_entries = by_key_iter->second;\n      RIEGELI_ASSERT(!by_key_entries.empty())\n          << \"Failed invariant of KeyedRecyclingPool: \"\n             \"empty by_key_ value\";\n      if (by_key_entries.front().object == nullptr) {\n        // Finish erasing the cached entry.\n        RIEGELI_ASSERT(cache_ == by_key_iter)\n            << \"Failed invariant of KeyedRecyclingPool: \"\n               \"nullptr object not pointed to by cache_\";\n        RIEGELI_ASSERT(cache_->first == by_age_entry.key)\n            << \"Failed invariant of KeyedRecyclingPool: \"\n               \"nullptr object not pointed to by cache_\";\n        RIEGELI_ASSERT_EQ(by_key_entries.size(), 1u)\n            << \"Failed invariant of KeyedRecyclingPool: \"\n               \"nullptr object not at the end of by_key_ value\";\n        by_key_.erase(by_key_iter);\n        cache_ = by_key_.end();\n        continue;\n      }\n      evicted.push_back(std::move(by_key_entries.front().object));\n      by_key_entries.pop_front();\n      if (by_key_entries.empty()) by_key_.erase(by_key_iter);\n    }\n  }\n  // Schedule cleaning and destroy `evicted` after releasing `mutex_`.\n  cleaner_->ScheduleCleaning(cleaner_token_, deadline);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_RECYCLING_POOL_H_\n"
  },
  {
    "path": "riegeli/base/ref_count.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_REF_COUNT_H_\n#define RIEGELI_BASE_REF_COUNT_H_\n\n#include <stddef.h>\n\n#include <atomic>\n#include <type_traits>\n\n#include \"riegeli/base/ownership.h\"\n\nnamespace riegeli {\n\n// `RefCount` provides operations on an atomic reference count.\nclass RefCount {\n public:\n  RefCount() = default;\n\n  RefCount(const RefCount&) = delete;\n  RefCount& operator=(const RefCount&) = delete;\n\n  // Increments the reference count.\n  //\n  // Does nothing if `Ownership` is `PassOwnership`.\n  template <typename Ownership = ShareOwnership,\n            std::enable_if_t<IsOwnership<Ownership>::value, int> = 0>\n  void Ref() const {\n    if (std::is_same_v<Ownership, ShareOwnership>) {\n      ref_count_.fetch_add(1, std::memory_order_relaxed);\n    }\n  }\n\n  // Decrements the reference count. Returns `true` when this was the last\n  // reference.\n  //\n  // Does nothing and returns `false` if `Ownership` is `ShareOwnership`.\n  //\n  // When `Unref()` returns `true`, the decrement can be skipped and the actual\n  // value of the reference count is unspecified. This avoids an expensive\n  // atomic read-modify-write operation, making the last `Unref()` much faster,\n  // at the cost of making a non-last `Unref()` a bit slower. This is in\n  // contrast to `std::shared_ptr` in libc++ and libstdc++.\n  template <typename Ownership = PassOwnership,\n            std::enable_if_t<IsOwnership<Ownership>::value, int> = 0>\n  bool Unref() const {\n    return std::is_same_v<Ownership, PassOwnership> &&\n           (HasUniqueOwner() ||\n            ref_count_.fetch_sub(1, std::memory_order_acq_rel) == 1);\n  }\n\n  // Returns `true` if there is only one owner of the object.\n  //\n  // This can be used to check if the object may be modified.\n  bool HasUniqueOwner() const {\n    return ref_count_.load(std::memory_order_acquire) == 1;\n  }\n\n  // Returns the current count.\n  //\n  // If the `RefCount` is accessed by multiple threads, this is a snapshot of\n  // the count which may change asynchronously, hence usage of `GetCount()`\n  // should be limited to cases not important for correctness, like producing\n  // debugging output.\n  //\n  // The count can be reliably compared against 1 with `HasUniqueOwner()`.\n  size_t GetCount() const { return ref_count_.load(std::memory_order_relaxed); }\n\n private:\n  mutable std::atomic<size_t> ref_count_ = 1;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_REF_COUNT_H_\n"
  },
  {
    "path": "riegeli/base/reset.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_RESET_H_\n#define RIEGELI_BASE_RESET_H_\n\n#include <stddef.h>\n\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// To make an existing `dest` object of some class `T` equivalent to a newly\n// constructed `T`, certain classes use a convention of providing member\n// functions `Reset()`, which mirror constructors of these classes (except for\n// copy and move constructors which are mirrored by the assignment operators).\n// This avoids constructing a temporary `T` and moving from it.\n//\n// If it is not known whether the given class provides member functions\n// `Reset()`, generic code can use `riegeli::Reset(dest, args...)`.\n// This calls the first defined form among the following:\n//  * `RiegeliReset(dest, args...)`\n//  * `dest.Reset(args...)`\n//  * `dest = args...` (if `args...` has a single element)\n//  * `dest = T(args...)`\n//\n// Hence to customize `riegeli::Reset()` for a class `T`, define overloads of\n// either a member function `void T::Reset(...)`, or a free function\n// `friend void RiegeliReset(T& dest, ...)` as a friend of `T` inside class\n// definition or in the same namespace as `T`, so that it can be found via ADL.\n//\n// `RiegeliReset()` is predefined for `std::string` and `absl::Cord`.\n\ninline void RiegeliReset(std::string& dest) { dest.clear(); }\n\ninline void RiegeliReset(std::string& dest, size_t size, char fill) {\n  dest.assign(size, fill);\n}\n\ninline void RiegeliReset(std::string& dest, absl::string_view src) {\n  dest.assign(src);\n}\n\ninline void RiegeliReset(std::string& dest, const char* src) {\n  dest.assign(src);\n}\n\ninline void RiegeliReset(std::string& dest, const char* absl_nullable src,\n                         size_t length) {\n  RIEGELI_ASSERT(src != nullptr || length == 0)\n      << \"Failed precondition of RiegeliReset(): non-empty span from nullptr\";\n  dest.assign(src, length);\n}\n\ninline void RiegeliReset(absl::Cord& dest) { dest.Clear(); }\n\ninline void RiegeliReset(absl::Cord& dest, absl::string_view src) {\n  dest = src;\n}\n\nnamespace reset_internal {\n\ntemplate <typename Enable, typename T, typename... Args>\nstruct HasRiegeliResetImpl : std::false_type {};\n\ntemplate <typename T, typename... Args>\nstruct HasRiegeliResetImpl<std::void_t<decltype(RiegeliReset(\n                               std::declval<T&>(), std::declval<Args>()...))>,\n                           T, Args...> : std::true_type {};\n\ntemplate <typename T, typename... Args>\nstruct HasRiegeliReset : HasRiegeliResetImpl<void, T, Args...> {};\n\ntemplate <typename Enable, typename T, typename... Args>\nstruct HasResetImpl : std::false_type {};\n\ntemplate <typename T, typename... Args>\nstruct HasResetImpl<\n    std::void_t<decltype(std::declval<T&>().Reset(std::declval<Args>()...))>, T,\n    Args...> : std::true_type {};\n\ntemplate <typename T, typename... Args>\nstruct HasReset : HasResetImpl<void, T, Args...> {};\n\ntemplate <typename Enable, typename T, typename... Args>\nstruct HasAssignmentImpl : std::false_type {};\n\ntemplate <typename T, typename Arg>\nstruct HasAssignmentImpl<\n    std::void_t<decltype(std::declval<T&>() = std::declval<Arg>())>, T, Arg>\n    : std::true_type {};\n\ntemplate <typename T, typename... Args>\nstruct HasAssignment : HasAssignmentImpl<void, T, Args...> {};\n\n}  // namespace reset_internal\n\n// `SupportsReset<T, Args...>::value` is true if `riegeli::Reset(T&, Args...)`\n// is supported.\ntemplate <typename T, typename... Args>\nstruct SupportsReset\n    : std::disjunction<reset_internal::HasRiegeliReset<T, Args...>,\n                       reset_internal::HasReset<T, Args...>,\n                       reset_internal::HasAssignment<T, Args...>,\n                       std::conjunction<std::is_constructible<T, Args...>,\n                                        std::is_move_assignable<T>>> {};\n\ntemplate <typename T, typename... Args,\n          std::enable_if_t<SupportsReset<T, Args...>::value, int> = 0>\ninline void Reset(T& dest, Args&&... args) {\n  if constexpr (reset_internal::HasRiegeliReset<T, Args...>::value) {\n    RiegeliReset(dest, std::forward<Args>(args)...);\n  } else if constexpr (reset_internal::HasReset<T, Args...>::value) {\n    dest.Reset(std::forward<Args>(args)...);\n  } else if constexpr (reset_internal::HasAssignment<T, Args...>::value) {\n    static_assert(sizeof...(Args) == 1, \"Implied by HasAssignment\");\n    dest = std::forward<Args...>(args...);\n  } else if constexpr (std::conjunction_v<std::is_constructible<T, Args...>,\n                                          std::is_move_assignable<T>>) {\n    dest = T(std::forward<Args>(args)...);\n  } else {\n    static_assert(std::is_void_v<std::void_t<T>>,\n                  \"Parameters excluded by function signature\");\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_RESET_H_\n"
  },
  {
    "path": "riegeli/base/shared_buffer.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/shared_buffer.h\"\n\n#include <stddef.h>\n\n#include <ostream>\n\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n\nnamespace riegeli {\n\nvoid SharedBuffer::DumpStructure(absl::string_view substr,\n                                 std::ostream& dest) const {\n  dest << \"[shared_buffer] {\";\n  const size_t ref_count = GetRefCount();\n  if (ref_count != 1) dest << \" ref_count: \" << ref_count;\n  if (!substr.empty()) {\n    if (substr.data() != data()) {\n      dest << \" space_before: \" << PtrDistance(data(), substr.data());\n    }\n    dest << \" space_after: \"\n         << PtrDistance(substr.data() + substr.size(), data() + capacity());\n  }\n  dest << \" }\";\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/shared_buffer.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_SHARED_BUFFER_H_\n#define RIEGELI_BASE_SHARED_BUFFER_H_\n\n#include <stddef.h>\n\n#include <iosfwd>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/external_data.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/shared_ptr.h\"\n\nnamespace riegeli {\n\n// Dynamically allocated byte buffer.\n//\n// Like `Buffer`, but ownership of the data can be shared.\nclass SharedBuffer {\n public:\n  SharedBuffer() = default;\n\n  // Ensures at least `min_capacity` of space.\n  explicit SharedBuffer(size_t min_capacity);\n\n  SharedBuffer(const SharedBuffer& that) = default;\n  SharedBuffer& operator=(const SharedBuffer& that) = default;\n\n  // The source `SharedBuffer` is left deallocated.\n  SharedBuffer(SharedBuffer&& that) = default;\n  SharedBuffer& operator=(SharedBuffer&& that) = default;\n\n  // Ensures at least `min_capacity` of space, and unique ownership of the data\n  // if `min_capacity > 0`. Existing contents are lost.\n  //\n  // Drops the allocation if the resulting capacity would be wasteful for\n  // `min_capacity`.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(size_t min_capacity = 0);\n\n  // Returns `true` if `*this` is the only owner of the data.\n  //\n  // If `capacity() == 0`, returns `false`.\n  bool IsUnique() const { return buffer_.IsUnique(); }\n\n  // Returns the current reference count.\n  //\n  // If the `SharedBuffer` is accessed by multiple threads, this is a snapshot\n  // of the count which may change asynchronously, hence usage of\n  // `GetRefCount()` should be limited to cases not important for correctness,\n  // like producing debugging output.\n  //\n  // The reference count can be reliably compared against 1 with `IsUnique()`.\n  size_t GetRefCount() const { return buffer_.GetRefCount(); }\n\n  // Returns the mutable data pointer.\n  //\n  // Precondition: `IsUnique()`.\n  char* mutable_data() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns the const data pointer.\n  const char* data() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns the usable data size. It can be greater than the requested size.\n  size_t capacity() const;\n\n  // Indicates support for:\n  //  * `ExternalRef(const SharedBuffer&, substr)`\n  //  * `ExternalRef(SharedBuffer&&, substr)`\n  friend void RiegeliSupportsExternalRef(const SharedBuffer*) {}\n\n  // Supports `ExternalRef`.\n  friend ExternalStorage RiegeliToExternalStorage(SharedBuffer* self) {\n    return RiegeliToExternalStorage(&self->buffer_);\n  }\n\n  // Supports `ExternalRef` and `Chain::Block`.\n  friend void RiegeliDumpStructure(const SharedBuffer* self,\n                                   absl::string_view substr,\n                                   std::ostream& dest) {\n    self->DumpStructure(substr, dest);\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const SharedBuffer* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->buffer_);\n  }\n\n private:\n  void DumpStructure(absl::string_view substr, std::ostream& dest) const;\n\n  SharedPtr<Buffer> buffer_;  // `nullptr` means `capacity() == 0`.\n};\n\n// Implementation details follow.\n\ninline SharedBuffer::SharedBuffer(size_t min_capacity)\n    : buffer_(min_capacity == 0\n                  ? nullptr\n                  : SharedPtr<Buffer>(riegeli::Maker(min_capacity))) {}\n\ninline void SharedBuffer::Reset(size_t min_capacity) {\n  if (min_capacity == 0) {\n    if (!buffer_.IsUnique()) buffer_.Reset();\n  } else {\n    buffer_.Reset(riegeli::Maker(min_capacity));\n  }\n}\n\ninline char* SharedBuffer::mutable_data() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT(IsUnique())\n      << \"Failed precondition of SharedBuffer::mutable_data(): \"\n         \"ownership is shared\";\n  return buffer_->data();\n}\n\ninline const char* SharedBuffer::data() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  if (buffer_ == nullptr) return nullptr;\n  return buffer_->data();\n}\n\ninline size_t SharedBuffer::capacity() const {\n  if (buffer_ == nullptr) return 0;\n  return buffer_->capacity();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_SHARED_BUFFER_H_\n"
  },
  {
    "path": "riegeli/base/shared_ptr.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_SHARED_PTR_H_\n#define RIEGELI_BASE_SHARED_PTR_H_\n\n#include <stddef.h>\n\n#include <cstddef>\n#include <memory>\n#include <new>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/external_data.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/new_aligned.h\"\n#include \"riegeli/base/ref_count.h\"\n\nnamespace riegeli {\n\n// `SharedPtr<T>` implements shared ownership of an object of type `T`.\n// It can also be empty, with the pointer being `nullptr`.\n//\n// The actual object can be of a subtype of `T`, as long as `T` has a virtual\n// destructor and is a leftmost non-virtual base class. Otherwise the object\n// must have the same type as `T`, except for possibly different cv-qualifiers.\n//\n// Compared to `std::shared_ptr`, `SharedPtr` supports `IsUnique()`, and has a\n// smaller memory overhead (the pointer has 1 word instead of 2, the allocated\n// header before the object has 1 word if `T` does not have a virtual destructor\n// and 2 words otherwise, instead of 3 words in either case). Also, the last\n// decrement of the reference count is faster than for `std::shared_ptr` in\n// libc++ and libstdc++. OTOH `SharedPtr` has fewer features, e.g. no custom\n// allocation or deletion, no aliasing constructor, no weak pointers, and it has\n// the leftmost non-virtual base class restriction.\n//\n// Compared to `IntrusiveSharedPtr`, `SharedPtr` is easier to use, because\n// it does not require the object to maintain its own reference count. OTOH\n// `IntrusiveSharedPtr` supports custom allocation and deallocation, and\n// conversion to an `IntrusiveSharedPtr` to a non-leftmost or virtual base\n// class. Prefer `SharedPtr` unless `IntrusiveSharedPtr` is needed.\ntemplate <typename T>\nclass ABSL_ATTRIBUTE_TRIVIAL_ABI ABSL_NULLABILITY_COMPATIBLE SharedPtr\n    : public WithEqual<SharedPtr<T>> {\n private:\n  template <typename SubT>\n  struct IsCompatibleProperSubtype\n      : std::conjunction<std::negation<std::is_same<SubT, T>>,\n                         std::is_convertible<SubT*, T*>,\n                         std::disjunction<std::is_same<std::remove_cv_t<SubT>,\n                                                       std::remove_cv_t<T>>,\n                                          std::has_virtual_destructor<T>>> {};\n\n public:\n  // Creates an empty `SharedPtr`.\n  constexpr SharedPtr() = default;\n  /*implicit*/ constexpr SharedPtr(std::nullptr_t) noexcept {}\n  SharedPtr& operator=(std::nullptr_t) {\n    Reset();\n    return *this;\n  }\n\n  // Creates a `SharedPtr` holding a constructed value.\n  explicit SharedPtr(Initializer<T> value) : ptr_(New(std::move(value))) {}\n\n  // Creates a `SharedPtr` holding a constructed value of a compatible type.\n  template <\n      typename SubInitializer,\n      std::enable_if_t<\n          IsCompatibleProperSubtype<TargetT<SubInitializer>>::value, int> = 0>\n  explicit SharedPtr(SubInitializer&& value)\n      : ptr_(UpCast(New<TargetT<SubInitializer>>(\n            std::forward<SubInitializer>(value)))) {}\n\n  // Converts from a `SharedPtr` with a compatible type.\n  template <typename SubT,\n            std::enable_if_t<IsCompatibleProperSubtype<SubT>::value, int> = 0>\n  /*implicit*/ SharedPtr(const SharedPtr<SubT>& that) noexcept\n      : ptr_(UpCast(Ref(that.ptr_.get()))) {}\n  template <typename SubT,\n            std::enable_if_t<IsCompatibleProperSubtype<SubT>::value, int> = 0>\n  SharedPtr& operator=(const SharedPtr<SubT>& that) noexcept {\n    ptr_.reset(UpCast(Ref(that.ptr_.get())));\n    return *this;\n  }\n\n  // Converts from a `SharedPtr` with a compatible type.\n  //\n  // The source `SharedPtr` is left empty.\n  template <typename SubT,\n            std::enable_if_t<IsCompatibleProperSubtype<SubT>::value, int> = 0>\n  /*implicit*/ SharedPtr(SharedPtr<SubT>&& that) noexcept\n      : ptr_(UpCast(that.Release())) {}\n  template <typename SubT,\n            std::enable_if_t<IsCompatibleProperSubtype<SubT>::value, int> = 0>\n  SharedPtr& operator=(SharedPtr<SubT>&& that) noexcept {\n    ptr_.reset(UpCast(that.Release()));\n    return *this;\n  }\n\n  SharedPtr(const SharedPtr& that) noexcept : ptr_(Ref(that.ptr_.get())) {}\n  SharedPtr& operator=(const SharedPtr& that) noexcept {\n    ptr_.reset(Ref(that.ptr_.get()));\n    return *this;\n  }\n\n  // The source `SharedPtr` is left empty.\n  SharedPtr(SharedPtr&& that) = default;\n  SharedPtr& operator=(SharedPtr&& that) = default;\n\n  // Makes `*this` empty.\n  //\n  // The old object, if any, is destroyed afterwards.\n  ABSL_ATTRIBUTE_REINITIALIZES\n  void Reset(std::nullptr_t = nullptr) { ptr_.reset(); }\n\n  // Replaces the object with a constructed value.\n  //\n  // The old object, if any, is destroyed afterwards.\n  //\n  // If `*this` is the only owner of an object known to have the same\n  // move-assignable type, the existing object is assigned or reset instead of\n  // allocating and constructing a new object.\n  ABSL_ATTRIBUTE_REINITIALIZES\n  void Reset(Initializer<T> value) { ResetImpl(std::move(value)); }\n\n  // Replaces the object with a constructed value of a compatible type.\n  //\n  // The old object, if any, is destroyed afterwards.\n  template <\n      typename SubInitializer,\n      std::enable_if_t<\n          IsCompatibleProperSubtype<TargetT<SubInitializer>>::value, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(SubInitializer&& value) {\n    ptr_.reset(UpCast(\n        New<TargetT<SubInitializer>>(std::forward<SubInitializer>(value))));\n  }\n\n  // Returns `true` if `*this` is the only owner of the object.\n  //\n  // This can be used to check if the object may be modified (in contrast to\n  // `std::shared_ptr::unique()`).\n  //\n  // If `*this` is empty, returns `false`.\n  bool IsUnique() const {\n    return ptr_ != nullptr && ref_count(ptr_.get()).HasUniqueOwner();\n  }\n\n  // Returns the current reference count.\n  //\n  // If the `SharedPtr` is accessed by multiple threads, this is a snapshot of\n  // the count which may change asynchronously, hence usage of `GetRefCount()`\n  // should be limited to cases not important for correctness, like producing\n  // debugging output.\n  //\n  // The reference count can be reliably compared against 1 with `IsUnique()`.\n  size_t GetRefCount() const {\n    if (ptr_ == nullptr) return 0;\n    return ref_count(ptr_.get()).GetCount();\n  }\n\n  // Returns the pointer.\n  T* get() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return ptr_.get(); }\n\n  // Dereferences the pointer.\n  T& operator*() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_NE(ptr_, nullptr)\n        << \"Failed precondition of SharedPtr::operator*: null pointer\";\n    return *ptr_;\n  }\n  T* operator->() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_NE(ptr_, nullptr)\n        << \"Failed precondition of SharedPtr::operator->: null pointer\";\n    return ptr_.get();\n  }\n\n  // Returns the pointer, releasing its ownership; the `SharedPtr` is left\n  // empty. The returned pointer must be deleted using `DeleteReleased()`.\n  //\n  // If the returned pointer is `nullptr`, it allowed but not required to call\n  // `DeleteReleased()`.\n  T* Release() { return ptr_.release(); }\n\n  // Deletes the pointer obtained by `Release()`.\n  //\n  // Does nothing if `ptr == nullptr`.\n  static void DeleteReleased(T* ptr) {\n    if (ptr != nullptr) Unrefer()(ptr);\n  }\n\n  template <typename OtherT>\n  friend bool operator==(const SharedPtr& a, const SharedPtr<OtherT>& b) {\n    return a.ptr_ == b.ptr_;\n  }\n  friend bool operator==(const SharedPtr& a, std::nullptr_t) {\n    return a.ptr_ == nullptr;\n  }\n\n  // Indicates support for:\n  //  * `ExternalRef(const SharedPtr&, substr)`\n  //  * `ExternalRef(SharedPtr&&, substr)`\n  friend void RiegeliSupportsExternalRef(const SharedPtr*) {}\n\n  // Supports `ExternalRef`.\n  friend ExternalStorage RiegeliToExternalStorage(SharedPtr* self) {\n    return ExternalStorage(\n        const_cast<std::remove_cv_t<T>*>(self->Release()),\n        [](void* ptr) { SharedPtr::DeleteReleased(static_cast<T*>(ptr)); });\n  }\n\n  // Supports `riegeli::Debug()`.\n  template <typename DebugStream>\n  friend void RiegeliDebug(const SharedPtr& src, DebugStream& dest) {\n    dest.Debug(src.get());\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const SharedPtr* self,\n                                        MemoryEstimator& memory_estimator) {\n    if (memory_estimator.RegisterNode(self->get())) {\n      self->RegisterSubobjects(memory_estimator);\n    }\n  }\n\n private:\n  // For converting from a `SharedPtr` with a compatible type.\n  template <typename SubT>\n  friend class SharedPtr;\n\n  using pointer = T*;  // For `ABSL_NULLABILITY_COMPATIBLE`.\n\n  // An object of type `SubT` is allocated together with `RefCount` if\n  // `!std::has_virtual_destructor_v<SubT>`, or `Control` otherwise.\n  //\n  // `RefCount` or `Control` immediately precede the object. If the object has\n  // a higher alignment requirement than `RefCount` or `Control`, there can be\n  // padding at the beginning of the allocation, before `RefCount` or `Control`.\n  // Hence if `std::has_virtual_destructor_v<SubT>` then the beginning of the\n  // allocation is known only to `Control::destroy()`.\n  struct Control {\n    explicit Control(void (*destroy)(void* ptr)) : destroy(destroy) {}\n\n    void (*destroy)(void* ptr);\n    RefCount ref_count;\n  };\n\n  struct Unrefer {\n    void operator()(T* ptr) const {\n      if (ref_count(ptr).Unref()) Delete(ptr);\n    }\n  };\n\n  template <typename SubT>\n  static void DestroyMethod(void* ptr) {\n    static_cast<SubT*>(ptr)->SubT::~SubT();\n    static constexpr size_t kOffset = RoundUp<alignof(SubT)>(sizeof(Control));\n    void* const allocated_ptr = static_cast<char*>(ptr) - kOffset;\n    DeleteAligned<void, UnsignedMax(alignof(Control), alignof(SubT))>(\n        allocated_ptr, kOffset + sizeof(SubT));\n  }\n\n  template <typename SubT>\n  static T* UpCast(SubT* ptr) {\n    T* const super_ptr = ptr;\n    RIEGELI_CHECK(\n        static_cast<void*>(const_cast<std::remove_cv_t<T>*>(super_ptr)) ==\n        static_cast<void*>(const_cast<std::remove_cv_t<SubT>*>(ptr)))\n        << \"SharedPtr does not support upcasting \"\n           \"to a non-leftmost or virtual base class\";\n    return super_ptr;\n  }\n\n  template <typename SubT>\n  static SubT* New(Initializer<SubT> value) {\n    if constexpr (!std::has_virtual_destructor_v<SubT>) {\n      static constexpr size_t kOffset =\n          RoundUp<alignof(SubT)>(sizeof(RefCount));\n      void* const allocated_ptr =\n          NewAligned<void, UnsignedMax(alignof(RefCount), alignof(SubT))>(\n              kOffset + sizeof(SubT));\n      void* const ptr = static_cast<char*>(allocated_ptr) + kOffset;\n      new (static_cast<RefCount*>(ptr) - 1) RefCount();\n      new (ptr) SubT(std::move(value).Construct());\n      return std::launder(static_cast<SubT*>(ptr));\n    } else {\n      static constexpr size_t kOffset = RoundUp<alignof(SubT)>(sizeof(Control));\n      void* const allocated_ptr =\n          NewAligned<void, UnsignedMax(alignof(Control), alignof(SubT))>(\n              kOffset + sizeof(SubT));\n      void* const ptr = static_cast<char*>(allocated_ptr) + kOffset;\n      new (static_cast<Control*>(ptr) - 1) Control(DestroyMethod<SubT>);\n      new (ptr) SubT(std::move(value).Construct());\n      return std::launder(static_cast<SubT*>(ptr));\n    }\n  }\n\n  static void Delete(T* ptr) {\n    if constexpr (!std::has_virtual_destructor_v<T>) {\n      ptr->~T();\n      static constexpr size_t kOffset = RoundUp<alignof(T)>(sizeof(RefCount));\n      void* const allocated_ptr =\n          reinterpret_cast<char*>(const_cast<std::remove_cv_t<T>*>(ptr)) -\n          kOffset;\n      DeleteAligned<void, UnsignedMax(alignof(RefCount), alignof(T))>(\n          allocated_ptr, kOffset + sizeof(T));\n    } else if constexpr (std::is_final_v<T>) {\n      ptr->~T();\n      static constexpr size_t kOffset = RoundUp<alignof(T)>(sizeof(Control));\n      void* const allocated_ptr =\n          reinterpret_cast<char*>(const_cast<std::remove_cv_t<T>*>(ptr)) -\n          kOffset;\n      DeleteAligned<void, UnsignedMax(alignof(Control), alignof(T))>(\n          allocated_ptr, kOffset + sizeof(T));\n    } else {\n      control(ptr).destroy(const_cast<std::remove_cv_t<T>*>(ptr));\n    }\n  }\n\n  template <typename SubT>\n  static Control& control(SubT* ptr) {\n    static_assert(\n        std::has_virtual_destructor_v<SubT>,\n        \"control() is used only with a type with a virtual destructor\");\n    return *std::launder(\n        reinterpret_cast<Control*>(const_cast<std::remove_cv_t<SubT>*>(ptr)) -\n        1);\n  }\n\n  template <typename SubT>\n  static RefCount& ref_count(SubT* ptr) {\n    if constexpr (!std::has_virtual_destructor_v<SubT>) {\n      return *std::launder(reinterpret_cast<RefCount*>(\n                               const_cast<std::remove_cv_t<SubT>*>(ptr)) -\n                           1);\n    } else {\n      return control(ptr).ref_count;\n    }\n  }\n\n  template <typename SubT>\n  static SubT* Ref(SubT* ptr) {\n    if (ptr != nullptr) ref_count(ptr).Ref();\n    return ptr;\n  }\n\n  template <typename DependentT>\n  struct IsAssignable\n      : public std::conjunction<\n            std::disjunction<\n                std::negation<std::has_virtual_destructor<DependentT>>,\n                std::is_final<DependentT>>,\n            std::is_move_assignable<DependentT>> {};\n\n  void ResetImpl(Initializer<T> value) {\n    if constexpr (IsAssignable<T>::value) {\n      if (IsUnique()) {\n        *ptr_ = std::move(value);\n        return;\n      }\n    }\n    ptr_.reset(New(std::move(value)));\n  }\n\n  template <typename MemoryEstimator>\n  void RegisterSubobjects(MemoryEstimator& memory_estimator) const {\n    if constexpr (!std::has_virtual_destructor_v<T>) {\n      static constexpr size_t kOffset = RoundUp<alignof(T)>(sizeof(RefCount));\n      void* const allocated_ptr =\n          reinterpret_cast<char*>(\n              const_cast<std::remove_cv_t<T>*>(ptr_.get())) -\n          kOffset;\n      memory_estimator.RegisterDynamicMemory(allocated_ptr,\n                                             kOffset + sizeof(T));\n      memory_estimator.RegisterSubobjects(ptr_.get());\n    } else if constexpr (std::is_final_v<T>) {\n      static constexpr size_t kOffset = RoundUp<alignof(T)>(sizeof(Control));\n      void* const allocated_ptr =\n          reinterpret_cast<char*>(\n              const_cast<std::remove_cv_t<T>*>(ptr_.get())) -\n          kOffset;\n      memory_estimator.RegisterDynamicMemory(allocated_ptr,\n                                             kOffset + sizeof(T));\n      memory_estimator.RegisterSubobjects(ptr_.get());\n    } else {\n      static constexpr size_t kOffset = RoundUp<alignof(T)>(sizeof(Control));\n      // `kOffset` is not necessarily accurate because the object can be of a\n      // subtype of `T`, so do not pass `allocated_ptr` to\n      // `RegisterDynamicMemory()`.\n      memory_estimator.RegisterDynamicMemory(\n          kOffset + memory_estimator.DynamicSizeOf(ptr_.get()));\n      memory_estimator.RegisterSubobjects(ptr_.get());\n    }\n  }\n\n  std::unique_ptr<T, Unrefer> ptr_;\n};\n\ntemplate <typename T>\nexplicit SharedPtr(T&& value) -> SharedPtr<TargetT<T>>;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_SHARED_PTR_H_\n"
  },
  {
    "path": "riegeli/base/sized_shared_buffer.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/sized_shared_buffer.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <limits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/shared_buffer.h\"\n\nnamespace riegeli {\n\nvoid SizedSharedBuffer::ShrinkSlow(size_t max_size) {\n  RIEGELI_ASSERT_GE(max_size, size_)\n      << \"Failed precondition of SizedSharedBuffer::ShrinkSlow(): \"\n         \"max_size less than current size\";\n  if (size_ == 0) {\n    buffer_ = SharedBuffer();\n    data_ = nullptr;\n    return;\n  }\n  SharedBuffer new_buffer(max_size);\n  char* const new_data = new_buffer.mutable_data();\n  std::memcpy(new_data, data_, size_);\n  data_ = new_data;\n  buffer_ = std::move(new_buffer);\n}\n\ninline size_t SizedSharedBuffer::space_before() const {\n  RIEGELI_ASSERT(data_ != nullptr || buffer_.data() == nullptr)\n      << \"Failed precondition of SizedSharedBuffer::space_before(): null data_\";\n  return PtrDistance(buffer_.data(), data_);\n}\n\ninline size_t SizedSharedBuffer::space_after() const {\n  RIEGELI_ASSERT(data_ != nullptr || buffer_.data() == nullptr)\n      << \"Failed precondition of SizedSharedBuffer::space_after(): null data_\";\n  return PtrDistance(data_ + size_, buffer_.data() + buffer_.capacity());\n}\n\ninline bool SizedSharedBuffer::CanAppendMovingData(size_t length,\n                                                   size_t& min_length_if_not) {\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - size_)\n      << \"Failed precondition of SizedSharedBuffer::CanAppendMovingData(): \"\n         \"SizedSharedBuffer size overflow\";\n  if (buffer_.IsUnique()) {\n    if (empty()) data_ = buffer_.mutable_data();\n    if (space_after() >= length) return true;\n    if (size_ + length <= capacity() && 2 * size_ <= capacity()) {\n      // Existing array has enough capacity and is at most half full: move\n      // contents to the beginning of the array. This is enough to make the\n      // amortized cost of adding one element constant as long as prepending\n      // leaves space at both ends.\n      char* const new_data = buffer_.mutable_data();\n      std::memmove(new_data, data_, size_);\n      data_ = new_data;\n      return true;\n    }\n    min_length_if_not = UnsignedClamp(\n        SaturatingAdd(empty() ? capacity() : space_after(), capacity() / 2),\n        length, std::numeric_limits<size_t>::max() - size_);\n  } else {\n    min_length_if_not = length;\n  }\n  return false;\n}\n\ninline bool SizedSharedBuffer::CanPrependMovingData(size_t length,\n                                                    size_t& space_after_if_not,\n                                                    size_t& min_length_if_not) {\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - size_)\n      << \"Failed precondition of SizedSharedBuffer::CanPrependMovingData(): \"\n         \"SizedSharedBuffer size overflow\";\n  if (buffer_.IsUnique()) {\n    if (empty()) data_ = buffer_.mutable_data() + buffer_.capacity();\n    if (space_before() >= length) return true;\n    if (size_ + length <= capacity() && 2 * size_ <= capacity()) {\n      // Existing array has enough capacity and is at most half full: move\n      // contents to the middle of the array. This makes the amortized cost of\n      // adding one element constant.\n      char* const new_data =\n          buffer_.mutable_data() + (capacity() - size_ + length) / 2;\n      std::memmove(new_data, data_, size_);\n      data_ = new_data;\n      return true;\n    }\n    min_length_if_not = UnsignedClamp(\n        SaturatingAdd(empty() ? capacity() : space_before(), capacity() / 2),\n        length, std::numeric_limits<size_t>::max() - size_);\n    space_after_if_not =\n        UnsignedMin(space_after(), std::numeric_limits<size_t>::max() - size_ -\n                                       min_length_if_not);\n  } else {\n    min_length_if_not = length;\n    space_after_if_not = 0;\n  }\n  return false;\n}\n\ninline size_t SizedSharedBuffer::NewCapacity(size_t extra_space,\n                                             size_t min_length,\n                                             size_t recommended_length) const {\n  RIEGELI_ASSERT_LE(extra_space, std::numeric_limits<size_t>::max() - size_)\n      << \"Failed precondition of SizedSharedBuffer::NewCapacity(): \"\n         \"SizedSharedBuffer size overflow\";\n  const size_t existing_space = size_ + extra_space;\n  RIEGELI_ASSERT_LE(min_length,\n                    std::numeric_limits<size_t>::max() - existing_space)\n      << \"Failed precondition of SizedSharedBuffer::NewCapacity(): \"\n         \"SizedSharedBuffer size overflow\";\n  return existing_space +\n         UnsignedClamp(recommended_length, min_length,\n                       std::numeric_limits<size_t>::max() - existing_space);\n}\n\nabsl::Span<char> SizedSharedBuffer::AppendBuffer(\n    size_t min_length, size_t recommended_length,\n    size_t max_length) ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT_LE(min_length, max_length)\n      << \"Failed precondition of SizedSharedBuffer::AppendBuffer(): \"\n         \"min_length > max_length\";\n  RIEGELI_CHECK_LE(min_length, std::numeric_limits<size_t>::max() - size())\n      << \"Failed precondition of SizedSharedBuffer::AppendBuffer(): \"\n         \"SizedSharedBuffer size overflow\";\n  size_t new_min_length;\n  if (!CanAppendMovingData(min_length, new_min_length)) {\n    if (min_length == 0) return absl::Span<char>();\n    // Reallocate the array, without keeping space before the contents. This\n    // is enough to make the amortized cost of adding one element constant if\n    // prepending leaves space at both ends.\n    const size_t new_capacity =\n        NewCapacity(0, new_min_length, recommended_length);\n    if (empty()) {\n      buffer_.Reset(new_capacity);\n    } else {\n      SharedBuffer new_buffer(new_capacity);\n      std::memcpy(new_buffer.mutable_data(), data_, size_);\n      buffer_ = std::move(new_buffer);\n    }\n    data_ = buffer_.mutable_data();\n  }\n  const size_t length = UnsignedMin(space_after(), max_length);\n  const absl::Span<char> buffer(data_ + size_, length);\n  size_ += length;\n  return buffer;\n}\n\nabsl::Span<char> SizedSharedBuffer::PrependBuffer(\n    size_t min_length, size_t recommended_length,\n    size_t max_length) ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT_LE(min_length, max_length)\n      << \"Failed precondition of SizedSharedBuffer::PrependBuffer(): \"\n         \"min_length > max_length\";\n  RIEGELI_CHECK_LE(min_length, std::numeric_limits<size_t>::max() - size())\n      << \"Failed precondition of SizedSharedBuffer::PrependBuffer(): \"\n         \"SizedSharedBuffer size overflow\";\n  size_t space_after, new_min_length;\n  if (!CanPrependMovingData(min_length, space_after, new_min_length)) {\n    if (min_length == 0) return absl::Span<char>();\n    // Reallocate the array, keeping space after the contents unchanged. This\n    // makes the amortized cost of adding one element constant.\n    const size_t new_capacity =\n        NewCapacity(space_after, new_min_length, recommended_length);\n    char* new_data;\n    if (empty()) {\n      buffer_.Reset(new_capacity);\n      new_data = buffer_.mutable_data() + buffer_.capacity() - space_after;\n    } else {\n      SharedBuffer new_buffer(new_capacity);\n      new_data = new_buffer.mutable_data() + new_buffer.capacity() -\n                 space_after - size_;\n      std::memcpy(new_data, data_, size_);\n      buffer_ = std::move(new_buffer);\n    }\n    data_ = new_data;\n  }\n  const size_t length = UnsignedMin(space_before(), max_length);\n  data_ -= length;\n  size_ += length;\n  return absl::Span<char>(data_, length);\n}\n\nabsl::Span<char> SizedSharedBuffer::AppendBufferIfExisting(size_t length)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  size_t new_min_length;\n  if (ABSL_PREDICT_FALSE(length >\n                         std::numeric_limits<size_t>::max() - size()) ||\n      !CanAppendMovingData(length, new_min_length)) {\n    return absl::Span<char>();\n  }\n  const absl::Span<char> buffer(data_ + size_, length);\n  size_ += length;\n  return buffer;\n}\n\nabsl::Span<char> SizedSharedBuffer::PrependBufferIfExisting(size_t length)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  size_t space_after, new_min_length;\n  if (ABSL_PREDICT_FALSE(length >\n                         std::numeric_limits<size_t>::max() - size()) ||\n      !CanPrependMovingData(length, space_after, new_min_length)) {\n    return absl::Span<char>();\n  }\n  data_ -= length;\n  size_ += length;\n  return absl::Span<char>(data_, length);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/sized_shared_buffer.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_SIZED_SHARED_BUFFER_H_\n#define RIEGELI_BASE_SIZED_SHARED_BUFFER_H_\n\n#include <stddef.h>\n\n#include <limits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/shared_buffer.h\"\n\nnamespace riegeli {\n\n// Dynamically allocated byte buffer.\n//\n// Like `SharedBuffer`, but keeps track of the substring which is used.\nclass ABSL_ATTRIBUTE_TRIVIAL_ABI SizedSharedBuffer {\n public:\n  // A sentinel value for the `max_length` parameter of\n  // `AppendBuffer()`/`PrependBuffer()`.\n  static constexpr size_t kAnyLength = std::numeric_limits<size_t>::max();\n\n  SizedSharedBuffer() = default;\n\n  SizedSharedBuffer(const SizedSharedBuffer& that) = default;\n  SizedSharedBuffer& operator=(const SizedSharedBuffer& that) = default;\n\n  // The source `SizedSharedBuffer` is left empty.\n  SizedSharedBuffer(SizedSharedBuffer&& that) noexcept;\n  SizedSharedBuffer& operator=(SizedSharedBuffer&& that) noexcept;\n\n  // Removes all data.\n  ABSL_ATTRIBUTE_REINITIALIZES void Clear();\n\n  /*implicit*/ operator absl::string_view() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return absl::string_view(data_, size_);\n  }\n\n  // Returns `true` if the data size is 0.\n  bool empty() const { return size_ == 0; }\n\n  // Returns the data pointer.\n  const char* data() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return data_; }\n\n  // Returns the data size.\n  size_t size() const { return size_; }\n\n  // Returns the allocated size, to which the `SizedSharedBuffer` can be resized\n  // without reallocation.\n  size_t capacity() const { return buffer_.capacity(); }\n\n  // Reduces the allocation if the capacity would be wasteful for\n  // `max(size(), max_size)`, assuming that `max_size` will be needed later.\n  void Shrink(size_t max_size = 0);\n\n  // Removes all data.\n  //\n  // Drops the allocation if the capacity would be wasteful for `max_size`.\n  ABSL_ATTRIBUTE_REINITIALIZES void ClearAndShrink(size_t max_size = 0);\n\n  // Appends/prepends some uninitialized space. The buffer will have length at\n  // least `min_length`, preferably `recommended_length`, and at most\n  // `max_length`.\n  //\n  // If `min_length == 0`, returns whatever space was already allocated\n  // (possibly an empty buffer) without invalidating existing pointers. If the\n  // `SizedSharedBuffer` was empty then the empty contents can be moved.\n  //\n  // If `recommended_length < min_length`, `recommended_length` is assumed to be\n  // `min_length`.\n  //\n  // If `max_length == kAnyLength`, there is no maximum.\n  //\n  // Precondition: `min_length <= max_length`\n  absl::Span<char> AppendBuffer(\n      size_t min_length, size_t recommended_length = 0,\n      size_t max_length = kAnyLength) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  absl::Span<char> PrependBuffer(\n      size_t min_length, size_t recommended_length = 0,\n      size_t max_length = kAnyLength) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Equivalent to `AppendBuffer()`/`PrependBuffer()` with\n  // `min_length == max_length`.\n  absl::Span<char> AppendFixedBuffer(size_t length)\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  absl::Span<char> PrependFixedBuffer(size_t length)\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Appends/prepends some uninitialized space with the given `length` if this\n  // is possible without invalidating existing pointers, otherwise returns an\n  // empty buffer. If the `SizedSharedBuffer` was empty then the empty contents\n  // can be moved.\n  //\n  // In contrast to `AppendBuffer(0, length, length)`, the returned buffer has\n  // size either 0 or `length`, nothing between.\n  absl::Span<char> AppendBufferIfExisting(size_t length)\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  absl::Span<char> PrependBufferIfExisting(size_t length)\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Removes suffix/prefix of the given length.\n  //\n  // Precondition: `length <= size()`\n  void RemoveSuffix(size_t length);\n  void RemovePrefix(size_t length);\n\n  // Indicates support for:\n  //  * `ExternalRef(const SizedSharedBuffer&)`\n  //  * `ExternalRef(SizedSharedBuffer&&)`\n  //  * `ExternalRef(const SizedSharedBuffer&, substr)`\n  //  * `ExternalRef(SizedSharedBuffer&&, substr)`\n  friend void RiegeliSupportsExternalRef(const SizedSharedBuffer*) {}\n\n  // Supports `ExternalRef`.\n  template <typename Callback>\n  friend void RiegeliExternalDelegate(SizedSharedBuffer* self,\n                                      absl::string_view substr,\n                                      Callback&& delegate_to) {\n    self->data_ = nullptr;\n    self->size_ = 0;\n    std::forward<Callback>(delegate_to)(std::move(self->buffer_), substr);\n  }\n  template <typename Callback>\n  friend void RiegeliExternalDelegate(const SizedSharedBuffer* self,\n                                      absl::string_view substr,\n                                      Callback&& delegate_to) {\n    std::forward<Callback>(delegate_to)(self->buffer_, substr);\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const SizedSharedBuffer* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->buffer_);\n  }\n\n private:\n  explicit SizedSharedBuffer(SharedBuffer buffer, char* data, size_t size)\n      : buffer_(std::move(buffer)), data_(data), size_(size) {}\n\n  void ShrinkSlow(size_t max_size);\n\n  size_t space_before() const;\n  size_t space_after() const;\n  bool CanAppendMovingData(size_t length, size_t& min_length_if_not);\n  bool CanPrependMovingData(size_t length, size_t& space_after_if_not,\n                            size_t& min_length_if_not);\n\n  size_t NewCapacity(size_t extra_space, size_t min_length,\n                     size_t recommended_length) const;\n\n  void RemoveSuffixSlow(size_t length);\n  void RemovePrefixSlow(size_t length);\n\n  SharedBuffer buffer_;\n  // Invariant:\n  //   `(data_ == nullptr && size_ == 0) ||\n  //    (data_ >= buffer_.data() &&\n  //     data_ + size_ <= buffer_.data() + buffer_.capacity())`\n  char* data_ = nullptr;\n  size_t size_ = 0;\n};\n\n// Implementation details follow.\n\ninline SizedSharedBuffer::SizedSharedBuffer(SizedSharedBuffer&& that) noexcept\n    : buffer_(std::move(that.buffer_)),\n      data_(std::exchange(that.data_, nullptr)),\n      size_(std::exchange(that.size_, 0)) {}\n\ninline SizedSharedBuffer& SizedSharedBuffer::operator=(\n    SizedSharedBuffer&& that) noexcept {\n  buffer_ = std::move(that.buffer_);\n  data_ = std::exchange(that.data_, nullptr);\n  size_ = std::exchange(that.size_, 0);\n  return *this;\n}\n\ninline void SizedSharedBuffer::Clear() { size_ = 0; }\n\ninline void SizedSharedBuffer::Shrink(size_t max_size) {\n  max_size = UnsignedMax(max_size, size_);\n  if (capacity() > max_size && Wasteful(capacity(), max_size)) {\n    ShrinkSlow(max_size);\n  }\n}\n\ninline void SizedSharedBuffer::ClearAndShrink(size_t max_size) {\n  size_ = 0;\n  if (capacity() > max_size && Wasteful(capacity(), max_size)) {\n    buffer_ = SharedBuffer();\n    data_ = nullptr;\n  }\n}\n\ninline absl::Span<char> SizedSharedBuffer::AppendFixedBuffer(size_t length)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return AppendBuffer(length, length, length);\n}\n\ninline absl::Span<char> SizedSharedBuffer::PrependFixedBuffer(size_t length)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return PrependBuffer(length, length, length);\n}\n\ninline void SizedSharedBuffer::RemoveSuffix(size_t length) {\n  RIEGELI_CHECK_LE(length, size())\n      << \"Failed precondition of SizedSharedBuffer::RemoveSuffix(): \"\n      << \"length to remove greater than current size\";\n  size_ -= length;\n}\n\ninline void SizedSharedBuffer::RemovePrefix(size_t length) {\n  RIEGELI_CHECK_LE(length, size())\n      << \"Failed precondition of SizedSharedBuffer::RemovePrefix(): \"\n      << \"length to remove greater than current size\";\n  data_ += length;\n  size_ -= length;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_SIZED_SHARED_BUFFER_H_\n"
  },
  {
    "path": "riegeli/base/stable_dependency.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_STABLE_DEPENDENCY_H_\n#define RIEGELI_BASE_STABLE_DEPENDENCY_H_\n\n#include <atomic>\n#include <memory>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/type_traits.h\"\n\nnamespace riegeli {\n\n// `StableDependency<Handle, Manager>` is similar to\n// `Dependency<Handle, Manager>`, but ensures that `Handle` stays unchanged when\n// the `StableDependency<Handle, Manager>` is moved.\n//\n// `StableDependency` can be used instead of `Dependency` if `Handle` stability\n// is required, e.g. if background threads access the `Handle`.\n\n// This template is specialized but does not have a primary definition.\ntemplate <typename Handle, typename Manager, typename Enable = void>\nclass StableDependency;\n\nnamespace dependency_internal {\n\ntemplate <typename Handle, typename Manager>\nclass StableDependencyDefault\n    : public PropagateStaticIsOwning<Dependency<Handle, Manager>> {\n public:\n  StableDependencyDefault() = default;\n\n  explicit StableDependencyDefault(Initializer<Manager> manager)\n      : dep_(new Dependency<Handle, Manager>(std::move(manager))),\n        ensure_allocated_(AssumeAllocatedSlow) {}\n\n  StableDependencyDefault(StableDependencyDefault&& that) noexcept\n      : dep_(that.dep_.exchange(nullptr, std::memory_order_relaxed)),\n        ensure_allocated_(\n            std::exchange(that.ensure_allocated_, EnsureAllocatedSlow)) {}\n  StableDependencyDefault& operator=(StableDependencyDefault&& that) noexcept {\n    delete dep_.exchange(that.dep_.exchange(nullptr, std::memory_order_relaxed),\n                         std::memory_order_relaxed);\n    ensure_allocated_ =\n        std::exchange(that.ensure_allocated_, EnsureAllocatedSlow);\n    return *this;\n  }\n\n  ~StableDependencyDefault() { delete dep_.load(std::memory_order_relaxed); }\n\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset() {\n    Dependency<Handle, Manager>* const dep =\n        dep_.load(std::memory_order_relaxed);\n    if (dep != nullptr) dep->Reset();\n  }\n\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Manager> manager) {\n    Dependency<Handle, Manager>* const dep =\n        dep_.load(std::memory_order_relaxed);\n    if (dep == nullptr) {\n      // A race would violate the contract because this is not a const method.\n      dep_.store(new Dependency<Handle, Manager>(std::move(manager)),\n                 std::memory_order_relaxed);\n    } else {\n      dep->Reset(std::move(manager));\n    }\n  }\n\n  Manager& manager() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return EnsureAllocated().manager();\n  }\n  const Manager& manager() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return EnsureAllocated().manager();\n  }\n\n  typename Dependency<Handle, Manager>::Subhandle get() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return EnsureAllocated().get();\n  }\n\n  bool IsOwning() const { return EnsureAllocated().IsOwning(); }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const StableDependencyDefault* self,\n                                        MemoryEstimator& memory_estimator) {\n    Dependency<Handle, Manager>* const dep =\n        self->dep_.load(std::memory_order_acquire);\n    if (dep != nullptr) memory_estimator.RegisterDynamicObject(dep);\n  }\n\n private:\n  Dependency<Handle, Manager>& EnsureAllocated() const {\n    Dependency<Handle, Manager>* const dep =\n        dep_.load(std::memory_order_acquire);\n    if (ABSL_PREDICT_TRUE(dep != nullptr)) return *dep;\n    return ensure_allocated_(*this);\n  }\n\n  static Dependency<Handle, Manager>& EnsureAllocatedSlow(\n      const StableDependencyDefault& self);\n  static Dependency<Handle, Manager>& AssumeAllocatedSlow(\n      const StableDependencyDefault& self);\n\n  // Owned. `nullptr` is equivalent to a default constructed `Dependency`.\n  mutable std::atomic<Dependency<Handle, Manager>*> dep_ = nullptr;\n  // Handles the case when `dep_ == nullptr`.\n  //\n  // The indirection allows initialization from `Initializer<Manager>` even if\n  // `Dependency<Handle, Manager>` appears to be default-constructible but its\n  // default constructor does not compile, by avoiding dead code which would\n  // call the default constructor.\n  //\n  // Invariant:\n  //   if `dep_ == nullptr` then `ensure_allocated_ == EnsureAllocatedSlow`\n  Dependency<Handle, Manager>& (*ensure_allocated_)(\n      const StableDependencyDefault&) = EnsureAllocatedSlow;\n};\n\ntemplate <typename Handle, typename Manager>\nclass StableDependencyNoDefault\n    : public PropagateStaticIsOwning<Dependency<Handle, Manager>> {\n public:\n  explicit StableDependencyNoDefault(Initializer<Manager> manager)\n      : dep_(\n            std::make_unique<Dependency<Handle, Manager>>(std::move(manager))) {\n  }\n\n  StableDependencyNoDefault(StableDependencyNoDefault&& that) noexcept\n      : dep_(std::make_unique<Dependency<Handle, Manager>>(\n            std::move(*that.dep_))) {}\n  StableDependencyNoDefault& operator=(\n      StableDependencyNoDefault&& that) noexcept {\n    *dep_ = std::move(*that.dep_);\n    return *this;\n  }\n\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Manager> manager) {\n    dep_->Reset(std::move(manager));\n  }\n\n  Manager& manager() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dep_->manager(); }\n  const Manager& manager() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dep_->manager();\n  }\n\n  typename Dependency<Handle, Manager>::Subhandle get() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dep_->get();\n  }\n\n  bool IsOwning() const { return dep_->IsOwning(); }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const StableDependencyNoDefault* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->dep_);\n  }\n\n private:\n  // Never `nullptr`.\n  std::unique_ptr<Dependency<Handle, Manager>> dep_;\n};\n\n}  // namespace dependency_internal\n\n// Specialization of `StableDependency<Handle, Manager>` when\n// `Dependency<Handle, Manager>` is already stable: delegate to it.\ntemplate <typename Handle, typename Manager>\nclass StableDependency<Handle, Manager,\n                       std::enable_if_t<Dependency<Handle, Manager>::kIsStable>>\n    : public Dependency<Handle, Manager> {\n public:\n  using StableDependency::Dependency::Dependency;\n\n  StableDependency(StableDependency&& that) = default;\n  StableDependency& operator=(StableDependency&& that) = default;\n};\n\n// Specialization of `StableDependency<Handle, Manager>` when\n// `Dependency<Handle, Manager>` is not stable but default-constructible:\n// allocate the dependency dynamically and conditionally.\ntemplate <typename Handle, typename Manager>\nclass StableDependency<\n    Handle, Manager,\n    std::enable_if_t<std::conjunction_v<\n        std::bool_constant<!Dependency<Handle, Manager>::kIsStable>,\n        std::is_default_constructible<Dependency<Handle, Manager>>>>>\n    : public dependency_internal::DependencyDerived<\n          dependency_internal::StableDependencyDefault<Handle, Manager>, Handle,\n          Manager> {\n public:\n  using StableDependency::DependencyDerived::DependencyDerived;\n\n  StableDependency(StableDependency&& that) = default;\n  StableDependency& operator=(StableDependency&& that) = default;\n};\n\n// Specialization of `StableDependency<Handle, Manager>` when\n// `Dependency<Handle, Manager>` is not stable and not default-constructible:\n// allocate the dependency dynamically and always keep it allocated.\ntemplate <typename Handle, typename Manager>\nclass StableDependency<\n    Handle, Manager,\n    std::enable_if_t<std::conjunction_v<\n        std::bool_constant<!Dependency<Handle, Manager>::kIsStable>,\n        std::negation<\n            std::is_default_constructible<Dependency<Handle, Manager>>>>>>\n    : public dependency_internal::DependencyDerived<\n          dependency_internal::StableDependencyNoDefault<Handle, Manager>,\n          Handle, Manager>,\n      public ConditionallyConstructible<\n          false, std::is_move_constructible_v<Dependency<Handle, Manager>>>,\n      public ConditionallyAssignable<\n          false, std::is_move_assignable_v<Dependency<Handle, Manager>>> {\n public:\n  using StableDependency::DependencyDerived::DependencyDerived;\n\n  StableDependency(StableDependency&& that) = default;\n  StableDependency& operator=(StableDependency&& that) = default;\n};\n\n// Implementation details follow.\n\nnamespace dependency_internal {\n\ntemplate <typename Handle, typename Manager>\nDependency<Handle, Manager>&\nStableDependencyDefault<Handle, Manager>::EnsureAllocatedSlow(\n    const StableDependencyDefault<Handle, Manager>& self) {\n  Dependency<Handle, Manager>* const dep = new Dependency<Handle, Manager>();\n  Dependency<Handle, Manager>* other_dep = nullptr;\n  if (ABSL_PREDICT_FALSE(!self.dep_.compare_exchange_strong(\n          other_dep, dep, std::memory_order_acq_rel))) {\n    // We lost the race.\n    delete dep;\n    return *other_dep;\n  }\n  return *dep;\n}\n\ntemplate <typename Handle, typename Manager>\nDependency<Handle, Manager>&\nStableDependencyDefault<Handle, Manager>::AssumeAllocatedSlow(\n    ABSL_ATTRIBUTE_UNUSED const StableDependencyDefault<Handle, Manager>&\n        self) {\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Failed invariant of StableDependency: \"\n         \"dep_ == nullptr but ensure_allocated_ == AssumeAllocatedSlow\";\n}\n\n}  // namespace dependency_internal\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_STABLE_DEPENDENCY_H_\n"
  },
  {
    "path": "riegeli/base/status.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/status.h\"\n\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n\nnamespace riegeli {\n\nabsl::Status SetMessage(const absl::Status& status, absl::string_view message) {\n  absl::Status result(status.code(), message);\n  status.ForEachPayload(\n      [&](absl::string_view type_url, const absl::Cord& payload) {\n        result.SetPayload(type_url, payload);\n      });\n  return result;\n}\n\nabsl::Status Annotate(const absl::Status& status, absl::string_view detail) {\n  if (status.ok() || detail.empty()) return status;\n  return SetMessage(status, status.message().empty()\n                                ? detail\n                                : absl::StrCat(status.message(), \"; \", detail));\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/status.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_STATUS_H_\n#define RIEGELI_BASE_STATUS_H_\n\n#include <utility>\n\n#include \"absl/status/status.h\"\n#include \"absl/status/statusor.h\"\n#include \"absl/strings/string_view.h\"\n\nnamespace riegeli {\n\n// Returns an `absl::Status` that is identical to `status` except that the\n// `message()` has been replaced with `message`.\n//\n// OK status values have no message and therefore if `status` is OK, the result\n// is unchanged.\nabsl::Status SetMessage(const absl::Status& status, absl::string_view message);\n\n// Returns an `absl::Status` that is identical to `status` except that the\n// `message()` has been augmented by adding `detail` to the end of the original\n// message.\n//\n// `Annotate()` adds the appropriate separators, so callers should not include a\n// separator in `detail`. The exact formatting is subject to change, so you\n// should not depend on it in your tests.\n//\n// OK status values have no message and therefore if `status` is OK, the result\n// is unchanged.\nabsl::Status Annotate(const absl::Status& status, absl::string_view detail);\n\n// `StatusOrMaker<T>::type` and `StatusOrMakerT<T>` generalize\n// `absl::StatusOr<T>` for types where that is not applicable:\n//  * `absl::StatusOr<const T>`           -> `absl::StatusOr<T>`\n//  * `absl::StatusOr<T&>`                -> `absl::StatusOr<T>`\n//  * `absl::StatusOr<T&&>`               -> `absl::StatusOr<T>`\n//  * `absl::StatusOr<void>`              -> `absl::Status`\n//  * `absl::StatusOr<absl::Status>`      -> `absl::Status`\n//  * `absl::StatusOr<absl::StatusOr<T>>` -> `absl::StatusOr<T>`\n//\n// The primary template provides the default implementation. Specializations\n// follow.\n\ntemplate <typename T>\nstruct StatusOrMaker {\n  // The combined type.\n  using type = absl::StatusOr<T>;\n\n  // Returns `status`.\n  static type FromStatus(const absl::Status& status) { return status; }\n  static type FromStatus(absl::Status&& status) { return std::move(status); }\n\n  // Returns `work()`.\n  template <typename Work>\n  static type FromWork(Work&& work) {\n    return std::forward<Work>(work)();\n  }\n\n  // Replaces `result` with `status` if `result` is OK.\n  static void Update(type& result, const absl::Status& status) {\n    if (result.ok()) result = status;\n  }\n};\n\ntemplate <typename T>\nstruct StatusOrMaker<const T> : StatusOrMaker<T> {};\n\ntemplate <typename T>\nstruct StatusOrMaker<T&> : StatusOrMaker<T> {};\n\ntemplate <typename T>\nstruct StatusOrMaker<T&&> : StatusOrMaker<T> {};\n\ntemplate <>\nstruct StatusOrMaker<void> {\n  using type = absl::Status;\n\n  static type FromStatus(const absl::Status& status) { return status; }\n  static type FromStatus(absl::Status&& status) { return std::move(status); }\n\n  template <typename Work>\n  static type FromWork(Work&& work) {\n    std::forward<Work>(work)();\n    return absl::OkStatus();\n  }\n\n  static void Update(type& result, const absl::Status& status) {\n    result.Update(status);\n  }\n};\n\ntemplate <>\nstruct StatusOrMaker<absl::Status> {\n  using type = absl::Status;\n\n  static type FromStatus(const absl::Status& status) { return status; }\n  static type FromStatus(absl::Status&& status) { return std::move(status); }\n\n  template <typename Work>\n  static type FromWork(Work&& work) {\n    return std::forward<Work>(work)();\n  }\n\n  static void Update(type& result, const absl::Status& status) {\n    result.Update(status);\n  }\n};\n\ntemplate <typename T>\nstruct StatusOrMaker<absl::StatusOr<T>> {\n  using type = absl::StatusOr<T>;\n\n  static type FromStatus(const absl::Status& status) { return status; }\n  static type FromStatus(absl::Status&& status) { return std::move(status); }\n\n  template <typename Work>\n  static type FromWork(Work&& work) {\n    return std::forward<Work>(work)();\n  }\n\n  static void Update(type& result, const absl::Status& status) {\n    if (result.ok()) result = status;\n  }\n};\n\ntemplate <typename T>\nusing StatusOrMakerT = typename StatusOrMaker<T>::type;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_STATUS_H_\n"
  },
  {
    "path": "riegeli/base/stream_utils.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/stream_utils.h\"\n\n#include <stddef.h>\n\n#include <cassert>\n#include <cstring>\n#include <ios>\n#include <ostream>\n\n#include \"absl/base/nullability.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nvoid WritePadding(std::ostream& dest, size_t length, char fill) {\n  char buffer[64];\n  std::memset(buffer, fill, sizeof(buffer));\n  while (length > sizeof(buffer)) {\n    dest.write(buffer, std::streamsize{sizeof(buffer)});\n    length -= sizeof(buffer);\n  }\n  dest.write(buffer, static_cast<std::streamsize>(length));\n}\n\nint StringifyOStream<StringStringifySink>::StringStreambuf::overflow(int src) {\n  if (src != traits_type::eof()) dest_->push_back(static_cast<char>(src));\n  return traits_type::not_eof(src);\n}\n\nstd::streamsize StringifyOStream<StringStringifySink>::StringStreambuf::xsputn(\n    const char* absl_nullable src, std::streamsize length) {\n  assert(length >= 0);\n  dest_->append(src, static_cast<size_t>(length));\n  return length;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/base/stream_utils.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_STREAM_UTILS_H_\n#define RIEGELI_BASE_STREAM_UTILS_H_\n\n#include <stddef.h>\n\n#include <cassert>\n#include <cstring>\n#include <ios>\n#include <ostream>\n#include <streambuf>\n#include <string>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/types.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// Writes `length` copies of `fill` to `dest`.\nvoid WritePadding(std::ostream& dest, size_t length, char fill);\n\n// Writes a value to `dest`, including padding configured in `dest`.\n// Resets `dest.width()` to 0 afterwards.\n//\n// `length` is the number of characters in the value. `callback()` is called\n// to write the value; it should use unformatted output, i.e. `dest.write()`.\ntemplate <typename Callback>\nvoid WriteWithPadding(std::ostream& dest, Position length,\n                      Callback&& callback) {\n  std::ostream::sentry sentry(dest);\n  if (sentry) {\n    size_t left_pad = 0;\n    size_t right_pad = 0;\n    if (dest.width() > 0) {\n      if (static_cast<size_t>(dest.width()) > length) {\n        const size_t pad =\n            static_cast<size_t>(dest.width()) - static_cast<size_t>(length);\n        if ((dest.flags() & dest.adjustfield) == dest.left) {\n          right_pad = pad;\n        } else {\n          left_pad = pad;\n        }\n      }\n      dest.width(0);\n    }\n    if (left_pad > 0) WritePadding(dest, left_pad, dest.fill());\n    std::forward<Callback>(callback)();\n    if (right_pad > 0) WritePadding(dest, right_pad, dest.fill());\n  }\n}\n\n// A sink for `AbslStringify()` which appends to a `std::string`.\nclass StringStringifySink {\n public:\n  explicit StringStringifySink(std::string* dest ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : dest_(dest) {}\n\n  StringStringifySink(const StringStringifySink& that) = default;\n  StringStringifySink& operator=(const StringStringifySink& that) = default;\n\n  std::string* dest() const { return dest_; }\n\n  void Append(size_t length, char fill) { dest_->append(length, fill); }\n  void Append(absl::string_view src) { dest_->append(src); }\n  friend void AbslFormatFlush(StringStringifySink* dest,\n                              absl::string_view src) {\n    dest->Append(src);\n  }\n\n private:\n  std::string* dest_;\n};\n\n// Adapts `std::ostream` to a sink for `AbslStringify()`.\nclass OStreamStringifySink {\n public:\n  explicit OStreamStringifySink(\n      std::ostream* dest ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : dest_(dest) {}\n\n  OStreamStringifySink(const OStreamStringifySink& that) = default;\n  OStreamStringifySink& operator=(const OStreamStringifySink& that) = default;\n\n  std::ostream* dest() const { return dest_; }\n\n  void Append(size_t length, char fill) { WritePadding(*dest_, length, fill); }\n  void Append(absl::string_view src) {\n    dest_->write(src.data(), static_cast<std::streamsize>(src.size()));\n  }\n  friend void AbslFormatFlush(OStreamStringifySink* dest,\n                              absl::string_view src) {\n    dest->Append(src);\n  }\n\n private:\n  std::ostream* dest_;\n};\n\n// Adapts a sink for `AbslStringify()` to `std::ostream`.\ntemplate <typename Sink>\nclass StringifyOStream final : public std::ostream {\n public:\n  explicit StringifyOStream(Sink* dest ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : std::ostream(&streambuf_), streambuf_(dest) {}\n\n  StringifyOStream(StringifyOStream&& that) noexcept\n      : std::ostream(static_cast<std::ostream&&>(that)),\n        streambuf_(std::move(that.streambuf_)) {\n    set_rdbuf(&streambuf_);\n  }\n  StringifyOStream& operator=(StringifyOStream&& that) noexcept {\n    std::ostream::operator=(static_cast<std::ostream&&>(that));\n    streambuf_ = std::move(that.streambuf_);\n    return *this;\n  }\n\n private:\n  class StringifyStreambuf;\n\n  StringifyStreambuf streambuf_;\n};\n\ntemplate <typename Sink>\nexplicit StringifyOStream(Sink* dest) -> StringifyOStream<Sink>;\n\ntemplate <>\nclass StringifyOStream<StringStringifySink> final : public std::ostream {\n public:\n  explicit StringifyOStream(std::string* dest ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : std::ostream(&streambuf_), streambuf_(dest) {}\n\n  explicit StringifyOStream(\n      StringStringifySink* sink ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : StringifyOStream(sink->dest()) {}\n\n  StringifyOStream(StringifyOStream&& that) noexcept\n      : std::ostream(static_cast<std::ostream&&>(that)),\n        streambuf_(std::move(that.streambuf_)) {\n    set_rdbuf(&streambuf_);\n  }\n  StringifyOStream& operator=(StringifyOStream&& that) noexcept {\n    std::ostream::operator=(static_cast<std::ostream&&>(that));\n    streambuf_ = std::move(that.streambuf_);\n    return *this;\n  }\n\n  std::string* dest() const { return streambuf_.dest(); }\n\n private:\n  class StringStreambuf final : public std::streambuf {\n   public:\n    explicit StringStreambuf(std::string* dest ABSL_ATTRIBUTE_LIFETIME_BOUND)\n        : dest_(dest) {}\n\n    StringStreambuf(const StringStreambuf& that) = default;\n    StringStreambuf& operator=(const StringStreambuf& that) = default;\n\n    std::string* dest() const { return dest_; }\n\n   protected:\n    int overflow(int src) override;\n    std::streamsize xsputn(const char* absl_nullable src,\n                           std::streamsize length) override;\n\n   private:\n    std::string* dest_;\n  };\n\n  StringStreambuf streambuf_;\n};\n\n// A faster version of `std::ostringstream`. It does not own the `std::string`\n// and does not support random access.\n//\n// This is similar to `absl::strings_internal::OStringStream`.\nusing StringOStream = StringifyOStream<StringStringifySink>;\n\n// Implementation details follow.\n\ntemplate <typename Sink>\nclass StringifyOStream<Sink>::StringifyStreambuf final : public std::streambuf {\n public:\n  explicit StringifyStreambuf(Sink* dest ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : dest_(dest) {}\n\n  StringifyStreambuf(const StringifyStreambuf& that) = default;\n  StringifyStreambuf& operator=(const StringifyStreambuf& that) = default;\n\n protected:\n  int overflow(int src) override;\n  std::streamsize xsputn(const char* absl_nullable src,\n                         std::streamsize length) override;\n\n private:\n  Sink* dest_;\n};\n\ntemplate <typename Sink>\nint StringifyOStream<Sink>::StringifyStreambuf::overflow(int src) {\n  if (src != traits_type::eof()) {\n    const char ch = static_cast<char>(src);\n    dest_->Append(absl::string_view(&ch, 1));\n  }\n  return traits_type::not_eof(src);\n}\n\ntemplate <typename Sink>\nstd::streamsize StringifyOStream<Sink>::StringifyStreambuf::xsputn(\n    const char* absl_nullable src, std::streamsize length) {\n  assert(length >= 0);\n  assert(src != nullptr || length == 0);\n  dest_->Append(absl::string_view(src, static_cast<size_t>(length)));\n  return length;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_STREAM_UTILS_H_\n"
  },
  {
    "path": "riegeli/base/string_ref.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_STRING_REF_H_\n#define RIEGELI_BASE_STRING_REF_H_\n\n#include <stddef.h>\n\n#include <ostream>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/config.h\"  // IWYU pragma: keep\n#include \"absl/base/nullability.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/temporary_storage.h\"\n#include \"riegeli/base/type_traits.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `StringRef` stores an `absl::string_view`, usually representing text data\n// (see `BytesRef` for binary data), possibly converted through temporary\n// `std::string`.\n//\n// It is intended for function parameters when the implementation needs\n// an `absl::string_view`, and the caller might have another representation\n// of the string.\n//\n// It is convertible from:\n//  * types convertible to `absl::string_view`\n//  * types convertible to `std::string`, e.g. `StringInitializer`\n//\n// `StringRef` does not own string contents and is efficiently copyable.\nclass StringRef : public WithCompare<StringRef> {\n public:\n  // Stores an empty `absl::string_view`.\n  StringRef() = default;\n\n  // Stores `str` converted to `absl::string_view`.\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  /*implicit*/ StringRef(const char* str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : str_(str) {}\n\n  // Stores `str`.\n  /*implicit*/ StringRef(absl::string_view str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : str_(str) {}\n\n  // Stores `str` converted to `absl::string_view`.\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<StringRef, T>,\n                                   NotSameRef<absl::string_view, T>,\n                                   std::is_convertible<T&&, absl::string_view>>,\n                int> = 0>\n  /*implicit*/ StringRef(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : str_(std::forward<T>(str)) {}\n\n  // Stores `str` materialized and then converted to `absl::string_view`.\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<\n                    NotSameRef<StringRef, T>,\n                    std::negation<std::is_convertible<T&&, absl::string_view>>,\n                    std::is_convertible<T&&, std::string>>,\n                int> = 0>\n  /*implicit*/ StringRef(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                         TemporaryStorage<std::string>&& storage\n                             ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : str_(std::move(storage).emplace(std::forward<T>(str))) {}\n\n  StringRef(const StringRef& that) = default;\n  StringRef& operator=(const StringRef&) = delete;\n\n  /*implicit*/ operator absl::string_view() const { return str_; }\n\n  bool empty() const { return size() == 0; }\n  const char* absl_nullable data() const { return str_.data(); };\n  size_t size() const {\n    RIEGELI_ASSUME_LE(str_.size(), str_.max_size());\n    return str_.size();\n  }\n\n  const char& operator[](size_t index) const;\n  const char& at(size_t index) const;\n  const char& front() const;\n  const char& back() const;\n\n  void remove_prefix(size_t length);\n  void remove_suffix(size_t length);\n\n  friend bool operator==(StringRef a, StringRef b) {\n    return absl::string_view(a) == absl::string_view(b);\n  }\n  friend riegeli::StrongOrdering RIEGELI_COMPARE(StringRef a, StringRef b) {\n    return riegeli::Compare(absl::string_view(a), absl::string_view(b));\n  }\n\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<StringRef, T>,\n                                   std::is_convertible<T&&, absl::string_view>>,\n                int> = 0>\n  friend bool operator==(StringRef a, T&& b) {\n    return a == StringRef(std::forward<T>(b));\n  }\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<StringRef, T>,\n                                   std::is_convertible<T&&, absl::string_view>>,\n                int> = 0>\n  friend riegeli::StrongOrdering RIEGELI_COMPARE(StringRef a, T&& b) {\n    return riegeli::Compare(a, StringRef(std::forward<T>(b)));\n  }\n\n  // Default stringification by `absl::StrCat()` etc.\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const StringRef& src) {\n    dest.Append(absl::string_view(src));\n  }\n\n  friend std::ostream& operator<<(std::ostream& dest, const StringRef& src) {\n    return dest << absl::string_view(src);\n  }\n\n private:\n  absl::string_view str_;\n};\n\n// `StringInitializer` is convertible from the same types as `StringRef`,\n// but efficiently takes ownership of `std::string`.\n//\n// `StringInitializer` behaves like `Initializer<std::string>`.\nclass StringInitializer : public Initializer<std::string> {\n public:\n  StringInitializer() = default;\n\n  // Stores `str` converted to `absl::string_view` and then to `std::string`.\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  /*implicit*/ StringInitializer(\n      const char* str ABSL_ATTRIBUTE_LIFETIME_BOUND,\n      TemporaryStorage<MakerType<absl::string_view>>&& storage\n          ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : Initializer(std::move(storage).emplace(absl::string_view(str))) {}\n\n  // Stores `str` converted to `std::string`.\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<StringInitializer, T>,\n                                   std::is_convertible<T&&, std::string>>,\n                int> = 0>\n  /*implicit*/ StringInitializer(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : Initializer(std::forward<T>(str)) {}\n\n  // Stores `str` converted to `StringRef`, then to `absl::string_view`, and\n  // then to `std::string`.\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<\n                           NotSameRef<StringInitializer, T>,\n                           std::negation<std::is_convertible<T&&, std::string>>,\n                           std::is_convertible<T&&, StringRef>>,\n                       int> = 0>\n  /*implicit*/ StringInitializer(\n      T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND,\n      TemporaryStorage<MakerType<absl::string_view>>&& storage\n          ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : Initializer(\n            std::move(storage).emplace(StringRef(std::forward<T>(str)))) {}\n\n  StringInitializer(StringInitializer&& that) = default;\n  StringInitializer& operator=(StringInitializer&&) = delete;\n};\n\n// Implementation details follow.\n\ninline const char& StringRef::operator[](size_t index) const {\n  RIEGELI_ASSERT_LT(index, size())\n      << \"Failed precondition of StringRef::operator[]: index out of range\";\n  return str_[index];\n}\n\ninline const char& StringRef::at(size_t index) const {\n  RIEGELI_ASSERT_LT(index, size())\n      << \"Failed precondition of StringRef::at(): index out of range\";\n  return str_[index];\n}\n\ninline const char& StringRef::front() const {\n  RIEGELI_ASSERT(!empty())\n      << \"Failed precondition of StringRef::front(): empty string\";\n  return str_.front();\n}\n\ninline const char& StringRef::back() const {\n  RIEGELI_ASSERT(!empty())\n      << \"Failed precondition of StringRef::back(): empty string\";\n  return str_.back();\n}\n\ninline void StringRef::remove_prefix(size_t length) {\n  RIEGELI_ASSERT_LE(length, size())\n      << \"Failed precondition of StringRef::remove_prefix(): \"\n         \"length out of range\";\n  str_.remove_prefix(length);\n}\n\ninline void StringRef::remove_suffix(size_t length) {\n  RIEGELI_ASSERT_LE(length, size())\n      << \"Failed precondition of StringRef::remove_suffix(): \"\n         \"length out of range\";\n  str_.remove_suffix(length);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_STRING_REF_H_\n"
  },
  {
    "path": "riegeli/base/string_utils.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/base/string_utils.h\"\n\n#include <stddef.h>\n\n#include <string>\n\n#include \"absl/base/nullability.h\"\n#include \"riegeli/base/arithmetic.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli::string_utils_internal {\n\nvoid ReserveAmortized(std::string& dest, size_t new_capacity) {\n  dest.reserve(dest.capacity() == std::string().capacity()\n                   ? new_capacity\n                   : UnsignedClamp(dest.capacity() + dest.capacity() / 2,\n                                   new_capacity, dest.max_size()));\n}\n\n}  // namespace riegeli::string_utils_internal\n"
  },
  {
    "path": "riegeli/base/string_utils.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_STRING_UTILS_H_\n#define RIEGELI_BASE_STRING_UTILS_H_\n\n#include <stddef.h>\n\n#include <string>\n#include <utility>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/strings/resize_and_overwrite.h\"\n#include \"riegeli/base/assert.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nnamespace string_utils_internal {\n\nvoid ReserveAmortized(std::string& dest, size_t new_capacity);\n\n}  // namespace string_utils_internal\n\n// Like `std::string::reserve()`, ensuring that repeated growth has the cost\n// proportional to the final size.\n//\n// If `new_capacity <= dest.capacity()`, does nothing like in C++20.\n// In earlier C++ versions this was a non-binding shrink request.\ninline void StringReserveAmortized(std::string& dest, size_t new_capacity) {\n  if (new_capacity > dest.capacity()) {\n    string_utils_internal::ReserveAmortized(dest, new_capacity);\n  }\n  RIEGELI_ASSUME_GE(dest.capacity(), new_capacity);\n}\n\n// Like `std::string::resize()`, ensuring that repeated growth has the cost\n// proportional to the final size.\ninline void StringResizeAmortized(std::string& dest, size_t new_size) {\n  StringReserveAmortized(dest, new_size);\n  dest.resize(new_size);\n}\n\n// Like `absl::StringResizeAndOverwrite()`, ensuring that repeated growth has\n// the cost proportional to the final size.\ntemplate <typename Op>\ninline void StringResizeAndOverwriteAmortized(std::string& dest,\n                                              size_t new_size, Op op) {\n  StringReserveAmortized(dest, new_size);\n  return absl::StringResizeAndOverwrite(dest, new_size, std::move(op));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_STRING_UTILS_H_\n"
  },
  {
    "path": "riegeli/base/temporary_storage.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_TEMPORARY_STORAGE_H_\n#define RIEGELI_BASE_TEMPORARY_STORAGE_H_\n\n#include <stdint.h>\n\n#include <new>  // IWYU pragma: keep\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"riegeli/base/assert.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// Internal storage used by functions which return a reference to either an\n// existing object or a newly constructed one.\n//\n// Such functions take a parameter\n//   `TemporaryStorage<T>&& storage ABSL_ATTRIBUTE_LIFETIME_BOUND = {}`\n// so that the default value is allocated as a temporary by the caller.\n//\n// The parameter can also be passed explicitly if a call to these functions\n// happens in a context which needs the returned reference to be valid longer\n// than the full expression containing the call. This passes the responsibility\n// for passing a `TemporaryStorage<T>` with a suitable lifetime to the caller of\n// that context.\ntemplate <typename T, typename Enable = void>\nclass TemporaryStorage {\n public:\n  TemporaryStorage() noexcept {}\n\n  TemporaryStorage(const TemporaryStorage&) = delete;\n  TemporaryStorage& operator=(const TemporaryStorage&) = delete;\n\n  ~TemporaryStorage() {\n    if (state_ != State::kUninitialized) value_.~T();\n  }\n\n  template <typename... Args,\n            std::enable_if_t<std::is_constructible_v<T, Args&&...>, int> = 0>\n      T& emplace(Args&&... args) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_EQ(state_, State::kUninitialized)\n        << \"Failed precondition of TemporaryStorage::emplace()\";\n    new (&value_) T(std::forward<Args>(args)...);\n    state_ = State::kValid;\n    return value_;\n  }\n  template <typename... Args,\n            std::enable_if_t<std::is_constructible_v<T, Args&&...>, int> = 0>\n      T&& emplace(Args&&... args) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_EQ(state_, State::kUninitialized)\n        << \"Failed precondition of TemporaryStorage::emplace()\";\n    new (&value_) T(std::forward<Args>(args)...);\n    state_ = State::kMovedFrom;\n    return std::move(value_);\n  }\n\n  T& operator*() & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_EQ(state_, State::kValid)\n        << \"Failed precondition of TemporaryStorage::operator*\";\n    return value_;\n  }\n  const T& operator*() const& ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_EQ(state_, State::kValid)\n        << \"Failed precondition of TemporaryStorage::operator*\";\n    return value_;\n  }\n  T&& operator*() && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_EQ(state_, State::kValid)\n        << \"Failed precondition of TemporaryStorage::operator*\";\n    state_ = State::kMovedFrom;\n    return std::move(value_);\n  }\n  const T&& operator*() const&& ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_EQ(state_, State::kValid)\n        << \"Failed precondition of TemporaryStorage::operator*\";\n    state_ = State::kMovedFrom;\n    return std::move(value_);\n  }\n\n private:\n  enum class State : uint8_t {\n    kUninitialized = 0,\n    kValid = 1,\n    kMovedFrom = 2,\n  };\n\n  union {\n    std::remove_cv_t<T> value_;\n  };\n  mutable State state_ = State::kUninitialized;\n};\n\n// Specialization of `TemporaryStorage<T>` for non-reference trivially\n// destructible but not trivially default constructible types. There is no need\n// to track whether the object was initialized.\ntemplate <typename T>\nclass TemporaryStorage<\n    T,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<std::is_reference<T>>, std::is_trivially_destructible<T>,\n        std::negation<std::is_trivially_default_constructible<T>>>>> {\n public:\n  TemporaryStorage() noexcept {}\n\n  TemporaryStorage(const TemporaryStorage&) = delete;\n  TemporaryStorage& operator=(const TemporaryStorage&) = delete;\n\n  template <typename... Args,\n            std::enable_if_t<std::is_constructible_v<T, Args&&...>, int> = 0>\n      T& emplace(Args&&... args) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    new (&value_) T(std::forward<Args>(args)...);\n    return value_;\n  }\n  template <typename... Args,\n            std::enable_if_t<std::is_constructible_v<T, Args&&...>, int> = 0>\n      T&& emplace(Args&&... args) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    new (&value_) T(std::forward<Args>(args)...);\n    return std::move(value_);\n  }\n\n  T& operator*() & ABSL_ATTRIBUTE_LIFETIME_BOUND { return value_; }\n  const T& operator*() const& ABSL_ATTRIBUTE_LIFETIME_BOUND { return value_; }\n  T&& operator*() && ABSL_ATTRIBUTE_LIFETIME_BOUND { return std::move(value_); }\n  const T&& operator*() const&& ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(value_);\n  }\n\n private:\n  union {\n    std::remove_cv_t<T> value_;\n  };\n};\n\n// Specialization of `TemporaryStorage<T>` for non-reference trivially\n// destructible and trivially default constructible types. There is no need to\n// track whether the object was initialized, and\n// `ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS` can be applied.\ntemplate <typename T>\nclass TemporaryStorage<\n    T,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<std::is_reference<T>>, std::is_trivially_destructible<T>,\n        std::is_trivially_default_constructible<T>>>> {\n public:\n  TemporaryStorage() = default;\n\n  TemporaryStorage(const TemporaryStorage&) = delete;\n  TemporaryStorage& operator=(const TemporaryStorage&) = delete;\n\n  template <typename... Args,\n            std::enable_if_t<std::is_constructible_v<T, Args&&...>, int> = 0>\n      T& emplace(Args&&... args) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    new (&value_) T(std::forward<Args>(args)...);\n    return value_;\n  }\n  template <typename... Args,\n            std::enable_if_t<std::is_constructible_v<T, Args&&...>, int> = 0>\n      T&& emplace(Args&&... args) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    new (&value_) T(std::forward<Args>(args)...);\n    return std::move(value_);\n  }\n\n  T& operator*() & ABSL_ATTRIBUTE_LIFETIME_BOUND { return value_; }\n  const T& operator*() const& ABSL_ATTRIBUTE_LIFETIME_BOUND { return value_; }\n  T&& operator*() && ABSL_ATTRIBUTE_LIFETIME_BOUND { return std::move(value_); }\n  const T&& operator*() const&& ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(value_);\n  }\n\n private:\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS std::remove_cv_t<T> value_;\n};\n\n// Specialization of `TemporaryStorage<T>` for reference types.\ntemplate <typename T>\nclass TemporaryStorage<T, std::enable_if_t<std::is_reference_v<T>>> {\n public:\n  TemporaryStorage() = default;\n\n  TemporaryStorage(const TemporaryStorage&) = delete;\n  TemporaryStorage& operator=(const TemporaryStorage&) = delete;\n\n  template <typename Arg,\n            std::enable_if_t<std::is_convertible_v<Arg&&, T>, int> = 0>\n      T& emplace(Arg&& arg) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    value_ = &arg;\n    return *value_;\n  }\n  template <typename Arg,\n            std::enable_if_t<std::is_convertible_v<Arg&&, T>, int> = 0>\n      T&& emplace(Arg&& arg) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    value_ = &arg;\n    return std::forward<T>(*value_);\n  }\n\n  T& operator*() & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT(value_ != nullptr)\n        << \"Failed precondition of TemporaryStorage::operator*\";\n    return *value_;\n  }\n  const T& operator*() const& ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT(value_ != nullptr)\n        << \"Failed precondition of TemporaryStorage::operator*\";\n    return *value_;\n  }\n  T&& operator*() && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT(value_ != nullptr)\n        << \"Failed precondition of TemporaryStorage::operator*\";\n    return std::forward<T>(*value_);\n  }\n  const T&& operator*() const&& ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT(value_ != nullptr)\n        << \"Failed precondition of TemporaryStorage::operator*\";\n    return std::forward<T>(*value_);\n  }\n\n private:\n  std::remove_reference_t<T>* absl_nullable value_;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_TEMPORARY_STORAGE_H_\n"
  },
  {
    "path": "riegeli/base/type_erased_ref.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_TYPE_ERASED_REF_H_\n#define RIEGELI_BASE_TYPE_ERASED_REF_H_\n\n#include <memory>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/meta/type_traits.h\"\n#include \"riegeli/base/type_traits.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `TypeErasedRef` wraps a reference, and allows to recover the original\n// reference as long as its original type is provided.\n//\n// `TypeErasedRef(std::forward<T>(value)).Cast<T>()` recovers the value of\n// `std::forward<T>(value)`.\n//\n// This is like converting the reference to a pointer and casting it to `void*`,\n// and casting back for recovery, but this correctly handles const references\n// and references to functions.\n//\n// Specifying `T` or `T&&` for recovery is interchangeable.\nclass TypeErasedRef {\n private:\n  template <typename T>\n  struct IsFunctionRef : std::false_type {};\n\n  template <typename T>\n  struct IsFunctionRef<T&> : std::is_function<T> {};\n\n public:\n  // Creates an empty `TypeErasedRef`. It cannot be recovered as any type.\n  //\n  // Conversion from `std::nullptr_t` is not supported because that binds\n  // to a reference to `nullptr` instead of being empty.\n  TypeErasedRef() = default;\n\n  // Wraps `std::forward<T>(value)`.\n  template <typename T, std::enable_if_t<\n                            std::conjunction_v<NotSameRef<TypeErasedRef, T>,\n                                               std::negation<IsFunctionRef<T>>>,\n                            int> = 0>\n  explicit TypeErasedRef(T&& value ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : ptr_(const_cast<absl::remove_cvref_t<T>*>(std::addressof(value))) {}\n\n  // Wraps a function reference.\n  //\n  // The implementation relies on the assumption that a function pointer can be\n  // `reinterpret_cast` to `void*` and back.\n  template <typename T,\n            std::enable_if_t<\n                // `NotSameRef` is not needed because `T` is a function\n                // reference, so it is never `TypeErasedRef`.\n                IsFunctionRef<T>::value, int> = 0>\n  explicit TypeErasedRef(T&& value) : ptr_(reinterpret_cast<void*>(&value)) {}\n\n  TypeErasedRef(const TypeErasedRef& that) = default;\n  TypeErasedRef& operator=(const TypeErasedRef& that) = default;\n\n  // Recovers the `T&&`.\n  template <typename T>\n  T&& Cast() const {\n    if constexpr (!IsFunctionRef<T>::value) {\n      return std::forward<T>(\n          *reinterpret_cast<std::remove_reference_t<T>*>(ptr_));\n    } else {\n      return *reinterpret_cast<std::remove_reference_t<T>*>(ptr_);\n    }\n  }\n\n  // Returns `true` if the `TypeErasedRef` is empty, i.e. default-constructed.\n  bool empty() const { return ptr_ == nullptr; }\n\n private:\n  void* absl_nullable ptr_ = nullptr;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_TYPE_ERASED_REF_H_\n"
  },
  {
    "path": "riegeli/base/type_id.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_TYPE_ID_H_\n#define RIEGELI_BASE_TYPE_ID_H_\n\n#include <cstddef>\n#include <functional>\n#include <utility>\n\n#include \"absl/base/nullability.h\"\n#include \"riegeli/base/compare.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `TypeId::For<A>()` is a token which is equal to `TypeId::For<B>()` whenever\n// `A` and `B` are the same type.\n//\n// `TypeId()` is another value not equal to any other.\nclass ABSL_NULLABILITY_COMPATIBLE TypeId : public WithCompare<TypeId> {\n public:\n  constexpr TypeId() = default;\n  /*implicit*/ constexpr TypeId(std::nullptr_t) noexcept {}\n\n  TypeId(const TypeId& that) = default;\n  TypeId& operator=(const TypeId& that) = default;\n\n  template <typename T>\n  static constexpr TypeId For();\n\n  friend constexpr bool operator==(TypeId a, TypeId b) {\n    return a.ptr_ == b.ptr_;\n  }\n  friend StrongOrdering RIEGELI_COMPARE(TypeId a, TypeId b) {\n    if (std::less<>()(a.ptr_, b.ptr_)) return StrongOrdering::less;\n    if (std::greater<>()(a.ptr_, b.ptr_)) return StrongOrdering::greater;\n    return StrongOrdering::equal;\n  }\n\n  template <typename HashState>\n  friend HashState AbslHashValue(HashState hash_state, TypeId self) {\n    return HashState::combine(std::move(hash_state), self.ptr_);\n  }\n\n private:\n  using pointer = void*;  // For `ABSL_NULLABILITY_COMPATIBLE`.\n\n  template <typename T>\n  struct TypeIdToken;\n\n  explicit constexpr TypeId(const void* ptr) : ptr_(ptr) {}\n\n  const void* absl_nullable ptr_ = nullptr;\n};\n\n// Implementation details follow.\n\ntemplate <typename T>\nstruct TypeId::TypeIdToken {\n  static const char token;\n};\n\ntemplate <typename T>\nconst char TypeId::TypeIdToken<T>::token = '\\0';\n\ntemplate <typename T>\nconstexpr TypeId TypeId::For() {\n  return TypeId(&TypeIdToken<T>::token);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_TYPE_ID_H_\n"
  },
  {
    "path": "riegeli/base/type_traits.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_TYPE_TRAITS_H_\n#define RIEGELI_BASE_TYPE_TRAITS_H_\n\n#include <stddef.h>\n\n#include <functional>\n#include <string>\n#include <tuple>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/utility/utility.h\"  // IWYU pragma: keep\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `type_identity<T>::type` and `type_identity_t<T>` are `T`, but do not deduce\n// the `T` in templates.\n\ntemplate <typename T>\nstruct type_identity {\n  using type = T;\n};\n\ntemplate <typename T>\nusing type_identity_t = typename type_identity<T>::type;\n\n// `unwrap_reference<T>::type` and `unwrap_reference_t<T>` changes\n// `std::reference_wrapper<U>` to `U&`, leaving other types unchanged.\n//\n// This is `std::unwrap_reference` from C++20.\n\n#if __cpp_lib_unwrap_ref\n\nusing std::unwrap_reference;\nusing std::unwrap_reference_t;\n\n#else\n\ntemplate <typename T>\nstruct unwrap_reference : type_identity<T> {};\n\ntemplate <typename T>\nstruct unwrap_reference<std::reference_wrapper<T>> : type_identity<T&> {};\n\ntemplate <typename T>\nusing unwrap_reference_t = typename unwrap_reference<T>::type;\n\n#endif\n\n// `unwrap_ref_decay<T>::type` and `unwrap_ref_decay_t<T>` changes\n// `std::reference_wrapper<U>` to `U&`, and other types like `std::decay`.\n//\n// This is `std::unwrap_ref_decay` from C++20.\n\n#if __cpp_lib_unwrap_ref\n\nusing std::unwrap_ref_decay;\nusing std::unwrap_ref_decay_t;\n\n#else\n\ntemplate <typename T>\nstruct unwrap_ref_decay : unwrap_reference<std::decay<T>> {};\n\ntemplate <typename T>\nusing unwrap_ref_decay_t = typename unwrap_ref_decay<T>::type;\n\n#endif\n\n// `IsConvertibleFromResult<T, Result>` is like\n// `std::is_convertible<Result, T>`, except that `Result` represents the\n// result of a function: due to copy elision, `T` and `Result` can also be the\n// same immovable type, possibly with different qualifiers.\ntemplate <typename T, typename Result>\nstruct IsConvertibleFromResult\n    : std::disjunction<\n          std::is_same<std::remove_cv_t<T>, std::remove_cv_t<Result>>,\n          std::is_convertible<Result, T>> {};\n\n// `IsConstructibleFromResult<T, Result>` is like\n// `std::is_constructible<T, Result>`, except that `Result` represents the\n// result of a function: due to copy elision, `T` and `Result` can also be the\n// same immovable type, possibly with different qualifiers.\ntemplate <typename T, typename Result>\nstruct IsConstructibleFromResult\n    : std::disjunction<\n          std::is_same<std::remove_cv_t<T>, std::remove_cv_t<Result>>,\n          std::is_constructible<T, Result>> {};\n\n// `SameRef<Self, Args...>::value` is `true` if a constructor or assignment of\n// `Self` from a reference to `Args...` would conflict with the copy or move\n// constructor or assignment.\n//\n// This means a single argument which is reference to `Self` or a class derived\n// from `Self`, ignoring `const`.\n\ntemplate <typename Self, typename... Args>\nstruct SameRef : std::false_type {};\n\ntemplate <typename Self, typename Arg>\nstruct SameRef<Self, Arg> : std::is_convertible<std::decay_t<Arg>*, Self*> {};\n\n// `NotSameRef` is the negation of `SameRef`.\n//\n// This should be included in constraints of a templated constructor or\n// assignment where such a conflict is possible. This makes argument types\n// compatible with copying or moving interpreted as the copy or move,\n// instead of passing them to the templated constructor or assignment.\ntemplate <typename Self, typename... Args>\nstruct NotSameRef : std::negation<SameRef<Self, Args...>> {};\n\nnamespace type_traits_internal {\n\n// Transforms a `std::tuple` type to another `std::tuple` type by selecting\n// element types corresponding to the given `std::index_sequence`.\ntemplate <typename Tuple, typename Indices>\nstruct SelectTypesFromTuple;\n\ntemplate <typename Tuple, size_t... indices>\nstruct SelectTypesFromTuple<Tuple, std::index_sequence<indices...>> {\n  using type = std::tuple<std::tuple_element_t<indices, Tuple>...>;\n};\n\n// Selects element types from a `std::tuple` type corresponding to the given\n// `std::index_sequence`.\ntemplate <typename Tuple, size_t... indices>\nstd::tuple<std::tuple_element_t<indices, Tuple>...> SelectFromTuple(\n    ABSL_ATTRIBUTE_UNUSED Tuple&& tuple, std::index_sequence<indices...>) {\n  return {std::forward<std::tuple_element_t<indices, Tuple>>(\n      std::get<indices>(tuple))...};\n}\n\n// SFINAE-friendly helper for `GetTypeFromEnd`.\ntemplate <typename Enable, size_t reverse_index, typename... T>\nstruct GetTypeFromEndImpl {};\ntemplate <size_t reverse_index, typename... T>\nstruct GetTypeFromEndImpl<\n    std::enable_if_t<(reverse_index > 0 && reverse_index <= sizeof...(T))>,\n    reverse_index, T...>\n    : std::tuple_element<sizeof...(T) - reverse_index, std::tuple<T...>> {};\n\n// SFINAE-friendly helper for `RemoveTypesFromEnd`.\ntemplate <typename Enable, size_t num_from_end, typename... T>\nstruct RemoveTypesFromEndImpl {};\ntemplate <size_t num_from_end, typename... T>\nstruct RemoveTypesFromEndImpl<std::enable_if_t<(num_from_end <= sizeof...(T))>,\n                              num_from_end, T...>\n    : SelectTypesFromTuple<std::tuple<T...>, std::make_index_sequence<\n                                                 sizeof...(T) - num_from_end>> {\n};\n\n// Concatenates `std::index_sequence` types.\ntemplate <typename... index_sequences>\nstruct ConcatIndexSequences;\n\ntemplate <>\nstruct ConcatIndexSequences<> {\n  using type = std::index_sequence<>;\n};\ntemplate <typename index_sequence>\nstruct ConcatIndexSequences<index_sequence> {\n  using type = index_sequence;\n};\ntemplate <size_t... indices1, size_t... indices2, typename... index_sequences>\nstruct ConcatIndexSequences<std::index_sequence<indices1...>,\n                            std::index_sequence<indices2...>,\n                            index_sequences...> {\n  using type = typename ConcatIndexSequences<\n      std::index_sequence<indices1..., indices2...>, index_sequences...>::type;\n};\n\n// Transforms a tuple type to a `std::index_sequence` type of indices of\n// elements satisfying a predicate.\ntemplate <template <typename...> class Predicate, typename Tuple,\n          typename Indices>\nstruct FilterTypeImpl;\n\ntemplate <template <typename...> class Predicate, typename Tuple,\n          size_t... indices>\nstruct FilterTypeImpl<Predicate, Tuple, std::index_sequence<indices...>>\n    : type_traits_internal::ConcatIndexSequences<std::conditional_t<\n          Predicate<std::tuple_element_t<indices, Tuple>>::value,\n          std::index_sequence<indices>, std::index_sequence<>>...> {};\n\n}  // namespace type_traits_internal\n\n// `GetTypeFromEnd<reverse_index, T...>::type` and\n// `GetTypeFromEndT<reverse_index, T...>` extract a type from a parameter pack\n// by its index from the end (1 = last).\ntemplate <size_t reverse_index, typename... T>\nstruct GetTypeFromEnd\n    : type_traits_internal::GetTypeFromEndImpl<void, reverse_index, T...> {};\ntemplate <size_t reverse_index, typename... T>\nusing GetTypeFromEndT = typename GetTypeFromEnd<reverse_index, T...>::type;\n\n// `GetFromEnd<reverse_index>(args...)` extracts an argument from a sequence of\n// arguments by its index from the end (1 = last).\ntemplate <size_t reverse_index, typename... Args,\n          std::enable_if_t<\n              (reverse_index > 0 && reverse_index <= sizeof...(Args)), int> = 0>\ninline GetTypeFromEndT<reverse_index, Args&&...> GetFromEnd(Args&&... args) {\n  return std::get<sizeof...(Args) - reverse_index>(\n      std::tuple<Args&&...>(std::forward<Args>(args)...));\n}\n\n// `RemoveTypesFromEnd<num_from_end, T...>::type` and\n// `RemoveTypesFromEndT<num_from_end, T...>` transform a parameter pack to a\n// `std::tuple` type by removing the given number of elements from the end.\ntemplate <size_t num_from_end, typename... T>\nstruct RemoveTypesFromEnd\n    : type_traits_internal::RemoveTypesFromEndImpl<void, num_from_end, T...> {};\ntemplate <size_t num_from_end, typename... T>\nusing RemoveTypesFromEndT =\n    typename RemoveTypesFromEnd<num_from_end, T...>::type;\n\n// `RemoveFromEnd<num_from_end>(args...)` transforms a sequence of arguments to\n// a `std::tuple` by removing the given number of arguments from the end.\ntemplate <size_t num_from_end, typename... Args,\n          std::enable_if_t<(num_from_end <= sizeof...(Args)), int> = 0>\ninline RemoveTypesFromEndT<num_from_end, Args&&...> RemoveFromEnd(\n    Args&&... args) {\n  return type_traits_internal::SelectFromTuple(\n      std::tuple<Args&&...>(std::forward<Args>(args)...),\n      std::make_index_sequence<sizeof...(Args) - num_from_end>());\n}\n\n// `ApplyToTupleElements<F, std::tuple<T...>>::type` and\n// `ApplyToTupleElementsT<F, std::tuple<T...>>` is `F<T...>`.\ntemplate <template <typename... Args> class F, typename Tuple>\nstruct ApplyToTupleElements;\ntemplate <template <typename... Args> class F, typename... T>\nstruct ApplyToTupleElements<F, std::tuple<T...>> {\n  using type = F<T...>;\n};\ntemplate <template <typename... Args> class F, typename Tuple>\nusing ApplyToTupleElementsT = typename ApplyToTupleElements<F, Tuple>::type;\n\n// `TupleElementsSatisfy<Tuple, Predicate>::value` checks if all element types\n// of a `std::tuple` type satisfy a predicate.\ntemplate <typename Tuple, template <typename...> class Predicate>\nstruct TupleElementsSatisfy;\n\ntemplate <typename... T, template <typename...> class Predicate>\nstruct TupleElementsSatisfy<std::tuple<T...>, Predicate>\n    : std::conjunction<Predicate<T>...> {};\n\n// `FilterType<Predicate, T...>::type` and\n// `FilterTypeT<Predicate, T...>` transform a parameter pack to a `std::tuple`\n// type by selecting types satisfying a predicate.\ntemplate <template <typename...> class Predicate, typename... T>\nstruct FilterType\n    : type_traits_internal::SelectTypesFromTuple<\n          std::tuple<T...>, typename type_traits_internal::FilterTypeImpl<\n                                Predicate, std::tuple<T...>,\n                                std::index_sequence_for<T...>>::type> {};\ntemplate <template <typename...> class Predicate, typename... T>\nusing FilterTypeT = typename FilterType<Predicate, T...>::type;\n\n// `Filter<Predicate>(args...)` transforms a sequence of arguments to a\n// `std::tuple` by selecting types satisfying a predicate.\ntemplate <template <typename...> class Predicate, typename... Args>\ninline FilterTypeT<Predicate, Args&&...> Filter(Args&&... args) {\n  return type_traits_internal::SelectFromTuple(\n      std::tuple<Args&&...>(std::forward<Args>(args)...),\n      typename type_traits_internal::FilterTypeImpl<\n          Predicate, std::tuple<Args&&...>,\n          std::index_sequence_for<Args&&...>>::type());\n}\n\n// `DecayTupleType<Tuple>::type` and `DecayTupleTypeT<Tuple>` transform a\n// `std::tuple` type by decaying all elements from references to values.\ntemplate <typename Tuple>\nstruct DecayTupleType;\n\ntemplate <typename... T>\nstruct DecayTupleType<std::tuple<T...>> {\n  using type = std::tuple<std::decay_t<T>...>;\n};\ntemplate <typename Tuple>\nusing DecayTupleTypeT = typename DecayTupleType<Tuple>::type;\n\n// `DecayTuple(tuple)` transforms a `std::tuple` by decaying all elements from\n// references to values.\ntemplate <typename Tuple>\ninline DecayTupleTypeT<Tuple> DecayTuple(Tuple&& tuple) {\n  return tuple;\n}\n\n// `DeduceClassTemplateArguments<Template, Args...>::type` and\n// `DeduceClassTemplateArgumentsT<Template, Args...>` deduce class template\n// arguments using CTAD from constructor arguments.\n//\n// Only class templates with solely type template parameters are supported.\n\ntemplate <template <typename...> class Template, typename... Args>\nstruct DeduceClassTemplateArguments {\n  using type = decltype(Template(std::declval<Args>()...));\n};\n\ntemplate <template <typename...> class Template, typename... Args>\nusing DeduceClassTemplateArgumentsT =\n    typename DeduceClassTemplateArguments<Template, Args...>::type;\n\n// `IntersectionType<Ts...>::type` and `IntersectionTypeT<Ts...>` compute the\n// smallest of unsigned integer types.\n\nnamespace type_traits_internal {\n\ntemplate <typename A, typename B, typename Common>\nstruct IntersectionTypeImpl;\n\ntemplate <typename A, typename B>\nstruct IntersectionTypeImpl<A, B, A> {\n  using type = B;\n};\n\ntemplate <typename A, typename B>\nstruct IntersectionTypeImpl<A, B, B> {\n  using type = A;\n};\n\ntemplate <typename A>\nstruct IntersectionTypeImpl<A, A, A> {\n  using type = A;\n};\n\n}  // namespace type_traits_internal\n\ntemplate <typename... T>\nstruct IntersectionType;\n\ntemplate <typename... T>\nusing IntersectionTypeT = typename IntersectionType<T...>::type;\n\ntemplate <typename A>\nstruct IntersectionType<A> {\n  using type = A;\n};\n\ntemplate <typename A, typename B>\nstruct IntersectionType<A, B>\n    : type_traits_internal::IntersectionTypeImpl<A, B,\n                                                 std::common_type_t<A, B>> {};\n\ntemplate <typename A, typename B, typename... Rest>\nstruct IntersectionType<A, B, Rest...>\n    : IntersectionType<IntersectionTypeT<A, B>, Rest...> {};\n\n// `HasDereference<T>::value` is `true` if a value of type `T` can be\n// dereferenced with `operator*`.\n\ntemplate <typename T, typename Enable = void>\nstruct HasDereference : std::false_type {};\n\ntemplate <typename T>\nstruct HasDereference<T, std::void_t<decltype(*std::declval<T>())>>\n    : std::true_type {};\n\n// `HasArrow<T>::value` is `true` if a value of type `T` can be dereferenced\n// with `operator->`.\n\ntemplate <typename T, typename Enable = void>\nstruct HasArrow : std::false_type {};\n\ntemplate <typename T>\nstruct HasArrow<T, std::enable_if_t<std::is_pointer_v<\n                       std::decay_t<decltype(std::declval<T>())>>>>\n    : std::true_type {};\n\ntemplate <typename T>\nstruct HasArrow<T, std::void_t<decltype(std::declval<T>().operator->())>>\n    : std::true_type {};\n\n// `IsComparableAgainstNullptr<T>::value` is `true` if a value of type `T` can\n// be compared against `nullptr`.\n\ntemplate <typename T, typename Enable = void>\nstruct IsComparableAgainstNullptr : std::false_type {};\n\ntemplate <typename CharT, typename Traits, typename Alloc>\nstruct IsComparableAgainstNullptr<std::basic_string<CharT, Traits, Alloc>>\n    : std::false_type {};\n\ntemplate <>\nstruct IsComparableAgainstNullptr<absl::string_view> : std::false_type {};\n\ntemplate <typename T>\nstruct IsComparableAgainstNullptr<\n    T, std::enable_if_t<\n           std::is_convertible_v<decltype(std::declval<T>() == nullptr), bool>>>\n    : std::true_type {};\n\n// Deriving a class from\n// `ConditionallyConstructible<copy_constructible, move_constructible>`\n// disables copy constructor if `!copy_constructible`, and\n// disables move constructor if `!move_constructible`.\n//\n// A derived class should have the constructors defaulted, so that they get\n// effectively conditionally defaulted or deleted. An explicit definition of\n// constructors in the derived class would make them available unconditionally;\n// if explicit definitions are desired, they must come from another base class.\n//\n// This is useful when explicitly defined constructors are only conditionally\n// valid (SFINAE is not applicable to special member functions). Since C++20,\n// `requires` can be used instead.\n//\n// This is also useful, together with `ConditionallyAssignable`, if the class\n// has a member pointing to another member. With defaulted constructors and\n// assignments, the pointer would not be repositioned, while implementing the\n// repositioning explicitly is not always feasible.\n\ntemplate <bool copy_constructible, bool move_constructible = copy_constructible>\nclass ConditionallyConstructible {};\n\ntemplate <>\nclass ConditionallyConstructible<false, false> {\n public:\n  ConditionallyConstructible() = default;\n\n  ConditionallyConstructible(const ConditionallyConstructible&) = delete;\n  ConditionallyConstructible(ConditionallyConstructible&&) = delete;\n\n  ConditionallyConstructible& operator=(const ConditionallyConstructible&) =\n      default;\n  ConditionallyConstructible& operator=(ConditionallyConstructible&&) = default;\n};\n\ntemplate <>\nclass ConditionallyConstructible<false, true> {\n public:\n  ConditionallyConstructible() = default;\n\n  ConditionallyConstructible(const ConditionallyConstructible&) = delete;\n  ConditionallyConstructible(ConditionallyConstructible&&) = default;\n\n  ConditionallyConstructible& operator=(const ConditionallyConstructible&) =\n      default;\n  ConditionallyConstructible& operator=(ConditionallyConstructible&&) = default;\n};\n\ntemplate <>\nclass ConditionallyConstructible<true, true> {\n public:\n  ConditionallyConstructible() = default;\n\n  ConditionallyConstructible(const ConditionallyConstructible&) = default;\n  ConditionallyConstructible(ConditionallyConstructible&&) = default;\n\n  ConditionallyConstructible& operator=(const ConditionallyConstructible&) =\n      default;\n  ConditionallyConstructible& operator=(ConditionallyConstructible&&) = default;\n};\n\n// Deriving a class from\n// `ConditionallyAssignable<copy_assignable, move_assignable>`\n// disables copy assignment if `!copy_assignable`, and\n// disables move assignment if `!move_assignable`.\n//\n// A derived class should have the assignments defaulted, so that they get\n// effectively conditionally defaulted or deleted. An explicit definition of\n// assignments in the derived class would make them available unconditionally;\n// if explicit definitions are desired, they must come from another base class.\n//\n// This is useful when explicitly defined assignments are only conditionally\n// valid (SFINAE is not applicable to special member functions). Since C++20,\n// `requires` can be used instead.\n//\n// This is also useful if the class has a `std::tuple` member whose elements\n// can be references. While a direct reference member would disable defaulted\n// assignments, a `std::tuple` containing references assigns through reference\n// elements, which might have undesirable semantics.\n\ntemplate <bool copy_assignable, bool move_assignable = copy_assignable>\nclass ConditionallyAssignable;\n\ntemplate <>\nclass ConditionallyAssignable<false, false> {\n public:\n  ConditionallyAssignable() = default;\n\n  ConditionallyAssignable(const ConditionallyAssignable&) = default;\n  ConditionallyAssignable(ConditionallyAssignable&&) = default;\n\n  ConditionallyAssignable& operator=(const ConditionallyAssignable&) = delete;\n  ConditionallyAssignable& operator=(ConditionallyAssignable&&) = delete;\n};\n\ntemplate <>\nclass ConditionallyAssignable<false, true> {\n public:\n  ConditionallyAssignable() = default;\n\n  ConditionallyAssignable(const ConditionallyAssignable&) = default;\n  ConditionallyAssignable(ConditionallyAssignable&&) = default;\n\n  ConditionallyAssignable& operator=(const ConditionallyAssignable&) = delete;\n  ConditionallyAssignable& operator=(ConditionallyAssignable&&) = default;\n};\n\ntemplate <>\nclass ConditionallyAssignable<true, true> {\n public:\n  ConditionallyAssignable() = default;\n\n  ConditionallyAssignable(const ConditionallyAssignable&) = default;\n  ConditionallyAssignable(ConditionallyAssignable&&) = default;\n\n  ConditionallyAssignable& operator=(const ConditionallyAssignable&) = default;\n  ConditionallyAssignable& operator=(ConditionallyAssignable&&) = default;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_TYPE_TRAITS_H_\n"
  },
  {
    "path": "riegeli/base/types.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_TYPES_H_\n#define RIEGELI_BASE_TYPES_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <ios>\n#include <type_traits>\n\n#include \"absl/base/nullability.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// Position in a stream of bytes, used also for stream sizes.\n//\n// This is an unsigned integer type at least as wide as `size_t`,\n// `std::streamoff`, and `uint64_t`.\nusing Position =\n    std::common_type_t<size_t, std::make_unsigned_t<std::streamoff>, uint64_t>;\n\n// Specifies the scope of objects to flush and the intended data durability\n// (without a guarantee).\nenum class FlushType {\n  // Makes data written so far visible in other objects, propagating flushing\n  // through owned dependencies of the given writer.\n  kFromObject = 0,\n  // Makes data written so far visible outside the process, propagating flushing\n  // through dependencies of the given writer. This is generally the default.\n  kFromProcess = 1,\n  // Makes data written so far visible outside the process and durable in case\n  // of operating system crash, propagating flushing through dependencies of the\n  // given writer.\n  kFromMachine = 2,\n};\n\n// Specifies the scope of objects to synchronize.\nenum class SyncType {\n  // Propagates synchronization through owned dependencies of the given reader.\n  kFromObject = 0,\n  // Propagates synchronization through all dependencies of the given reader.\n  // This is generally the default.\n  kFromProcess = 1,\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_TYPES_H_\n"
  },
  {
    "path": "riegeli/base/unicode.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifdef _WIN32\n#define WIN32_LEAN_AND_MEAN\n#endif\n\n#include \"riegeli/base/unicode.h\"  // IWYU pragma: keep\n\n#ifdef _WIN32\n\n#include <stddef.h>\n#include <windows.h>\n\n#include <limits>\n#include <string>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/resize_and_overwrite.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nbool Utf8ToWide(absl::string_view src, std::wstring& dest) {\n  dest.clear();\n  if (src.empty()) return true;\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         unsigned{std::numeric_limits<int>::max()})) {\n    return false;\n  }\n  const int dest_size =\n      MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, src.data(),\n                          IntCast<int>(src.size()), nullptr, 0);\n  if (ABSL_PREDICT_FALSE(dest_size == 0)) return false;\n  dest.resize(IntCast<size_t>(dest_size));\n  MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, src.data(),\n                      IntCast<int>(src.size()), dest.data(), dest_size);\n  return true;\n}\n\nbool WideToUtf8(absl::Span<const wchar_t> src, std::string& dest) {\n  dest.clear();\n  if (src.empty()) return true;\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         unsigned{std::numeric_limits<int>::max()})) {\n    return false;\n  }\n  const int dest_size = WideCharToMultiByte(\n      CP_UTF8, WC_ERR_INVALID_CHARS, src.data(), IntCast<int>(src.size()),\n      nullptr, 0, nullptr, nullptr);\n  if (ABSL_PREDICT_FALSE(dest_size == 0)) return false;\n  absl::StringResizeAndOverwrite(\n      dest, IntCast<size_t>(dest_size), [&](char* data, size_t size) {\n        WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, src.data(),\n                            IntCast<int>(src.size()), data, IntCast<int>(size),\n                            nullptr, nullptr);\n        return size;\n      });\n  return true;\n}\n\nstd::string WideToUtf8Lossy(absl::Span<const wchar_t> src) {\n  std::string dest;\n  if (src.empty()) return dest;\n  const int dest_size = WideCharToMultiByte(CP_UTF8, 0, src.data(),\n                                            SaturatingIntCast<int>(src.size()),\n                                            nullptr, 0, nullptr, nullptr);\n  absl::StringResizeAndOverwrite(\n      dest, IntCast<size_t>(dest_size), [&](char* data, size_t size) {\n        WideCharToMultiByte(CP_UTF8, 0, src.data(),\n                            SaturatingIntCast<int>(src.size()), data,\n                            IntCast<int>(size), nullptr, nullptr);\n        return size;\n      });\n  return dest;\n}\n\n}  // namespace riegeli\n\n#endif  // _WIN32\n"
  },
  {
    "path": "riegeli/base/unicode.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_UNICODE_H_\n#define RIEGELI_BASE_UNICODE_H_\n\n#ifdef _WIN32\n\n#include <string>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// Converts from UTF-8 string to `wchar_t[]`.\n//\n// Returns `false` on failure.\nbool Utf8ToWide(absl::string_view src, std::wstring& dest);\n\n// Converts from `wchar_t[]` to UTF-8 string.\n//\n// Returns `false` on failure.\nbool WideToUtf8(absl::Span<const wchar_t> src, std::string& dest);\n\n// Converts from `wchar_t[]` to UTF-8 string.\n//\n// Emits replacement characters on failure.\nstd::string WideToUtf8Lossy(absl::Span<const wchar_t> src);\n\n}  // namespace riegeli\n\n#endif  // _WIN32\n\n#endif  // RIEGELI_BASE_UNICODE_H_\n"
  },
  {
    "path": "riegeli/base/uninitialized_vector.h",
    "content": "// Copyright 2026 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BASE_UNINITIALIZED_VECTOR_H_\n#define RIEGELI_BASE_UNINITIALIZED_VECTOR_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/container/inlined_vector.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// An allocator which default-initializes a newly constructed element instead of\n// value-initializing it.\n//\n// When used with a container of a trivially constructible type,\n// newly allocated elements are left uninitialized. The container must use\n// `allocator::construct()` to construct the elements.\n//\n// This makes it faster to resize the container. This is meant for the cases\n// when its contents will be filled right after resizing. This also allows\n// detecting usages of unfilled elements by msan.\n//\n// Based on https://howardhinnant.github.io/allocator_boilerplate.html.\ntemplate <typename T>\nclass UninitializedAllocator : public std::allocator<T> {\n public:\n  using UninitializedAllocator::allocator::allocator;\n\n  template <typename U, typename... Args>\n  void construct(U* ptr, Args&&... args) {\n    if constexpr (sizeof...(args) == 0) {\n      ::new (ptr) U;  // Default initialization, not value initialization.\n    } else {\n      ::new (ptr) U(std::forward<Args>(args)...);\n    }\n  }\n};\n\n// Like `std::vector`, but newly allocated elements of trivially-constructible\n// types are left uninitialized.\n//\n// This makes it faster to resize the vector. This is meant for the cases when\n// its contents will be filled right after resizing. This also allows detecting\n// usages of unfilled elements by msan.\ntemplate <typename T>\nusing UninitializedVector = std::vector<T, UninitializedAllocator<T>>;\n\n// Like `absl::InlinedVector`, but newly allocated elements of\n// trivially-constructible types are left uninitialized.\n//\n// This makes it faster to resize the vector. This is meant for the cases when\n// its contents will be filled right after resizing. This also allows detecting\n// usages of unfilled elements by msan.\ntemplate <typename T, size_t inlined_size>\nusing UninitializedInlinedVector =\n    absl::InlinedVector<T, inlined_size, UninitializedAllocator<T>>;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BASE_UNINITIALIZED_VECTOR_H_\n"
  },
  {
    "path": "riegeli/brotli/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"brotli_reader\",\n    srcs = [\"brotli_reader.cc\"],\n    hdrs = [\"brotli_reader.h\"],\n    deps = [\n        \":brotli_allocator\",\n        \":brotli_dictionary\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:shared_ptr\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:pullable_reader\",\n        \"//riegeli/bytes:reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@org_brotli//:brotlicommon\",\n        \"@org_brotli//:brotlidec\",\n    ],\n)\n\ncc_library(\n    name = \"brotli_writer\",\n    srcs = [\"brotli_writer.cc\"],\n    hdrs = [\"brotli_writer.h\"],\n    deps = [\n        \":brotli_allocator\",\n        \":brotli_dictionary\",\n        \":brotli_reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:shared_ptr\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_writer\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@org_brotli//:brotlienc\",\n    ],\n)\n\ncc_library(\n    name = \"brotli_dictionary\",\n    srcs = [\"brotli_dictionary.cc\"],\n    hdrs = [\"brotli_dictionary.h\"],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:shared_ptr\",\n        \"@com_google_absl//absl/base\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n        \"@org_brotli//:brotlicommon\",\n        \"@org_brotli//:brotlienc\",\n    ],\n)\n\ncc_library(\n    name = \"brotli_allocator\",\n    srcs = [\"brotli_allocator.cc\"],\n    hdrs = [\"brotli_allocator.h\"],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \"//riegeli/base:shared_ptr\",\n        \"@org_brotli//:brotlicommon\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/brotli/brotli_allocator.cc",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/brotli/brotli_allocator.h\"\n\n#include <stddef.h>\n\nnamespace riegeli {\n\nnamespace brotli_internal {\n\nvoid* RiegeliBrotliAllocFunc(void* opaque, size_t size) {\n  return static_cast<const BrotliAllocator::Interface*>(opaque)->Alloc(size);\n}\n\nvoid RiegeliBrotliFreeFunc(void* opaque, void* ptr) {\n  static_cast<const BrotliAllocator::Interface*>(opaque)->Free(ptr);\n}\n\n}  // namespace brotli_internal\n\nBrotliAllocator::Interface::~Interface() {}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/brotli/brotli_allocator.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BROTLI_BROTLI_ALLOCATOR_H_\n#define RIEGELI_BROTLI_BROTLI_ALLOCATOR_H_\n\n// IWYU pragma: private, include \"riegeli/brotli/brotli_reader.h\"\n// IWYU pragma: private, include \"riegeli/brotli/brotli_writer.h\"\n\n#include <stddef.h>\n\n#include <type_traits>\n#include <utility>\n\n#include \"brotli/types.h\"\n#include \"riegeli/base/shared_ptr.h\"\n\nnamespace riegeli {\n\nnamespace brotli_internal {\n\n// `extern \"C\"` sets the C calling convention for compatibility with the Brotli\n// API.\nextern \"C\" {\nvoid* RiegeliBrotliAllocFunc(void* opaque, size_t size);\nvoid RiegeliBrotliFreeFunc(void* opaque, void* ptr);\n}  // extern \"C\"\n\n}  // namespace brotli_internal\n\n// Memory allocator used by the Brotli engine.\n//\n// `BrotliAllocator` adapts C++ functors to C function pointers required by the\n// Brotli engine.\nclass BrotliAllocator {\n public:\n  // Specifies the default allocator chosen by the Brotli engine.\n  BrotliAllocator() = default;\n\n  BrotliAllocator(const BrotliAllocator& that) = default;\n  BrotliAllocator& operator=(const BrotliAllocator& that) = default;\n\n  BrotliAllocator(BrotliAllocator&& that) = default;\n  BrotliAllocator& operator=(BrotliAllocator&& that) = default;\n\n  // Specifies functions to allocate and free a block of memory.\n  //\n  // Arguments should be functions of the following types, or equivalent\n  // functors:\n  //   `void* alloc_functor(size_t size)`\n  //   `void free_functor(void* ptr)`\n  template <typename AllocFunctor, typename FreeFunctor>\n  explicit BrotliAllocator(AllocFunctor&& alloc_functor,\n                           FreeFunctor&& free_functor);\n\n  // Returns parameters for `Brotli{Encoder,Decoder}CreateInstance()`.\n  brotli_alloc_func alloc_func() const;\n  brotli_free_func free_func() const;\n  void* opaque() const;\n\n private:\n  friend void* brotli_internal::RiegeliBrotliAllocFunc(void* opaque,\n                                                       size_t size);\n  friend void brotli_internal::RiegeliBrotliFreeFunc(void* opaque, void* ptr);\n\n  class Interface;\n\n  template <typename AllocFunctor, typename FreeFunctor>\n  class Implementation;\n\n  SharedPtr<const Interface> impl_;\n};\n\n// Implementation details follow.\n\nclass BrotliAllocator::Interface {\n public:\n  virtual ~Interface();\n\n  virtual void* Alloc(size_t size) const = 0;\n  virtual void Free(void* ptr) const = 0;\n};\n\ntemplate <typename AllocFunctor, typename FreeFunctor>\nclass BrotliAllocator::Implementation : public Interface {\n public:\n  template <typename AllocFunctorArg, typename FreeFunctorArg>\n  explicit Implementation(AllocFunctorArg&& alloc_functor,\n                          FreeFunctorArg&& free_functor)\n      : alloc_functor_(std::forward<AllocFunctorArg>(alloc_functor)),\n        free_functor_(std::forward<FreeFunctorArg>(free_functor)) {}\n\n  void* Alloc(size_t size) const override { return alloc_functor_(size); }\n  void Free(void* ptr) const override { free_functor_(ptr); }\n\n private:\n  AllocFunctor alloc_functor_;\n  FreeFunctor free_functor_;\n};\n\ntemplate <typename AllocFunctor, typename FreeFunctor>\ninline BrotliAllocator::BrotliAllocator(AllocFunctor&& alloc_functor,\n                                        FreeFunctor&& free_functor)\n    : impl_(riegeli::Maker<const Implementation<std::decay_t<AllocFunctor>,\n                                                std::decay_t<FreeFunctor>>>(\n          std::forward<AllocFunctor>(alloc_functor),\n          std::forward<FreeFunctor>(free_functor))) {}\n\ninline brotli_alloc_func BrotliAllocator::alloc_func() const {\n  return impl_ == nullptr ? nullptr : brotli_internal::RiegeliBrotliAllocFunc;\n}\n\ninline brotli_free_func BrotliAllocator::free_func() const {\n  return impl_ == nullptr ? nullptr : brotli_internal::RiegeliBrotliFreeFunc;\n}\n\ninline void* BrotliAllocator::opaque() const {\n  return const_cast<Interface*>(impl_.get());\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BROTLI_BROTLI_ALLOCATOR_H_\n"
  },
  {
    "path": "riegeli/brotli/brotli_dictionary.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/brotli/brotli_dictionary.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <memory>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/call_once.h\"\n#include \"absl/strings/string_view.h\"\n#include \"brotli/encode.h\"\n#include \"brotli/shared_dictionary.h\"\n#include \"riegeli/base/assert.h\"\n\nnamespace riegeli {\n\nconst BrotliEncoderPreparedDictionary*\nBrotliDictionary::Chunk::PrepareCompressionDictionary() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  absl::call_once(compression_once_, [&] {\n    if (type_ == Type::kNative) {\n      RIEGELI_ASSERT_NE(compression_dictionary_, nullptr)\n          << \"Failed invariant of BrotliDictionary::Chunk: \"\n             \"unprepared native chunk\";\n      return;\n    }\n    owned_compression_dictionary_.reset(BrotliEncoderPrepareDictionary(\n        static_cast<BrotliSharedDictionaryType>(type_), data_.size(),\n        reinterpret_cast<const uint8_t*>(data_.data()), BROTLI_MAX_QUALITY,\n        // `BrotliAllocator` is not supported here because the prepared\n        // dictionary may easily outlive the allocator.\n        nullptr, nullptr, nullptr));\n    compression_dictionary_ = owned_compression_dictionary_.get();\n  });\n  return compression_dictionary_;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/brotli/brotli_dictionary.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BROTLI_BROTLI_DICTIONARY_H_\n#define RIEGELI_BROTLI_BROTLI_DICTIONARY_H_\n\n// IWYU pragma: private, include \"riegeli/brotli/brotli_reader.h\"\n// IWYU pragma: private, include \"riegeli/brotli/brotli_writer.h\"\n\n#include <stddef.h>\n\n#include <memory>\n#include <string>\n#include <type_traits>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/call_once.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"brotli/encode.h\"\n#include \"brotli/shared_dictionary.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/shared_ptr.h\"\n\nnamespace riegeli {\n\n// Stores an optional Brotli dictionary for compression and decompression\n// (Shared Brotli).\n//\n// A dictionary is empty and is equivalent to having no dictionary, or contains\n// a number of raw chunks (data which should contain sequences that are commonly\n// seen in the data being compressed), or contains one serialized chunk\n// (prepared by shared_brotli_encode_dictionary tool).\n//\n// A `BrotliDictionary` object can own the dictionary data, or can hold a\n// pointer to unowned dictionary data which must not be changed until the last\n// `BrotliReader` or `BrotliWriter` using this dictionary is closed or no\n// longer used. A `BrotliDictionary` object also holds prepared structures\n// derived from dictionary data. If the same dictionary is needed for multiple\n// compression or decompression sessions, the `BrotliDictionary` object can be\n// reused to avoid preparing them again for compression.\n//\n// Copying a `BrotliDictionary` object is cheap, sharing the actual\n// dictionary.\nclass BrotliDictionary {\n public:\n  class Chunk;\n\n  enum class Type {\n    // Chunk data should contain sequences that are commonly seen in the data\n    // being compressed\n    kRaw = BROTLI_SHARED_DICTIONARY_RAW,\n    // Chunk data prepared by shared_brotli_encode_dictionary tool.\n    kSerialized = BROTLI_SHARED_DICTIONARY_SERIALIZED,\n    // Chunk represented by `BrotliEncoderPreparedDictionary` pointer.\n    kNative = 2,\n  };\n\n  static constexpr size_t kMaxRawChunks = SHARED_BROTLI_MAX_COMPOUND_DICTS;\n\n  // Creates an empty `BrotliDictionary`.\n  BrotliDictionary() = default;\n\n  BrotliDictionary(const BrotliDictionary& that) = default;\n  BrotliDictionary& operator=(const BrotliDictionary& that) = default;\n\n  BrotliDictionary(BrotliDictionary&& that) = default;\n  BrotliDictionary& operator=(BrotliDictionary&& that) = default;\n\n  // Resets the `BrotliDictionary` to the empty state.\n  BrotliDictionary& Reset() & ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  BrotliDictionary&& Reset() && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(Reset());\n  }\n\n  // Adds a raw chunk (data which should contain sequences that are commonly\n  // seen in the data being compressed). Up to `kMaxRawChunks` can be added.\n  BrotliDictionary& add_raw(BytesInitializer data) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  BrotliDictionary&& add_raw(BytesInitializer data) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(add_raw(std::move(data)));\n  }\n\n  // Like `add_raw()`, but does not take ownership of `data`, which must not\n  // be changed until the last `BrotliReader` or `BrotliWriter` using this\n  // dictionary is closed or no longer used.\n  BrotliDictionary& add_raw_unowned(\n      absl::string_view data ABSL_ATTRIBUTE_LIFETIME_BOUND) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  BrotliDictionary&& add_raw_unowned(\n      absl::string_view data ABSL_ATTRIBUTE_LIFETIME_BOUND) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(add_raw_unowned(data));\n  }\n\n  // Sets a serialized chunk (prepared by shared_brotli_encode_dictionary tool).\n  BrotliDictionary& set_serialized(BytesInitializer data) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  BrotliDictionary&& set_serialized(BytesInitializer data) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_serialized(std::move(data)));\n  }\n\n  // Like `set_serialized()`, but does not take ownership of `data`, which\n  // must not be changed until the last `BrotliWriter` or `BrotliReader` using\n  // this dictionary is closed or no longer used.\n  BrotliDictionary& set_serialized_unowned(\n      absl::string_view data ABSL_ATTRIBUTE_LIFETIME_BOUND) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  BrotliDictionary&& set_serialized_unowned(\n      absl::string_view data ABSL_ATTRIBUTE_LIFETIME_BOUND) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_serialized_unowned(data));\n  }\n\n  // Interoperability with the native Brotli engine: adds a chunk represented by\n  // `BrotliEncoderPreparedDictionary` pointer. It can be used for compression\n  // but not for decompression.\n  //\n  // Does not take ownedship of `prepared, which must be valid until the last\n  // `BrotliReader` or `BrotliWriter` using this dictionary is closed or no\n  // longer used.\n  BrotliDictionary& add_native_unowned(\n      const BrotliEncoderPreparedDictionary* prepared\n          ABSL_ATTRIBUTE_LIFETIME_BOUND) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  BrotliDictionary&& add_native_unowned(\n      const BrotliEncoderPreparedDictionary* prepared\n          ABSL_ATTRIBUTE_LIFETIME_BOUND) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(add_native_unowned(prepared));\n  }\n\n  // Returns `true` if no dictionary is present.\n  bool empty() const { return chunks_.empty(); }\n\n  // Returns the sequence of chunks the dictionary consists of.\n  absl::Span<const SharedPtr<const Chunk>> chunks() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return chunks_;\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const BrotliDictionary* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->chunks_);\n  }\n\n private:\n  enum class Ownership { kCopied, kUnowned };\n\n  std::vector<SharedPtr<const Chunk>> chunks_;\n};\n\nclass BrotliDictionary::Chunk {\n public:\n  // Owns a copy of `data`.\n  explicit Chunk(Type type, BytesInitializer data,\n                 std::integral_constant<Ownership, Ownership::kCopied>)\n      : type_(type), owned_data_(std::move(data)), data_(owned_data_) {}\n\n  // Does not take ownership of `data`, which must not be changed until the\n  // last `BrotliWriter` or `BrotliReader` using this dictionary is closed or\n  // no longer used.\n  explicit Chunk(Type type,\n                 absl::string_view data ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                 std::integral_constant<Ownership, Ownership::kUnowned>)\n      : type_(type), data_(data) {}\n\n  // Does not know the data. The chunk is represented by\n  // `BrotliEncoderPreparedDictionary` pointer.\n  explicit Chunk(const BrotliEncoderPreparedDictionary* prepared\n                     ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : type_(Type::kNative), compression_dictionary_(prepared) {}\n\n  Chunk(const Chunk&) = delete;\n  Chunk& operator=(const Chunk&) = delete;\n\n  Type type() const { return type_; }\n  absl::string_view data() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_NE(static_cast<int>(type_), static_cast<int>(Type::kNative))\n        << \"Original data are not available \"\n           \"for a native Brotli dictionary chunk\";\n    return data_;\n  }\n\n  // Returns the compression dictionary in the prepared form, or `nullptr` if\n  // `BrotliEncoderPrepareDictionary()` failed.\n  //\n  // The dictionary is owned by `*this`.\n  const BrotliEncoderPreparedDictionary* PrepareCompressionDictionary() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const Chunk* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->owned_data_);\n    if (const BrotliEncoderPreparedDictionary* const compression_dictionary =\n            self->PrepareCompressionDictionary()) {\n      memory_estimator.RegisterMemory(\n          BrotliEncoderGetPreparedDictionarySize(compression_dictionary));\n    }\n  }\n\n private:\n  struct BrotliEncoderDictionaryDeleter {\n    void operator()(BrotliEncoderPreparedDictionary* ptr) const {\n      BrotliEncoderDestroyPreparedDictionary(ptr);\n    }\n  };\n\n  Type type_;\n  std::string owned_data_;\n  absl::string_view data_;\n\n  mutable absl::once_flag compression_once_;\n  mutable std::unique_ptr<BrotliEncoderPreparedDictionary,\n                          BrotliEncoderDictionaryDeleter>\n      owned_compression_dictionary_;\n  mutable const BrotliEncoderPreparedDictionary* compression_dictionary_ =\n      nullptr;\n};\n\n// Implementation details follow.\n\ninline BrotliDictionary& BrotliDictionary::Reset() &\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  chunks_.clear();\n  return *this;\n}\n\ninline BrotliDictionary& BrotliDictionary::add_raw(BytesInitializer data) &\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  chunks_.emplace_back(\n      riegeli::Maker(Type::kRaw, std::move(data),\n                     std::integral_constant<Ownership, Ownership::kCopied>()));\n  return *this;\n}\n\ninline BrotliDictionary& BrotliDictionary::add_raw_unowned(\n    absl::string_view data ABSL_ATTRIBUTE_LIFETIME_BOUND) &\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  chunks_.emplace_back(\n      riegeli::Maker(Type::kRaw, data,\n                     std::integral_constant<Ownership, Ownership::kUnowned>()));\n  return *this;\n}\n\ninline BrotliDictionary& BrotliDictionary::set_serialized(\n    BytesInitializer data) &\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  Reset();\n  chunks_.emplace_back(\n      riegeli::Maker(Type::kSerialized, std::move(data),\n                     std::integral_constant<Ownership, Ownership::kCopied>()));\n  return *this;\n}\n\ninline BrotliDictionary& BrotliDictionary::set_serialized_unowned(\n    absl::string_view data ABSL_ATTRIBUTE_LIFETIME_BOUND) &\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  Reset();\n  chunks_.emplace_back(\n      riegeli::Maker(Type::kSerialized, data,\n                     std::integral_constant<Ownership, Ownership::kUnowned>()));\n  return *this;\n}\n\ninline BrotliDictionary& BrotliDictionary::add_native_unowned(\n    const BrotliEncoderPreparedDictionary* prepared\n        ABSL_ATTRIBUTE_LIFETIME_BOUND) &\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  chunks_.emplace_back(riegeli::Maker(prepared));\n  return *this;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BROTLI_BROTLI_DICTIONARY_H_\n"
  },
  {
    "path": "riegeli/brotli/brotli_reader.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/brotli/brotli_reader.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"brotli/decode.h\"\n#include \"brotli/shared_dictionary.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/shared_ptr.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pullable_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nvoid BrotliReaderBase::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of BrotliReader: null Reader pointer\";\n  if (ABSL_PREDICT_FALSE(!src->ok()) && src->available() == 0) {\n    FailWithoutAnnotation(AnnotateOverSrc(src->status()));\n    return;\n  }\n  initial_compressed_pos_ = src->pos();\n  InitializeDecompressor();\n}\n\ninline void BrotliReaderBase::InitializeDecompressor() {\n  decompressor_.reset(BrotliDecoderCreateInstance(\n      allocator_.alloc_func(), allocator_.free_func(), allocator_.opaque()));\n  if (ABSL_PREDICT_FALSE(decompressor_ == nullptr)) {\n    Fail(absl::InternalError(\"BrotliDecoderCreateInstance() failed\"));\n    return;\n  }\n  if (ABSL_PREDICT_FALSE(!BrotliDecoderSetParameter(\n          decompressor_.get(), BROTLI_DECODER_PARAM_LARGE_WINDOW,\n          uint32_t{true}))) {\n    Fail(absl::InternalError(\n        \"BrotliDecoderSetParameter(BROTLI_DECODER_PARAM_LARGE_WINDOW) failed\"));\n    return;\n  }\n  for (const SharedPtr<const BrotliDictionary::Chunk>& chunk :\n       dictionary_.chunks()) {\n    if (ABSL_PREDICT_FALSE(chunk->type() == BrotliDictionary::Type::kNative)) {\n      Fail(absl::InvalidArgumentError(\n          \"A native Brotli dictionary chunk cannot be used for decompression\"));\n      return;\n    }\n    if (ABSL_PREDICT_FALSE(!BrotliDecoderAttachDictionary(\n            decompressor_.get(),\n            static_cast<BrotliSharedDictionaryType>(chunk->type()),\n            chunk->data().size(),\n            reinterpret_cast<const uint8_t*>(chunk->data().data())))) {\n      Fail(absl::InternalError(\"BrotliDecoderAttachDictionary() failed\"));\n      return;\n    }\n  }\n}\n\nvoid BrotliReaderBase::Done() {\n  if (ABSL_PREDICT_FALSE(truncated_)) {\n    Reader& src = *SrcReader();\n    FailWithoutAnnotation(AnnotateOverSrc(src.AnnotateStatus(\n        absl::InvalidArgumentError(\"Truncated Brotli-compressed stream\"))));\n  }\n  PullableReader::Done();\n  decompressor_.reset();\n  allocator_ = BrotliAllocator();\n  dictionary_ = BrotliDictionary();\n}\n\nabsl::Status BrotliReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    if (ABSL_PREDICT_FALSE(truncated_)) {\n      status = Annotate(status, \"reading truncated Brotli-compressed stream\");\n    }\n    Reader& src = *SrcReader();\n    status = src.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `src` with the compressed position.\n  // Clarify that the current position is the uncompressed position instead of\n  // delegating to `PullableReader::AnnotateStatusImpl()`.\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status BrotliReaderBase::AnnotateOverSrc(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nbool BrotliReaderBase::PullBehindScratch(size_t recommended_length) {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"some data available, use Pull() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(decompressor_ == nullptr)) return false;\n  Reader& src = *SrcReader();\n  truncated_ = false;\n  size_t available_out = 0;\n  for (;;) {\n    size_t available_in = src.available();\n    const uint8_t* next_in = reinterpret_cast<const uint8_t*>(src.cursor());\n    const BrotliDecoderResult result = BrotliDecoderDecompressStream(\n        decompressor_.get(), &available_in, &next_in, &available_out, nullptr,\n        nullptr);\n    src.set_cursor(reinterpret_cast<const char*>(next_in));\n    switch (result) {\n      case BROTLI_DECODER_RESULT_ERROR:\n        set_buffer();\n        return Fail(absl::InvalidArgumentError(\n            absl::StrCat(\"BrotliDecoderDecompressStream() failed: \",\n                         BrotliDecoderErrorString(\n                             BrotliDecoderGetErrorCode(decompressor_.get())))));\n      case BROTLI_DECODER_RESULT_SUCCESS:\n        set_buffer();\n        decompressor_.reset();\n        return false;\n      case BROTLI_DECODER_RESULT_NEEDS_MORE_INPUT:\n      case BROTLI_DECODER_RESULT_NEEDS_MORE_OUTPUT: {\n        // Take the output first even if `BrotliDecoderDecompressStream()`\n        // returned `BROTLI_DECODER_RESULT_NEEDS_MORE_INPUT`, in order to be\n        // able to read data which have been written before a `Flush()` without\n        // waiting for data to be written after the `Flush()`.\n        size_t length = 0;\n        const char* const data = reinterpret_cast<const char*>(\n            BrotliDecoderTakeOutput(decompressor_.get(), &length));\n        if (length > 0) {\n          const Position max_length =\n              std::numeric_limits<Position>::max() - limit_pos();\n          if (ABSL_PREDICT_FALSE(length > max_length)) {\n            set_buffer(data, IntCast<size_t>(max_length));\n            move_limit_pos(available());\n            return FailOverflow();\n          }\n          set_buffer(data, length);\n          move_limit_pos(available());\n          return true;\n        }\n        RIEGELI_ASSERT_EQ(result, BROTLI_DECODER_RESULT_NEEDS_MORE_INPUT)\n            << \"BrotliDecoderDecompressStream() returned \"\n               \"BROTLI_DECODER_RESULT_NEEDS_MORE_OUTPUT but \"\n               \"BrotliDecoderTakeOutput() returned no data\";\n        if (ABSL_PREDICT_FALSE(!src.Pull())) {\n          set_buffer();\n          if (ABSL_PREDICT_FALSE(!src.ok())) {\n            return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n          }\n          truncated_ = true;\n          return false;\n        }\n        continue;\n      }\n    }\n    RIEGELI_ASSUME_UNREACHABLE()\n        << \"Unknown BrotliDecoderResult: \" << static_cast<int>(result);\n  }\n}\n\nbool BrotliReaderBase::ToleratesReadingAhead() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->ToleratesReadingAhead();\n}\n\nbool BrotliReaderBase::SupportsRewind() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRewind();\n}\n\nbool BrotliReaderBase::SeekBehindScratch(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"scratch used\";\n  if (new_pos <= limit_pos()) {\n    // Seeking backwards.\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    Reader& src = *SrcReader();\n    truncated_ = false;\n    set_buffer();\n    set_limit_pos(0);\n    decompressor_.reset();\n    if (ABSL_PREDICT_FALSE(!src.Seek(initial_compressed_pos_))) {\n      return FailWithoutAnnotation(AnnotateOverSrc(src.StatusOrAnnotate(\n          absl::DataLossError(\"Brotli-compressed stream got truncated\"))));\n    }\n    InitializeDecompressor();\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    if (new_pos == 0) return true;\n  }\n  return PullableReader::SeekBehindScratch(new_pos);\n}\n\nbool BrotliReaderBase::SupportsNewReader() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsNewReader();\n}\n\nstd::unique_ptr<Reader> BrotliReaderBase::NewReaderImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point\n  // if `SrcReader()->SupportsNewReader()`.\n  Reader& src = *SrcReader();\n  std::unique_ptr<Reader> compressed_reader =\n      src.NewReader(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    return nullptr;\n  }\n  std::unique_ptr<Reader> reader =\n      std::make_unique<BrotliReader<std::unique_ptr<Reader>>>(\n          std::move(compressed_reader), BrotliReaderBase::Options()\n                                            .set_dictionary(dictionary_)\n                                            .set_allocator(allocator_));\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/brotli/brotli_reader.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BROTLI_BROTLI_READER_H_\n#define RIEGELI_BROTLI_BROTLI_READER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"brotli/decode.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/brotli/brotli_allocator.h\"   // IWYU pragma: export\n#include \"riegeli/brotli/brotli_dictionary.h\"  // IWYU pragma: export\n#include \"riegeli/bytes/pullable_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `BrotliReader`.\nclass BrotliReaderBase : public PullableReader {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // Shared Brotli dictionary. The same dictionary must have been used\n    // for compression. If no dictionary was used for compression, then no\n    // dictionary must be supplied for decompression.\n    //\n    // Default: `BrotliDictionary()`.\n    Options& set_dictionary(BrotliDictionary dictionary) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      dictionary_ = std::move(dictionary);\n      return *this;\n    }\n    Options&& set_dictionary(BrotliDictionary dictionary) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_dictionary(std::move(dictionary)));\n    }\n    BrotliDictionary& dictionary() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n    const BrotliDictionary& dictionary() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n\n    // Memory allocator used by the Brotli engine.\n    //\n    // Default: `BrotliAllocator()`.\n    Options& set_allocator(BrotliAllocator allocator) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      allocator_ = std::move(allocator);\n      return *this;\n    }\n    Options&& set_allocator(BrotliAllocator allocator) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_allocator(std::move(allocator)));\n    }\n    BrotliAllocator& allocator() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return allocator_;\n    }\n    const BrotliAllocator& allocator() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return allocator_;\n    }\n\n   private:\n    BrotliDictionary dictionary_;\n    BrotliAllocator allocator_;\n  };\n\n  // Returns the compressed `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns `true` if the source is truncated (without a clean end of the\n  // compressed stream) at the current position. In such case, if the source\n  // does not grow, `Close()` will fail.\n  bool truncated() const { return truncated_ && available() == 0; }\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRewind() override;\n  bool SupportsNewReader() override;\n\n protected:\n  explicit BrotliReaderBase(Closed) noexcept : PullableReader(kClosed) {}\n\n  explicit BrotliReaderBase(BrotliDictionary&& dictionary,\n                            BrotliAllocator&& allocator);\n\n  BrotliReaderBase(BrotliReaderBase&& that) noexcept;\n  BrotliReaderBase& operator=(BrotliReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BrotliDictionary&& dictionary, BrotliAllocator&& allocator);\n  void Initialize(Reader* src);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PullBehindScratch(size_t recommended_length) override;\n  bool SeekBehindScratch(Position new_pos) override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  struct BrotliDecoderStateDeleter {\n    void operator()(BrotliDecoderState* ptr) const {\n      BrotliDecoderDestroyInstance(ptr);\n    }\n  };\n\n  void InitializeDecompressor();\n\n  BrotliDictionary dictionary_;\n  BrotliAllocator allocator_;\n  Position initial_compressed_pos_ = 0;\n  // If `true`, the source is truncated (without a clean end of the compressed\n  // stream) at the current position. If the source does not grow, `Close()`\n  // will fail.\n  bool truncated_ = false;\n  // If `ok()` but `decompressor_ == nullptr` then all data have been\n  // decompressed.\n  std::unique_ptr<BrotliDecoderState, BrotliDecoderStateDeleter> decompressor_;\n\n  // Invariant if scratch is not used:\n  //   `start()` and `limit()` point to the buffer returned by\n  //   `BrotliDecoderTakeOutput()` or are both `nullptr`\n};\n\n// A `Reader` which decompresses data with Brotli after getting it from another\n// `Reader`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Reader` must not be accessed until the `BrotliReader` is\n// closed or no longer used.\ntemplate <typename Src = Reader*>\nclass BrotliReader : public BrotliReaderBase {\n public:\n  // Creates a closed `BrotliReader`.\n  explicit BrotliReader(Closed) noexcept : BrotliReaderBase(kClosed) {}\n\n  // Will read from the compressed `Reader` provided by `src`.\n  explicit BrotliReader(Initializer<Src> src, Options options = Options());\n\n  BrotliReader(BrotliReader&& that) = default;\n  BrotliReader& operator=(BrotliReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `BrotliReader`. This avoids\n  // constructing a temporary `BrotliReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  void VerifyEndImpl() override;\n\n private:\n  // The object providing and possibly owning the compressed `Reader`.\n  Dependency<Reader*, Src> src_;\n};\n\nexplicit BrotliReader(Closed) -> BrotliReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit BrotliReader(\n    Src&& src, BrotliReaderBase::Options options = BrotliReaderBase::Options())\n    -> BrotliReader<TargetT<Src>>;\n\n// Implementation details follow.\n\ninline BrotliReaderBase::BrotliReaderBase(BrotliDictionary&& dictionary,\n                                          BrotliAllocator&& allocator)\n    : dictionary_(std::move(dictionary)), allocator_(std::move(allocator)) {}\n\ninline BrotliReaderBase::BrotliReaderBase(BrotliReaderBase&& that) noexcept\n    : PullableReader(static_cast<PullableReader&&>(that)),\n      dictionary_(std::move(that.dictionary_)),\n      allocator_(std::move(that.allocator_)),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      truncated_(that.truncated_),\n      decompressor_(std::move(that.decompressor_)) {}\n\ninline BrotliReaderBase& BrotliReaderBase::operator=(\n    BrotliReaderBase&& that) noexcept {\n  PullableReader::operator=(static_cast<PullableReader&&>(that));\n  dictionary_ = std::move(that.dictionary_);\n  allocator_ = std::move(that.allocator_);\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  truncated_ = that.truncated_;\n  decompressor_ = std::move(that.decompressor_);\n  return *this;\n}\n\ninline void BrotliReaderBase::Reset(Closed) {\n  PullableReader::Reset(kClosed);\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  decompressor_.reset();\n  // Must be destroyed after `decompressor_`.\n  dictionary_ = BrotliDictionary();\n  // Must be destroyed after `decompressor_`.\n  allocator_ = BrotliAllocator();\n}\n\ninline void BrotliReaderBase::Reset(BrotliDictionary&& dictionary,\n                                    BrotliAllocator&& allocator) {\n  PullableReader::Reset();\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  decompressor_.reset();\n  // Must be destroyed after `decompressor_`.\n  dictionary_ = std::move(dictionary);\n  // Must be destroyed after `decompressor_`.\n  allocator_ = std::move(allocator);\n}\n\ntemplate <typename Src>\ninline BrotliReader<Src>::BrotliReader(Initializer<Src> src, Options options)\n    : BrotliReaderBase(std::move(options.dictionary()),\n                       std::move(options.allocator())),\n      src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void BrotliReader<Src>::Reset(Closed) {\n  BrotliReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void BrotliReader<Src>::Reset(Initializer<Src> src, Options options) {\n  BrotliReaderBase::Reset(std::move(options.dictionary()),\n                          std::move(options.allocator()));\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\nvoid BrotliReader<Src>::Done() {\n  BrotliReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(AnnotateOverSrc(src_->status()));\n    }\n  }\n}\n\ntemplate <typename Src>\nvoid BrotliReader<Src>::SetReadAllHintImpl(bool read_all_hint) {\n  if (src_.IsOwning()) src_->SetReadAllHint(read_all_hint);\n}\n\ntemplate <typename Src>\nvoid BrotliReader<Src>::VerifyEndImpl() {\n  BrotliReaderBase::VerifyEndImpl();\n  if (src_.IsOwning() && ABSL_PREDICT_TRUE(ok())) src_->VerifyEnd();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BROTLI_BROTLI_READER_H_\n"
  },
  {
    "path": "riegeli/brotli/brotli_writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/brotli/brotli_writer.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"brotli/encode.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/shared_ptr.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/brotli/brotli_reader.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nnamespace {\n\nstruct BrotliEncoderDictionaryDeleter {\n  void operator()(BrotliEncoderPreparedDictionary* ptr) const {\n    BrotliEncoderDestroyPreparedDictionary(ptr);\n  }\n};\n\n}  // namespace\n\nvoid BrotliWriterBase::Initialize(Writer* dest, int compression_level,\n                                  int window_log) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of BrotliWriter: null Writer pointer\";\n  if (ABSL_PREDICT_FALSE(!dest->ok())) {\n    FailWithoutAnnotation(AnnotateOverDest(dest->status()));\n    return;\n  }\n  initial_compressed_pos_ = dest->pos();\n  compressor_.reset(BrotliEncoderCreateInstance(\n      allocator_.alloc_func(), allocator_.free_func(), allocator_.opaque()));\n  if (ABSL_PREDICT_FALSE(compressor_ == nullptr)) {\n    Fail(absl::InternalError(\"BrotliEncoderCreateInstance() failed\"));\n    return;\n  }\n  if (ABSL_PREDICT_FALSE(\n          !BrotliEncoderSetParameter(compressor_.get(), BROTLI_PARAM_QUALITY,\n                                     IntCast<uint32_t>(compression_level)))) {\n    Fail(absl::InternalError(\n        \"BrotliEncoderSetParameter(BROTLI_PARAM_QUALITY) failed\"));\n    return;\n  }\n  if (ABSL_PREDICT_FALSE(!BrotliEncoderSetParameter(\n          compressor_.get(), BROTLI_PARAM_LARGE_WINDOW,\n          uint32_t{window_log > BROTLI_MAX_WINDOW_BITS}))) {\n    Fail(absl::InternalError(\n        \"BrotliEncoderSetParameter(BROTLI_PARAM_LARGE_WINDOW) failed\"));\n    return;\n  }\n  if (ABSL_PREDICT_FALSE(\n          !BrotliEncoderSetParameter(compressor_.get(), BROTLI_PARAM_LGWIN,\n                                     IntCast<uint32_t>(window_log)))) {\n    Fail(absl::InternalError(\n        \"BrotliEncoderSetParameter(BROTLI_PARAM_LGWIN) failed\"));\n    return;\n  }\n  for (const SharedPtr<const BrotliDictionary::Chunk>& chunk :\n       dictionary_.chunks()) {\n    const BrotliEncoderPreparedDictionary* const compression_dictionary =\n        chunk->PrepareCompressionDictionary();\n    if (ABSL_PREDICT_FALSE(compression_dictionary == nullptr)) {\n      Fail(absl::InternalError(\"BrotliEncoderPrepareDictionary() failed\"));\n      return;\n    }\n    if (ABSL_PREDICT_FALSE(!BrotliEncoderAttachPreparedDictionary(\n            compressor_.get(), compression_dictionary))) {\n      Fail(absl::InternalError(\n          \"BrotliEncoderAttachPreparedDictionary() failed\"));\n      return;\n    }\n  }\n}\n\nvoid BrotliWriterBase::DoneBehindBuffer(absl::string_view src) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::DoneBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  Writer& dest = *DestWriter();\n  WriteInternal(src, dest, BROTLI_OPERATION_FINISH);\n}\n\nvoid BrotliWriterBase::Done() {\n  BufferedWriter::Done();\n  compressor_.reset();\n  dictionary_ = BrotliDictionary();\n  allocator_ = BrotliAllocator();\n  associated_reader_.Reset();\n}\n\nabsl::Status BrotliWriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    status = dest.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `dest` with the compressed\n  // position. Clarify that the current position is the uncompressed position\n  // instead of delegating to `BufferedWriter::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status BrotliWriterBase::AnnotateOverDest(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nvoid BrotliWriterBase::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  BufferedWriter::SetWriteSizeHintImpl(write_size_hint);\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  // Ignore failure if compression already started.\n  BrotliEncoderSetParameter(compressor_.get(), BROTLI_PARAM_SIZE_HINT,\n                            write_size_hint == std::nullopt\n                                ? 0\n                                : SaturatingIntCast<uint32_t>(\n                                      SaturatingAdd(pos(), *write_size_hint)));\n}\n\nbool BrotliWriterBase::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of BufferedWriter::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedWriter::WriteInternal()\";\n  Writer& dest = *DestWriter();\n  return WriteInternal(src, dest, BROTLI_OPERATION_PROCESS);\n}\n\ninline bool BrotliWriterBase::WriteInternal(absl::string_view src, Writer& dest,\n                                            BrotliEncoderOperation op) {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BrotliWriterBase::WriteInternal()\";\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  size_t available_in = src.size();\n  const uint8_t* next_in = reinterpret_cast<const uint8_t*>(src.data());\n  size_t available_out = 0;\n  for (;;) {\n    if (ABSL_PREDICT_FALSE(!BrotliEncoderCompressStream(\n            compressor_.get(), op, &available_in, &next_in, &available_out,\n            nullptr, nullptr))) {\n      return Fail(absl::InternalError(\"BrotliEncoderCompressStream() failed\"));\n    }\n    size_t length = 0;\n    const char* const data = reinterpret_cast<const char*>(\n        BrotliEncoderTakeOutput(compressor_.get(), &length));\n    if (length > 0) {\n      if (ABSL_PREDICT_FALSE(!dest.Write(absl::string_view(data, length)))) {\n        return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n      }\n    } else if (available_in == 0) {\n      move_start_pos(src.size());\n      return true;\n    }\n  }\n}\n\nbool BrotliWriterBase::FlushBehindBuffer(absl::string_view src,\n                                         FlushType flush_type) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::FlushBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  return WriteInternal(src, dest, BROTLI_OPERATION_FLUSH);\n}\n\nbool BrotliWriterBase::SupportsReadMode() {\n  Writer* const dest = DestWriter();\n  if (dest != nullptr && dest->SupportsReadMode()) {\n    for (const SharedPtr<const BrotliDictionary::Chunk>& chunk :\n         dictionary_.chunks()) {\n      if (chunk->type() == BrotliDictionary::Type::kNative) return false;\n    }\n    return true;\n  }\n  return false;\n}\n\nReader* BrotliWriterBase::ReadModeBehindBuffer(Position initial_pos) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::ReadModeBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!BrotliWriterBase::FlushBehindBuffer(\n          absl::string_view(), FlushType::kFromObject))) {\n    return nullptr;\n  }\n  Writer& dest = *DestWriter();\n  Reader* const compressed_reader = dest.ReadMode(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    return nullptr;\n  }\n  BrotliReader<>* const reader = associated_reader_.ResetReader(\n      compressed_reader, BrotliReaderBase::Options()\n                             .set_dictionary(dictionary_)\n                             .set_allocator(allocator_));\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/brotli/brotli_writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BROTLI_BROTLI_WRITER_H_\n#define RIEGELI_BROTLI_BROTLI_WRITER_H_\n\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"brotli/encode.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/brotli/brotli_allocator.h\"   // IWYU pragma: export\n#include \"riegeli/brotli/brotli_dictionary.h\"  // IWYU pragma: export\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\ntemplate <typename Src>\nclass BrotliReader;\nclass Reader;\n\n// Template parameter independent part of `BrotliWriter`.\nclass BrotliWriterBase : public BufferedWriter {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // Tunes the tradeoff between compression density and compression speed\n    // (higher = better density but slower).\n    //\n    // `compression_level` must be between `kMinCompressionLevel` (0) and\n    // `kMaxCompressionLevel` (11). Default: `kDefaultCompressionLevel` (6).\n    static constexpr int kMinCompressionLevel = BROTLI_MIN_QUALITY;\n    static constexpr int kMaxCompressionLevel = BROTLI_MAX_QUALITY;\n    static constexpr int kDefaultCompressionLevel = 6;\n    Options& set_compression_level(int compression_level) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(compression_level, kMinCompressionLevel)\n          << \"Failed precondition of \"\n             \"BrotliWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      RIEGELI_ASSERT_LE(compression_level, kMaxCompressionLevel)\n          << \"Failed precondition of \"\n             \"BrotliWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      compression_level_ = compression_level;\n      return *this;\n    }\n    Options&& set_compression_level(int compression_level) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_compression_level(compression_level));\n    }\n    int compression_level() const { return compression_level_; }\n\n    // Logarithm of the LZ77 sliding window size. This tunes the tradeoff\n    // between compression density and memory usage (higher = better density but\n    // more memory).\n    //\n    // `window_log` must be between `kMinWindowLog` (10) and `kMaxWindowLog`\n    // (30). Default: `kDefaultWindowLog` (22).\n    static constexpr int kMinWindowLog = BROTLI_MIN_WINDOW_BITS;\n    static constexpr int kMaxWindowLog = BROTLI_LARGE_MAX_WINDOW_BITS;\n    static constexpr int kDefaultWindowLog = BROTLI_DEFAULT_WINDOW;\n    Options& set_window_log(int window_log) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(window_log, kMinWindowLog)\n          << \"Failed precondition of \"\n             \"BrotliWriterBase::Options::set_window_log(): \"\n             \"window log out of range\";\n      RIEGELI_ASSERT_LE(window_log, kMaxWindowLog)\n          << \"Failed precondition of \"\n             \"BrotliWriterBase::Options::set_window_log(): \"\n             \"window log out of range\";\n      window_log_ = window_log;\n      return *this;\n    }\n    Options&& set_window_log(int window_log) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_window_log(window_log));\n    }\n    int window_log() const { return window_log_; }\n\n    // Shared Brotli dictionary. The same dictionary must have been used for\n    // compression.\n    //\n    // Default: `BrotliDictionary()`.\n    Options& set_dictionary(BrotliDictionary dictionary) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      dictionary_ = std::move(dictionary);\n      return *this;\n    }\n    Options&& set_dictionary(BrotliDictionary dictionary) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_dictionary(std::move(dictionary)));\n    }\n    BrotliDictionary& dictionary() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n    const BrotliDictionary& dictionary() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n\n    // Memory allocator used by the Brotli engine.\n    //\n    // Default: `BrotliAllocator()`.\n    Options& set_allocator(BrotliAllocator allocator) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      allocator_ = std::move(allocator);\n      return *this;\n    }\n    Options&& set_allocator(BrotliAllocator allocator) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_allocator(std::move(allocator)));\n    }\n    BrotliAllocator& allocator() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return allocator_;\n    }\n    const BrotliAllocator& allocator() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return allocator_;\n    }\n\n   private:\n    int compression_level_ = kDefaultCompressionLevel;\n    int window_log_ = kDefaultWindowLog;\n    BrotliDictionary dictionary_;\n    BrotliAllocator allocator_;\n  };\n\n  // Returns the compressed `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsReadMode() override;\n\n protected:\n  explicit BrotliWriterBase(Closed) noexcept : BufferedWriter(kClosed) {}\n\n  explicit BrotliWriterBase(BufferOptions buffer_options,\n                            BrotliDictionary&& dictionary,\n                            BrotliAllocator&& allocator);\n\n  BrotliWriterBase(BrotliWriterBase&& that) noexcept;\n  BrotliWriterBase& operator=(BrotliWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, BrotliDictionary&& dictionary,\n             BrotliAllocator&& allocator);\n  void Initialize(Writer* dest, int compression_level, int window_log);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  void DoneBehindBuffer(absl::string_view src) override;\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool WriteInternal(absl::string_view src) override;\n  bool FlushBehindBuffer(absl::string_view src, FlushType flush_type) override;\n  Reader* ReadModeBehindBuffer(Position initial_pos) override;\n\n private:\n  struct BrotliEncoderStateDeleter {\n    void operator()(BrotliEncoderState* ptr) const {\n      BrotliEncoderDestroyInstance(ptr);\n    }\n  };\n\n  bool WriteInternal(absl::string_view src, Writer& dest,\n                     BrotliEncoderOperation op);\n\n  BrotliDictionary dictionary_;\n  BrotliAllocator allocator_;\n  Position initial_compressed_pos_ = 0;\n  std::unique_ptr<BrotliEncoderState, BrotliEncoderStateDeleter> compressor_;\n\n  AssociatedReader<BrotliReader<Reader*>> associated_reader_;\n};\n\n// A `Writer` which compresses data with Brotli before passing it to another\n// `Writer`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Writer` must not be accessed until the `BrotliWriter` is\n// closed or no longer used, except that it is allowed to read the destination\n// of the compressed `Writer` immediately after `Flush()`.\ntemplate <typename Dest = Writer*>\nclass BrotliWriter : public BrotliWriterBase {\n public:\n  // Creates a closed `BrotliWriter`.\n  explicit BrotliWriter(Closed) noexcept : BrotliWriterBase(kClosed) {}\n\n  // Will write to the compressed `Writer` provided by `dest`.\n  explicit BrotliWriter(Initializer<Dest> dest, Options options = Options());\n\n  BrotliWriter(BrotliWriter&& that) = default;\n  BrotliWriter& operator=(BrotliWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `BrotliWriter`. This avoids\n  // constructing a temporary `BrotliWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  // The object providing and possibly owning the compressed `Writer`.\n  Dependency<Writer*, Dest> dest_;\n};\n\nexplicit BrotliWriter(Closed) -> BrotliWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit BrotliWriter(Dest&& dest, BrotliWriterBase::Options options =\n                                       BrotliWriterBase::Options())\n    -> BrotliWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline BrotliWriterBase::BrotliWriterBase(BufferOptions buffer_options,\n                                          BrotliDictionary&& dictionary,\n                                          BrotliAllocator&& allocator)\n    : BufferedWriter(buffer_options),\n      dictionary_(std::move(dictionary)),\n      allocator_(std::move(allocator)) {}\n\ninline BrotliWriterBase::BrotliWriterBase(BrotliWriterBase&& that) noexcept\n    : BufferedWriter(static_cast<BufferedWriter&&>(that)),\n      dictionary_(std::move(that.dictionary_)),\n      allocator_(std::move(that.allocator_)),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      compressor_(std::move(that.compressor_)),\n      associated_reader_(std::move(that.associated_reader_)) {}\n\ninline BrotliWriterBase& BrotliWriterBase::operator=(\n    BrotliWriterBase&& that) noexcept {\n  BufferedWriter::operator=(static_cast<BufferedWriter&&>(that));\n  dictionary_ = std::move(that.dictionary_);\n  allocator_ = std::move(that.allocator_);\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  compressor_ = std::move(that.compressor_);\n  associated_reader_ = std::move(that.associated_reader_);\n  return *this;\n}\n\ninline void BrotliWriterBase::Reset(Closed) {\n  BufferedWriter::Reset(kClosed);\n  initial_compressed_pos_ = 0;\n  compressor_.reset();\n  // Must be destroyed after `compressor_`.\n  dictionary_ = BrotliDictionary();\n  // Must be destroyed after `compressor_`.\n  allocator_ = BrotliAllocator();\n  associated_reader_.Reset();\n}\n\ninline void BrotliWriterBase::Reset(BufferOptions buffer_options,\n                                    BrotliDictionary&& dictionary,\n                                    BrotliAllocator&& allocator) {\n  BufferedWriter::Reset(buffer_options);\n  initial_compressed_pos_ = 0;\n  compressor_.reset();\n  // Must be destroyed after `compressor_`.\n  dictionary_ = std::move(dictionary);\n  // Must be destroyed after `compressor_`.\n  allocator_ = std::move(allocator);\n  associated_reader_.Reset();\n}\n\ntemplate <typename Dest>\ninline BrotliWriter<Dest>::BrotliWriter(Initializer<Dest> dest, Options options)\n    : BrotliWriterBase(options.buffer_options(),\n                       std::move(options.dictionary()),\n                       std::move(options.allocator())),\n      dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.compression_level(), options.window_log());\n}\n\ntemplate <typename Dest>\ninline void BrotliWriter<Dest>::Reset(Closed) {\n  BrotliWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void BrotliWriter<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  BrotliWriterBase::Reset(options.buffer_options(),\n                          std::move(options.dictionary()),\n                          std::move(options.allocator()));\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.compression_level(), options.window_log());\n}\n\ntemplate <typename Dest>\nvoid BrotliWriter<Dest>::Done() {\n  BrotliWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <typename Dest>\nbool BrotliWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!BrotliWriterBase::FlushImpl(flush_type))) {\n    return false;\n  }\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Flush(flush_type))) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n  return true;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BROTLI_BROTLI_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/BUILD",
    "content": "load(\"@bazel_skylib//rules:common_settings.bzl\", \"bool_flag\")\nload(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\nbool_flag(\n    name = \"use_copy_file_range\",\n    build_setting_default = True,\n)\n\nconfig_setting(\n    name = \"disable_copy_file_range\",\n    flag_values = {\":use_copy_file_range\": \"False\"},\n)\n\ncc_library(\n    name = \"reader\",\n    hdrs = [\"reader.h\"],\n    deps = [\n        \":reader_and_writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"writer\",\n    hdrs = [\"writer.h\"],\n    deps = [\n        \":reader_and_writer\",\n        \":stringify\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:int128\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"backward_writer\",\n    hdrs = [\"backward_writer.h\"],\n    deps = [\n        \":reader_and_writer\",\n        \":stringify\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:int128\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"reader_and_writer\",\n    srcs = [\n        \"backward_writer.cc\",\n        \"reader.cc\",\n        \"restricted_chain_writer.cc\",\n        \"writer.cc\",\n    ],\n    hdrs = [\n        \"backward_writer.h\",\n        \"reader.h\",\n        \"restricted_chain_writer.h\",\n        \"writer.h\",\n    ],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \":stringify\",\n        \":write_int_internal\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:cord_utils\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:string_utils\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:int128\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:str_format\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"read_all\",\n    srcs = [\"read_all.cc\"],\n    hdrs = [\"read_all.h\"],\n    deps = [\n        \":reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:resize_and_overwrite\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"copy_all\",\n    srcs = [\"copy_all.cc\"],\n    hdrs = [\"copy_all.h\"],\n    deps = [\n        \":backward_writer\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n    ],\n)\n\ncc_library(\n    name = \"write\",\n    hdrs = [\"write.h\"],\n    deps = [\n        \":backward_writer\",\n        \":stringify\",\n        \":writer\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n    ],\n)\n\ncc_library(\n    name = \"write_int_internal\",\n    srcs = [\"write_int_internal.cc\"],\n    hdrs = [\"write_int_internal.h\"],\n    visibility = [\"//riegeli:__subpackages__\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:bits\",\n        \"@com_google_absl//absl/numeric:int128\",\n    ],\n)\n\ncc_library(\n    name = \"stringify\",\n    hdrs = [\"stringify.h\"],\n    deps = [\n        \":write_int_internal\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:int128\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"pullable_reader\",\n    srcs = [\"pullable_reader.cc\"],\n    hdrs = [\"pullable_reader.h\"],\n    deps = [\n        \":backward_writer\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:cord_utils\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:sized_shared_buffer\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"pushable_writer\",\n    srcs = [\"pushable_writer.cc\"],\n    hdrs = [\"pushable_writer.h\"],\n    deps = [\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:cord_utils\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:sized_shared_buffer\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"pushable_backward_writer\",\n    srcs = [\"pushable_backward_writer.cc\"],\n    hdrs = [\"pushable_backward_writer.h\"],\n    deps = [\n        \":backward_writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:cord_utils\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:sized_shared_buffer\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"buffer_options\",\n    srcs = [\"buffer_options.cc\"],\n    hdrs = [\"buffer_options.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:bits\",\n    ],\n)\n\ncc_library(\n    name = \"buffered_reader\",\n    srcs = [\"buffered_reader.cc\"],\n    hdrs = [\"buffered_reader.h\"],\n    deps = [\n        \":backward_writer\",\n        \":buffer_options\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:sized_shared_buffer\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"buffered_writer\",\n    srcs = [\"buffered_writer.cc\"],\n    hdrs = [\"buffered_writer.h\"],\n    deps = [\n        \":buffer_options\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffer\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"wrapping_reader\",\n    srcs = [\"wrapping_reader.cc\"],\n    hdrs = [\"wrapping_reader.h\"],\n    deps = [\n        \":backward_writer\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n    ],\n)\n\ncc_library(\n    name = \"wrapping_writer\",\n    srcs = [\"wrapping_writer.cc\"],\n    hdrs = [\"wrapping_writer.h\"],\n    deps = [\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"wrapping_backward_writer\",\n    srcs = [\"wrapping_backward_writer.cc\"],\n    hdrs = [\"wrapping_backward_writer.h\"],\n    deps = [\n        \":backward_writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"limiting_reader\",\n    srcs = [\"limiting_reader.cc\"],\n    hdrs = [\"limiting_reader.h\"],\n    deps = [\n        \":backward_writer\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n    ],\n)\n\ncc_library(\n    name = \"limiting_writer\",\n    srcs = [\"limiting_writer.cc\"],\n    hdrs = [\"limiting_writer.h\"],\n    deps = [\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"limiting_backward_writer\",\n    srcs = [\"limiting_backward_writer.cc\"],\n    hdrs = [\"limiting_backward_writer.h\"],\n    deps = [\n        \":backward_writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"prefix_limiting_reader\",\n    srcs = [\"prefix_limiting_reader.cc\"],\n    hdrs = [\"prefix_limiting_reader.h\"],\n    deps = [\n        \":backward_writer\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n    ],\n)\n\ncc_library(\n    name = \"prefix_limiting_writer\",\n    srcs = [\"prefix_limiting_writer.cc\"],\n    hdrs = [\"prefix_limiting_writer.h\"],\n    deps = [\n        \":prefix_limiting_reader\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"prefix_limiting_backward_writer\",\n    srcs = [\"prefix_limiting_backward_writer.cc\"],\n    hdrs = [\"prefix_limiting_backward_writer.h\"],\n    deps = [\n        \":backward_writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"position_shifting_reader\",\n    srcs = [\"position_shifting_reader.cc\"],\n    hdrs = [\"position_shifting_reader.h\"],\n    deps = [\n        \":backward_writer\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n    ],\n)\n\ncc_library(\n    name = \"position_shifting_writer\",\n    srcs = [\"position_shifting_writer.cc\"],\n    hdrs = [\"position_shifting_writer.h\"],\n    deps = [\n        \":position_shifting_reader\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"position_shifting_backward_writer\",\n    srcs = [\"position_shifting_backward_writer.cc\"],\n    hdrs = [\"position_shifting_backward_writer.h\"],\n    deps = [\n        \":backward_writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"null_writer\",\n    srcs = [\"null_writer.cc\"],\n    hdrs = [\"null_writer.h\"],\n    deps = [\n        \":buffer_options\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffer\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"null_backward_writer\",\n    srcs = [\"null_backward_writer.cc\"],\n    hdrs = [\"null_backward_writer.h\"],\n    deps = [\n        \":backward_writer\",\n        \":buffer_options\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffer\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"joining_reader\",\n    srcs = [\"joining_reader.cc\"],\n    hdrs = [\"joining_reader.h\"],\n    deps = [\n        \":pullable_reader\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n    ],\n)\n\ncc_library(\n    name = \"splitting_writer\",\n    srcs = [\"splitting_writer.cc\"],\n    hdrs = [\"splitting_writer.h\"],\n    deps = [\n        \":chain_reader\",\n        \":cord_reader\",\n        \":pushable_writer\",\n        \":string_reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"reader_factory\",\n    srcs = [\"reader_factory.cc\"],\n    hdrs = [\"reader_factory.h\"],\n    deps = [\n        \":buffer_options\",\n        \":pullable_reader\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:stable_dependency\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/synchronization\",\n    ],\n)\n\ncc_library(\n    name = \"array_writer\",\n    srcs = [\"array_writer.cc\"],\n    hdrs = [\"array_writer.h\"],\n    deps = [\n        \":pushable_writer\",\n        \":reader\",\n        \":string_reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"array_backward_writer\",\n    srcs = [\"array_backward_writer.cc\"],\n    hdrs = [\"array_backward_writer.h\"],\n    deps = [\n        \":backward_writer\",\n        \":pushable_backward_writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"string_reader\",\n    srcs = [\"string_reader.cc\"],\n    hdrs = [\"string_reader.h\"],\n    deps = [\n        \":reader\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"string_writer\",\n    srcs = [\"string_writer.cc\"],\n    hdrs = [\"string_writer.h\"],\n    deps = [\n        \":buffer_options\",\n        \":reader\",\n        \":string_reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:cord_utils\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:string_utils\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"resizable_writer\",\n    srcs = [\"resizable_writer.cc\"],\n    hdrs = [\"resizable_writer.h\"],\n    deps = [\n        \":buffer_options\",\n        \":reader\",\n        \":string_reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:cord_utils\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"vector_writer\",\n    hdrs = [\"vector_writer.h\"],\n    deps = [\n        \":resizable_writer\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:uninitialized_vector\",\n    ],\n)\n\ncc_library(\n    name = \"compact_string_writer\",\n    hdrs = [\"compact_string_writer.h\"],\n    deps = [\n        \":resizable_writer\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:compact_string\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n    ],\n)\n\ncc_library(\n    name = \"chain_reader\",\n    srcs = [\"chain_reader.cc\"],\n    hdrs = [\"chain_reader.h\"],\n    deps = [\n        \":backward_writer\",\n        \":pullable_reader\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"chain_writer\",\n    srcs = [\"chain_writer.cc\"],\n    hdrs = [\"chain_writer.h\"],\n    deps = [\n        \":chain_reader\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"chain_backward_writer\",\n    srcs = [\"chain_backward_writer.cc\"],\n    hdrs = [\"chain_backward_writer.h\"],\n    deps = [\n        \":backward_writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"restricted_chain_writer\",\n    hdrs = [\"restricted_chain_writer.h\"],\n    deps = [\n        \":reader_and_writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:object\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"cord_reader\",\n    srcs = [\"cord_reader.cc\"],\n    hdrs = [\"cord_reader.h\"],\n    deps = [\n        \":backward_writer\",\n        \":pullable_reader\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"cord_writer\",\n    srcs = [\"cord_writer.cc\"],\n    hdrs = [\"cord_writer.h\"],\n    deps = [\n        \":cord_reader\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffer\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:cord_utils\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"cord_backward_writer\",\n    srcs = [\"cord_backward_writer.cc\"],\n    hdrs = [\"cord_backward_writer.h\"],\n    deps = [\n        \":backward_writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffer\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:cord_utils\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"file_mode_string\",\n    srcs = [\"file_mode_string.cc\"],\n    hdrs = [\"file_mode_string.h\"],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"path_ref\",\n    hdrs = [\"path_ref.h\"],\n    deps = [\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:string_ref\",\n        \"//riegeli/base:type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"fd_handle\",\n    srcs = [\"fd_handle.cc\"],\n    hdrs = [\"fd_handle.h\"],\n    # fd_handle.cc has #define before #include to influence what the included\n    # files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \":fd_internal\",\n        \":path_ref\",\n        \"//riegeli/base:any\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:shared_ptr\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:type_erased_ref\",\n        \"//riegeli/base:type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n    ] + select({\n        \"@platforms//os:windows\": [\n            \"//riegeli/base:unicode\",\n        ],\n        \"//conditions:default\": [],\n    }),\n)\n\ncc_library(\n    name = \"fd_reader\",\n    srcs = [\"fd_reader.cc\"] + select({\n        \"@platforms//os:windows\": [],\n        \"//conditions:default\": [\"fd_writer.h\"],\n    }),\n    hdrs = [\"fd_reader.h\"],\n    defines = select({\n        \"//riegeli/bytes:disable_copy_file_range\": [\"RIEGELI_DISABLE_COPY_FILE_RANGE\"],\n        \"//conditions:default\": [],\n    }),\n    # fd_reader.cc has #define before #include to influence what the included\n    # files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \":buffer_options\",\n        \":buffered_reader\",\n        \":fd_handle\",\n        \":fd_internal\",\n        \":path_ref\",\n        \":reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:global\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:bits\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ] + select({\n        \"@platforms//os:windows\": [],\n        \"//conditions:default\": [\n            \":buffered_writer\",\n            \":writer\",\n            \"//riegeli/base:type_id\",\n        ],\n    }),\n)\n\ncc_library(\n    name = \"fd_mmap_reader\",\n    srcs = [\"fd_mmap_reader.cc\"],\n    hdrs = [\"fd_mmap_reader.h\"],\n    # fd_mmap_reader.cc has #define before #include to influence what the\n    # included files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \":chain_reader\",\n        \":fd_handle\",\n        \":fd_internal\",\n        \":path_ref\",\n        \":reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ] + select({\n        \"@platforms//os:windows\": [],\n        \"//conditions:default\": [\n            \"//riegeli/base:global\",\n            \"@com_google_absl//absl/status:statusor\",\n        ],\n    }),\n)\n\ncc_library(\n    name = \"fd_writer\",\n    srcs = [\"fd_writer.cc\"],\n    hdrs = [\"fd_writer.h\"],\n    # fd_writer.cc has #define before #include to influence what the included\n    # files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \":buffer_options\",\n        \":buffered_writer\",\n        \":fd_handle\",\n        \":fd_internal\",\n        \":fd_reader\",\n        \":path_ref\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:global\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:type_id\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:bits\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"fd_internal\",\n    srcs = [\"fd_internal.cc\"],\n    hdrs = [\n        \"fd_internal.h\",\n        \"fd_internal_for_cc.h\",\n    ],\n    # fd_internal.cc has #define before #include to influence what the included\n    # files provide.\n    features = [\"-use_header_modules\"],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ] + select({\n        \"@platforms//os:windows\": [],\n        \"//conditions:default\": [\n            \"//riegeli/base:arithmetic\",\n            \"//riegeli/base:buffer\",\n            \"@com_google_absl//absl/base:core_headers\",\n        ],\n    }),\n)\n\ncc_library(\n    name = \"std_io\",\n    srcs = [\"std_io.cc\"],\n    hdrs = [\"std_io.h\"],\n    deps = [\n        \":fd_handle\",\n        \":fd_reader\",\n        \":fd_writer\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:global\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:sized_shared_buffer\",\n        \"@com_google_absl//absl/base:core_headers\",\n    ],\n)\n\ncc_library(\n    name = \"istream_reader\",\n    srcs = [\n        \"iostream_internal.h\",\n        \"istream_reader.cc\",\n    ],\n    hdrs = [\"istream_reader.h\"],\n    deps = [\n        \":buffer_options\",\n        \":buffered_reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:global\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"ostream_writer\",\n    srcs = [\n        \"iostream_internal.h\",\n        \"ostream_writer.cc\",\n    ],\n    hdrs = [\"ostream_writer.h\"],\n    deps = [\n        \":buffer_options\",\n        \":buffered_writer\",\n        \":istream_reader\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:global\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"reader_istream\",\n    srcs = [\"reader_istream.cc\"],\n    hdrs = [\"reader_istream.h\"],\n    deps = [\n        \":reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n    ],\n)\n\ncc_library(\n    name = \"writer_ostream\",\n    srcs = [\"writer_ostream.cc\"],\n    hdrs = [\"writer_ostream.h\"],\n    deps = [\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"cfile_handle\",\n    srcs = [\"cfile_handle.cc\"],\n    hdrs = [\"cfile_handle.h\"],\n    # cfile_handle.cc has #define before #include to influence what the\n    # included files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \":cfile_internal\",\n        \":path_ref\",\n        \"//riegeli/base:any\",\n        \"//riegeli/base:c_string_ref\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:shared_ptr\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:type_erased_ref\",\n        \"//riegeli/base:type_traits\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ] + select({\n        \"@platforms//os:windows\": [\n            \"//riegeli/base:unicode\",\n        ],\n        \"//conditions:default\": [],\n    }),\n)\n\ncc_library(\n    name = \"cfile_reader\",\n    srcs = [\"cfile_reader.cc\"],\n    hdrs = [\"cfile_reader.h\"],\n    features = select({\n        \"@platforms//os:windows\": [],\n        # cfile_reader.cc has #define before #include to influence what the\n        # included files provide.\n        \"//conditions:default\": [\"-use_header_modules\"],\n    }),\n    deps = [\n        \":buffer_options\",\n        \":buffered_reader\",\n        \":cfile_handle\",\n        \":cfile_internal\",\n        \":file_mode_string\",\n        \":path_ref\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:global\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:string_ref\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:dynamic_annotations\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"cfile_writer\",\n    srcs = [\"cfile_writer.cc\"],\n    hdrs = [\"cfile_writer.h\"],\n    features = select({\n        \"@platforms//os:windows\": [],\n        # cfile_writer.cc has #define before #include to influence what the\n        # included files provide.\n        \"//conditions:default\": [\"-use_header_modules\"],\n    }),\n    deps = [\n        \":buffer_options\",\n        \":buffered_writer\",\n        \":cfile_handle\",\n        \":cfile_internal\",\n        \":cfile_reader\",\n        \":file_mode_string\",\n        \":path_ref\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:global\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:string_ref\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"cfile_internal\",\n    srcs = [\"cfile_internal.cc\"],\n    hdrs = [\n        \"cfile_internal.h\",\n        \"cfile_internal_for_cc.h\",\n    ],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \":fd_internal\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"reader_cfile\",\n    srcs = [\"reader_cfile.cc\"],\n    hdrs = [\"reader_cfile.h\"],\n    # reader_cfile.cc has #define before #include to influence what the included\n    # files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \":cfile_handle\",\n        \":path_ref\",\n        \":reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n    ],\n)\n\ncc_library(\n    name = \"writer_cfile\",\n    srcs = [\n        \"writer_cfile.cc\",\n    ],\n    hdrs = [\"writer_cfile.h\"],\n    # writer_cfile.cc has #define before #include to influence what the included\n    # files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \":cfile_handle\",\n        \":path_ref\",\n        \":reader\",\n        \":writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:dynamic_annotations\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"stringify_writer\",\n    hdrs = [\"stringify_writer.h\"],\n    deps = [\n        \":buffered_writer\",\n        \":prefix_limiting_writer\",\n        \":writer\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/bytes/array_backward_writer.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/array_backward_writer.h\"\n\n#include <stddef.h>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/types.h\"\n\nnamespace riegeli {\n\nbool ArrayBackwardWriterBase::PushBehindScratch(size_t recommended_length) {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PushableBackwardWriter::PushBehindScratch(): \"\n         \"some space available, use Push() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableBackwardWriter::PushBehindScratch(): \"\n         \"scratch used\";\n  return ForcePushUsingScratch();\n}\n\nbool ArrayBackwardWriterBase::WriteBehindScratch(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(string_view): \"\n         \"scratch used\";\n  return FailOverflow();\n}\n\nbool ArrayBackwardWriterBase::FlushBehindScratch(FlushType flush_type) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableBackwardWriter::FlushBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  written_ = absl::MakeSpan(cursor(), start_to_cursor());\n  return true;\n}\n\nbool ArrayBackwardWriterBase::TruncateBehindScratch(Position new_size) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::TruncateBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(new_size > start_to_cursor())) return false;\n  set_cursor(start() - IntCast<size_t>(new_size));\n  return true;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/array_backward_writer.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_ARRAY_BACKWARD_WRITER_H_\n#define RIEGELI_BYTES_ARRAY_BACKWARD_WRITER_H_\n\n#include <stddef.h>\n\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/pushable_backward_writer.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `ArrayBackwardWriter`.\nclass ArrayBackwardWriterBase : public PushableBackwardWriter {\n public:\n  // Returns the array being written to. Unchanged by `Close()`.\n  virtual absl::Span<char> DestSpan() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns written data in a suffix of the original array. Valid only after\n  // `Close()` or `Flush()`.\n  absl::Span<char> written() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return written_;\n  }\n\n  bool SupportsTruncate() override { return true; }\n\n protected:\n  using PushableBackwardWriter::PushableBackwardWriter;\n\n  ArrayBackwardWriterBase(ArrayBackwardWriterBase&& that) noexcept;\n  ArrayBackwardWriterBase& operator=(ArrayBackwardWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(absl::Span<char> dest);\n  void set_written(absl::Span<char> written) { written_ = written; }\n\n  bool PushBehindScratch(size_t recommended_length) override;\n  using PushableBackwardWriter::WriteBehindScratch;\n  bool WriteBehindScratch(absl::string_view src) override;\n  bool FlushBehindScratch(FlushType flush_type) override;\n  bool TruncateBehindScratch(Position new_size) override;\n\n private:\n  // Written data. Valid only after `Close()` or `Flush()`.\n  absl::Span<char> written_;\n\n  // Invariants if `ok()` and scratch is not used:\n  //   `limit() == DestSpan().data()`\n  //   `start_to_limit() == DestSpan().size()`\n  //   `start_pos() == 0`\n};\n\n// A `BackwardWriter` which writes to a preallocated array with a known size\n// limit.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the array being written to. `Dest` must support\n// `Dependency<absl::Span<char>, Dest>`, e.g.\n// `absl::Span<char>` (not owned, default), `std::string*` (not owned),\n// `std::string` (owned), `Any<absl::Span<char>>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument, except that CTAD is deleted if the\n// first constructor argument is a reference to a type that `absl::Span<char>`\n// would be constructible from, other than `absl::Span<char>` itself (to avoid\n// writing to an unintentionally separate copy of an existing object).\n//\n// The array must not be destroyed until the `ArrayBackwardWriter` is closed or\n// no longer used.\ntemplate <typename Dest = absl::Span<char>>\nclass ArrayBackwardWriter : public ArrayBackwardWriterBase {\n public:\n  // Creates a closed `ArrayBackwardWriter`.\n  explicit ArrayBackwardWriter(Closed) noexcept\n      : ArrayBackwardWriterBase(kClosed) {}\n\n  // Will write to the array provided by `dest`.\n  explicit ArrayBackwardWriter(Initializer<Dest> dest);\n\n  // Will write to `absl::MakeSpan(dest, size)`. This constructor is present\n  // only if `Dest` is `absl::Span<char>`.\n  template <typename DependentDest = Dest,\n            std::enable_if_t<std::is_same_v<DependentDest, absl::Span<char>>,\n                             int> = 0>\n  explicit ArrayBackwardWriter(char* dest ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                               size_t size);\n\n  ArrayBackwardWriter(ArrayBackwardWriter&& that) = default;\n  ArrayBackwardWriter& operator=(ArrayBackwardWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `ArrayBackwardWriter`. This\n  // avoids constructing a temporary `ArrayBackwardWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest);\n  template <typename DependentDest = Dest,\n            std::enable_if_t<std::is_same_v<DependentDest, absl::Span<char>>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(char* dest, size_t size);\n\n  // Returns the object providing and possibly owning the array being written\n  // to. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  absl::Span<char> DestSpan() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the array being written to.\n  MovingDependency<absl::Span<char>, Dest, Mover> dest_;\n};\n\nexplicit ArrayBackwardWriter(Closed) -> ArrayBackwardWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit ArrayBackwardWriter(Dest&& dest)\n    -> ArrayBackwardWriter<std::conditional_t<\n        std::conjunction_v<\n            std::negation<std::is_same<std::decay_t<Dest>, absl::Span<char>>>,\n            std::is_lvalue_reference<Dest>,\n            std::is_constructible<absl::Span<char>, Dest>>,\n        DeleteCtad<Dest&&>, TargetT<Dest>>>;\nexplicit ArrayBackwardWriter(char* dest ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                             size_t size)\n    -> ArrayBackwardWriter<absl::Span<char>>;\n\n// Implementation details follow.\n\ninline ArrayBackwardWriterBase::ArrayBackwardWriterBase(\n    ArrayBackwardWriterBase&& that) noexcept\n    : PushableBackwardWriter(static_cast<PushableBackwardWriter&&>(that)),\n      written_(that.written_) {}\n\ninline ArrayBackwardWriterBase& ArrayBackwardWriterBase::operator=(\n    ArrayBackwardWriterBase&& that) noexcept {\n  PushableBackwardWriter::operator=(\n      static_cast<PushableBackwardWriter&&>(that));\n  written_ = that.written_;\n  return *this;\n}\n\ninline void ArrayBackwardWriterBase::Reset(Closed) {\n  PushableBackwardWriter::Reset(kClosed);\n  written_ = absl::Span<char>();\n}\n\ninline void ArrayBackwardWriterBase::Reset() {\n  PushableBackwardWriter::Reset();\n  written_ = absl::Span<char>();\n}\n\ninline void ArrayBackwardWriterBase::Initialize(absl::Span<char> dest) {\n  set_buffer(dest.data(), dest.size());\n}\n\ntemplate <typename Dest>\nclass ArrayBackwardWriter<Dest>::Mover {\n public:\n  static auto member() { return &ArrayBackwardWriter::dest_; }\n\n  explicit Mover(ArrayBackwardWriter& self, ArrayBackwardWriter& that)\n      : behind_scratch_(&self),\n        uses_buffer_(self.start() != nullptr),\n        start_to_cursor_(self.start_to_cursor()),\n        has_written_(self.written().data() != nullptr),\n        written_size_(self.written().size()) {\n    if (uses_buffer_) {\n      RIEGELI_ASSERT_EQ(that.dest_.get().data(), self.limit())\n          << \"ArrayBackwardWriter destination changed unexpectedly\";\n      RIEGELI_ASSERT_EQ(that.dest_.get().size(), self.start_to_limit())\n          << \"ArrayBackwardWriter destination changed unexpectedly\";\n    }\n    if (has_written_) {\n      RIEGELI_ASSERT(that.dest_.get().data() + that.dest_.get().size() ==\n                     self.written().data() + self.written().size())\n          << \"ArrayBackwardWriter destination changed unexpectedly\";\n    }\n  }\n\n  void Done(ArrayBackwardWriter& self) {\n    if (uses_buffer_) {\n      const absl::Span<char> dest = self.dest_.get();\n      self.set_buffer(dest.data(), dest.size(), start_to_cursor_);\n    }\n    if (has_written_) {\n      const absl::Span<char> dest = self.dest_.get();\n      self.set_written(absl::MakeSpan(dest.data() + dest.size() - written_size_,\n                                      written_size_));\n    }\n  }\n\n private:\n  BehindScratch behind_scratch_;\n  bool uses_buffer_;\n  size_t start_to_cursor_;\n  bool has_written_;\n  size_t written_size_;\n};\n\ntemplate <typename Dest>\ninline ArrayBackwardWriter<Dest>::ArrayBackwardWriter(Initializer<Dest> dest)\n    : dest_(std::move(dest)) {\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\ntemplate <\n    typename DependentDest,\n    std::enable_if_t<std::is_same_v<DependentDest, absl::Span<char>>, int>>\ninline ArrayBackwardWriter<Dest>::ArrayBackwardWriter(char* dest, size_t size)\n    : ArrayBackwardWriter(absl::MakeSpan(dest, size)) {}\n\ntemplate <typename Dest>\ninline void ArrayBackwardWriter<Dest>::Reset(Closed) {\n  ArrayBackwardWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void ArrayBackwardWriter<Dest>::Reset(Initializer<Dest> dest) {\n  ArrayBackwardWriterBase::Reset();\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\ntemplate <\n    typename DependentDest,\n    std::enable_if_t<std::is_same_v<DependentDest, absl::Span<char>>, int>>\ninline void ArrayBackwardWriter<Dest>::Reset(char* dest, size_t size) {\n  Reset(absl::MakeSpan(dest, size));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_ARRAY_BACKWARD_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/array_writer.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/array_writer.h\"\n\n#include <stddef.h>\n\n#include <optional>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pushable_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/string_reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid ArrayWriterBase::Done() {\n  PushableWriter::Done();\n  associated_reader_.Reset();\n}\n\nbool ArrayWriterBase::PushBehindScratch(size_t recommended_length) {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PushableWriter::PushBehindScratch(): \"\n         \"some space available, use Push() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::PushBehindScratch(): \"\n         \"scratch used\";\n  return ForcePushUsingScratch();\n}\n\nbool ArrayWriterBase::WriteBehindScratch(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of \"\n         \"PushableWriter::WriteBehindScratch(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableWriter::WriteBehindScratch(string_view): \"\n         \"scratch used\";\n  return FailOverflow();\n}\n\nbool ArrayWriterBase::FlushBehindScratch(FlushType flush_type) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::FlushBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const size_t size = UnsignedMax(start_to_cursor(), written_.size());\n  written_ = absl::MakeSpan(start(), size);\n  return true;\n}\n\nbool ArrayWriterBase::SeekBehindScratch(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of PushableWriter::SeekBehindScratch(): \"\n         \"position unchanged, use Seek() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::SeekBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const size_t size = UnsignedMax(start_to_cursor(), written_.size());\n  if (ABSL_PREDICT_FALSE(new_pos > size)) {\n    set_cursor(start() + size);\n    return false;\n  }\n  written_ = absl::MakeSpan(start(), size);\n  set_cursor(start() + IntCast<size_t>(new_pos));\n  return true;\n}\n\nstd::optional<Position> ArrayWriterBase::SizeBehindScratch() {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::SizeBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  return UnsignedMax(start_to_cursor(), written_.size());\n}\n\nbool ArrayWriterBase::TruncateBehindScratch(Position new_size) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::TruncateBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const size_t size = UnsignedMax(start_to_cursor(), written_.size());\n  if (ABSL_PREDICT_FALSE(new_size > size)) {\n    set_cursor(start() + size);\n    return false;\n  }\n  written_ = absl::MakeSpan(start(), IntCast<size_t>(new_size));\n  set_cursor(start() + IntCast<size_t>(new_size));\n  return true;\n}\n\nReader* ArrayWriterBase::ReadModeBehindScratch(Position initial_pos) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::ReadModeBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  const size_t size = UnsignedMax(start_to_cursor(), written_.size());\n  StringReader<>* const reader = associated_reader_.ResetReader(start(), size);\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/array_writer.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_ARRAY_WRITER_H_\n#define RIEGELI_BYTES_ARRAY_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pushable_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nclass Reader;\ntemplate <typename Src>\nclass StringReader;\n\n// Template parameter independent part of `ArrayWriter`.\nclass ArrayWriterBase : public PushableWriter {\n public:\n  // Returns the array being written to. Unchanged by `Close()`.\n  virtual absl::Span<char> DestSpan() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns written data in a prefix of the original array. Valid only after\n  // `Close()` or `Flush()`.\n  absl::Span<char> written() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return written_;\n  }\n  absl::Span<char> Digest() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    Flush();\n    return written();\n  }\n\n  bool SupportsRandomAccess() override { return true; }\n  bool SupportsReadMode() override { return true; }\n\n protected:\n  using PushableWriter::PushableWriter;\n\n  ArrayWriterBase(ArrayWriterBase&& that) noexcept;\n  ArrayWriterBase& operator=(ArrayWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(absl::Span<char> dest);\n  void set_written(absl::Span<char> written) { written_ = written; }\n\n  void Done() override;\n  bool PushBehindScratch(size_t recommended_length) override;\n  using PushableWriter::WriteBehindScratch;\n  bool WriteBehindScratch(absl::string_view src) override;\n  bool FlushBehindScratch(FlushType flush_type) override;\n  bool SeekBehindScratch(Position new_pos) override;\n  std::optional<Position> SizeBehindScratch() override;\n  bool TruncateBehindScratch(Position new_size) override;\n  Reader* ReadModeBehindScratch(Position initial_pos) override;\n\n private:\n  // Written data. Valid only after `Close()` or `Flush()`.\n  //\n  // Size of written data is always `UnsignedMax(pos(), written_.size())`.\n  // This is used to determine the size after seeking backwards.\n  absl::Span<char> written_;\n\n  AssociatedReader<StringReader<absl::string_view>> associated_reader_;\n\n  // Invariants if `ok()` and scratch is not used:\n  //   `start() == DestSpan().data()`\n  //   `start_to_limit() == DestSpan().size()`\n  //   `start_pos() == 0`\n};\n\n// A `Writer` which writes to a preallocated array with a known size limit.\n//\n// It supports `ReadMode()`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the array being written to. `Dest` must support\n// `Dependency<absl::Span<char>, Dest>`, e.g.\n// `absl::Span<char>` (not owned, default), `std::string*` (not owned),\n// `std::string` (owned), `Any<absl::Span<char>>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument, except that CTAD is deleted if the\n// first constructor argument is a reference to a type that `absl::Span<char>`\n// would be constructible from, other than `absl::Span<char>` itself (to avoid\n// writing to an unintentionally separate copy of an existing object).\n//\n// The array must not be destroyed until the `ArrayWriter` is closed or no\n// longer used.\ntemplate <typename Dest = absl::Span<char>>\nclass ArrayWriter : public ArrayWriterBase {\n public:\n  // Creates a closed `ArrayWriter`.\n  explicit ArrayWriter(Closed) noexcept : ArrayWriterBase(kClosed) {}\n\n  // Will write to the array provided by `dest`.\n  explicit ArrayWriter(Initializer<Dest> dest);\n\n  // Will write to `absl::MakeSpan(dest, size)`. This constructor is present\n  // only if `Dest` is `absl::Span<char>`.\n  template <typename DependentDest = Dest,\n            std::enable_if_t<std::is_same_v<DependentDest, absl::Span<char>>,\n                             int> = 0>\n  explicit ArrayWriter(char* dest ABSL_ATTRIBUTE_LIFETIME_BOUND, size_t size);\n\n  ArrayWriter(ArrayWriter&& that) = default;\n  ArrayWriter& operator=(ArrayWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `ArrayWriter`. This avoids\n  // constructing a temporary `ArrayWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest);\n  template <typename DependentDest = Dest,\n            std::enable_if_t<std::is_same_v<DependentDest, absl::Span<char>>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(char* dest, size_t size);\n\n  // Returns the object providing and possibly owning the array being written\n  // to. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  absl::Span<char> DestSpan() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the array being written to.\n  MovingDependency<absl::Span<char>, Dest, Mover> dest_;\n};\n\nexplicit ArrayWriter(Closed) -> ArrayWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit ArrayWriter(Dest&& dest) -> ArrayWriter<std::conditional_t<\n    std::conjunction_v<\n        std::negation<std::is_same<std::decay_t<Dest>, absl::Span<char>>>,\n        std::is_lvalue_reference<Dest>,\n        std::is_constructible<absl::Span<char>, Dest>>,\n    DeleteCtad<Dest&&>, TargetT<Dest>>>;\nexplicit ArrayWriter(char* dest, size_t size) -> ArrayWriter<absl::Span<char>>;\n\n// Implementation details follow.\n\ninline ArrayWriterBase::ArrayWriterBase(ArrayWriterBase&& that) noexcept\n    : PushableWriter(static_cast<PushableWriter&&>(that)),\n      written_(that.written_),\n      associated_reader_(std::move(that.associated_reader_)) {}\n\ninline ArrayWriterBase& ArrayWriterBase::operator=(\n    ArrayWriterBase&& that) noexcept {\n  PushableWriter::operator=(static_cast<PushableWriter&&>(that));\n  written_ = that.written_;\n  associated_reader_ = std::move(that.associated_reader_);\n  return *this;\n}\n\ninline void ArrayWriterBase::Reset(Closed) {\n  PushableWriter::Reset(kClosed);\n  written_ = absl::Span<char>();\n  associated_reader_.Reset();\n}\n\ninline void ArrayWriterBase::Reset() {\n  PushableWriter::Reset();\n  written_ = absl::Span<char>();\n  associated_reader_.Reset();\n}\n\ninline void ArrayWriterBase::Initialize(absl::Span<char> dest) {\n  set_buffer(dest.data(), dest.size());\n}\n\ntemplate <typename Dest>\nclass ArrayWriter<Dest>::Mover {\n public:\n  static auto member() { return &ArrayWriter::dest_; }\n\n  explicit Mover(ArrayWriter& self, ArrayWriter& that)\n      : behind_scratch_(&self),\n        uses_buffer_(self.start() != nullptr),\n        start_to_cursor_(self.start_to_cursor()),\n        has_written_(self.written().data() != nullptr),\n        written_size_(self.written().size()) {\n    if (uses_buffer_) {\n      RIEGELI_ASSERT_EQ(that.dest_.get().data(), self.start())\n          << \"ArrayWriter destination changed unexpectedly\";\n      RIEGELI_ASSERT_EQ(that.dest_.get().size(), self.start_to_limit())\n          << \"ArrayWriter destination changed unexpectedly\";\n    }\n    if (has_written_) {\n      RIEGELI_ASSERT_EQ(that.dest_.get().data(), self.written().data())\n          << \"ArrayWriter destination changed unexpectedly\";\n    }\n  }\n\n  void Done(ArrayWriter& self) {\n    if (uses_buffer_) {\n      const absl::Span<char> dest = self.dest_.get();\n      self.set_buffer(dest.data(), dest.size(), start_to_cursor_);\n    }\n    if (has_written_) {\n      const absl::Span<char> dest = self.dest_.get();\n      self.set_written(absl::MakeSpan(dest.data(), written_size_));\n    }\n  }\n\n private:\n  BehindScratch behind_scratch_;\n  bool uses_buffer_;\n  size_t start_to_cursor_;\n  bool has_written_;\n  size_t written_size_;\n};\n\ntemplate <typename Dest>\ninline ArrayWriter<Dest>::ArrayWriter(Initializer<Dest> dest)\n    : dest_(std::move(dest)) {\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\ntemplate <\n    typename DependentDest,\n    std::enable_if_t<std::is_same_v<DependentDest, absl::Span<char>>, int>>\ninline ArrayWriter<Dest>::ArrayWriter(char* dest ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                                      size_t size)\n    : ArrayWriter(absl::MakeSpan(dest, size)) {}\n\ntemplate <typename Dest>\ninline void ArrayWriter<Dest>::Reset(Closed) {\n  ArrayWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void ArrayWriter<Dest>::Reset(Initializer<Dest> dest) {\n  ArrayWriterBase::Reset();\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\ntemplate <\n    typename DependentDest,\n    std::enable_if_t<std::is_same_v<DependentDest, absl::Span<char>>, int>>\ninline void ArrayWriter<Dest>::Reset(char* dest, size_t size) {\n  Reset(absl::MakeSpan(dest, size));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_ARRAY_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/backward_writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/backward_writer.h\"\n\n#include <stddef.h>\n\n#include <cmath>\n#include <cstring>\n#include <limits>\n#include <optional>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/int128.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/str_format.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_utils.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/write_int_internal.h\"\n\nnamespace riegeli {\n\nnamespace {\n\ntemplate <typename T>\ninline bool WriteUnsigned(T src, BackwardWriter& dest) {\n  // `digits10` is rounded down, `kMaxNumDigits` is rounded up, hence `+ 1`.\n  constexpr size_t kMaxNumDigits = std::numeric_limits<T>::digits10 + 1;\n  if (ABSL_PREDICT_FALSE(!dest.Push(kMaxNumDigits))) return false;\n  dest.set_cursor(\n      write_int_internal::WriteDecUnsignedBackward(src, dest.cursor()));\n  return true;\n}\n\ntemplate <typename T>\ninline bool WriteSigned(T src, BackwardWriter& dest) {\n  // `digits10` is rounded down, `kMaxNumDigits` is rounded up, hence `+ 1`.\n  constexpr size_t kMaxNumDigits = std::numeric_limits<T>::digits10 + 1;\n  // `+ 1` for the minus sign.\n  if (ABSL_PREDICT_FALSE(!dest.Push(kMaxNumDigits + 1))) return false;\n  dest.set_cursor(\n      write_int_internal::WriteDecSignedBackward(src, dest.cursor()));\n  return true;\n}\n\n}  // namespace\n\nvoid BackwardWriter::OnFail() { set_buffer(start()); }\n\nabsl::Status BackwardWriter::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) return Annotate(status, absl::StrCat(\"at byte \", pos()));\n  return status;\n}\n\nbool BackwardWriter::FailOverflow() {\n  return Fail(absl::ResourceExhaustedError(\"BackwardWriter position overflow\"));\n}\n\nbool BackwardWriter::Write(const Chain& src) {\n#ifdef MEMORY_SANITIZER\n  for (const absl::string_view fragment : src.blocks()) {\n    AssertInitialized(fragment.data(), fragment.size());\n  }\n#endif\n  if (ABSL_PREDICT_TRUE(available() >= src.size() &&\n                        src.size() <= kMaxBytesToCopy)) {\n    move_cursor(src.size());\n    src.CopyTo(cursor());\n    return true;\n  }\n  AssertInitialized(cursor(), start_to_cursor());\n  return WriteSlow(src);\n}\n\nbool BackwardWriter::Write(Chain&& src) {\n#ifdef MEMORY_SANITIZER\n  for (const absl::string_view fragment : src.blocks()) {\n    AssertInitialized(fragment.data(), fragment.size());\n  }\n#endif\n  if (ABSL_PREDICT_TRUE(available() >= src.size() &&\n                        src.size() <= kMaxBytesToCopy)) {\n    move_cursor(src.size());\n    src.CopyTo(cursor());\n    return true;\n  }\n  AssertInitialized(cursor(), start_to_cursor());\n  return WriteSlow(std::move(src));\n}\n\nbool BackwardWriter::Write(const absl::Cord& src) {\n#ifdef MEMORY_SANITIZER\n  for (const absl::string_view fragment : src.Chunks()) {\n    AssertInitialized(fragment.data(), fragment.size());\n  }\n#endif\n  if (ABSL_PREDICT_TRUE(available() >= src.size() &&\n                        src.size() <= kMaxBytesToCopy)) {\n    move_cursor(src.size());\n    cord_internal::CopyCordToArray(src, cursor());\n    return true;\n  }\n  AssertInitialized(cursor(), start_to_cursor());\n  return WriteSlow(src);\n}\n\nbool BackwardWriter::Write(absl::Cord&& src) {\n#ifdef MEMORY_SANITIZER\n  for (const absl::string_view fragment : src.Chunks()) {\n    AssertInitialized(fragment.data(), fragment.size());\n  }\n#endif\n  if (ABSL_PREDICT_TRUE(available() >= src.size() &&\n                        src.size() <= kMaxBytesToCopy)) {\n    move_cursor(src.size());\n    cord_internal::CopyCordToArray(src, cursor());\n    return true;\n  }\n  AssertInitialized(cursor(), start_to_cursor());\n  return WriteSlow(std::move(src));\n}\n\nbool BackwardWriter::Write(signed char src) { return WriteSigned(src, *this); }\n\nbool BackwardWriter::Write(unsigned char src) {\n  return WriteUnsigned(src, *this);\n}\n\nbool BackwardWriter::Write(short src) { return WriteSigned(src, *this); }\n\nbool BackwardWriter::Write(unsigned short src) {\n  return WriteUnsigned(src, *this);\n}\n\nbool BackwardWriter::Write(int src) { return WriteSigned(src, *this); }\n\nbool BackwardWriter::Write(unsigned src) { return WriteUnsigned(src, *this); }\n\nbool BackwardWriter::Write(long src) { return WriteSigned(src, *this); }\n\nbool BackwardWriter::Write(unsigned long src) {\n  return WriteUnsigned(src, *this);\n}\n\nbool BackwardWriter::Write(long long src) { return WriteSigned(src, *this); }\n\nbool BackwardWriter::Write(unsigned long long src) {\n  return WriteUnsigned(src, *this);\n}\n\nbool BackwardWriter::Write(absl::int128 src) { return WriteSigned(src, *this); }\n\nbool BackwardWriter::Write(absl::uint128 src) {\n  return WriteUnsigned(src, *this);\n}\n\n// TODO: Optimize implementations below.\nbool BackwardWriter::Write(float src) { return Write(absl::StrCat(src)); }\n\nbool BackwardWriter::Write(double src) { return Write(absl::StrCat(src)); }\n\nbool BackwardWriter::Write(long double src) {\n  return Write(\n      absl::StrFormat(\"%g\",\n                      // Consistently use \"nan\", never \"-nan\".\n                      ABSL_PREDICT_FALSE(std::isnan(src))\n                          ? std::numeric_limits<long double>::quiet_NaN()\n                          : src));\n}\n\nbool BackwardWriter::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  do {\n    const size_t available_length = available();\n    move_cursor(available_length);\n    riegeli::null_safe_memcpy(\n        cursor(), src.data() + src.size() - available_length, available_length);\n    src.remove_suffix(available_length);\n    if (ABSL_PREDICT_FALSE(!PushSlow(1, src.size()))) return false;\n  } while (src.size() > available());\n  move_cursor(src.size());\n  std::memcpy(cursor(), src.data(), src.size());\n  return true;\n}\n\nbool BackwardWriter::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  return Write(absl::string_view(src));\n}\n\nbool BackwardWriter::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  for (Chain::Blocks::const_reverse_iterator iter = src.blocks().crbegin();\n       iter != src.blocks().crend(); ++iter) {\n    if (ABSL_PREDICT_FALSE(!Write(absl::string_view(*iter)))) return false;\n  }\n  return true;\n}\n\nbool BackwardWriter::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  // Not `std::move(src)`: forward to `WriteSlow(const Chain&)`.\n  return WriteSlow(src);\n}\n\nbool BackwardWriter::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (const std::optional<absl::string_view> flat = src.TryFlat();\n      flat != std::nullopt) {\n    return Write(*flat);\n  }\n  if (src.size() <= available()) {\n    move_cursor(src.size());\n    cord_internal::CopyCordToArray(src, cursor());\n    return true;\n  }\n  std::vector<absl::string_view> fragments(src.chunk_begin(), src.chunk_end());\n  for (std::vector<absl::string_view>::const_reverse_iterator iter =\n           fragments.crbegin();\n       iter != fragments.crend(); ++iter) {\n    if (ABSL_PREDICT_FALSE(!Write(*iter))) return false;\n  }\n  return true;\n}\n\nbool BackwardWriter::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  // Not `std::move(src)`: forward to `WriteSlow(const absl::Cord&)`.\n  return WriteSlow(src);\n}\n\nbool BackwardWriter::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  while (src.size() > available()) {\n    const size_t available_length = available();\n    move_cursor(available_length);\n    riegeli::null_safe_memset(cursor(), src.fill(), available_length);\n    src.Extract(available_length);\n    if (ABSL_PREDICT_FALSE(!Push(1, SaturatingIntCast<size_t>(src.size())))) {\n      return false;\n    }\n  }\n  move_cursor(IntCast<size_t>(src.size()));\n  std::memset(cursor(), src.fill(), IntCast<size_t>(src.size()));\n  return true;\n}\n\nbool BackwardWriter::FlushImpl(FlushType flush_type) { return ok(); }\n\nbool BackwardWriter::TruncateImpl(Position new_size) {\n  return Fail(\n      absl::UnimplementedError(\"BackwardWriter::Truncate() not supported\"));\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/backward_writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_BACKWARD_WRITER_H_\n#define RIEGELI_BYTES_BACKWARD_WRITER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <optional>\n#include <tuple>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/int128.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/has_absl_stringify.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/restricted_chain_writer.h\"\n#include \"riegeli/bytes/stringify.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// Abstract class `BackwardWriter` writes sequences of bytes to a destination,\n// like `Writer`, but back to front.\n//\n// Sequential writing is supported, random access is not supported, truncation\n// is optionally supported.\nclass BackwardWriter : public Object {\n public:\n  // The same as `Object::Close()`.\n  //\n  // The implementation in this class adds an assertion.\n  bool Close();\n\n  // If `write_size_hint` is not `std::nullopt`, hints that this amount of data\n  // will be written sequentially from the current position, then `Close()` will\n  // be called.\n  //\n  // This may improve performance and memory usage:\n  //  * Larger buffer sizes may be used before reaching the size hint, and\n  //    a smaller buffer size may be used when reaching the size hint.\n  //  * This hint may be propagated to owned destinations.\n  //  * Other consequences are possible.\n  //\n  // If the hint turns out to not match reality, nothing breaks. It is better if\n  // `write_size_hint` is slightly too large than slightly too small.\n  //\n  // `SetWriteSizeHint()` is usually be called from the same abstraction layer\n  // which later calls `Close()`.\n  void SetWriteSizeHint(std::optional<Position> write_size_hint);\n\n  // Ensures that enough space is available in the buffer: if less than\n  // `min_length` of space is available, pushes previously written data to the\n  // destination, and points `cursor()` and `limit()` to space following the\n  // current position with length at least `min_length`, preferably\n  // `recommended_length`.\n  //\n  // The current position does not change with `Push()`. It changes with e.g.\n  // `move_cursor()` and `Write()`.\n  //\n  // If `recommended_length < min_length`, `recommended_length` is assumed to be\n  // `min_length`.\n  //\n  // Return values:\n  //  * `true`  - success (`available() >= min_length`)\n  //  * `false` - failure (`available() < min_length`, `!ok()`)\n  bool Push(size_t min_length = 1, size_t recommended_length = 0);\n\n  // Buffer pointers. Space between `start()` (exclusive upper bound) and\n  // `limit()` (inclusive lower bound) is available for immediate writing data\n  // to it, with `cursor()` pointing to the current position going downwards\n  // (past the next byte to write).\n  //\n  // Memory before the address to which `cursor()` is eventually moved must not\n  // be clobbered.\n  //\n  // Non-const member functions may change buffer pointers, including changing\n  // how much data around the current position are buffered.\n  //\n  // Invariants:\n  //   `start() >= cursor() >= limit()` (possibly all `nullptr`)\n  //   if `!ok()` then `start() == cursor() == limit() == nullptr`\n  char* start() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return start_; }\n  char* cursor() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return cursor_; }\n  char* limit() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return limit_; }\n\n  // Decrements the value of `cursor()`. Does not change `start()` nor\n  // `limit()`. Call this during writing data under `cursor()` to indicate how\n  // much was written.\n  //\n  // Precondition: `length <= available()`\n  void move_cursor(size_t length);\n\n  // Sets the value of `cursor()`. Does not change `start()` nor `limit()`. Call\n  // this during writing data under `cursor()` to indicate how much was written.\n  //\n  // Precondition: `start() >= cursor >= limit()`\n  void set_cursor(char* cursor);\n\n  // Returns the amount of space available in the buffer, between `cursor()` and\n  // `limit()`.\n  //\n  // Invariant: if `!ok()` then `available() == 0`\n  size_t available() const { return PtrDistance(limit_, cursor_); }\n\n  // Returns the buffer size, between `start()` and `limit()`.\n  //\n  // Invariant: if `!ok()` then `start_to_limit() == 0`\n  size_t start_to_limit() const { return PtrDistance(limit_, start_); }\n\n  // Returns the amount of data written to the buffer, between `start()` and\n  // `cursor()`.\n  //\n  // Invariant: if `!ok()` then `start_to_cursor() == 0`\n  size_t start_to_cursor() const { return PtrDistance(cursor_, start_); }\n\n  // Writes a single byte to the buffer or the destination.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool WriteByte(uint8_t src) { return Write(static_cast<char>(src)); }\n\n  // Writes a fixed number of bytes from `src` to the buffer and/or the\n  // destination. The whole `src` is prepended, bytes are not reversed.\n  //\n  // Return values:\n  //  * `true`  - success (`src.size()` bytes written)\n  //  * `false` - failure (a suffix of less than `src.size()` bytes written,\n  //                       `!ok()`)\n  bool Write(char src);\n#if __cpp_char8_t\n  bool Write(char8_t src) { return Write(static_cast<char>(src)); }\n#endif\n  bool Write(BytesRef src);\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  bool Write(const char* src) { return Write(absl::string_view(src)); }\n  bool Write(ExternalRef src);\n  template <typename Src,\n            std::enable_if_t<SupportsExternalRefWhole<Src>::value, int> = 0>\n  bool Write(Src&& src);\n  bool Write(const Chain& src);\n  bool Write(Chain&& src);\n  bool Write(const absl::Cord& src);\n  bool Write(absl::Cord&& src);\n  bool Write(ByteFill src);\n\n  // Writes a stringified value to the buffer and/or the destination.\n  //\n  // Return values:\n  //  * `true`  - success\n  //  * `false` - failure (`!ok()`)\n  bool Write(signed char src);\n  bool Write(unsigned char src);\n  bool Write(short src);\n  bool Write(unsigned short src);\n  bool Write(int src);\n  bool Write(unsigned src);\n  bool Write(long src);\n  bool Write(unsigned long src);\n  bool Write(long long src);\n  bool Write(unsigned long long src);\n  bool Write(absl::int128 src);\n  bool Write(absl::uint128 src);\n  bool Write(float src);\n  bool Write(double src);\n  bool Write(long double src);\n  template <\n      typename Src,\n      std::enable_if_t<\n          std::conjunction_v<\n              absl::HasAbslStringify<Src>,\n              std::negation<std::is_convertible<Src&&, BytesRef>>,\n              std::negation<std::is_convertible<Src&&, const Chain&>>,\n              std::negation<std::is_convertible<Src&&, const absl::Cord&>>,\n              std::negation<std::is_convertible<Src&&, ByteFill>>>,\n          int> = 0>\n  bool Write(Src&& src);\n\n  // Other integer types are is not supported. Delete overloads to avoid\n  // implicit conversions.\n  bool Write(bool src) = delete;\n  bool Write(wchar_t src) = delete;\n  bool Write(char16_t src) = delete;\n  bool Write(char32_t src) = delete;\n\n  // Writes stringified values to the buffer and/or the destination.\n  //\n  // `srcs` are prepended in the reverse order, so that they appear in the\n  // destination in the same order as arguments of `Write()`.\n  //\n  // Return values:\n  //  * `true`  - success\n  //  * `false` - failure (`!ok()`)\n  template <typename... Srcs\n#if !__cpp_concepts\n            ,\n            std::enable_if_t<\n                std::conjunction_v<std::bool_constant<sizeof...(Srcs) != 1>,\n                                   IsStringifiable<Srcs...>>,\n                int> = 0\n#endif\n            >\n  bool Write(Srcs&&... srcs)\n#if __cpp_concepts\n      // For conjunctions, `requires` gives better error messages than\n      // `std::enable_if_t`, indicating the relevant argument.\n    requires(sizeof...(Srcs) != 1) && (IsStringifiable<Srcs>::value && ...)\n#endif\n  {\n    return WriteInternal<sizeof...(Srcs)>(\n        std::forward_as_tuple(std::forward<Srcs>(srcs)...));\n  }\n\n  // Pushes buffered data to the destination.\n  //\n  // This makes data written so far visible, but in contrast to `Close()`,\n  // keeps the possibility to write more data later. What exactly does it mean\n  // for data to be visible depends on the destination. If this is not\n  // applicable or not feasible, does nothing.\n  //\n  // The scope of objects to flush and the intended data durability (without a\n  // guarantee) are specified by `flush_type`:\n  //  * `FlushType::kFromObject`  - Makes data written so far visible in other\n  //                                objects, propagating flushing through owned\n  //                                dependencies of the given writer.\n  //  * `FlushType::kFromProcess` - Makes data written so far visible outside\n  //                                the process, propagating flushing through\n  //                                dependencies of the given writer.\n  //                                This is the default.\n  //  * `FlushType::kFromMachine` - Makes data written so far visible outside\n  //                                the process and durable in case of operating\n  //                                system crash, propagating flushing through\n  //                                dependencies of the given writer.\n  //\n  // Return values:\n  //  * `true ` - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool Flush(FlushType flush_type = FlushType::kFromProcess);\n\n  // Returns the current position (increasing as data are prepended).\n  //\n  // This is not necessarily 0 after creating the `BackwardWriter` if it\n  // prepends to a destination with existing contents, or if the\n  // `BackwardWriter` wraps another writer or output stream propagating its\n  // position.\n  //\n  // This may decrease when the `BackwardWriter` becomes not OK (due to\n  // buffering, previously written but unflushed data may be lost).\n  //\n  // `pos()` is unchanged by a successful `Close()`.\n  Position pos() const;\n\n  // Returns the position corresponding to `start()`,\n  // i.e. `pos() - start_to_cursor()`.\n  Position start_pos() const { return start_pos_; }\n\n  // Returns the position corresponding to `limit()`,\n  // i.e. `pos() + available()`.\n  Position limit_pos() const;\n\n  // Returns `true` if this `BackwardWriter` supports `Truncate()`.\n  virtual bool SupportsTruncate() { return false; }\n\n  // Discards the part of the destination after the given position. Sets the\n  // current position to the new end.\n  //\n  // Return values:\n  //  * `true`                 - success (destination truncated, `ok()`)\n  //  * `false` (when `ok()`)  - destination is smaller than `new_size`\n  //                             (position is set to end)\n  //  * `false` (when `!ok()`) - failure\n  //\n  // `Truncate()` is supported if `SupportsTruncate()` is `true`.\n  bool Truncate(Position new_size);\n\n protected:\n  using Object::Object;\n\n  // Moves the part of the object defined in this class.\n  //\n  // Buffer pointers do not need to satisfy their invariants during this part of\n  // the move, here they are merely exchanged with `nullptr` and copied.\n  BackwardWriter(BackwardWriter&& that) noexcept;\n  BackwardWriter& operator=(BackwardWriter&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `BackwardWriter`. This\n  // avoids constructing a temporary `BackwardWriter` and moving from it.\n  // Derived classes which redefine `Reset()` should include a call to\n  // `BackwardWriter::Reset()`.\n  void Reset(Closed);\n  void Reset();\n\n  // `BackwardWriter` overrides `Object::Done()` to set buffer pointers to\n  // `nullptr`. Derived classes which override it further should include a call\n  // to `BackwardWriter::Done()`.\n  void Done() override;\n\n  // `BackwardWriter` overrides `Object::OnFail()` to set buffer pointers to\n  // `nullptr`. Derived classes which override it further should include a call\n  // to `BackwardWriter::OnFail()`.\n  //\n  // `pos()` decreases by `start_to_cursor()` to indicate that any buffered\n  // data have been lost.\n  ABSL_ATTRIBUTE_COLD void OnFail() override;\n\n  // `BackwardWriter` overrides `Object::AnnotateStatusImpl()` to annotate the\n  // status with the current position.\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n\n  // Marks the `BackwardWriter` as failed with message\n  // \"BackwardWriter position overflow\". Always returns `false`.\n  //\n  // This can be called if the destination would exceed its maximum possible\n  // size or if `start_pos()` would overflow.\n  ABSL_ATTRIBUTE_COLD bool FailOverflow();\n\n  // Implementation of `SetWriteSizeHint()`.\n  virtual void SetWriteSizeHintImpl(\n      ABSL_ATTRIBUTE_UNUSED std::optional<Position> write_size_hint) {}\n\n  // Implementation of the slow part of `Push()`.\n  //\n  // Precondition: `available() < min_length`\n  virtual bool PushSlow(size_t min_length, size_t recommended_length) = 0;\n\n  // Sets the values of:\n  //  * `start()`  - to `limit + start_to_limit`\n  //  * `cursor()` - to `start() - start_to_cursor`\n  //  * `limit()`  - to `limit`\n  //\n  // Preconditions:\n  //   [`limit`..`limit + start_to_limit`) is a valid byte range\n  //   `start_to_cursor <= start_to_limit`\n  void set_buffer(char* limit = nullptr, size_t start_to_limit = 0,\n                  size_t start_to_cursor = 0);\n\n  // Implementation of the slow part of `Write()`.\n  //\n  // By default:\n  //  * `WriteSlow(absl::string_view)` and `WriteSlow(ByteFill)` are\n  //    implemented in terms of `PushSlow()`\n  //  * `WriteSlow(ExternalRef)`, `WriteSlow(const Chain&)`, and\n  //    `WriteSlow(const absl::Cord&)` are implemented in terms of\n  //    `WriteSlow(absl::string_view)`\n  //  * `WriteSlow(Chain&&)` is implemented in terms of\n  //    `WriteSlow(const Chain&)`\n  //  * `WriteSlow(absl::Cord&&)` is implemented in terms of\n  //    `WriteSlow(const absl::Cord&)`\n  //\n  // Precondition for `WriteSlow(absl::string_view)`:\n  //   `available() < src.size()`\n  //\n  // Precondition for `WriteSlow(ExternalRef)`, `WriteSlow(const Chain&)`,\n  // `WriteSlow(Chain&&)`, `WriteSlow(const absl::Cord&)`,\n  // `WriteSlow(absl::Cord&&), and `WriteSlow(ByteFill)`:\n  //   `UnsignedMin(available(), kMaxBytesToCopy) < src.size()`\n  virtual bool WriteSlow(absl::string_view src);\n  virtual bool WriteSlow(ExternalRef src);\n  virtual bool WriteSlow(const Chain& src);\n  virtual bool WriteSlow(Chain&& src);\n  virtual bool WriteSlow(const absl::Cord& src);\n  virtual bool WriteSlow(absl::Cord&& src);\n  virtual bool WriteSlow(ByteFill src);\n\n  // Implementation of `Flush()`, except that the parameter is not defaulted,\n  // which is problematic for virtual functions.\n  //\n  // By default does nothing and returns `ok()`.\n  virtual bool FlushImpl(FlushType flush_type);\n\n  // Increments the value of `start_pos()`.\n  void move_start_pos(Position length);\n\n  // Sets the value of `start_pos()`.\n  void set_start_pos(Position start_pos);\n\n  // Implementation of `Truncate()`.\n  //\n  // By default fails.\n  virtual bool TruncateImpl(Position new_size);\n\n private:\n  template <size_t index, typename... Srcs>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool WriteInternal(std::tuple<Srcs...>&& srcs) {\n    if constexpr (index > 0) {\n      return Write(std::forward<\n                   std::tuple_element_t<index - 1, std::tuple<Srcs...>>>(\n                 std::get<index - 1>(srcs))) &&\n             WriteInternal<index - 1>(std::move(srcs));\n    } else {\n      return true;\n    }\n  }\n\n  char* start_ = nullptr;\n  char* cursor_ = nullptr;\n  char* limit_ = nullptr;\n\n  // Destination position corresponding to `start_`.\n  //\n  // Invariant:\n  //   `start_pos_ <= std::numeric_limits<Position>::max() - start_to_limit()`\n  Position start_pos_ = 0;\n};\n\n// Implementation details follow.\n\ninline BackwardWriter::BackwardWriter(BackwardWriter&& that) noexcept\n    : Object(static_cast<Object&&>(that)),\n      start_(std::exchange(that.start_, nullptr)),\n      cursor_(std::exchange(that.cursor_, nullptr)),\n      limit_(std::exchange(that.limit_, nullptr)),\n      start_pos_(std::exchange(that.start_pos_, 0)) {}\n\ninline BackwardWriter& BackwardWriter::operator=(\n    BackwardWriter&& that) noexcept {\n  Object::operator=(static_cast<Object&&>(that));\n  start_ = std::exchange(that.start_, nullptr);\n  cursor_ = std::exchange(that.cursor_, nullptr);\n  limit_ = std::exchange(that.limit_, nullptr);\n  start_pos_ = std::exchange(that.start_pos_, 0);\n  return *this;\n}\n\ninline void BackwardWriter::Reset(Closed) {\n  Object::Reset(kClosed);\n  start_ = nullptr;\n  cursor_ = nullptr;\n  limit_ = nullptr;\n  start_pos_ = 0;\n}\n\ninline void BackwardWriter::Reset() {\n  Object::Reset();\n  start_ = nullptr;\n  cursor_ = nullptr;\n  limit_ = nullptr;\n  start_pos_ = 0;\n}\n\ninline bool BackwardWriter::Close() {\n  AssertInitialized(cursor(), start_to_cursor());\n  return Object::Close();\n}\n\ninline void BackwardWriter::Done() {\n  start_pos_ = pos();\n  set_buffer();\n}\n\ninline void BackwardWriter::SetWriteSizeHint(\n    std::optional<Position> write_size_hint) {\n  AssertInitialized(cursor(), start_to_cursor());\n  SetWriteSizeHintImpl(write_size_hint);\n}\n\ninline bool BackwardWriter::Push(size_t min_length, size_t recommended_length) {\n  if (ABSL_PREDICT_TRUE(available() >= min_length)) return true;\n  AssertInitialized(cursor(), start_to_cursor());\n  if (ABSL_PREDICT_FALSE(!PushSlow(min_length, recommended_length))) {\n    return false;\n  }\n  RIEGELI_ASSERT_GE(available(), min_length)\n      << \"Failed postcondition of BackwardWriter::PushSlow(): \"\n         \"not enough space available\";\n  return true;\n}\n\ninline void BackwardWriter::move_cursor(size_t length) {\n  RIEGELI_ASSERT_LE(length, available())\n      << \"Failed precondition of BackwardWriter::move_cursor(): \"\n         \"length out of range\";\n  cursor_ -= length;\n}\n\ninline void BackwardWriter::set_cursor(char* cursor) {\n  RIEGELI_ASSERT_LE(cursor, start())\n      << \"Failed precondition of BackwardWriter::set_cursor(): \"\n         \"pointer out of range\";\n  RIEGELI_ASSERT_GE(cursor, limit())\n      << \"Failed precondition of BackwardWriter::set_cursor(): \"\n         \"pointer out of range\";\n  cursor_ = cursor;\n}\n\ninline void BackwardWriter::set_buffer(char* limit, size_t start_to_limit,\n                                       size_t start_to_cursor) {\n  RIEGELI_ASSERT_LE(start_to_cursor, start_to_limit)\n      << \"Failed precondition of BackwardWriter::set_buffer(): \"\n         \"length out of range\";\n  start_ = limit + start_to_limit;\n  cursor_ = start_ - start_to_cursor;\n  limit_ = limit;\n}\n\ninline bool BackwardWriter::Write(char src) {\n  if (ABSL_PREDICT_FALSE(!Push())) return false;\n  move_cursor(1);\n  *cursor() = src;\n  return true;\n}\n\ninline bool BackwardWriter::Write(BytesRef src) {\n  AssertInitialized(src.data(), src.size());\n  if (ABSL_PREDICT_TRUE(available() >= src.size())) {\n    move_cursor(src.size());\n    riegeli::null_safe_memcpy(cursor(), src.data(), src.size());\n    return true;\n  }\n  AssertInitialized(cursor(), start_to_cursor());\n  return WriteSlow(src);\n}\n\ninline bool BackwardWriter::Write(ExternalRef src) {\n  if (ABSL_PREDICT_TRUE(available() >= src.size() &&\n                        src.size() <= kMaxBytesToCopy)) {\n    move_cursor(src.size());\n    riegeli::null_safe_memcpy(cursor(), src.data(), src.size());\n    return true;\n  }\n  AssertInitialized(cursor(), start_to_cursor());\n  return WriteSlow(std::move(src));\n}\n\ntemplate <typename Src,\n          std::enable_if_t<SupportsExternalRefWhole<Src>::value, int>>\ninline bool BackwardWriter::Write(Src&& src) {\n  return Write(ExternalRef(std::forward<Src>(src)));\n}\n\ninline bool BackwardWriter::Write(ByteFill src) {\n  if (ABSL_PREDICT_TRUE(available() >= src.size() &&\n                        src.size() <= kMaxBytesToCopy)) {\n    move_cursor(IntCast<size_t>(src.size()));\n    riegeli::null_safe_memset(cursor(), src.fill(),\n                              IntCast<size_t>(src.size()));\n    return true;\n  }\n  AssertInitialized(cursor(), start_to_cursor());\n  return WriteSlow(src);\n}\n\ntemplate <typename Src,\n          std::enable_if_t<\n              std::conjunction_v<\n                  absl::HasAbslStringify<Src>,\n                  std::negation<std::is_convertible<Src&&, BytesRef>>,\n                  std::negation<std::is_convertible<Src&&, const Chain&>>,\n                  std::negation<std::is_convertible<Src&&, const absl::Cord&>>,\n                  std::negation<std::is_convertible<Src&&, ByteFill>>>,\n              int>>\nbool BackwardWriter::Write(Src&& src) {\n  RestrictedChainWriter chain_writer;\n  WriterStringifySink sink(&chain_writer);\n  AbslStringify(sink, std::forward<Src>(src));\n  if (ABSL_PREDICT_FALSE(!chain_writer.Close())) {\n    return Fail(chain_writer.status());\n  }\n  return Write(std::move(chain_writer.dest()));\n}\n\ninline bool BackwardWriter::Flush(FlushType flush_type) {\n  AssertInitialized(cursor(), start_to_cursor());\n  return FlushImpl(flush_type);\n}\n\ninline Position BackwardWriter::pos() const {\n  RIEGELI_ASSERT_LE(start_pos_,\n                    std::numeric_limits<Position>::max() - start_to_limit())\n      << \"Failed invariant of BackwardWriter: \"\n         \"position of buffer limit overflow\";\n  return start_pos_ + start_to_cursor();\n}\n\ninline Position BackwardWriter::limit_pos() const {\n  RIEGELI_ASSERT_LE(start_pos_,\n                    std::numeric_limits<Position>::max() - start_to_limit())\n      << \"Failed invariant of BackwardWriter: \"\n         \"position of buffer limit overflow\";\n  return start_pos_ + start_to_limit();\n}\n\ninline void BackwardWriter::move_start_pos(Position length) {\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<Position>::max() - start_pos_)\n      << \"Failed precondition of BackwardWriter::move_start_pos(): \"\n         \"position out of range\";\n  start_pos_ += length;\n}\n\ninline void BackwardWriter::set_start_pos(Position start_pos) {\n  start_pos_ = start_pos;\n}\n\ninline bool BackwardWriter::Truncate(Position new_size) {\n  AssertInitialized(cursor(), start_to_cursor());\n  return TruncateImpl(new_size);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_BACKWARD_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/buffer_options.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/buffer_options.h\"\n\n#include <stddef.h>\n\n#include <optional>\n\n#include \"absl/numeric/bits.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/types.h\"\n\nnamespace riegeli {\n\nnamespace {\n\n// Recommends the length of a buffer.\n//\n// The following constraints are applied, in the order of weakest to strongest:\n//  * If `single_run` and `pos` did not pass `size_hint` yet, the remaining\n//    length, otherwise the base recommendation of `length`.\n//  * At least `max(min_length, recommended_length)`.\n//  * At most `max_length`.\n//  * Aligned so that the next position is a multiple of the length so far\n//    rounded up to the nearest power of 2, but at least `min_length`.\ninline size_t ApplySizeHintAndRoundPos(Position base_length, size_t min_length,\n                                       size_t recommended_length,\n                                       size_t max_length,\n                                       std::optional<Position> size_hint,\n                                       Position pos, bool single_run) {\n  if (single_run) base_length = ApplySizeHint(base_length, size_hint, pos);\n  const size_t length_for_rounding = UnsignedMin(\n      UnsignedMax(base_length, min_length, recommended_length), max_length);\n  if (length_for_rounding == 0) return min_length;\n  const size_t rounding_mask = absl::bit_ceil(length_for_rounding) - 1;\n  const size_t rounded_length = (~pos & rounding_mask) + 1;\n  if (rounded_length < min_length) {\n    // Return at least `min_length`, keeping the same remainder modulo\n    // `rounding_mask + 1` as `rounded_length`.\n    return ((min_length - rounded_length + rounding_mask) & ~rounding_mask) +\n           rounded_length;\n  }\n  return rounded_length;\n}\n\n}  // namespace\n\nsize_t ReadBufferSizer::BufferLength(Position pos, size_t min_length,\n                                     size_t recommended_length) const {\n  RIEGELI_ASSERT_GE(pos, base_pos_)\n      << \"Failed precondition of ReadBufferSizer::ReadBufferLength(): \"\n      << \"position earlier than base position of the run\";\n  const size_t length = ApplySizeHintAndRoundPos(\n      UnsignedMax(pos - base_pos_, buffer_length_from_last_run_,\n                  buffer_options_.min_buffer_size()),\n      min_length, recommended_length, buffer_options_.max_buffer_size(),\n      exact_size(), pos, read_all_hint_);\n  if (exact_size() != std::nullopt) {\n    return UnsignedMin(length, SaturatingSub(*exact_size(), pos));\n  }\n  return length;\n}\n\nsize_t WriteBufferSizer::BufferLength(Position pos, size_t min_length,\n                                      size_t recommended_length) const {\n  RIEGELI_ASSERT_GE(pos, base_pos_)\n      << \"Failed precondition of WriteBufferSizer::WriteBufferLength(): \"\n      << \"position earlier than base position of the run\";\n  const size_t length = ApplySizeHintAndRoundPos(\n      UnsignedMax(pos - base_pos_, buffer_length_from_last_run_,\n                  buffer_options_.min_buffer_size()),\n      min_length, recommended_length, buffer_options_.max_buffer_size(),\n      size_hint(), pos, buffer_length_from_last_run_ == 0);\n  if (size_hint() != std::nullopt && pos <= *size_hint()) {\n    return UnsignedClamp(length, min_length, *size_hint() - pos);\n  }\n  return length;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/buffer_options.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_BUFFER_OPTIONS_H_\n#define RIEGELI_BYTES_BUFFER_OPTIONS_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/types.h\"\n\nnamespace riegeli {\n\n// Common options related to buffering in a `Reader` or `Writer`.\nclass BufferOptions {\n public:\n  BufferOptions() noexcept {}\n\n  // Tunes the minimal buffer size, which determines how much data at a time is\n  // typically read from the source / written to the destination.\n  //\n  // The actual buffer size changes between `min_buffer_size()` and\n  // `max_buffer_size()` depending on the access pattern.\n  //\n  // Default: `kDefaultMinBufferSize` (4K).\n  static constexpr size_t kDefaultMinBufferSize = size_t{4} << 10;\n  BufferOptions& set_min_buffer_size(size_t min_buffer_size) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    min_buffer_size_ = UnsignedMin(min_buffer_size, uint32_t{1} << 31);\n    return *this;\n  }\n  BufferOptions&& set_min_buffer_size(size_t min_buffer_size) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_min_buffer_size(min_buffer_size));\n  }\n  size_t min_buffer_size() const { return min_buffer_size_; }\n\n  // Tunes the maximal buffer size, which determines how much data at a time is\n  // typically read from the source / written to the destination.\n  //\n  // The actual buffer size changes between `min_buffer_size()` and\n  // `max_buffer_size()` depending on the access pattern.\n  //\n  // Default: `kDefaultMaxBufferSize` (64K).\n  static constexpr size_t kDefaultMaxBufferSize = size_t{64} << 10;\n  BufferOptions& set_max_buffer_size(size_t max_buffer_size) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_GT(max_buffer_size, 0u)\n        << \"Failed precondition of BufferOptions::set_max_buffer_size(): \"\n           \"zero buffer size\";\n    max_buffer_size_ = UnsignedMin(max_buffer_size, uint32_t{1} << 31);\n    return *this;\n  }\n  BufferOptions&& set_max_buffer_size(size_t max_buffer_size) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_max_buffer_size(max_buffer_size));\n  }\n  size_t max_buffer_size() const { return max_buffer_size_; }\n\n  // A shortcut for `set_min_buffer_size(buffer_size)` with\n  // `set_max_buffer_size(buffer_size)`.\n  BufferOptions& set_buffer_size(size_t buffer_size) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return set_min_buffer_size(buffer_size).set_max_buffer_size(buffer_size);\n  }\n  BufferOptions&& set_buffer_size(size_t buffer_size) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_buffer_size(buffer_size));\n  }\n\n private:\n  // Use `uint32_t` instead of `size_t` to reduce the object size.\n  uint32_t min_buffer_size_ = uint32_t{kDefaultMinBufferSize};\n  uint32_t max_buffer_size_ = uint32_t{kDefaultMaxBufferSize};\n};\n\n// Deriving `Options` from `BufferOptionsBase<Options>` makes it easier to\n// provide options related to buffering among `Options` (deriving from\n// `BufferOptions` would yield wrong result types of setters).\n//\n// Default values of `{min,max}_buffer_size()` can be overridden by\n// `Options::kDefault{Min,Max}BufferSize`.\ntemplate <typename Options>\nclass BufferOptionsBase {\n public:\n  BufferOptionsBase() noexcept {\n    static_assert(std::is_base_of_v<BufferOptionsBase<Options>, Options>,\n                  \"The template argument Options in BufferOptionsBase<Options> \"\n                  \"must be the class derived from BufferOptionsBase<Options>\");\n    set_min_buffer_size(Options::kDefaultMinBufferSize);\n    set_max_buffer_size(Options::kDefaultMaxBufferSize);\n  }\n\n  // See `BufferOptions::set_min_buffer_size()`.\n  static constexpr size_t kDefaultMinBufferSize =\n      BufferOptions::kDefaultMinBufferSize;\n  Options& set_min_buffer_size(size_t min_buffer_size) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    buffer_options_.set_min_buffer_size(min_buffer_size);\n    return static_cast<Options&>(*this);\n  }\n  Options&& set_min_buffer_size(size_t min_buffer_size) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_min_buffer_size(min_buffer_size));\n  }\n  size_t min_buffer_size() const { return buffer_options_.min_buffer_size(); }\n\n  // See `BufferOptions::set_max_buffer_size()`.\n  static constexpr size_t kDefaultMaxBufferSize =\n      BufferOptions::kDefaultMaxBufferSize;\n  Options& set_max_buffer_size(size_t max_buffer_size) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    buffer_options_.set_max_buffer_size(max_buffer_size);\n    return static_cast<Options&>(*this);\n  }\n  Options&& set_max_buffer_size(size_t max_buffer_size) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_max_buffer_size(max_buffer_size));\n  }\n  size_t max_buffer_size() const { return buffer_options_.max_buffer_size(); }\n\n  // See `BufferOptions::set_buffer_size()`.\n  Options& set_buffer_size(size_t buffer_size) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    buffer_options_.set_buffer_size(buffer_size);\n    return static_cast<Options&>(*this);\n  }\n  Options&& set_buffer_size(size_t buffer_size) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_buffer_size(buffer_size));\n  }\n\n  // Grouped options related to buffering.\n  Options& set_buffer_options(BufferOptions buffer_options) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    buffer_options_ = buffer_options;\n    return static_cast<Options&>(*this);\n  }\n  Options&& set_buffer_options(BufferOptions buffer_options) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_buffer_options(buffer_options));\n  }\n  BufferOptions& buffer_options() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return buffer_options_;\n  }\n  const BufferOptions& buffer_options() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return buffer_options_;\n  }\n\n protected:\n  BufferOptionsBase(const BufferOptionsBase& that) = default;\n  BufferOptionsBase& operator=(const BufferOptionsBase& that) = default;\n\n  ~BufferOptionsBase() = default;\n\n private:\n  BufferOptions buffer_options_;\n};\n\n// Recommends an adaptive buffer length based on the access pattern of a\n// `Reader`.\n//\n// The buffer length grows geometrically from `min_buffer_size` to\n// `max_buffer_size` through each run of sequential reading operations.\n//\n// A new run may begin from a function which forces using a new buffer\n// (`Reader::Seek()` or `Reader::Sync()`). The buffer length for the new run is\n// optimized for the case when the new run will have a similar length to the\n// previous non-empty run.\n//\n// The lengths aim at letting absolute positions be multiples of sufficiently\n// large powers of 2. Aligned positions might make reading from the source more\n// efficient. This is also important for filling the remaining part of the\n// buffer if the source returned less data than asked for.\nclass ReadBufferSizer {\n public:\n  ReadBufferSizer() = default;\n\n  explicit ReadBufferSizer(BufferOptions buffer_options);\n\n  ReadBufferSizer(const ReadBufferSizer& that) = default;\n  ReadBufferSizer& operator=(const ReadBufferSizer& that) = default;\n\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(BufferOptions buffer_options);\n\n  // Returns the options passed to the constructor.\n  BufferOptions buffer_options() const { return buffer_options_; }\n\n  // Intended storage for the hint set by `Reader::SetReadAllHint()`.\n  //\n  // If `true` and `exact_size()` is not `absl::nullptr`, this causes larger\n  // buffer sizes to be used before reaching `*exact_size()`.\n  void set_read_all_hint(bool read_all_hint) { read_all_hint_ = read_all_hint; }\n  bool read_all_hint() const { return read_all_hint_; }\n\n  // Intended storage for an exact size of the source, as discovered by the\n  // `Reader` itself.\n  //\n  // If not `absl::nullptr` and `read_all_hint()` is `true`, this causes larger\n  // buffer sizes to be used before reaching `*exact_size()`.\n  //\n  // Also, if not `absl::nullptr`, this causes a smaller buffer size to be used\n  // when reaching `*exact_size()`.\n  void set_exact_size(std::optional<Position> exact_size) {\n    exact_size_ = exact_size;\n  }\n  std::optional<Position> exact_size() const { return exact_size_; }\n\n  // Called at the beginning of a run.\n  //\n  // This must be called during initialization if reading starts from a position\n  // greater than 0.\n  //\n  // `BeginRun()` may be called again without an intervening `EndRun()`.\n  void BeginRun(Position pos) { base_pos_ = pos; }\n\n  // Called at the end of a run.\n  //\n  // `EndRun()` may be called again without an intervening `BeginRun()`.\n  //\n  // Precondition:\n  //   `pos >= base_pos`, where `base_pos` is the argument of the last call to\n  //       `BeginRun()`, if `BeginRun()` has been called\n  void EndRun(Position pos);\n\n  // Proposed a buffer length for reading at `pos`.\n  //\n  // The length will not let the next position exceed `exact_size()`,\n  // in particular it is 0 if `exact_size() != nullptr && pos >= *exact_size()`.\n  //\n  // It will be at least `min_length` unless `exact_size()` is reached,\n  // preferably `recommended_length`.\n  //\n  // Precondition:\n  //   `pos >= base_pos`, where `base_pos` is the argument of the last call to\n  //       `BeginRun()`, if `BeginRun()` has been called\n  size_t BufferLength(Position pos, size_t min_length = 0,\n                      size_t recommended_length = 0) const;\n\n private:\n  BufferOptions buffer_options_;\n  // Position where the current run started.\n  Position base_pos_ = 0;\n  // Buffer size recommended by the previous run.\n  size_t buffer_length_from_last_run_ = 0;\n  bool read_all_hint_ = false;\n  std::optional<Position> exact_size_;\n};\n\n// Recommends an adaptive buffer length based on the access pattern of a\n// `Writer`.\n//\n// The buffer length grows geometrically from `min_buffer_size` to\n// `max_buffer_size` through each run of sequential writing operations.\n//\n// A new run may begin from a function which forces using a new buffer (mainly\n// `Writer::Seek()` or `Writer::Flush()`). The buffer length for the new run is\n// optimized for the case when the new run will have a similar length to the\n// previous non-empty run.\n//\n// The lengths aim at letting absolute positions be multiples of sufficiently\n// large powers of 2. Aligned positions might make writing to the destination\n// more efficient.\nclass WriteBufferSizer {\n public:\n  WriteBufferSizer() = default;\n\n  explicit WriteBufferSizer(BufferOptions buffer_options);\n\n  WriteBufferSizer(const WriteBufferSizer& that) = default;\n  WriteBufferSizer& operator=(const WriteBufferSizer& that) = default;\n\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(BufferOptions buffer_options);\n\n  // Returns the options passed to the constructor.\n  BufferOptions buffer_options() const { return buffer_options_; }\n\n  // Intended storage for the hint set by\n  // `{,Backward}Writer::SetWriteSizeHint()`.\n  //\n  // If not `absl::nullptr`, this causes larger buffer sizes to be used before\n  // reaching `*size_hint()`, and a smaller buffer size to be used when reaching\n  // `*size_hint()`.\n  void set_write_size_hint(Position pos,\n                           std::optional<Position> write_size_hint) {\n    if (write_size_hint == std::nullopt) {\n      size_hint_ = std::nullopt;\n    } else {\n      size_hint_ = SaturatingAdd(pos, *write_size_hint);\n    }\n  }\n  std::optional<Position> size_hint() const { return size_hint_; }\n\n  // Called at the beginning of a run.\n  //\n  // This must be called during initialization if writing starts from a position\n  // greater than 0.\n  //\n  // `BeginRun()` may be called again without an intervening `EndRun()`.\n  void BeginRun(Position pos) { base_pos_ = pos; }\n\n  // Called at the end of a run.\n  //\n  // `EndRun()` may be called again without an intervening `BeginRun()`.\n  //\n  // Precondition:\n  //   `pos >= base_pos`, where `base_pos` is the argument of the last call to\n  //       `BeginRun()`, if `BeginRun()` has been called\n  void EndRun(Position pos);\n\n  // Proposed a buffer length for writing at `pos`.\n  //\n  // The length will be at least `min_length`, preferably `recommended_length`.\n  //\n  // Precondition:\n  //   `pos >= base_pos`, where `base_pos` is the argument of the last call to\n  //       `BeginRun()`, if `BeginRun()` has been called\n  size_t BufferLength(Position pos, size_t min_length = 0,\n                      size_t recommended_length = 0) const;\n\n private:\n  BufferOptions buffer_options_;\n  // Position where the current run started.\n  Position base_pos_ = 0;\n  // Buffer size recommended by the previous run.\n  size_t buffer_length_from_last_run_ = 0;\n  std::optional<Position> size_hint_;\n};\n\n// Implementation details follow.\n\ninline ReadBufferSizer::ReadBufferSizer(BufferOptions buffer_options)\n    : buffer_options_(buffer_options) {}\n\ninline void ReadBufferSizer::Reset() {\n  buffer_options_ = BufferOptions();\n  base_pos_ = 0;\n  buffer_length_from_last_run_ = 0;\n  read_all_hint_ = false;\n  exact_size_ = std::nullopt;\n}\n\ninline void ReadBufferSizer::Reset(BufferOptions buffer_options) {\n  buffer_options_ = buffer_options;\n  base_pos_ = 0;\n  buffer_length_from_last_run_ = 0;\n  read_all_hint_ = false;\n  exact_size_ = std::nullopt;\n}\n\ninline void ReadBufferSizer::EndRun(Position pos) {\n  RIEGELI_ASSERT_GE(pos, base_pos_)\n      << \"Failed precondition of ReadBufferSizer::EndRun(): \"\n      << \"position earlier than base position of the run\";\n  if (pos == base_pos_) return;\n  const size_t length = SaturatingIntCast<size_t>(pos - base_pos_);\n  // Increase the length to compensate for variability of the lengths, and for\n  // rounding the positions so that even after rounding the length down to a\n  // power of 2 the last length is covered.\n  buffer_length_from_last_run_ = SaturatingAdd(length, length - 1);\n}\n\ninline WriteBufferSizer::WriteBufferSizer(BufferOptions buffer_options)\n    : buffer_options_(buffer_options) {}\n\ninline void WriteBufferSizer::Reset() {\n  buffer_options_ = BufferOptions();\n  base_pos_ = 0;\n  buffer_length_from_last_run_ = 0;\n  size_hint_ = std::nullopt;\n}\n\ninline void WriteBufferSizer::Reset(BufferOptions buffer_options) {\n  buffer_options_ = buffer_options;\n  base_pos_ = 0;\n  buffer_length_from_last_run_ = 0;\n  size_hint_ = std::nullopt;\n}\n\ninline void WriteBufferSizer::EndRun(Position pos) {\n  RIEGELI_ASSERT_GE(pos, base_pos_)\n      << \"Failed precondition of WriteBufferSizer::EndRun(): \"\n      << \"position earlier than base position of the run\";\n  if (pos == base_pos_) return;\n  const size_t length = SaturatingIntCast<size_t>(pos - base_pos_);\n  // Increase the length to compensate for variability of the lengths, and for\n  // rounding the positions so that even after rounding the length down to a\n  // power of 2 the last length is covered.\n  buffer_length_from_last_run_ = SaturatingAdd(length, length - 1);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_BUFFER_OPTIONS_H_\n"
  },
  {
    "path": "riegeli/bytes/buffered_reader.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/buffered_reader.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/sized_shared_buffer.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid BufferedReader::Done() {\n  if (available() > 0) {\n    if (!SupportsRandomAccess()) {\n      // Seeking back is not feasible.\n      Reader::Done();\n      buffer_ = SizedSharedBuffer();\n      return;\n    }\n    const Position new_pos = pos();\n    set_buffer();\n    SeekBehindBuffer(new_pos);\n  }\n  Reader::Done();\n  buffer_ = SizedSharedBuffer();\n}\n\ninline void BufferedReader::SyncBuffer() {\n  set_buffer();\n  buffer_.Clear();\n}\n\nvoid BufferedReader::SetReadAllHintImpl(bool read_all_hint) {\n  buffer_sizer_.set_read_all_hint(read_all_hint);\n}\n\nvoid BufferedReader::ExactSizeReached() {}\n\nbool BufferedReader::PullSlow(size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::PullSlow(): \"\n         \"enough data available, use Pull() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const size_t available_length = available();\n  const size_t buffer_length = buffer_sizer_.BufferLength(\n      limit_pos(), min_length - available_length,\n      SaturatingSub(recommended_length, available_length));\n  if (ABSL_PREDICT_FALSE(buffer_length == 0)) {\n    ExactSizeReached();\n    return false;\n  }\n  size_t cursor_index = start_to_cursor();\n  absl::Span<char> flat_buffer = buffer_.AppendBufferIfExisting(buffer_length);\n  if (flat_buffer.empty()) {\n    // Not enough space in `buffer_`. Resize `buffer_`, keeping available data.\n    buffer_.RemovePrefix(cursor_index);\n    buffer_.Shrink(available_length + buffer_length);\n    cursor_index = 0;\n    flat_buffer = buffer_.AppendFixedBuffer(buffer_length);\n  }\n  // Read more data into `buffer_`.\n  const size_t min_length_to_read =\n      ToleratesReadingAhead()\n          ? buffer_length\n          : UnsignedMin(min_length - available_length, buffer_length);\n  const Position pos_before = limit_pos();\n  const bool read_ok =\n      ReadInternal(min_length_to_read, buffer_length, flat_buffer.data());\n  RIEGELI_ASSERT_GE(limit_pos(), pos_before)\n      << \"BufferedReader::ReadInternal() decreased limit_pos()\";\n  const Position length_read = limit_pos() - pos_before;\n  RIEGELI_ASSERT_LE(length_read, buffer_length)\n      << \"BufferedReader::ReadInternal() read more than requested\";\n  if (read_ok) {\n    RIEGELI_ASSERT_GE(length_read, min_length_to_read)\n        << \"BufferedReader::ReadInternal() succeeded but \"\n           \"read less than requested\";\n  } else {\n    RIEGELI_ASSERT_LT(length_read, min_length_to_read)\n        << \"BufferedReader::ReadInternal() failed but read enough\";\n  }\n  buffer_.RemoveSuffix(flat_buffer.size() - IntCast<size_t>(length_read));\n  set_buffer(buffer_.data(), buffer_.size(), cursor_index);\n  return available() >= min_length;\n}\n\nbool BufferedReader::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(new_pos <= limit_pos())) {\n    return Fail(\n        absl::UnimplementedError(\"Reader::Seek() backwards not supported\"));\n  }\n  // Seeking forwards.\n  do {\n    move_cursor(available());\n    if (ABSL_PREDICT_FALSE(!PullSlow(1, 0))) return false;\n  } while (new_pos > limit_pos());\n  const Position available_length = limit_pos() - new_pos;\n  RIEGELI_ASSERT_LE(available_length, start_to_limit())\n      << \"Reader::PullSlow() skipped some data\";\n  set_cursor(limit() - available_length);\n  return true;\n}\n\nbool BufferedReader::ReadSlow(size_t length, char* dest) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of Reader::ReadSlow(char*): \"\n         \"enough data available, use Read(char*) instead\";\n  if (length >= buffer_sizer_.BufferLength(pos())) {\n    // Read directly to `dest`.\n    const size_t available_length = available();\n    riegeli::null_safe_memcpy(dest, cursor(), available_length);\n    dest += available_length;\n    length -= available_length;\n    SyncBuffer();\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    size_t length_to_read = length;\n    if (exact_size() != std::nullopt) {\n      if (ABSL_PREDICT_FALSE(limit_pos() >= *exact_size())) {\n        ExactSizeReached();\n        return false;\n      }\n      length_to_read = UnsignedMin(length_to_read, *exact_size() - limit_pos());\n    }\n    if (ABSL_PREDICT_FALSE(\n            !ReadInternal(length_to_read, length_to_read, dest))) {\n      return false;\n    }\n    return length_to_read >= length;\n  }\n  return Reader::ReadSlow(length, dest);\n}\n\nbool BufferedReader::ReadSlow(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"Chain size overflow\";\n  bool enough_read = true;\n  while (length > available()) {\n    size_t available_length = available();\n    if (ABSL_PREDICT_FALSE(!ok())) {\n      // Read as much as is available.\n      enough_read = false;\n      length = available_length;\n      break;\n    }\n    const size_t buffer_length =\n        buffer_sizer_.BufferLength(limit_pos(), 1, length - available_length);\n    size_t cursor_index = start_to_cursor();\n    absl::Span<char> flat_buffer =\n        buffer_.AppendBufferIfExisting(buffer_length);\n    if (flat_buffer.empty()) {\n      // Not enough space in `buffer_`. Append available data to `dest` and make\n      // a new buffer.\n      dest.Append(ExternalRef(std::move(buffer_),\n                              absl::string_view(cursor(), available_length)));\n      length -= available_length;\n      buffer_.ClearAndShrink(buffer_length);\n      if (ABSL_PREDICT_FALSE(buffer_length == 0)) {\n        set_buffer();\n        ExactSizeReached();\n        return false;\n      }\n      available_length = 0;\n      cursor_index = 0;\n      flat_buffer = buffer_.AppendFixedBuffer(buffer_length);\n    }\n    // Read more data into `buffer_`.\n    const size_t min_length_to_read =\n        ToleratesReadingAhead()\n            ? buffer_length\n            : UnsignedMin(length - available_length, buffer_length);\n    const Position pos_before = limit_pos();\n    const bool read_ok =\n        ReadInternal(min_length_to_read, buffer_length, flat_buffer.data());\n    RIEGELI_ASSERT_GE(limit_pos(), pos_before)\n        << \"BufferedReader::ReadInternal() decreased limit_pos()\";\n    const Position length_read = limit_pos() - pos_before;\n    RIEGELI_ASSERT_LE(length_read, buffer_length)\n        << \"BufferedReader::ReadInternal() read more than requested\";\n    if (read_ok) {\n      RIEGELI_ASSERT_GE(length_read, min_length_to_read)\n          << \"BufferedReader::ReadInternal() succeeded but \"\n             \"read less than requested\";\n    } else {\n      RIEGELI_ASSERT_LT(length_read, min_length_to_read)\n          << \"BufferedReader::ReadInternal() failed but read enough\";\n    }\n    buffer_.RemoveSuffix(flat_buffer.size() - IntCast<size_t>(length_read));\n    set_buffer(buffer_.data(), buffer_.size(), cursor_index);\n    if (ABSL_PREDICT_FALSE(!read_ok)) {\n      // Read as much as is available.\n      enough_read = available() >= length;\n      if (ABSL_PREDICT_FALSE(!enough_read)) length = available();\n      break;\n    }\n  }\n  dest.Append(ExternalRef(buffer_, absl::string_view(cursor(), length)));\n  move_cursor(length);\n  return enough_read;\n}\n\nbool BufferedReader::ReadSlow(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"Cord size overflow\";\n  bool enough_read = true;\n  while (length > available()) {\n    size_t available_length = available();\n    if (ABSL_PREDICT_FALSE(!ok())) {\n      // Read as much as is available.\n      enough_read = false;\n      length = available_length;\n      break;\n    }\n    const size_t buffer_length =\n        buffer_sizer_.BufferLength(limit_pos(), 1, length - available_length);\n    size_t cursor_index = start_to_cursor();\n    absl::Span<char> flat_buffer =\n        buffer_.AppendBufferIfExisting(buffer_length);\n    if (flat_buffer.empty()) {\n      // Not enough space in `buffer_`. Append available data to `dest` and make\n      // a new buffer.\n      ExternalRef(std::move(buffer_),\n                  absl::string_view(cursor(), available_length))\n          .AppendTo(dest);\n      length -= available_length;\n      buffer_.ClearAndShrink(buffer_length);\n      if (ABSL_PREDICT_FALSE(buffer_length == 0)) {\n        set_buffer();\n        ExactSizeReached();\n        return false;\n      }\n      available_length = 0;\n      cursor_index = 0;\n      flat_buffer = buffer_.AppendFixedBuffer(buffer_length);\n    }\n    // Read more data into `buffer_`.\n    const size_t min_length_to_read =\n        ToleratesReadingAhead()\n            ? buffer_length\n            : UnsignedMin(length - available_length, buffer_length);\n    const Position pos_before = limit_pos();\n    const bool read_ok =\n        ReadInternal(min_length_to_read, buffer_length, flat_buffer.data());\n    RIEGELI_ASSERT_GE(limit_pos(), pos_before)\n        << \"BufferedReader::ReadInternal() decreased limit_pos()\";\n    const Position length_read = limit_pos() - pos_before;\n    RIEGELI_ASSERT_LE(length_read, buffer_length)\n        << \"BufferedReader::ReadInternal() read more than requested\";\n    if (read_ok) {\n      RIEGELI_ASSERT_GE(length_read, min_length_to_read)\n          << \"BufferedReader::ReadInternal() succeeded but \"\n             \"read less than requested\";\n    } else {\n      RIEGELI_ASSERT_LT(length_read, min_length_to_read)\n          << \"BufferedReader::ReadInternal() failed but read enough\";\n    }\n    buffer_.RemoveSuffix(flat_buffer.size() - IntCast<size_t>(length_read));\n    set_buffer(buffer_.data(), buffer_.size(), cursor_index);\n    if (ABSL_PREDICT_FALSE(!read_ok)) {\n      // Read as much as is available.\n      enough_read = available() >= length;\n      if (ABSL_PREDICT_FALSE(!enough_read)) length = available();\n      break;\n    }\n  }\n  ExternalRef(buffer_, absl::string_view(cursor(), length)).AppendTo(dest);\n  move_cursor(length);\n  return enough_read;\n}\n\nbool BufferedReader::CopySlow(Position length, Writer& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  bool enough_read = true;\n  while (length > available()) {\n    size_t available_length = available();\n    if (ABSL_PREDICT_FALSE(!ok())) {\n      // Copy as much as is available.\n      enough_read = false;\n      length = available_length;\n      break;\n    }\n    const bool read_directly = length >= buffer_sizer_.BufferLength(pos());\n    if (read_directly) {\n      if (available_length <= kMaxBytesToCopy) {\n        if (ABSL_PREDICT_FALSE(\n                !dest.Write(absl::string_view(cursor(), available_length)))) {\n          move_cursor(available_length);\n          return false;\n        }\n        length -= available_length;\n        SyncBuffer();\n        return CopyUsingPush(length, dest);\n      }\n      // It is better to write available data from `buffer_` as a `Chain` before\n      // reading directly to `dest`. Before that, `buffer_` might need to be\n      // filled more to avoid attaching a wasteful `Chain`.\n    }\n    const size_t buffer_length =\n        buffer_sizer_.BufferLength(limit_pos(), 1, length - available_length);\n    size_t cursor_index = start_to_cursor();\n    absl::Span<char> flat_buffer =\n        buffer_.AppendBufferIfExisting(buffer_length);\n    if (flat_buffer.empty()) {\n      // Not enough space in `buffer_`. Append available data to `dest` and make\n      // a new buffer.\n      if (available_length > 0) {\n        const bool write_ok = dest.Write(ExternalRef(\n            std::move(buffer_), absl::string_view(cursor(), available_length)));\n        if (ABSL_PREDICT_FALSE(!write_ok)) {\n          buffer_.ClearAndShrink(buffer_length);\n          set_buffer();\n          return false;\n        }\n        length -= available_length;\n      }\n      buffer_.ClearAndShrink(buffer_length);\n      if (ABSL_PREDICT_FALSE(buffer_length == 0)) {\n        set_buffer();\n        ExactSizeReached();\n        return false;\n      }\n      if (read_directly) {\n        set_buffer();\n        return CopyUsingPush(length, dest);\n      }\n      available_length = 0;\n      cursor_index = 0;\n      flat_buffer = buffer_.AppendFixedBuffer(buffer_length);\n    }\n    // Read more data into `buffer_`.\n    const size_t min_length_to_read =\n        ToleratesReadingAhead()\n            ? buffer_length\n            : UnsignedMin(length - available_length, buffer_length);\n    const Position pos_before = limit_pos();\n    const bool read_ok =\n        ReadInternal(min_length_to_read, buffer_length, flat_buffer.data());\n    RIEGELI_ASSERT_GE(limit_pos(), pos_before)\n        << \"BufferedReader::ReadInternal() decreased limit_pos()\";\n    const Position length_read = limit_pos() - pos_before;\n    RIEGELI_ASSERT_LE(length_read, buffer_length)\n        << \"BufferedReader::ReadInternal() read more than requested\";\n    if (read_ok) {\n      RIEGELI_ASSERT_GE(length_read, min_length_to_read)\n          << \"BufferedReader::ReadInternal() succeeded but \"\n             \"read less than requested\";\n    } else {\n      RIEGELI_ASSERT_LT(length_read, min_length_to_read)\n          << \"BufferedReader::ReadInternal() failed but read enough\";\n    }\n    buffer_.RemoveSuffix(flat_buffer.size() - IntCast<size_t>(length_read));\n    set_buffer(buffer_.data(), buffer_.size(), cursor_index);\n    if (ABSL_PREDICT_FALSE(!read_ok)) {\n      // Copy as much as is available.\n      enough_read = available() >= length;\n      if (ABSL_PREDICT_FALSE(!enough_read)) length = available();\n      break;\n    }\n  }\n  const bool write_ok = dest.Write(ExternalRef(\n      buffer_, absl::string_view(cursor(), IntCast<size_t>(length))));\n  move_cursor(IntCast<size_t>(length));\n  return write_ok && enough_read;\n}\n\ninline bool BufferedReader::CopyUsingPush(Position length, Writer& dest) {\n  RIEGELI_ASSERT_GT(length, 0u)\n      << \"Failed precondition of BufferedReader::CopyUsingPush(): \"\n         \"nothing to copy\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::CopyUsingPush()\";\n  Position length_to_read = length;\n  if (exact_size() != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(limit_pos() >= *exact_size())) {\n      ExactSizeReached();\n      return false;\n    }\n    length_to_read = UnsignedMin(length_to_read, *exact_size() - limit_pos());\n  }\n  return CopyInternal(length_to_read, dest) && length_to_read == length;\n}\n\nbool BufferedReader::CopyInternal(Position length, Writer& dest) {\n  RIEGELI_ASSERT_GT(length, 0u)\n      << \"Failed precondition of BufferedReader::CopyInternal(): \"\n         \"nothing to copy\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::CopyInternal()\";\n  size_t length_to_read = SaturatingIntCast<size_t>(length);\n  // In the first iteration `exact_size()` was taken into account by\n  // `CopyUsingPush()`, so that `CopyInternal()` overrides do not need to.\n  for (;;) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(1, length_to_read))) return false;\n    const size_t length_to_copy = UnsignedMin(length_to_read, dest.available());\n    const Position pos_before = limit_pos();\n    const bool read_ok =\n        ReadInternal(length_to_copy, length_to_copy, dest.cursor());\n    RIEGELI_ASSERT_GE(limit_pos(), pos_before)\n        << \"BufferedReader::ReadInternal() decreased limit_pos()\";\n    const Position length_read = limit_pos() - pos_before;\n    RIEGELI_ASSERT_LE(length_read, length_to_copy)\n        << \"BufferedReader::ReadInternal() read more than requested\";\n    if (read_ok) {\n      RIEGELI_ASSERT_GE(length_read, length_to_copy)\n          << \"BufferedReader::ReadInternal() succeeded but \"\n             \"read less than requested\";\n    } else {\n      RIEGELI_ASSERT_LT(length_read, length_to_copy)\n          << \"BufferedReader::ReadInternal() failed but read enough\";\n    }\n    dest.move_cursor(IntCast<size_t>(length_read));\n    if (ABSL_PREDICT_FALSE(!read_ok)) return false;\n    length -= IntCast<size_t>(length_read);\n    if (length == 0) return true;\n    // `ReadInternal()` might have set `exact_size()`, so this implementation of\n    // `CopyInternal()` needs to take `exact_size()` into account for remaining\n    // iterations.\n    length_to_read = SaturatingIntCast<size_t>(length);\n    if (exact_size() != std::nullopt) {\n      if (ABSL_PREDICT_FALSE(limit_pos() >= *exact_size())) {\n        ExactSizeReached();\n        return false;\n      }\n      length_to_read = UnsignedMin(length_to_read, *exact_size() - limit_pos());\n    }\n  }\n}\n\nbool BufferedReader::CopySlow(size_t length, BackwardWriter& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(BackwardWriter&): \"\n         \"enough data available, use Copy(BackwardWriter&) instead\";\n  if (length <= kMaxBytesToCopy) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(length))) return false;\n    dest.move_cursor(length);\n    if (ABSL_PREDICT_FALSE(!ReadSlow(length, dest.cursor()))) {\n      dest.set_cursor(dest.cursor() + length);\n      return false;\n    }\n    return true;\n  }\n  Chain data;\n  if (ABSL_PREDICT_FALSE(!ReadSlow(length, data))) return false;\n  return dest.Write(std::move(data));\n}\n\nbool BufferedReader::ReadSomeSlow(size_t max_length, char* dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"nothing to read, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"some data available, use ReadSome(char*) instead\";\n  if (max_length >= buffer_sizer_.BufferLength(limit_pos())) {\n    // Read directly to `dest`.\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    SyncBuffer();\n    if (exact_size() != std::nullopt) {\n      if (ABSL_PREDICT_FALSE(limit_pos() >= *exact_size())) {\n        ExactSizeReached();\n        return false;\n      }\n      max_length = UnsignedMin(max_length, *exact_size() - limit_pos());\n    }\n    const size_t min_length_to_read = ToleratesReadingAhead() ? max_length : 1;\n    const Position pos_before = limit_pos();\n    ReadInternal(min_length_to_read, max_length, dest);\n    RIEGELI_ASSERT_GE(limit_pos(), pos_before)\n        << \"BufferedReader::ReadInternal() decreased limit_pos()\";\n    return limit_pos() != pos_before;\n  }\n  return Reader::ReadSomeSlow(max_length, dest);\n}\n\nbool BufferedReader::CopySomeSlow(size_t max_length, Writer& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"nothing to read, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"some data available, use CopySome(Writer&) instead\";\n  if (max_length >= buffer_sizer_.BufferLength(limit_pos())) {\n    // Copy directly to `dest`.\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    SyncBuffer();\n    if (exact_size() != std::nullopt) {\n      if (ABSL_PREDICT_FALSE(limit_pos() >= *exact_size())) {\n        ExactSizeReached();\n        return false;\n      }\n      max_length = UnsignedMin(max_length, *exact_size() - limit_pos());\n    }\n    if (ABSL_PREDICT_FALSE(!dest.Push(1, max_length))) return false;\n    max_length = UnsignedMin(max_length, dest.available());\n    const size_t min_length_to_read = ToleratesReadingAhead() ? max_length : 1;\n    const Position pos_before = limit_pos();\n    ReadInternal(min_length_to_read, max_length, dest.cursor());\n    RIEGELI_ASSERT_GE(limit_pos(), pos_before)\n        << \"BufferedReader::ReadInternal() decreased limit_pos()\";\n    const Position length_read = limit_pos() - pos_before;\n    RIEGELI_ASSERT_LE(length_read, max_length)\n        << \"BufferedReader::ReadInternal() read more than requested\";\n    dest.move_cursor(IntCast<size_t>(length_read));\n    return length_read > 0;\n  }\n  return Reader::CopySomeSlow(max_length, dest);\n}\n\nvoid BufferedReader::ReadHintSlow(size_t min_length,\n                                  size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::ReadHintSlow(): \"\n         \"enough data available, use ReadHint() instead\";\n  PullSlow(min_length, recommended_length);\n}\n\nbool BufferedReader::SyncImpl(SyncType sync_type) {\n  if (available() > 0 && !SupportsRandomAccess()) {\n    // Seeking back is not feasible.\n    return ok();\n  }\n  const Position new_pos = pos();\n  buffer_sizer_.EndRun(new_pos);\n  SyncBuffer();\n  const bool result = new_pos == limit_pos() ? ok() : SeekBehindBuffer(new_pos);\n  buffer_sizer_.BeginRun(start_pos());\n  return result;\n}\n\nbool BufferedReader::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of Reader::SeekSlow(): \"\n         \"position in the buffer, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!SupportsRandomAccess())) {\n    SyncBuffer();\n    return SeekBehindBuffer(new_pos);\n  }\n  buffer_sizer_.EndRun(pos());\n  SyncBuffer();\n  const bool result = SeekBehindBuffer(new_pos);\n  buffer_sizer_.BeginRun(start_pos());\n  return result;\n}\n\nstd::optional<Position> BufferedReader::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  if (ABSL_PREDICT_FALSE(exact_size() == std::nullopt)) {\n    // Delegate to the base class to avoid repeating the error message.\n    return Reader::SizeImpl();\n  }\n  return *exact_size();\n}\n\nvoid BufferedReader::ShareBufferTo(BufferedReader& reader) const {\n  const Position new_pos = reader.pos();\n  if (new_pos >= start_pos() && new_pos < limit_pos()) {\n    reader.buffer_ = buffer_;\n    reader.set_buffer(start(), start_to_limit(),\n                      IntCast<size_t>(new_pos - start_pos()));\n    reader.set_limit_pos(limit_pos());\n  }\n}\n\nSizedSharedBuffer BufferedReader::SaveBuffer() {\n  set_limit_pos(pos());\n  buffer_.RemovePrefix(start_to_cursor());\n  set_buffer();\n  return std::move(buffer_);\n}\n\nvoid BufferedReader::RestoreBuffer(SizedSharedBuffer buffer) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::RestoreBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  buffer_ = std::move(buffer);\n  set_buffer(buffer_.data(), buffer_.size());\n  move_limit_pos(available());\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/buffered_reader.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_BUFFERED_READER_H_\n#define RIEGELI_BYTES_BUFFERED_READER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/strings/cord.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/sized_shared_buffer.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nclass BackwardWriter;\nclass Writer;\n\n// Abstract class `BufferedReader` helps to implement a `Reader` for an\n// underlying source which provides data by copying to external byte arrays,\n// e.g. like in the `read()` syscall.\n//\n// `BufferedReader` accumulates data which has been pulled in a flat buffer.\n// Reading a large enough array bypasses the buffer.\nclass BufferedReader : public Reader {\n public:\n  // Derived classes which override `ToleratesReadingAhead()` further should\n  // return `true` when `BufferedReader::ToleratesReadingAhead()`, and possibly\n  // also in some other cases.\n  bool ToleratesReadingAhead() override { return read_all_hint(); }\n\n  bool SupportsSize() override { return exact_size() != std::nullopt; }\n\n protected:\n  // Creates a closed `BufferedReader`.\n  explicit BufferedReader(Closed) noexcept : Reader(kClosed) {}\n\n  explicit BufferedReader(\n      BufferOptions buffer_options = BufferOptions()) noexcept;\n\n  BufferedReader(BufferedReader&& that) noexcept;\n  BufferedReader& operator=(BufferedReader&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `BufferedReader`. This\n  // avoids constructing a temporary `BufferedReader` and moving from it.\n  // Derived classes which redefine `Reset()` should include a call to\n  // `BufferedReader::Reset()`.\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options = BufferOptions());\n\n  void Done() override;\n\n  // Returns the options passed to the constructor.\n  BufferOptions buffer_options() const {\n    return buffer_sizer_.buffer_options();\n  }\n\n  // Storage for the hint set by `Reader::SetReadAllHint()`.\n  //\n  // If `true` and `exact_size()` is not `absl::nullptr`, this causes larger\n  // buffer sizes to be used before reaching `*exact_size()`.\n  bool read_all_hint() const { return buffer_sizer_.read_all_hint(); }\n\n  // Storage for an exact size of the source, as discovered by the `Reader`\n  // itself.\n  //\n  // If not `absl::nullptr` and `read_all_hint()` is `true`, this causes larger\n  // buffer sizes to be used before reaching `*exact_size()`.\n  //\n  // Also, if not `absl::nullptr`, this causes a smaller buffer size to be used\n  // when reaching `*exact_size()`.\n  void set_exact_size(std::optional<Position> exact_size) {\n    buffer_sizer_.set_exact_size(exact_size);\n  }\n  std::optional<Position> exact_size() const {\n    return buffer_sizer_.exact_size();\n  }\n\n  // In derived classes this must be called during initialization if reading\n  // starts from a position greater than 0.\n  void BeginRun() { buffer_sizer_.BeginRun(start_pos()); }\n\n  // `BufferedReader::{Done,SyncImpl}()` seek the source back to the current\n  // position if not all buffered data were read. This is feasible only if\n  // `SupportsRandomAccess()`.\n  //\n  // Warning: if `!SupportsRandomAccess()`, the source will have an\n  // unpredictable amount of extra data consumed because of buffering.\n  //\n  // For propagating `{Close,Sync}()` to dependencies, `{Done,SyncImpl}()`\n  // should be overridden to call `BufferedReader::{Done,SyncImpl}()` and then\n  // close/sync the dependencies.\n\n  // Reads data from the source, from the physical source position which is\n  // `limit_pos()`.\n  //\n  // Tries to read at most `max_length`, but can return successfully after\n  // reading at least `min_length` if less data was available in the source at\n  // the moment.\n  //\n  // Does not use buffer pointers. Increments `limit_pos()` by the length read,\n  // which must be in the range [`min_length`..`max_length`] on success. Returns\n  // `true` on success.\n  //\n  // Preconditions:\n  //   `0 < min_length <= max_length`\n  //   `ok()`\n  virtual bool ReadInternal(size_t min_length, size_t max_length,\n                            char* dest) = 0;\n\n  // Copies data from the source, from the physical source position which is\n  // `limit_pos()`, to `dest`.\n  //\n  // Does not use buffer pointers of `*this`. Increments `limit_pos()` by the\n  // length read, which must be `length` on success. Returns `true` on success.\n  //\n  // By default uses `Writer::Push()` and `ReadInternal()`.\n  //\n  // Preconditions:\n  //   `length > 0`\n  //   `ok()`\n  virtual bool CopyInternal(Position length, Writer& dest);\n\n  // Called when `exact_size()` was reached but reading more is requested.\n  // In this case `ReadInternal()` was not called.\n  //\n  // By default does nothing. This can be overridden e.g. to ensure that a\n  // compressed stream is fully consumed after decompressing all data.\n  virtual void ExactSizeReached();\n\n  // Implementation of `SeekSlow()`, called while no data are buffered.\n  //\n  // By default it is implemented analogously to the corresponding `Reader`\n  // function.\n  //\n  // Preconditions:\n  //   like the corresponding `Reader` function\n  //   `start_to_limit() == 0`\n  virtual bool SeekBehindBuffer(Position new_pos);\n\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  bool PullSlow(size_t min_length, size_t recommended_length) override;\n  using Reader::ReadSlow;\n  bool ReadSlow(size_t length, char* dest) override;\n  bool ReadSlow(size_t length, Chain& dest) override;\n  bool ReadSlow(size_t length, absl::Cord& dest) override;\n  using Reader::CopySlow;\n  bool CopySlow(Position length, Writer& dest) override;\n  bool CopySlow(size_t length, BackwardWriter& dest) override;\n  using Reader::ReadSomeSlow;\n  bool ReadSomeSlow(size_t max_length, char* dest) override;\n  using Reader::CopySomeSlow;\n  bool CopySomeSlow(size_t max_length, Writer& dest) override;\n  void ReadHintSlow(size_t min_length, size_t recommended_length) override;\n  bool SyncImpl(SyncType sync_type) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n\n  // Reuses `buffer_` as `reader.buffer_` if `reader.pos()` falls inside.\n  void ShareBufferTo(BufferedReader& reader) const;\n\n  // Extracts available data from the buffer, leaving it empty.\n  //\n  // `SaveBuffer()` is meant to be called in `Done()` to preserve pending data\n  // across instances reading from the same source.\n  SizedSharedBuffer SaveBuffer();\n\n  // Restores available data to the buffer.\n  //\n  // `RestoreBuffer()` is meant to be called in a constructor or `Reset()` with\n  // data previously returned by `SaveBuffer()`, to preserve pending data across\n  // instances reading from the same source.\n  //\n  // Precondition: `start_to_limit() == 0`\n  void RestoreBuffer(SizedSharedBuffer buffer);\n\n private:\n  // Discards buffer contents and sets buffer pointers to `nullptr`.\n  //\n  // This can move `pos()` forwards to account for skipping over previously\n  // buffered data. `limit_pos()` remains unchanged.\n  void SyncBuffer();\n\n  // Implementation of `CopySlow(Writer&)` in terms of `Writer::Push()` and\n  // `ReadInternal()`. Does not use buffer pointers.\n  //\n  // Precondition: `length > 0`\n  bool CopyUsingPush(Position length, Writer& dest);\n\n  ReadBufferSizer buffer_sizer_;\n  // Buffered data, read directly before the physical source position which is\n  // `limit_pos()`.\n  SizedSharedBuffer buffer_;\n\n  // Invariants:\n  //   if `!buffer_.empty()` then `start() == buffer_.data()`\n  //   `start_to_limit() == buffer_.size()`\n};\n\n// Implementation details follow.\n\ninline BufferedReader::BufferedReader(BufferOptions buffer_options) noexcept\n    : buffer_sizer_(buffer_options) {}\n\ninline BufferedReader::BufferedReader(BufferedReader&& that) noexcept\n    : Reader(static_cast<Reader&&>(that)),\n      buffer_sizer_(that.buffer_sizer_),\n      buffer_(std::move(that.buffer_)) {}\n\ninline BufferedReader& BufferedReader::operator=(\n    BufferedReader&& that) noexcept {\n  Reader::operator=(static_cast<Reader&&>(that));\n  buffer_sizer_ = that.buffer_sizer_;\n  buffer_ = std::move(that.buffer_);\n  return *this;\n}\n\ninline void BufferedReader::Reset(Closed) {\n  Reader::Reset(kClosed);\n  buffer_sizer_.Reset();\n  buffer_ = SizedSharedBuffer();\n}\n\ninline void BufferedReader::Reset(BufferOptions buffer_options) {\n  Reader::Reset();\n  buffer_sizer_.Reset(buffer_options);\n  buffer_.Clear();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_BUFFERED_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/buffered_writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/buffered_writer.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <limits>\n#include <optional>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid BufferedWriter::Done() {\n  const absl::string_view src(start(),\n                              UnsignedMax(start_to_cursor(), written_));\n  const Position new_pos = pos();\n  set_buffer();\n  written_ = 0;\n  DoneBehindBuffer(src);\n  if (ABSL_PREDICT_FALSE(start_pos() != new_pos) && ABSL_PREDICT_TRUE(ok())) {\n    SeekBehindBuffer(new_pos);\n  }\n  Writer::Done();\n  buffer_ = Buffer();\n}\n\nvoid BufferedWriter::DoneBehindBuffer(absl::string_view src) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::DoneBehindBuffer(): \"\n         \"buffer not empty\";\n  FlushBehindBuffer(src, FlushType::kFromObject);\n}\n\ninline bool BufferedWriter::SyncBuffer() {\n  const absl::string_view data(start(),\n                               UnsignedMax(start_to_cursor(), written_));\n  const Position new_pos = pos();\n  set_buffer();\n  written_ = 0;\n  if (data.empty()) return true;\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!WriteInternal(data))) return false;\n  if (ABSL_PREDICT_FALSE(start_pos() != new_pos)) {\n    return SeekBehindBuffer(new_pos);\n  }\n  return true;\n}\n\nvoid BufferedWriter::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  buffer_sizer_.set_write_size_hint(pos(), write_size_hint);\n}\n\nbool BufferedWriter::PushSlow(size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(min_length >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  const size_t buffer_length = UnsignedMin(\n      buffer_sizer_.BufferLength(start_pos(), min_length, recommended_length),\n      std::numeric_limits<Position>::max() - start_pos());\n  buffer_.Reset(buffer_length);\n  set_buffer(buffer_.data(), buffer_length);\n  return true;\n}\n\nbool BufferedWriter::FlushBehindBuffer(absl::string_view src,\n                                       FlushType flush_type) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::FlushBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (src.empty()) return true;\n  return WriteInternal(src);\n}\n\nbool BufferedWriter::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of BufferedWriter::SeekBehindBuffer(): \"\n         \"position unchanged, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  return Fail(absl::UnimplementedError(\"Writer::Seek() not supported\"));\n}\n\nstd::optional<Position> BufferedWriter::SizeBehindBuffer() {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::SizeBehindBuffer(): \"\n         \"buffer not empty\";\n  Fail(absl::UnimplementedError(\"Writer::Size() not supported\"));\n  return std::nullopt;\n}\n\nbool BufferedWriter::TruncateBehindBuffer(Position new_size) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::TruncateBehindBuffer(): \"\n         \"buffer not empty\";\n  return Fail(absl::UnimplementedError(\"Writer::Truncate() not supported\"));\n}\n\nReader* BufferedWriter::ReadModeBehindBuffer(Position initial_pos) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::ReadModeBehindBuffer(): \"\n         \"buffer not empty\";\n  Fail(absl::UnimplementedError(\"Writer::ReadMode() not supported\"));\n  return nullptr;\n}\n\nbool BufferedWriter::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of Writer::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  if (src.size() >= buffer_sizer_.BufferLength(pos())) {\n    // Write directly from `src`.\n    if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    return WriteInternal(src);\n  }\n  return Writer::WriteSlow(src);\n}\n\nbool BufferedWriter::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  for (const absl::string_view fragment : src.blocks()) {\n    if (ABSL_PREDICT_FALSE(!Write(fragment))) return false;\n  }\n  return true;\n}\n\nbool BufferedWriter::FlushImpl(FlushType flush_type) {\n  buffer_sizer_.EndRun(start_pos() + UnsignedMax(start_to_cursor(), written_));\n  const absl::string_view src(start(),\n                              UnsignedMax(start_to_cursor(), written_));\n  const Position new_pos = pos();\n  set_buffer();\n  written_ = 0;\n  if (ABSL_PREDICT_FALSE(!FlushBehindBuffer(src, flush_type))) return false;\n  const bool result = start_pos() == new_pos || SeekBehindBuffer(new_pos);\n  buffer_sizer_.BeginRun(start_pos());\n  return result;\n}\n\nbool BufferedWriter::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of Writer::SeekSlow(): \"\n         \"position unchanged, use Seek() instead\";\n  if (ABSL_PREDICT_TRUE(\n          SupportsRandomAccess() && new_pos >= start_pos() &&\n          new_pos <= start_pos() + UnsignedMax(start_to_cursor(), written_))) {\n    written_ = UnsignedMax(start_to_cursor(), written_);\n    set_cursor(start() + IntCast<size_t>(new_pos - start_pos()));\n    return true;\n  }\n  buffer_sizer_.EndRun(start_pos() + UnsignedMax(start_to_cursor(), written_));\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n  const bool result = SeekBehindBuffer(new_pos);\n  buffer_sizer_.BeginRun(start_pos());\n  return result;\n}\n\nstd::optional<Position> BufferedWriter::SizeImpl() {\n  buffer_sizer_.EndRun(start_pos() + UnsignedMax(start_to_cursor(), written_));\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return std::nullopt;\n  const std::optional<Position> size = SizeBehindBuffer();\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) return std::nullopt;\n  buffer_sizer_.BeginRun(start_pos());\n  return *size;\n}\n\nbool BufferedWriter::TruncateImpl(Position new_size) {\n  buffer_sizer_.EndRun(start_pos() + UnsignedMax(start_to_cursor(), written_));\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n  const bool result = TruncateBehindBuffer(new_size);\n  buffer_sizer_.BeginRun(start_pos());\n  return result;\n}\n\nReader* BufferedWriter::ReadModeImpl(Position initial_pos) {\n  buffer_sizer_.EndRun(start_pos() + UnsignedMax(start_to_cursor(), written_));\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return nullptr;\n  Reader* const reader = ReadModeBehindBuffer(initial_pos);\n  if (ABSL_PREDICT_FALSE(reader == nullptr)) return nullptr;\n  buffer_sizer_.BeginRun(start_pos());\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/buffered_writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_BUFFERED_WRITER_H_\n#define RIEGELI_BYTES_BUFFERED_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nclass Reader;\n\n// Abstract class `BufferedWriter` helps to implement a `Writer` for an\n// underlying destination which accepts data by copying from external byte\n// arrays, e.g. like in the `write()` syscall.\n//\n// `BufferedWriter` accumulates data to be pushed in a flat buffer. Writing a\n// large enough array bypasses the buffer.\nclass BufferedWriter : public Writer {\n protected:\n  // Creates a closed `BufferedWriter`.\n  explicit BufferedWriter(Closed) noexcept : Writer(kClosed) {}\n\n  explicit BufferedWriter(\n      BufferOptions buffer_options = BufferOptions()) noexcept;\n\n  BufferedWriter(BufferedWriter&& that) noexcept;\n  BufferedWriter& operator=(BufferedWriter&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `BufferedWriter`. This\n  // avoids constructing a temporary `BufferedWriter` and moving from it.\n  // Derived classes which redefine `Reset()` should include a call to\n  // `BufferedWriter::Reset()`.\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options = BufferOptions());\n\n  void Done() override;\n\n  // Returns the options passed to the constructor.\n  BufferOptions buffer_options() const {\n    return buffer_sizer_.buffer_options();\n  }\n\n  // In derived classes this must be called during initialization if writing\n  // starts from a position greater than 0.\n  void BeginRun() { buffer_sizer_.BeginRun(start_pos()); }\n\n  // `BufferedWriter::{Done,FlushImpl}()` call `{Done,Flush}BehindBuffer()` to\n  // write the last piece of data and close/flush the destination.\n  //\n  // For propagating `{Close,Flush}()` to dependencies, `{Done,FlushImpl}()`\n  // should be overridden to call `BufferedWriter::{Done,FlushImpl}()` and then\n  // close/flush the dependencies.\n\n  // Implementation of `Done()`, called with the last piece of data.\n  //\n  // By default calls `FlushBehindBuffer(FlushType::kFromObject)`, which by\n  // default writes data to the destination. Can be overridden if writing\n  // coupled with closing can be implemented better.\n  //\n  // Precondition: `start_to_limit() == 0`\n  virtual void DoneBehindBuffer(absl::string_view src);\n\n  // Writes data to the destination, to the physical destination position which\n  // is `start_pos()`.\n  //\n  // Does not use buffer pointers. Increments `start_pos()` by the length\n  // written, which must be `src.size()` on success. Returns `true` on success.\n  //\n  // Preconditions:\n  //   `!src.empty()`\n  //   `ok()`\n  virtual bool WriteInternal(absl::string_view src) = 0;\n\n  // Implementation of `FlushImpl()`, called with the last piece of data.\n  //\n  // By default writes data to the destination. Can be overridden if writing\n  // coupled with flushing can be implemented better.\n  //\n  // Precondition: `start_to_limit() == 0`\n  virtual bool FlushBehindBuffer(absl::string_view src, FlushType flush_type);\n\n  // Implementation of `SeekSlow()`, `SizeImpl()`, `TruncateImpl()`, and\n  // `ReadModeImpl()`, called while no data are buffered.\n  //\n  // By default they are implemented analogously to the corresponding `Writer`\n  // functions.\n  //\n  // Preconditions:\n  //   like the corresponding `Writer` functions\n  //   `start_to_limit() == 0`\n  virtual bool SeekBehindBuffer(Position new_pos);\n  virtual std::optional<Position> SizeBehindBuffer();\n  virtual bool TruncateBehindBuffer(Position new_size);\n  virtual Reader* ReadModeBehindBuffer(Position initial_pos);\n\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool FlushImpl(FlushType flush_type) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  bool TruncateImpl(Position new_size) override;\n  Reader* ReadModeImpl(Position initial_pos) override;\n\n private:\n  // Writes `buffer_` to the destination. Sets buffer pointers to `nullptr`.\n  bool SyncBuffer();\n\n  WriteBufferSizer buffer_sizer_;\n  // Contains buffered data, to be written directly after the physical\n  // destination position which is `start_pos()`.\n  Buffer buffer_;\n  // Size of buffered data is `UnsignedMax(start_to_cursor(), written_)`.\n  size_t written_ = 0;\n};\n\n// Implementation details follow.\n\ninline BufferedWriter::BufferedWriter(BufferOptions buffer_options) noexcept\n    : buffer_sizer_(buffer_options) {}\n\ninline BufferedWriter::BufferedWriter(BufferedWriter&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)),\n      buffer_sizer_(that.buffer_sizer_),\n      buffer_(std::move(that.buffer_)),\n      written_(that.written_) {}\n\ninline BufferedWriter& BufferedWriter::operator=(\n    BufferedWriter&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  buffer_sizer_ = that.buffer_sizer_;\n  buffer_ = std::move(that.buffer_);\n  written_ = that.written_;\n  return *this;\n}\n\ninline void BufferedWriter::Reset(Closed) {\n  Writer::Reset(kClosed);\n  buffer_sizer_.Reset();\n  buffer_ = Buffer();\n  written_ = 0;\n}\n\ninline void BufferedWriter::Reset(BufferOptions buffer_options) {\n  Writer::Reset();\n  buffer_sizer_.Reset(buffer_options);\n  written_ = 0;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_BUFFERED_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/cfile_handle.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// Make `O_CLOEXEC` available on Darwin.\n#if !defined(_XOPEN_SOURCE) || _XOPEN_SOURCE < 700\n#undef _XOPEN_SOURCE\n#define _XOPEN_SOURCE 700\n#endif\n\n#include \"riegeli/bytes/cfile_handle.h\"\n\n#ifdef __APPLE__\n#include <fcntl.h>\n#endif\n#include <stdio.h>\n\n#include <cerrno>\n#ifdef _WIN32\n#include <string>\n#endif\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/c_string_ref.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/type_erased_ref.h\"\n#ifdef _WIN32\n#include \"riegeli/base/unicode.h\"\n#endif\n#include \"riegeli/bytes/path_ref.h\"\n\nnamespace riegeli {\n\nnamespace cfile_internal {\n\ntemplate class CFileBase<UnownedCFileDeleter>;\ntemplate class CFileBase<OwnedCFileDeleter>;\n\n}  // namespace cfile_internal\n\nFILE* CFileHandle::GetMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target) {\n  return nullptr;\n}\n\nbool CFileHandle::IsOwningMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target) {\n  return false;\n}\n\nabsl::string_view CFileHandle::FilenameMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target) {\n  return kDefaultFilename;\n}\n\nabsl::Status CFileHandle::CloseMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target) {\n  return absl::OkStatus();\n}\n\nabsl::Status OwnedCFile::Open(PathInitializer filename, CStringRef mode) {\n  Reset(nullptr, std::move(filename));\n#ifndef _WIN32\n#ifndef __APPLE__\n  FILE* const file = fopen(c_filename(), mode.c_str());\n  if (ABSL_PREDICT_FALSE(file == nullptr)) {\n    const int error_number = errno;\n    return Annotate(absl::ErrnoToStatus(error_number, \"fopen() failed\"),\n                    absl::StrCat(\"opening \", this->filename()));\n  }\n#else   // __APPLE__\n  // Emulate `fopen()` with `open()` + `fdopen()`, adding support for 'e'\n  // (`O_CLOEXEC`).\n  mode_t open_mode;\n  const char* mode_ptr = mode.c_str();\n  switch (mode_ptr[0]) {\n    case 'r':\n      open_mode = O_RDONLY;\n      break;\n    case 'w':\n      open_mode = O_WRONLY | O_CREAT | O_TRUNC;\n      break;\n    case 'a':\n      open_mode = O_WRONLY | O_CREAT | O_APPEND;\n      break;\n    default:\n      return absl::InvalidArgumentError(absl::StrCat(\n          \"Mode must begin with 'r', 'w', or 'a': \", mode.c_str()));\n  }\n  for (++mode_ptr; *mode_ptr != '\\0' && *mode_ptr != ','; ++mode_ptr) {\n    switch (*mode_ptr) {\n      case '+':\n        open_mode = (open_mode & ~O_ACCMODE) | O_RDWR;\n        break;\n      case 'b':\n        break;\n      case 'x':\n        open_mode |= O_EXCL;\n        break;\n      case 'e':\n        open_mode |= O_CLOEXEC;\n        break;\n      default:\n        break;\n    }\n  }\nagain:\n  const int fd = open(c_filename(), open_mode, 0666);\n  if (ABSL_PREDICT_FALSE(fd < 0)) {\n    const int error_number = errno;\n    if (error_number == EINTR) goto again;\n    return Annotate(absl::ErrnoToStatus(error_number, \"open() failed\"),\n                    absl::StrCat(\"opening \", this->filename()));\n  }\n  FILE* const file = fdopen(fd, mode.c_str());\n  if (ABSL_PREDICT_FALSE(file == nullptr)) {\n    const int error_number = errno;\n    close(fd);\n    return Annotate(absl::ErrnoToStatus(error_number, \"fdopen() failed\"),\n                    absl::StrCat(\"opening \", this->filename()));\n  }\n#endif  // __APPLE__\n#else   // _WIN32\n  std::wstring filename_wide;\n  if (ABSL_PREDICT_FALSE(!Utf8ToWide(this->filename(), filename_wide))) {\n    return absl::InvalidArgumentError(\n        absl::StrCat(\"Filename not valid UTF-8: \", this->filename()));\n  }\n  std::wstring mode_wide;\n  if (ABSL_PREDICT_FALSE(!Utf8ToWide(mode.c_str(), mode_wide))) {\n    return absl::InvalidArgumentError(\n        absl::StrCat(\"Mode not valid UTF-8: \", mode.c_str()));\n  }\n  FILE* const file = _wfopen(filename_wide.c_str(), mode_wide.c_str());\n  if (ABSL_PREDICT_FALSE(file == nullptr)) {\n    const int error_number = errno;\n    return Annotate(absl::ErrnoToStatus(error_number, \"_wfopen() failed\"),\n                    absl::StrCat(\"opening \", this->filename()));\n  }\n#endif  // _WIN32\n  SetFileKeepFilename(file);\n  return absl::OkStatus();\n}\n\nabsl::Status OwnedCFile::Close() {\n  FILE* const file = Release();\n  if (file == nullptr) return absl::OkStatus();\n  if (ABSL_PREDICT_FALSE(fclose(file) != 0)) {\n    const int error_number = errno;\n    return Annotate(absl::ErrnoToStatus(error_number, \"fclose() failed\"),\n                    absl::StrCat(\"closing \", filename()));\n  }\n  return absl::OkStatus();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/cfile_handle.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_CFILE_HANDLE_H_\n#define RIEGELI_BYTES_CFILE_HANDLE_H_\n\n#include <stdio.h>\n\n#include <cstddef>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/any.h\"\n#include \"riegeli/base/c_string_ref.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/shared_ptr.h\"\n#include \"riegeli/base/type_erased_ref.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/bytes/cfile_internal.h\"\n#include \"riegeli/bytes/path_ref.h\"\n\nnamespace riegeli {\n\n// `SupportsCFileHandle<T>::value` is `true` if `T&` is a valid constructor\n// argument for `CFileHandle`.\n\ntemplate <typename T, typename Enable = void>\nstruct SupportsCFileHandle : std::false_type {};\n\ntemplate <typename T>\nstruct SupportsCFileHandle<\n    T,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<std::is_const<T>>,\n        std::is_convertible<decltype(std::declval<const T&>().get()), FILE*>>>>\n    : std::true_type {};\n\n// `CFileSupportsOpen<T>::value` is `true` if `T` supports `Open()` with the\n// signature like in `OwnedCFile`, but taking `const char* mode` instead of\n// `CStringRef mode` is sufficient.\n\ntemplate <typename T, typename Enable = void>\nstruct CFileSupportsOpen : std::false_type {};\n\ntemplate <typename T>\nstruct CFileSupportsOpen<\n    T, std::enable_if_t<std::is_convertible_v<\n           decltype(std::declval<T&>().Open(std::declval<PathInitializer>(),\n                                            std::declval<const char*>())),\n           absl::Status>>> : std::true_type {};\n\n// Type-erased pointer to a target object like `UnownedCFile` or `OwnedCFile`\n// which stores and possibly owns a `FILE*`.\n//\n// The target should support:\n//\n// ```\n//   // Returns the `FILE*`.\n//   FILE* get() const;\n//\n//   // Returns `true` if the target owns the `FILE*`, i.e. is responsible for\n//   // closing it and the `FILE*` is present.\n//   //\n//   // Optional. If absent, the presence of `Close()` determines whether the\n//   // target is considered to own the `FILE*`.\n//   bool IsOwning() const;\n//\n//   // Opens a new `FILE*`, like with `fopen()` but taking\n//   // `PathInitializer filename` instead of `const char* filename` and\n//   // returning `absl::Status` instead of `FILE*`.\n//   //\n//   // Optional. Not used by `CFileHandle` itself. Used by `CFileReader` and\n//   // `CFileWriter` constructors from the filename.\n//   absl::Status Open(PathInitializer filename, const char* mode);\n//\n//   // Returns the filename of the `FILE*`, or \"<none>\" for\n//   // default-constructed or moved-from target. Unchanged by `Close()`.\n//   //\n//   // If `Open()` was used, this is the filename passed to `Open()`, otherwise\n//   // a filename is inferred from the `FILE*`. This can be a placeholder\n//   // instead of a real filename if the `FILE*` does not refer to a named file\n//   // or inferring the filename is not supported.\n//   //\n//   // Optional. If absent, \"<unsupported>\" is assumed.\n//   absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n//\n//   // If `IsOwning()`, closes the `FILE*`.\n//   //\n//   // If `!IsOwning()`, does nothing and returns `absl::OkStatus()`.\n//   //\n//   // Optional. If absent, `absl::OkStatus()` is assumed.\n//   absl::Status Close();\n// ```\nclass CFileHandle : public WithEqual<CFileHandle> {\n public:\n  // Creates a `CFileHandle` which does not refer to a target.\n  CFileHandle() = default;\n  /*implicit*/ CFileHandle(std::nullptr_t) {}\n\n  // Creates a `CFileHandle` which refers to `target`.\n  template <typename T,\n            std::enable_if_t<std::conjunction_v<NotSameRef<CFileHandle, T&>,\n                                                SupportsCFileHandle<T>>,\n                             int> = 0>\n  /*implicit*/ CFileHandle(T& target ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : methods_(&kMethods<T>), target_(target) {}\n\n  CFileHandle(const CFileHandle& that) = default;\n  CFileHandle& operator=(const CFileHandle& that) = default;\n\n  // Returns `true` if the `FILE*` is present.\n  bool is_open() const { return *this != nullptr; }\n\n  // Returns the `FILE*`.\n  FILE* get() const { return methods_->get(target_); }\n\n  // Returns `true` if the `CFileHandle` owns the `FILE*`, i.e. is responsible\n  // for closing it and the `FILE*` is present.\n  bool IsOwning() const { return methods_->is_owning(target_); }\n\n  // Returns the filename of the `FILE*`, or \"<none>\" for default-constructed or\n  // moved-from target. Unchanged by `Close()`.\n  //\n  // If `Open()` was used, this is the filename passed to `Open()`, otherwise a\n  // filename is inferred from the `FILE*`. This can be a placeholder instead of\n  // a real filename if the `FILE*` does not refer to a named file or inferring\n  // the filename is not supported.\n  //\n  // If the target does not support `filename()`, returns \"<unsupported>\".\n  absl::string_view filename() const { return methods_->filename(target_); }\n\n  // If `IsOwning()`, closes the `FILE*`.\n  //\n  // If `!IsOwning()`, does nothing and returns `absl::OkStatus()`.\n  absl::Status Close() { return methods_->close(target_); }\n\n  friend bool operator==(CFileHandle a, CFileHandle b) {\n    return a.get() == b.get();\n  }\n  friend bool operator==(CFileHandle a, FILE* b) { return a.get() == b; }\n  friend bool operator==(CFileHandle a, std::nullptr_t) {\n    return a.target_.empty() || a.get() == nullptr;\n  }\n\n private:\n  struct Methods {\n    FILE* (*get)(TypeErasedRef target);\n    bool (*is_owning)(TypeErasedRef target);\n    absl::string_view (*filename)(TypeErasedRef target);\n    absl::Status (*close)(TypeErasedRef target);\n  };\n\n  template <typename T, typename Enable = void>\n  struct HasIsOwning : std::false_type {};\n  template <typename T>\n  struct HasIsOwning<T,\n                     std::enable_if_t<std::is_convertible_v<\n                         decltype(std::declval<const T&>().IsOwning()), bool>>>\n      : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct HasFilename : std::false_type {};\n  template <typename T>\n  struct HasFilename<\n      T, std::enable_if_t<std::is_convertible_v<\n             decltype(std::declval<const T&>().filename()), absl::string_view>>>\n      : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct HasClose : std::false_type {};\n  template <typename T>\n  struct HasClose<T, std::enable_if_t<std::is_convertible_v<\n                         decltype(std::declval<T&>().Close()), absl::Status>>>\n      : std::true_type {};\n\n  static FILE* GetMethodDefault(TypeErasedRef target);\n  static bool IsOwningMethodDefault(TypeErasedRef target);\n  static absl::string_view FilenameMethodDefault(TypeErasedRef target);\n  static absl::Status CloseMethodDefault(TypeErasedRef target);\n\n  static constexpr Methods kMethodsDefault = {\n      GetMethodDefault, IsOwningMethodDefault, FilenameMethodDefault,\n      CloseMethodDefault};\n\n  template <typename T>\n  static FILE* GetMethod(TypeErasedRef target) {\n    return target.Cast<const T&>().get();\n  }\n\n  template <typename T>\n  static bool IsOwningMethod(TypeErasedRef target) {\n    if constexpr (HasIsOwning<T>::value) {\n      return target.Cast<const T&>().IsOwning();\n    } else if constexpr (HasClose<T>::value) {\n      return target.Cast<const T&>().get() != nullptr;\n    } else {\n      return false;\n    }\n  }\n\n  template <typename T>\n  static absl::string_view FilenameMethod(TypeErasedRef target) {\n    if constexpr (HasFilename<T>::value) {\n      return target.Cast<const T&>().filename();\n    } else {\n      return \"<unsupported>\";\n    }\n  }\n\n  template <typename T>\n  static absl::Status CloseMethod(TypeErasedRef target) {\n    if constexpr (HasClose<T>::value) {\n      return target.Cast<T&>().Close();\n    } else {\n      return absl::OkStatus();\n    }\n  }\n\n  template <typename T>\n  static constexpr Methods kMethods = {GetMethod<T>, IsOwningMethod<T>,\n                                       FilenameMethod<T>, CloseMethod<T>};\n\n  const Methods* methods_ = &kMethodsDefault;\n  TypeErasedRef target_;\n};\n\nnamespace cfile_internal {\n\nclass UnownedCFileDeleter;\n\n// Common parts of `UnownedCFileDeleter` and `OwnedCFileDeleter`.\nclass ABSL_ATTRIBUTE_TRIVIAL_ABI CFileDeleterBase {\n public:\n  CFileDeleterBase() = default;\n\n  explicit CFileDeleterBase(PathInitializer filename)\n      : metadata_(riegeli::Maker<Metadata>(std::move(filename))) {}\n\n  // Supports creating a `CFileBase` converted from `UnownedCFile`.\n  explicit CFileDeleterBase(const UnownedCFileDeleter& that);\n  explicit CFileDeleterBase(UnownedCFileDeleter&& that);\n\n  void Reset() { metadata_ = nullptr; }\n\n  void Reset(PathInitializer filename) {\n    if (!metadata_.IsUnique()) {\n      metadata_.Reset(riegeli::Maker<Metadata>(std::move(filename)));\n    } else {\n      riegeli::Reset(metadata_->filename, std::move(filename));\n    }\n  }\n\n  // Supports creating a `CFileBase` converted from `UnownedCFile`.\n  void Reset(const UnownedCFileDeleter& that);\n\n  // Supports creating a `CFileBase` converted from `UnownedCFile`, and\n  // resetting `CFileBase` from the same `CFileBase`.\n  void Reset(CFileDeleterBase&& that);\n\n  absl::string_view filename() const {\n    if (ABSL_PREDICT_FALSE(metadata_ == nullptr)) return kDefaultFilename;\n    return metadata_->filename;\n  }\n\n  const char* c_filename() const {\n    if (ABSL_PREDICT_FALSE(metadata_ == nullptr)) return kDefaultFilenameCStr;\n    return metadata_->filename.c_str();\n  }\n\n protected:\n  CFileDeleterBase(const CFileDeleterBase& that) = default;\n  CFileDeleterBase& operator=(const CFileDeleterBase& that) = default;\n\n  CFileDeleterBase(CFileDeleterBase&& that) = default;\n  CFileDeleterBase& operator=(CFileDeleterBase&& that) = default;\n\n private:\n  struct Metadata {\n    explicit Metadata(PathInitializer filename)\n        : filename(std::move(filename)) {}\n\n    std::string filename;\n  };\n\n  // `nullptr` means `filename = kDefaultFilename`.\n  SharedPtr<Metadata> metadata_;\n};\n\nclass UnownedCFileDeleter : public CFileDeleterBase {\n public:\n  using CFileDeleterBase::CFileDeleterBase;\n\n  // Supports creating an `UnownedCFile` converted from any `CFileBase`.\n  explicit UnownedCFileDeleter(const CFileDeleterBase& that)\n      : CFileDeleterBase(that) {}\n\n  UnownedCFileDeleter(const UnownedCFileDeleter& that) = default;\n  UnownedCFileDeleter& operator=(const UnownedCFileDeleter& that) = default;\n\n  UnownedCFileDeleter(UnownedCFileDeleter&& that) = default;\n  UnownedCFileDeleter& operator=(UnownedCFileDeleter&& that) = default;\n\n  using CFileDeleterBase::Reset;\n  // Supports creating an `UnownedCFile` converted from any `CFileBase`.\n  void Reset(const CFileDeleterBase& that) {\n    CFileDeleterBase::operator=(that);\n  }\n\n  static void Destroy(ABSL_ATTRIBUTE_UNUSED FILE* file) {}\n};\n\nclass OwnedCFileDeleter : public CFileDeleterBase {\n public:\n  using CFileDeleterBase::CFileDeleterBase;\n\n  OwnedCFileDeleter(OwnedCFileDeleter&& that) = default;\n  OwnedCFileDeleter& operator=(OwnedCFileDeleter&& that) = default;\n\n  static void Destroy(FILE* file) { fclose(file); }\n};\n\ninline CFileDeleterBase::CFileDeleterBase(const UnownedCFileDeleter& that)\n    : metadata_(that.metadata_) {}\n\ninline CFileDeleterBase::CFileDeleterBase(UnownedCFileDeleter&& that)\n    : metadata_(std::move(that.metadata_)) {}\n\ninline void CFileDeleterBase::Reset(const UnownedCFileDeleter& that) {\n  metadata_ = that.metadata_;\n}\n\ninline void CFileDeleterBase::Reset(CFileDeleterBase&& that) {\n  metadata_ = std::move(that.metadata_);\n}\n\n// Common parts of `UnownedCFile` and `OwnedCFile`.\ntemplate <typename Deleter>\nclass ABSL_ATTRIBUTE_TRIVIAL_ABI CFileBase {\n public:\n  // Creates a `CFileBase` which does not store a `FILE*` and stores \"<none>\"\n  // as the filename.\n  CFileBase() = default;\n  /*implicit*/ CFileBase(std::nullptr_t) {}\n\n  // Creates a `CFileBase` which stores `file` with the filename inferred from\n  // the `FILE*` (or \"<none>\" if `file == nullptr`).\n  explicit CFileBase(FILE* file ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : file_(file),\n        deleter_(file_ == nullptr ? Deleter()\n                                  : Deleter(FilenameForCFile(file_))) {}\n\n  // Creates a `CFileBase` which stores `file` with `filename`.\n  explicit CFileBase(FILE* file ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                     PathInitializer filename)\n      : file_(file), deleter_(std::move(filename)) {}\n\n  // Creates a `CFileBase` converted from `UnownedCFile`.\n  template <\n      typename DependentDeleter = Deleter,\n      std::enable_if_t<!std::is_same_v<DependentDeleter, UnownedCFileDeleter>,\n                       int> = 0>\n  explicit CFileBase(const CFileBase<UnownedCFileDeleter>& that)\n      : file_(that.file_), deleter_(that.deleter_) {}\n  template <\n      typename DependentDeleter = Deleter,\n      std::enable_if_t<!std::is_same_v<DependentDeleter, UnownedCFileDeleter>,\n                       int> = 0>\n  explicit CFileBase(CFileBase<UnownedCFileDeleter>&& that)\n      : file_(that.Release()), deleter_(std::move(that.deleter_)) {}\n\n  // Creates an `UnownedCFile` converted from any `CFileBase`.\n  template <\n      typename OtherDeleter,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::is_same<Deleter, UnownedCFileDeleter>,\n              std::negation<std::is_same<OtherDeleter, UnownedCFileDeleter>>>,\n          int> = 0>\n  /*implicit*/ CFileBase(\n      const CFileBase<OtherDeleter>& that ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : file_(that.file_), deleter_(that.deleter_) {}\n\n  // Makes `*this` equivalent to a newly constructed `CFileBase`.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(std::nullptr_t = nullptr) {\n    SetFileKeepFilename();\n    deleter_.Reset();\n  }\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(FILE* file) {\n    SetFileKeepFilename(file);\n    if (file == nullptr) {\n      deleter_.Reset();\n    } else {\n      deleter_.Reset(FilenameForCFile(file));\n    }\n  }\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(FILE* file,\n                                          PathInitializer filename) {\n    SetFileKeepFilename(file);\n    deleter_.Reset(std::move(filename));\n  }\n  template <\n      typename OtherDeleter,\n      std::enable_if_t<\n          std::disjunction_v<std::is_same<Deleter, UnownedCFileDeleter>,\n                             std::is_same<OtherDeleter, UnownedCFileDeleter>>,\n          int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(const CFileBase<OtherDeleter>& that) {\n    SetFileKeepFilename(that.file_);\n    deleter_.Reset(that.deleter_);\n  }\n  template <\n      typename OtherDeleter,\n      std::enable_if_t<\n          std::disjunction_v<std::is_same<OtherDeleter, UnownedCFileDeleter>,\n                             std::is_same<OtherDeleter, Deleter>>,\n          int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(CFileBase<OtherDeleter>&& that) {\n    SetFileKeepFilename(that.Release());\n    deleter_.Reset(std::move(that.deleter_));\n  }\n\n  // Sets the `FILE*`, keeping `filename()` unchanged.\n  void SetFileKeepFilename(FILE* file = nullptr) {\n    Destroy();\n    file_ = file;\n  }\n\n  // Returns `true` if the `FILE*` is present.\n  bool is_open() const { return file_ != nullptr; }\n\n  // Returns the `FILE*`.\n  FILE* get() const { return file_; }\n\n  // Returns the filename of the `FILE*`, or \"<none>\" for default-constructed or\n  // moved-from `CFileBase`. Unchanged by `Close()` and `Release()`.\n  //\n  // If `Open()` was used, this is the filename passed to `Open()`, otherwise\n  // a filename is inferred from the `FILE*`. This can be a placeholder instead\n  // of a real filename if the `FILE*` does not refer to a named file or\n  // inferring the filename is not supported.\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return deleter_.filename();\n  }\n\n  // Returns `filename()` as a NUL-terminated string.\n  const char* c_filename() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return deleter_.c_filename();\n  }\n\n protected:\n  CFileBase(const CFileBase& that) = default;\n  CFileBase& operator=(const CFileBase& that) = default;\n\n  CFileBase(CFileBase&& that) noexcept\n      : file_(that.Release()), deleter_(std::move(that.deleter_)) {}\n  CFileBase& operator=(CFileBase&& that) noexcept {\n    FILE* const file = that.Release();\n    Destroy();\n    file_ = file;\n    deleter_ = std::move(that.deleter_);\n    return *this;\n  }\n\n  ~CFileBase() { Destroy(); }\n\n  // Returns the file. The stored `FILE*` is left absent, without modifying\n  // `filename()`.\n  FILE* Release() { return std::exchange(file_, nullptr); }\n\n private:\n  template <typename OtherDeleter>\n  friend class CFileBase;  // For conversions.\n\n  void Destroy() {\n    if (is_open()) deleter_.Destroy(file_);\n  }\n\n  FILE* file_ = nullptr;\n  Deleter deleter_;\n};\n\nextern template class CFileBase<UnownedCFileDeleter>;\nextern template class CFileBase<OwnedCFileDeleter>;\n\n}  // namespace cfile_internal\n\n// Stores a `FILE*` but does not own it, i.e. is not responsible for closing it.\n//\n// The `FILE*` can be `nullptr` which means absent.\nclass UnownedCFile\n    : public cfile_internal::CFileBase<cfile_internal::UnownedCFileDeleter>,\n      public WithEqual<UnownedCFile> {\n public:\n  using CFileBase::CFileBase;\n\n  // Overridden to make implicit.\n  /*implicit*/ UnownedCFile(FILE* file ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : CFileBase(file) {}\n\n  // Creates an `UnownedCFile` which stores `file.get()` with `file.filename()`.\n  explicit UnownedCFile(CFileHandle file)\n      : CFileBase(file.get(), file.filename()) {}\n\n  UnownedCFile(const UnownedCFile& that) = default;\n  UnownedCFile& operator=(const UnownedCFile& that) = default;\n\n  // The moved-from `FILE*` is left absent.\n  UnownedCFile(UnownedCFile&& that) = default;\n  UnownedCFile& operator=(UnownedCFile&& that) = default;\n\n  using CFileBase::Reset;\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(CFileHandle file) {\n    Reset(file.get(), file.filename());\n  }\n\n  friend bool operator==(const UnownedCFile& a, const UnownedCFile& b) {\n    return a.get() == b.get();\n  }\n  friend bool operator==(const UnownedCFile& a, FILE* b) {\n    return a.get() == b;\n  }\n};\n\n// Owns a `FILE*`, i.e. stores it and is responsible for closing it.\n//\n// The `FILE*` can be `nullptr` which means absent.\nclass OwnedCFile\n    : public cfile_internal::CFileBase<cfile_internal::OwnedCFileDeleter>,\n      public WithEqual<OwnedCFile> {\n public:\n  using CFileBase::CFileBase;\n\n  // The moved-from `FILE*` is left absent.\n  OwnedCFile(OwnedCFile&& that) = default;\n  OwnedCFile& operator=(OwnedCFile&& that) = default;\n\n  // Overridden to apply `ABSL_ATTRIBUTE_LIFETIME_BOUND`.\n  FILE* get() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return CFileBase::get(); }\n\n  using CFileBase::Release;\n\n  // Opens a new `FILE*`, like with `fopen()`, but taking\n  // `PathInitializer filename` instead of `const char* filename`,\n  // `CStringRef mode` instead of `const char* mode`, and returning\n  // `absl::Status` instead of `FILE*`.\n  ABSL_ATTRIBUTE_REINITIALIZES absl::Status Open(PathInitializer filename,\n                                                 CStringRef mode);\n\n  // Closes the `FILE*` if present.\n  //\n  // Returns `absl::OkStatus()` if absent.\n  absl::Status Close();\n\n  friend bool operator==(const OwnedCFile& a, FILE* b) { return a.get() == b; }\n};\n\n// Type-erased object like `UnownedCFile` or `OwnedCFile` which stores and\n// possibly owns a `FILE*`.\nusing AnyCFile = Any<CFileHandle>::Inlining<UnownedCFile, OwnedCFile>;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_CFILE_HANDLE_H_\n"
  },
  {
    "path": "riegeli/bytes/cfile_internal.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/cfile_internal.h\"\n\n#include <stdio.h>\n\n#include <string>\n\n#include \"absl/base/optimization.h\"\n#include \"riegeli/bytes/fd_internal.h\"\n\nnamespace riegeli::cfile_internal {\n\nstd::string FilenameForCFile(FILE* file) {\n  const int fd = fileno(file);\n  if (ABSL_PREDICT_FALSE(fd < 0)) {\n    return \"<unknown>\";\n  } else {\n    return fd_internal::FilenameForFd(fd);\n  }\n}\n\n}  // namespace riegeli::cfile_internal\n"
  },
  {
    "path": "riegeli/bytes/cfile_internal.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_CFILE_INTERNAL_H_\n#define RIEGELI_BYTES_CFILE_INTERNAL_H_\n\n#include <stdio.h>\n\n#include <string>\n\nnamespace riegeli::cfile_internal {\n\n// Infers a filename from the fd corresponding to the `FILE` by reading the\n// symlink target for `absl::StrCat(\"/proc/self/fd/\", fd)` (on Windows returns\n// a `absl::StrCat(\"<fd \", fd, \">\")` placeholder instead), or returning\n// \"<unknown>\" if there is no corresponding fd.\nstd::string FilenameForCFile(FILE* file);\n\n}  // namespace riegeli::cfile_internal\n\n#endif  // RIEGELI_BYTES_CFILE_INTERNAL_H_\n"
  },
  {
    "path": "riegeli/bytes/cfile_internal_for_cc.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_CFILE_INTERNAL_FOR_CC_H_\n#define RIEGELI_BYTES_CFILE_INTERNAL_FOR_CC_H_\n\n// Warning: Do not include this header in other headers, because the definition\n// of `off_t` depends on `_FILE_OFFSET_BITS` which can reliably be set only\n// in a standalone compilation unit.\n\n#include <stdio.h>\n#ifndef _WIN32\n#include <sys/types.h>\n#endif\n\n#include <type_traits>\n#include <utility>\n\n#include \"absl/strings/string_view.h\"\n\nnamespace riegeli::cfile_internal {\n\n#ifndef _WIN32\n\n// Use `fseeko()` and `ftello()` when available, otherwise `fseek()` and\n// `ftell()`.\n\ntemplate <typename DependentFILE, typename Enable = void>\nstruct HaveFSeekO : std::false_type {};\n\ntemplate <typename DependentFILE>\nstruct HaveFSeekO<\n    DependentFILE,\n    std::void_t<decltype(fseeko(std::declval<DependentFILE*>(),\n                                std::declval<off_t>(), std::declval<int>())),\n                decltype(ftello(std::declval<DependentFILE*>()))>>\n    : std::true_type {};\n\nusing Offset = std::conditional_t<HaveFSeekO<FILE>::value, off_t, long>;\n\ntemplate <typename DependentFILE>\ninline int FSeek(DependentFILE* file, Offset offset, int whence) {\n  if constexpr (HaveFSeekO<DependentFILE>::value) {\n    return fseeko(file, offset, whence);\n  } else {\n    return fseek(file, offset, whence);\n  }\n}\n\ninline constexpr absl::string_view kFSeekFunctionName =\n    HaveFSeekO<FILE>::value ? \"fseeko()\" : \"fseek()\";\n\ntemplate <typename DependentFILE>\ninline Offset FTell(DependentFILE* file) {\n  if constexpr (HaveFSeekO<DependentFILE>::value) {\n    return ftello(file);\n  } else {\n    return ftell(file);\n  }\n}\n\ninline constexpr absl::string_view kFTellFunctionName =\n    HaveFSeekO<FILE>::value ? \"ftello()\" : \"ftell()\";\n\n#else  // _WIN32\n\nusing Offset = __int64;\n\ninline int FSeek(FILE* file, Offset offset, int whence) {\n  return _fseeki64(file, offset, whence);\n}\n\ninline constexpr absl::string_view kFSeekFunctionName = \"_fseeki64\";\n\ninline Offset FTell(FILE* file) { return _ftelli64(file); }\n\ninline constexpr absl::string_view kFTellFunctionName = \"_ftelli64\";\n\n#endif  // _WIN32\n\n}  // namespace riegeli::cfile_internal\n\n#endif  // RIEGELI_BYTES_CFILE_INTERNAL_FOR_CC_H_\n"
  },
  {
    "path": "riegeli/bytes/cfile_reader.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef _WIN32\n\n// Make `fseeko()` and `ftello()` available.\n#if !defined(_XOPEN_SOURCE) || _XOPEN_SOURCE < 500\n#undef _XOPEN_SOURCE\n#define _XOPEN_SOURCE 500\n#endif\n\n// Make `off_t` 64-bit even on 32-bit systems.\n#undef _FILE_OFFSET_BITS\n#define _FILE_OFFSET_BITS 64\n\n#endif\n\n#include \"riegeli/bytes/cfile_reader.h\"\n\n#ifdef _WIN32\n#include <fcntl.h>\n#include <io.h>\n#endif\n#include <stddef.h>\n#include <stdio.h>\n\n#include <cerrno>\n#include <limits>\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/dynamic_annotations.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/global.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/cfile_internal_for_cc.h\"\n\nnamespace riegeli {\n\nnamespace {\n\n// There is no portable way to apply efficient buffering over `FILE` while\n// avoiding reading ahead more than eventually needed, because `fread()`\n// performs multiple low level reads until the whole requested length is read.\n//\n// To solve this, `AvailableLength()` tries to determine how much data is\n// available in the buffer of the `FILE` in a glibc-specific way, with a\n// portable fallback.\n\ntemplate <typename DependentFILE, typename Enable = void>\nstruct HasAvailableLength : std::false_type {};\n\ntemplate <typename DependentFILE>\nstruct HasAvailableLength<\n    DependentFILE,\n    std::void_t<decltype(std::declval<DependentFILE>()._IO_read_end -\n                         std::declval<DependentFILE>()._IO_read_ptr)>>\n    : std::true_type {};\n\ntemplate <typename DependentFILE>\ninline size_t AvailableLength(DependentFILE* src) {\n  if constexpr (HasAvailableLength<DependentFILE>::value) {\n    // Msan does not properly track initialization performed by precompiled\n    // libraries.\n    ABSL_ANNOTATE_MEMORY_IS_INITIALIZED(src, sizeof(DependentFILE));\n    return PtrDistance(src->_IO_read_ptr, src->_IO_read_end);\n  } else {\n    return 0;\n  }\n}\n\n}  // namespace\n\nvoid CFileReaderBase::Initialize(FILE* src, Options&& options) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of CFileReader: null FILE pointer\";\n  InitializePos(src, std::move(options)\n#ifdef _WIN32\n                         ,\n                /*mode_was_passed_to_fopen=*/false\n#endif\n  );\n}\n\nvoid CFileReaderBase::InitializePos(FILE* src, Options&& options\n#ifdef _WIN32\n                                    ,\n                                    bool mode_was_passed_to_fopen\n#endif\n) {\n  RIEGELI_ASSERT(!supports_random_access_)\n      << \"Failed precondition of CFileReaderBase::InitializePos(): \"\n         \"supports_random_access_ not reset\";\n  RIEGELI_ASSERT_OK(random_access_status_)\n      << \"Failed precondition of CFileReaderBase::InitializePos(): \"\n         \"random_access_status_ not reset\";\n#ifdef _WIN32\n  RIEGELI_ASSERT_EQ(original_mode_, std::nullopt)\n      << \"Failed precondition of CFileReaderBase::InitializePos(): \"\n         \"original_mode_ not reset\";\n#endif\n  if (ABSL_PREDICT_FALSE(ferror(src))) {\n    FailOperation(\"FILE\");\n    return;\n  }\n#ifdef _WIN32\n  int text_mode = file_internal::GetTextAsFlags(options.mode());\n  if (!mode_was_passed_to_fopen && text_mode != 0) {\n    const int fd = _fileno(src);\n    if (ABSL_PREDICT_FALSE(fd < 0)) {\n      FailOperation(\"_fileno()\");\n      return;\n    }\n    const int original_mode = _setmode(fd, text_mode);\n    if (ABSL_PREDICT_FALSE(original_mode < 0)) {\n      FailOperation(\"_setmode()\");\n      return;\n    }\n    original_mode_ = original_mode;\n  }\n  if (options.assumed_pos() == std::nullopt) {\n    if (text_mode == 0) {\n      const int fd = _fileno(src);\n      if (ABSL_PREDICT_FALSE(fd < 0)) {\n        FailOperation(\"_fileno()\");\n        return;\n      }\n      // There is no `_getmode()`, but `_setmode()` returns the previous mode.\n      text_mode = _setmode(fd, _O_BINARY);\n      if (ABSL_PREDICT_FALSE(text_mode < 0)) {\n        FailOperation(\"_setmode()\");\n        return;\n      }\n      if (ABSL_PREDICT_FALSE(_setmode(fd, text_mode) < 0)) {\n        FailOperation(\"_setmode()\");\n        return;\n      }\n    }\n    if (text_mode != _O_BINARY) options.set_assumed_pos(0);\n  }\n#endif  // _WIN32\n  if (options.assumed_pos() != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(\n            *options.assumed_pos() >\n            Position{std::numeric_limits<cfile_internal::Offset>::max()})) {\n      FailOverflow();\n      return;\n    }\n    set_limit_pos(*options.assumed_pos());\n    // `supports_random_access_` is left as `false`.\n    random_access_status_ = Global([] {\n      return absl::UnimplementedError(\n          \"CFileReaderBase::Options::assumed_pos() excludes random access\");\n    });\n  } else {\n    const cfile_internal::Offset file_pos = cfile_internal::FTell(src);\n    if (file_pos < 0) {\n      // Random access is not supported. Assume 0 as the initial position.\n      // `supports_random_access_` is left as `false`.\n      random_access_status_ =\n          FailedOperationStatus(cfile_internal::kFTellFunctionName);\n      clearerr(src);\n      return;\n    }\n    set_limit_pos(IntCast<Position>(file_pos));\n\n    // Check the size, and whether random access is supported.\n    if (cfile_internal::FSeek(src, 0, SEEK_END) != 0) {\n      // Random access is not supported. `supports_random_access_` is left as\n      // `false`.\n      random_access_status_ =\n          FailedOperationStatus(cfile_internal::kFSeekFunctionName);\n      clearerr(src);\n      return;\n    }\n    cfile_internal::Offset file_size = cfile_internal::FTell(src);\n    if (ABSL_PREDICT_FALSE(file_size < 0)) {\n      FailOperation(cfile_internal::kFTellFunctionName);\n      return;\n    }\n    if (limit_pos() != IntCast<Position>(file_size)) {\n      if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(\n                                 src,\n                                 IntCast<cfile_internal::Offset>(limit_pos()),\n                                 SEEK_SET) != 0)) {\n        FailOperation(cfile_internal::kFSeekFunctionName);\n        return;\n      }\n    }\n#ifndef _WIN32\n    if (file_size == 0 && limit_pos() == 0) {\n      // Some \"/proc\" and \"/sys\" files claim to have zero size but have\n      // non-empty contents when read.\n      if (BufferedReader::PullSlow(1, 0)) {\n        if (growing_source_) {\n          // Check the size again. Maybe the file has grown.\n          if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(src, 0, SEEK_END) !=\n                                 0)) {\n            FailOperation(cfile_internal::kFSeekFunctionName);\n            return;\n          }\n          file_size = cfile_internal::FTell(src);\n          if (ABSL_PREDICT_FALSE(file_size < 0)) {\n            FailOperation(cfile_internal::kFTellFunctionName);\n            return;\n          }\n          if (limit_pos() != IntCast<Position>(file_size)) {\n            if (ABSL_PREDICT_FALSE(\n                    cfile_internal::FSeek(\n                        src, IntCast<cfile_internal::Offset>(limit_pos()),\n                        SEEK_SET) != 0)) {\n              FailOperation(cfile_internal::kFSeekFunctionName);\n              return;\n            }\n          }\n          if (file_size > 0) goto regular;\n        }\n        // This is one of \"/proc\" or \"/sys\" files which claim to have zero size\n        // but have non-empty contents when read. Random access is not\n        // supported. `supports_random_access_` is left as `false`.\n        random_access_status_ = Global([] {\n          return absl::UnimplementedError(\n              \"Random access is not supported because \"\n              \"the file claims zero size but has non-empty contents when read\");\n        });\n        return;\n      }\n      if (ABSL_PREDICT_FALSE(!ok())) return;\n      // This is a regular empty file.\n    }\n  regular:\n#endif\n    // Random access is supported.\n    supports_random_access_ = true;\n    if (!growing_source_) set_exact_size(IntCast<Position>(file_size));\n  }\n  BeginRun();\n}\n\nvoid CFileReaderBase::Done() {\n  BufferedReader::Done();\n#ifdef _WIN32\n  if (original_mode_ != std::nullopt) {\n    FILE* const src = SrcFile();\n    const int fd = _fileno(src);\n    if (ABSL_PREDICT_FALSE(fd < 0)) {\n      FailOperation(\"_fileno()\");\n    } else if (ABSL_PREDICT_FALSE(_setmode(fd, *original_mode_) < 0)) {\n      FailOperation(\"_setmode()\");\n    }\n  }\n#endif  // !_WIN32\n  random_access_status_ = absl::OkStatus();\n}\n\ninline absl::Status CFileReaderBase::FailedOperationStatus(\n    absl::string_view operation) {\n  const int error_number = errno;\n  RIEGELI_ASSERT_NE(error_number, 0)\n      << \"Failed precondition of CFileReaderBase::FailedOperationStatus(): \"\n         \"zero errno\";\n  return absl::ErrnoToStatus(error_number, absl::StrCat(operation, \" failed\"));\n}\n\nbool CFileReaderBase::FailOperation(absl::string_view operation) {\n  return Fail(FailedOperationStatus(operation));\n}\n\nabsl::Status CFileReaderBase::AnnotateStatusImpl(absl::Status status) {\n  return BufferedReader::AnnotateStatusImpl(\n      Annotate(status, absl::StrCat(\"reading \", filename())));\n}\n\nbool CFileReaderBase::ReadInternal(size_t min_length, size_t max_length,\n                                   char* dest) {\n  RIEGELI_ASSERT_GT(min_length, 0u)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"nothing to read\";\n  RIEGELI_ASSERT_GE(max_length, min_length)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"max_length < min_length\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::ReadInternal()\";\n  FILE* const src = SrcFile();\n  for (;;) {\n    if (ABSL_PREDICT_FALSE(\n            limit_pos() >=\n            Position{std::numeric_limits<cfile_internal::Offset>::max()})) {\n      return FailOverflow();\n    }\n    const size_t length_to_read = UnsignedMin(\n        UnsignedClamp(AvailableLength(src), min_length, max_length),\n        Position{std::numeric_limits<cfile_internal::Offset>::max()} -\n            limit_pos());\n    const size_t length_read = fread(dest, 1, length_to_read, src);\n    RIEGELI_ASSERT_LE(length_read, length_to_read)\n        << \"fread() read more than requested\";\n    move_limit_pos(length_read);\n    if (ABSL_PREDICT_FALSE(length_read < length_to_read)) {\n      RIEGELI_ASSERT_LT(length_read, min_length)\n          << \"fread() read less than was available\";\n      if (ABSL_PREDICT_FALSE(ferror(src))) return FailOperation(\"fread()\");\n      RIEGELI_ASSERT(feof(src))\n          << \"fread() succeeded but read less than requested\";\n      clearerr(src);\n      if (!growing_source_) set_exact_size(limit_pos());\n      return false;\n    }\n    if (length_read >= min_length) return true;\n    dest += length_read;\n    min_length -= length_read;\n    max_length -= length_read;\n  }\n}\n\nbool CFileReaderBase::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!CFileReaderBase::SupportsRandomAccess())) {\n    if (ABSL_PREDICT_FALSE(new_pos < start_pos())) {\n      if (ok()) Fail(random_access_status_);\n      return false;\n    }\n    return BufferedReader::SeekBehindBuffer(new_pos);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  FILE* const src = SrcFile();\n  if (new_pos > limit_pos()) {\n    // Seeking forwards.\n    if (exact_size() != std::nullopt) {\n      if (ABSL_PREDICT_FALSE(new_pos > *exact_size())) {\n        // File ends.\n        if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(\n                src, IntCast<cfile_internal::Offset>(*exact_size()),\n                SEEK_SET)) != 0) {\n          return FailOperation(cfile_internal::kFSeekFunctionName);\n        }\n        set_limit_pos(*exact_size());\n        return false;\n      }\n    } else {\n      if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(src, 0, SEEK_END) != 0)) {\n        return FailOperation(cfile_internal::kFSeekFunctionName);\n      }\n      const cfile_internal::Offset file_size = cfile_internal::FTell(src);\n      if (ABSL_PREDICT_FALSE(file_size < 0)) {\n        return FailOperation(cfile_internal::kFTellFunctionName);\n      }\n      if (!growing_source_) set_exact_size(IntCast<Position>(file_size));\n      if (ABSL_PREDICT_FALSE(new_pos > IntCast<Position>(file_size))) {\n        // File ends.\n        set_limit_pos(IntCast<Position>(file_size));\n        return false;\n      }\n    }\n  }\n  if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(\n          src, IntCast<cfile_internal::Offset>(new_pos), SEEK_SET)) != 0) {\n    return FailOperation(cfile_internal::kFSeekFunctionName);\n  }\n  set_limit_pos(new_pos);\n  return true;\n}\n\nstd::optional<Position> CFileReaderBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  if (exact_size() != std::nullopt) return *exact_size();\n  if (ABSL_PREDICT_FALSE(!CFileReaderBase::SupportsRandomAccess())) {\n    Fail(random_access_status_);\n    return std::nullopt;\n  }\n  FILE* const src = SrcFile();\n  if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(src, 0, SEEK_END)) != 0) {\n    FailOperation(cfile_internal::kFSeekFunctionName);\n    return std::nullopt;\n  }\n  const cfile_internal::Offset file_size = cfile_internal::FTell(src);\n  if (ABSL_PREDICT_FALSE(file_size < 0)) {\n    FailOperation(cfile_internal::kFTellFunctionName);\n    return std::nullopt;\n  }\n  if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(\n                             src, IntCast<cfile_internal::Offset>(limit_pos()),\n                             SEEK_SET) != 0)) {\n    FailOperation(cfile_internal::kFSeekFunctionName);\n    return std::nullopt;\n  }\n  if (!growing_source_) set_exact_size(IntCast<Position>(file_size));\n  return IntCast<Position>(file_size);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/cfile_reader.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_CFILE_READER_H_\n#define RIEGELI_BYTES_CFILE_READER_H_\n\n#include <stddef.h>\n#include <stdio.h>\n\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/string_ref.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/cfile_handle.h\"\n#include \"riegeli/bytes/file_mode_string.h\"\n#include \"riegeli/bytes/path_ref.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `CFileReader`.\nclass CFileReaderBase : public BufferedReader {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // If `CFileReader` opens a `FILE` with a filename, `mode()` is the second\n    // argument of `fopen()` and specifies the open mode, typically \"r\" (on\n    // Windows: \"rb\").\n    //\n    // `mode()` can also be changed with `set_inheritable()` and `set_text()`.\n    //\n    // Default: \"re\" (on Windows: \"rbN\").\n    Options& set_mode(StringInitializer mode) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      riegeli::Reset(mode_, std::move(mode));\n      return *this;\n    }\n    Options&& set_mode(StringInitializer mode) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_mode(std::move(mode)));\n    }\n    const std::string& mode() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return mode_;\n    }\n\n    // If `false`, `execve()` (`CreateProcess()` on Windows) will close the\n    // file. This is not supported by all systems, but it is supported at least\n    // on Linux, Windows, FreeBSD, OpenBSD, NetBSD, and it is planned for POSIX\n    // (https://www.austingroupbugs.net/view.php?id=411). For MacOS X this is\n    // emulated by `CFileReader`.\n    //\n    // If `true`, the file will remain open across `execve()` (`CreateProcess()`\n    // on Windows).\n    //\n    // If `CFileReader` reads from an already open file, `inheritable()` has no\n    // effect.\n    //\n    // `set_inheritable()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_inheritable(bool inheritable) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      file_internal::SetInheritableReading(inheritable, mode_);\n      return *this;\n    }\n    Options&& set_inheritable(bool inheritable) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_inheritable(inheritable));\n    }\n    bool inheritable() const { return file_internal::GetInheritable(mode_); }\n\n    // If `false`, data will be read directly from the file. This is called the\n    // binary mode.\n    //\n    // If `true`, text mode translation will be applied on Windows:\n    // CR-LF character pairs are translated to LF, and a ^Z character is\n    // interpreted as end of file.\n    //\n    // It is recommended to use `ReadLine()` or `TextReader` instead, which\n    // expect a binary mode `Reader`.\n    //\n    // `set_text()` has an effect only on Windows. It is applicable whenever\n    // `CFileReader` opens a `FILE` with a filename or reads from an already\n    // open `FILE`.\n    //\n    // `set_text()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_text(bool text) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      file_internal::SetTextReading(text, mode_);\n      return *this;\n    }\n    Options&& set_text(bool text) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_text(text));\n    }\n    // No `text()` getter is provided. On Windows `mode()` can have unspecified\n    // text mode, resolved using `_get_fmode()`. Not on Windows the concept does\n    // not exist.\n\n    // If `std::nullopt`, the current position reported by `pos()` corresponds\n    // to the current `FILE` position if possible, otherwise 0 is assumed as the\n    // initial position. Random access is supported if the `FILE` supports\n    // random access.\n    //\n    // If not `std::nullopt`, this position is assumed initially, to be reported\n    // by `pos()`. It does not need to correspond to the current `FILE`\n    // position. Random access is not supported.\n    //\n    // Default: `std::nullopt`.\n    Options& set_assumed_pos(std::optional<Position> assumed_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      assumed_pos_ = assumed_pos;\n      return *this;\n    }\n    Options&& set_assumed_pos(std::optional<Position> assumed_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_pos(assumed_pos));\n    }\n    std::optional<Position> assumed_pos() const { return assumed_pos_; }\n\n    // If `true`, supports reading up to the end of the file, then retrying when\n    // the file has grown. This disables caching the file size.\n    //\n    // Default: `false`.\n    Options& set_growing_source(bool growing_source) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      growing_source_ = growing_source;\n      return *this;\n    }\n    Options&& set_growing_source(bool growing_source) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_growing_source(growing_source));\n    }\n    bool growing_source() const { return growing_source_; }\n\n   private:\n#ifndef _WIN32\n    std::string mode_ = \"re\";\n#else\n    std::string mode_ = \"rbN\";\n#endif\n    std::optional<Position> assumed_pos_;\n    bool growing_source_ = false;\n  };\n\n  // Returns the `CFileHandle` being read from. Unchanged by `Close()`.\n  virtual CFileHandle SrcCFileHandle() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the `FILE*` being read from. If the `FILE*` is owned then changed\n  // to `nullptr` by `Close()`, otherwise unchanged.\n  virtual FILE* SrcFile() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the filename of the `FILE*` being read from, or \"<none>\" for\n  // closed-constructed or moved-from `CFileReader`. Unchanged by `Close()`.\n  //\n  // If the constructor from filename was used, this is the filename passed to\n  // the constructor, otherwise a filename is inferred from the `FILE*`. This\n  // can be a placeholder instead of a real filename if the `FILE*` does not\n  // refer to a named file or inferring the filename is not supported.\n  //\n  // If `Src` does not support `filename()`, returns \"<unsupported>\".\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return SrcCFileHandle().filename();\n  }\n\n  bool ToleratesReadingAhead() override {\n    return BufferedReader::ToleratesReadingAhead() ||\n           CFileReaderBase::SupportsRandomAccess();\n  }\n  bool SupportsRandomAccess() override { return supports_random_access_; }\n\n protected:\n  explicit CFileReaderBase(Closed) noexcept : BufferedReader(kClosed) {}\n\n  explicit CFileReaderBase(BufferOptions buffer_options, bool growing_source);\n\n  CFileReaderBase(CFileReaderBase&& that) noexcept;\n  CFileReaderBase& operator=(CFileReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, bool growing_source);\n  void Initialize(FILE* src, Options&& options);\n  void InitializePos(FILE* src, Options&& options\n#ifdef _WIN32\n                     ,\n                     bool mode_was_passed_to_fopen\n#endif\n  );\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation);\n\n  void Done() override;\n  absl::Status AnnotateStatusImpl(absl::Status status) override;\n  bool ReadInternal(size_t min_length, size_t max_length, char* dest) override;\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n\n private:\n  absl::Status FailedOperationStatus(absl::string_view operation);\n\n  bool growing_source_ = false;\n  bool supports_random_access_ = false;\n  absl::Status random_access_status_;\n#ifdef _WIN32\n  std::optional<int> original_mode_;\n#endif\n\n  // Invariant:\n  //   `limit_pos() <= std::numeric_limits<cfile_internal::Offset>::max()`\n};\n\n// A `Reader` which reads from a `FILE`.\n//\n// `CFileReader` supports random access if\n// `Options::assumed_pos() == std::nullopt` and the `FILE` supports random\n// access (this is checked by calling `ftell()` and `fseek(SEEK_END)`).\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `FILE` being read from. `Src` must support\n// `Dependency<CFileHandle, Src>`, e.g. `OwnedCFile` (owned, default),\n// `UnownedCFile` (not owned), `AnyCFile` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `OwnedCFile` if\n// the first constructor argument is a filename or a `FILE*`, otherwise as\n// `TargetT` of the type of the first constructor argument.\n//\n// Warning: if random access is not supported and the `FILE` is not owned, it\n// will have an unpredictable amount of extra data consumed because of\n// buffering.\n//\n// Until the `CFileReader` is closed or no longer used, the `FILE` must not be\n// closed nor have its position changed.\ntemplate <typename Src = OwnedCFile>\nclass CFileReader : public CFileReaderBase {\n public:\n  // Creates a closed `CFileReader`.\n  explicit CFileReader(Closed) noexcept : CFileReaderBase(kClosed) {}\n\n  // Will read from the `FILE` provided by `src`.\n  explicit CFileReader(Initializer<Src> src, Options options = Options());\n\n  // Will read from `src`.\n  template <\n      typename DependentSrc = Src,\n      std::enable_if_t<std::is_constructible_v<DependentSrc, FILE*>, int> = 0>\n  explicit CFileReader(FILE* src ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                       Options options = Options());\n\n  // Opens a file for reading.\n  //\n  // If opening the file fails, `CFileReader` will be failed and closed.\n  //\n  // This constructor is present only if `Src` supports `Open()`.\n  template <typename DependentSrc = Src,\n            std::enable_if_t<\n                std::conjunction_v<CFileSupportsOpen<DependentSrc>,\n                                   std::is_default_constructible<DependentSrc>>,\n                int> = 0>\n  explicit CFileReader(PathInitializer filename, Options options = Options());\n\n  CFileReader(CFileReader&& that) = default;\n  CFileReader& operator=(CFileReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `CFileReader`. This avoids\n  // constructing a temporary `CFileReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n  template <\n      typename DependentSrc = Src,\n      std::enable_if_t<std::is_constructible_v<DependentSrc, FILE*>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(FILE* src,\n                                          Options options = Options());\n  template <typename DependentSrc = Src,\n            std::enable_if_t<std::conjunction_v<CFileSupportsOpen<DependentSrc>,\n                                                SupportsReset<DependentSrc>>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(PathInitializer filename,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the `FILE` being read\n  // from. Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  CFileHandle SrcCFileHandle() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n  FILE* SrcFile() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get().get();\n  }\n\n  // An optimized implementation in a derived class, avoiding a virtual call.\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.get().filename();\n  }\n\n protected:\n  void Done() override;\n\n private:\n  template <typename DependentSrc = Src,\n            std::enable_if_t<CFileSupportsOpen<DependentSrc>::value, int> = 0>\n  void OpenImpl(PathInitializer filename, Options&& options);\n\n  // The object providing and possibly owning the `FILE` being read from.\n  Dependency<CFileHandle, Src> src_;\n};\n\nexplicit CFileReader(Closed) -> CFileReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit CFileReader(\n    Src&& src, CFileReaderBase::Options options = CFileReaderBase::Options())\n    -> CFileReader<std::conditional_t<\n        std::disjunction_v<std::is_convertible<Src&&, FILE*>,\n                           std::is_convertible<Src&&, PathInitializer>>,\n        OwnedCFile, TargetT<Src>>>;\n\n// Implementation details follow.\n\ninline CFileReaderBase::CFileReaderBase(BufferOptions buffer_options,\n                                        bool growing_source)\n    : BufferedReader(buffer_options), growing_source_(growing_source) {}\n\ninline CFileReaderBase::CFileReaderBase(CFileReaderBase&& that) noexcept\n    : BufferedReader(static_cast<BufferedReader&&>(that)),\n      growing_source_(that.growing_source_),\n      supports_random_access_(\n          std::exchange(that.supports_random_access_, false)),\n      random_access_status_(std::move(that.random_access_status_))\n#ifdef _WIN32\n      ,\n      original_mode_(that.original_mode_)\n#endif\n{\n}\n\ninline CFileReaderBase& CFileReaderBase::operator=(\n    CFileReaderBase&& that) noexcept {\n  BufferedReader::operator=(static_cast<BufferedReader&&>(that));\n  growing_source_ = that.growing_source_;\n  supports_random_access_ = std::exchange(that.supports_random_access_, false);\n  random_access_status_ = std::move(that.random_access_status_);\n#ifdef _WIN32\n  original_mode_ = that.original_mode_;\n#endif\n  return *this;\n}\n\ninline void CFileReaderBase::Reset(Closed) {\n  BufferedReader::Reset(kClosed);\n  growing_source_ = false;\n  supports_random_access_ = false;\n  random_access_status_ = absl::OkStatus();\n#ifdef _WIN32\n  original_mode_ = std::nullopt;\n#endif\n}\n\ninline void CFileReaderBase::Reset(BufferOptions buffer_options,\n                                   bool growing_source) {\n  BufferedReader::Reset(buffer_options);\n  growing_source_ = growing_source;\n  supports_random_access_ = false;\n  random_access_status_ = absl::OkStatus();\n#ifdef _WIN32\n  original_mode_ = std::nullopt;\n#endif\n}\n\ntemplate <typename Src>\ninline CFileReader<Src>::CFileReader(Initializer<Src> src, Options options)\n    : CFileReaderBase(options.buffer_options(), options.growing_source()),\n      src_(std::move(src)) {\n  Initialize(src_.get().get(), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<std::is_constructible_v<DependentSrc, FILE*>, int>>\ninline CFileReader<Src>::CFileReader(FILE* src ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                                     Options options)\n    : CFileReader(riegeli::Maker(src), std::move(options)) {}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<\n              std::conjunction_v<CFileSupportsOpen<DependentSrc>,\n                                 std::is_default_constructible<DependentSrc>>,\n              int>>\ninline CFileReader<Src>::CFileReader(PathInitializer filename, Options options)\n    : CFileReaderBase(options.buffer_options(), options.growing_source()),\n      src_(riegeli::Maker()) {\n  OpenImpl(std::move(filename), std::move(options));\n}\n\ntemplate <typename Src>\ninline void CFileReader<Src>::Reset(Closed) {\n  CFileReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void CFileReader<Src>::Reset(Initializer<Src> src, Options options) {\n  CFileReaderBase::Reset(options.buffer_options(), options.growing_source());\n  src_.Reset(std::move(src));\n  Initialize(src_.get().get(), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<std::is_constructible_v<DependentSrc, FILE*>, int>>\ninline void CFileReader<Src>::Reset(FILE* src, Options options) {\n  Reset(riegeli::Maker(src), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<std::conjunction_v<CFileSupportsOpen<DependentSrc>,\n                                              SupportsReset<DependentSrc>>,\n                           int>>\ninline void CFileReader<Src>::Reset(PathInitializer filename, Options options) {\n  // In case `filename` is owned by `src_` and gets invalidated.\n  std::string filename_copy = std::move(filename);\n  riegeli::Reset(src_.manager());\n  CFileReaderBase::Reset(options.buffer_options(), options.growing_source());\n  OpenImpl(std::move(filename_copy), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<CFileSupportsOpen<DependentSrc>::value, int>>\nvoid CFileReader<Src>::OpenImpl(PathInitializer filename, Options&& options) {\n  absl::Status status =\n      src_.manager().Open(std::move(filename), options.mode());\n  if (ABSL_PREDICT_FALSE(!status.ok())) {\n    CFileReaderBase::Reset(kClosed);\n    FailWithoutAnnotation(std::move(status));\n    return;\n  }\n  InitializePos(src_.get().get(), std::move(options)\n#ifdef _WIN32\n                                      ,\n                /*mode_was_passed_to_fopen=*/true\n#endif\n  );\n}\n\ntemplate <typename Src>\nvoid CFileReader<Src>::Done() {\n  CFileReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (absl::Status status = src_.get().Close();\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      Fail(std::move(status));\n    }\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_CFILE_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/cfile_writer.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef _WIN32\n\n// Make `fseeko()` and `ftello()` available.\n#if !defined(_XOPEN_SOURCE) || _XOPEN_SOURCE < 500\n#undef _XOPEN_SOURCE\n#define _XOPEN_SOURCE 500\n#endif\n\n// Make `off_t` 64-bit even on 32-bit systems.\n#undef _FILE_OFFSET_BITS\n#define _FILE_OFFSET_BITS 64\n\n#endif\n\n#include \"riegeli/bytes/cfile_writer.h\"\n\n#ifdef _WIN32\n#include <fcntl.h>\n#include <io.h>\n#endif\n#include <stddef.h>\n#include <stdio.h>\n#ifndef _WIN32\n#include <unistd.h>\n#endif\n\n#include <cerrno>\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/global.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/cfile_handle.h\"\n#include \"riegeli/bytes/cfile_internal_for_cc.h\"\n#include \"riegeli/bytes/cfile_reader.h\"\n#include \"riegeli/bytes/file_mode_string.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid CFileWriterBase::Initialize(FILE* dest, Options&& options) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of CFileReader: null FILE pointer\";\n  InitializePos(dest, std::move(options), /*mode_was_passed_to_fopen=*/false);\n}\n\nvoid CFileWriterBase::InitializePos(FILE* dest, Options&& options,\n                                    bool mode_was_passed_to_fopen) {\n  RIEGELI_ASSERT_EQ(supports_random_access_, LazyBoolState::kUnknown)\n      << \"Failed precondition of CFileWriterBase::InitializePos(): \"\n         \"supports_random_access_ not reset\";\n  RIEGELI_ASSERT_EQ(supports_read_mode_, LazyBoolState::kUnknown)\n      << \"Failed precondition of CFileWriterBase::InitializePos(): \"\n         \"supports_read_mode_ not reset\";\n  RIEGELI_ASSERT_OK(random_access_status_)\n      << \"Failed precondition of CFileWriterBase::InitializePos(): \"\n         \"random_access_status_ not reset\";\n  RIEGELI_ASSERT_OK(read_mode_status_)\n      << \"Failed precondition of CFileWriterBase::InitializePos(): \"\n         \"read_mode_status_ not reset\";\n#ifdef _WIN32\n  RIEGELI_ASSERT_EQ(original_mode_, std::nullopt)\n      << \"Failed precondition of CFileWriterBase::InitializePos(): \"\n         \"original_mode_ not reset\";\n#endif\n  if (ABSL_PREDICT_FALSE(ferror(dest))) {\n    FailOperation(\"FILE\");\n    return;\n  }\n#ifdef _WIN32\n  int text_mode = file_internal::GetTextAsFlags(options.mode());\n#endif  // _WIN32\n  if (mode_was_passed_to_fopen) {\n    if (!file_internal::GetRead(options.mode())) {\n      supports_read_mode_ = LazyBoolState::kFalse;\n      read_mode_status_ = Global(\n          [] { return absl::UnimplementedError(\"Mode does not include '+'\"); });\n    }\n  }\n#ifdef _WIN32\n  else if (text_mode != 0) {\n    const int fd = _fileno(dest);\n    if (ABSL_PREDICT_FALSE(fd < 0)) {\n      FailOperation(\"_fileno()\");\n      return;\n    }\n    const int original_mode = _setmode(fd, text_mode);\n    if (ABSL_PREDICT_FALSE(original_mode < 0)) {\n      FailOperation(\"_setmode()\");\n      return;\n    }\n    original_mode_ = original_mode;\n  }\n  if (options.assumed_pos() == std::nullopt) {\n    if (text_mode == 0) {\n      const int fd = _fileno(dest);\n      if (ABSL_PREDICT_FALSE(fd < 0)) {\n        FailOperation(\"_fileno()\");\n        return;\n      }\n      // There is no `_getmode()`, but `_setmode()` returns the previous mode.\n      text_mode = _setmode(fd, _O_BINARY);\n      if (ABSL_PREDICT_FALSE(text_mode < 0)) {\n        FailOperation(\"_setmode()\");\n        return;\n      }\n      if (ABSL_PREDICT_FALSE(_setmode(fd, text_mode) < 0)) {\n        FailOperation(\"_setmode()\");\n        return;\n      }\n    }\n    if (text_mode != _O_BINARY) options.set_assumed_pos(0);\n  }\n#endif  // _WIN32\n  if (options.assumed_pos() != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(\n            *options.assumed_pos() >\n            Position{std::numeric_limits<cfile_internal::Offset>::max()})) {\n      FailOverflow();\n      return;\n    }\n    set_start_pos(*options.assumed_pos());\n    supports_random_access_ = LazyBoolState::kFalse;\n    supports_read_mode_ = LazyBoolState::kFalse;\n    random_access_status_ = Global([] {\n      return absl::UnimplementedError(\n          \"CFileWriterBase::Options::assumed_pos() excludes random access\");\n    });\n    read_mode_status_.Update(random_access_status_);\n  } else {\n    if (file_internal::GetAppend(options.mode())) {\n      if (cfile_internal::FSeek(dest, 0, SEEK_END) != 0) {\n        // Random access is not supported. Assume 0 as the initial position.\n        supports_random_access_ = LazyBoolState::kFalse;\n        supports_read_mode_ = LazyBoolState::kFalse;\n        random_access_status_ =\n            FailedOperationStatus(cfile_internal::kFSeekFunctionName);\n        read_mode_status_.Update(random_access_status_);\n        clearerr(dest);\n        return;\n      }\n    }\n    const cfile_internal::Offset file_pos = cfile_internal::FTell(dest);\n    if (file_pos < 0) {\n      // Random access is not supported. Assume 0 as the initial position.\n      supports_random_access_ = LazyBoolState::kFalse;\n      supports_read_mode_ = LazyBoolState::kFalse;\n      random_access_status_ =\n          FailedOperationStatus(cfile_internal::kFTellFunctionName);\n      read_mode_status_.Update(random_access_status_);\n      clearerr(dest);\n      return;\n    }\n    set_start_pos(IntCast<Position>(file_pos));\n    if (file_internal::GetAppend(options.mode())) {\n      // `cfile_internal::FSeek(SEEK_END)` succeeded.\n      supports_random_access_ = LazyBoolState::kFalse;\n      if (mode_was_passed_to_fopen &&\n          supports_read_mode_ == LazyBoolState::kUnknown) {\n        supports_read_mode_ = LazyBoolState::kTrue;\n      }\n      random_access_status_ = Global([] {\n        return absl::UnimplementedError(\"Append mode excludes random access\");\n      });\n    } else {\n      // `cfile_internal::FTell()` succeeded, and\n      // `cfile_internal::FSeek(SEEK_END)` will be checked later.\n      // `supports_random_access_` and `supports_read_mode_` are left as\n      // `LazyBoolState::kUnknown`.\n    }\n  }\n  BeginRun();\n}\n\nvoid CFileWriterBase::Done() {\n  BufferedWriter::Done();\n#ifdef _WIN32\n  if (original_mode_ != std::nullopt) {\n    FILE* const dest = DestFile();\n    const int fd = _fileno(dest);\n    if (ABSL_PREDICT_FALSE(fd < 0)) {\n      FailOperation(\"_fileno()\");\n    } else if (ABSL_PREDICT_FALSE(_setmode(fd, *original_mode_) < 0)) {\n      FailOperation(\"_setmode()\");\n    }\n  }\n#endif  // _WIN32\n  random_access_status_ = absl::OkStatus();\n  read_mode_status_.Update(absl::OkStatus());\n  associated_reader_.Reset();\n}\n\ninline absl::Status CFileWriterBase::FailedOperationStatus(\n    absl::string_view operation) {\n  const int error_number = errno;\n  RIEGELI_ASSERT_NE(error_number, 0)\n      << \"Failed precondition of CFileWriterBase::FailedOperationStatus(): \"\n         \"zero errno\";\n  return absl::ErrnoToStatus(error_number, absl::StrCat(operation, \" failed\"));\n}\n\nbool CFileWriterBase::FailOperation(absl::string_view operation) {\n  return Fail(FailedOperationStatus(operation));\n}\n\nabsl::Status CFileWriterBase::AnnotateStatusImpl(absl::Status status) {\n  return BufferedWriter::AnnotateStatusImpl(\n      Annotate(status, absl::StrCat(\"writing \", filename())));\n}\n\ninline absl::Status CFileWriterBase::SizeStatus() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of CFileWriterBase::SizeStatus(): \";\n  FILE* const dest = DestFile();\n  if (cfile_internal::FSeek(dest, 0, SEEK_END) != 0) {\n    // Not supported.\n    const absl::Status status =\n        FailedOperationStatus(cfile_internal::kFSeekFunctionName);\n    clearerr(dest);\n    return status;\n  }\n  const cfile_internal::Offset file_size = cfile_internal::FTell(dest);\n  if (ABSL_PREDICT_FALSE(file_size < 0)) {\n    FailOperation(cfile_internal::kFTellFunctionName);\n    return status();\n  }\n  if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(\n                             dest, IntCast<cfile_internal::Offset>(limit_pos()),\n                             SEEK_SET) != 0)) {\n    FailOperation(cfile_internal::kFSeekFunctionName);\n    return status();\n  }\n  // Supported.\n  return absl::OkStatus();\n}\n\nbool CFileWriterBase::SupportsRandomAccess() {\n  if (ABSL_PREDICT_TRUE(supports_random_access_ != LazyBoolState::kUnknown)) {\n    return supports_random_access_ == LazyBoolState::kTrue;\n  }\n  RIEGELI_ASSERT_NE(supports_read_mode_, LazyBoolState::kTrue)\n      << \"Failed invariant of CFileWriterBase: \"\n         \"supports_random_access_ is unknown but supports_read_mode_ is true\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Status status = SizeStatus();\n  if (!status.ok()) {\n    // Not supported.\n    supports_random_access_ = LazyBoolState::kFalse;\n    supports_read_mode_ = LazyBoolState::kFalse;\n    random_access_status_ = std::move(status);\n    read_mode_status_.Update(random_access_status_);\n    return false;\n  }\n  // Supported.\n  supports_random_access_ = LazyBoolState::kTrue;\n  return true;\n}\n\nbool CFileWriterBase::SupportsTruncate() {\n  if (!SupportsRandomAccess()) return false;\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  FILE* const dest = DestFile();\n#ifndef _WIN32\n  return fileno(dest) >= 0;\n#else   // _WIN32\n  return _fileno(dest) >= 0;\n#endif  // _WIN32\n}\n\nbool CFileWriterBase::SupportsReadMode() {\n  if (ABSL_PREDICT_TRUE(supports_read_mode_ != LazyBoolState::kUnknown)) {\n    return supports_read_mode_ == LazyBoolState::kTrue;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (supports_random_access_ == LazyBoolState::kUnknown) {\n    // It is unknown whether even size is supported.\n    absl::Status status = SizeStatus();\n    if (!status.ok()) {\n      // Not supported.\n      supports_random_access_ = LazyBoolState::kFalse;\n      supports_read_mode_ = LazyBoolState::kFalse;\n      random_access_status_ = std::move(status);\n      read_mode_status_ = random_access_status_;\n      return false;\n    }\n    // Size is supported.\n    supports_random_access_ = LazyBoolState::kTrue;\n  }\n\n  FILE* const dest = DestFile();\n  if (cfile_internal::FSeek(dest, 0, SEEK_END) != 0) {\n    // Read mode is not supported.\n    supports_read_mode_ = LazyBoolState::kFalse;\n    read_mode_status_ =\n        FailedOperationStatus(cfile_internal::kFSeekFunctionName);\n    clearerr(dest);\n    return false;\n  }\n  char buf[1];\n  fread(buf, 1, 1, dest);\n  if (ferror(dest)) {\n    // Not supported.\n    supports_read_mode_ = LazyBoolState::kFalse;\n    read_mode_status_ = FailedOperationStatus(\"fread()\");\n    clearerr(dest);\n  } else {\n    // Supported.\n    supports_read_mode_ = LazyBoolState::kTrue;\n  }\n  if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(\n                             dest, IntCast<cfile_internal::Offset>(start_pos()),\n                             SEEK_SET) != 0)) {\n    return FailOperation(cfile_internal::kFSeekFunctionName);\n  }\n  return supports_read_mode_ == LazyBoolState::kTrue;\n}\n\ninline bool CFileWriterBase::WriteMode() {\n  if (ABSL_PREDICT_TRUE(!read_mode_)) return true;\n  read_mode_ = false;\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  FILE* const dest = DestFile();\n  if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(\n                             dest, IntCast<cfile_internal::Offset>(start_pos()),\n                             SEEK_SET) != 0)) {\n    return FailOperation(cfile_internal::kFSeekFunctionName);\n  }\n  return true;\n}\n\nbool CFileWriterBase::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of BufferedWriter::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedWriter::WriteInternal()\";\n  if (ABSL_PREDICT_FALSE(!WriteMode())) return false;\n  FILE* const dest = DestFile();\n  if (ABSL_PREDICT_FALSE(\n          src.size() >\n          Position{std::numeric_limits<cfile_internal::Offset>::max()} -\n              start_pos())) {\n    return FailOverflow();\n  }\n  const size_t length_written = fwrite(src.data(), 1, src.size(), dest);\n  RIEGELI_ASSERT_LE(length_written, src.size())\n      << \"fwrite() wrote more than requested\";\n  move_start_pos(length_written);\n  if (ABSL_PREDICT_FALSE(length_written < src.size())) {\n    RIEGELI_ASSERT(ferror(dest))\n        << \"fwrite() succeeded but wrote less than requested\";\n    return FailOperation(\"fwrite()\");\n  }\n  return true;\n}\n\nbool CFileWriterBase::FlushBehindBuffer(absl::string_view src,\n                                        FlushType flush_type) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::FlushBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!WriteMode())) return false;\n  return BufferedWriter::FlushBehindBuffer(src, flush_type);\n}\n\nbool CFileWriterBase::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of BufferedWriter::SeekBehindBuffer(): \"\n         \"position unchanged, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!CFileWriterBase::SupportsRandomAccess())) {\n    if (ok()) Fail(random_access_status_);\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  read_mode_ = false;\n  FILE* const dest = DestFile();\n  if (new_pos > start_pos()) {\n    // Seeking forwards.\n    if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(dest, 0, SEEK_END) != 0)) {\n      return FailOperation(cfile_internal::kFSeekFunctionName);\n    }\n    const cfile_internal::Offset file_size = cfile_internal::FTell(dest);\n    if (ABSL_PREDICT_FALSE(file_size < 0)) {\n      return FailOperation(cfile_internal::kFTellFunctionName);\n    }\n    if (ABSL_PREDICT_FALSE(new_pos > IntCast<Position>(file_size))) {\n      // File ends.\n      set_start_pos(IntCast<Position>(file_size));\n      return false;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(\n          cfile_internal::FSeek(dest, IntCast<cfile_internal::Offset>(new_pos),\n                                SEEK_SET) != 0)) {\n    return FailOperation(cfile_internal::kFSeekFunctionName);\n  }\n  set_start_pos(new_pos);\n  return true;\n}\n\nstd::optional<Position> CFileWriterBase::SizeBehindBuffer() {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::SizeBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!CFileWriterBase::SupportsRandomAccess())) {\n    if (ok()) Fail(random_access_status_);\n    return std::nullopt;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  read_mode_ = false;\n  FILE* const dest = DestFile();\n  if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(dest, 0, SEEK_END) != 0)) {\n    FailOperation(cfile_internal::kFSeekFunctionName);\n    return std::nullopt;\n  }\n  const cfile_internal::Offset file_size = cfile_internal::FTell(dest);\n  if (ABSL_PREDICT_FALSE(file_size < 0)) {\n    FailOperation(cfile_internal::kFTellFunctionName);\n    return std::nullopt;\n  }\n  if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(\n                             dest, IntCast<cfile_internal::Offset>(start_pos()),\n                             SEEK_SET) != 0)) {\n    FailOperation(cfile_internal::kFSeekFunctionName);\n    return std::nullopt;\n  }\n  return IntCast<Position>(file_size);\n}\n\nbool CFileWriterBase::TruncateBehindBuffer(Position new_size) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::TruncateBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  read_mode_ = false;\n  FILE* const dest = DestFile();\n#ifndef _WIN32\n  const int fd = fileno(dest);\n  if (ABSL_PREDICT_FALSE(fd < 0)) return FailOperation(\"fileno()\");\n#else   // _WIN32\n  const int fd = _fileno(dest);\n  if (ABSL_PREDICT_FALSE(fd < 0)) return FailOperation(\"_fileno()\");\n#endif  // _WIN32\n  if (new_size >= start_pos()) {\n    // Seeking forwards.\n    if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(dest, 0, SEEK_END) != 0)) {\n      return FailOperation(cfile_internal::kFSeekFunctionName);\n    }\n    const cfile_internal::Offset file_size = cfile_internal::FTell(dest);\n    if (ABSL_PREDICT_FALSE(file_size < 0)) {\n      return FailOperation(cfile_internal::kFTellFunctionName);\n    }\n    if (ABSL_PREDICT_FALSE(new_size > IntCast<Position>(file_size))) {\n      // File ends.\n      set_start_pos(IntCast<Position>(file_size));\n      return false;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(fflush(dest) != 0)) {\n    return FailOperation(\"fflush()\");\n  }\n#ifndef _WIN32\nagain:\n  if (ABSL_PREDICT_FALSE(ftruncate(fd, IntCast<off_t>(new_size)) < 0)) {\n    if (errno == EINTR) goto again;\n    return FailOperation(\"ftruncate()\");\n  }\n#else   // _WIN32\n  if (ABSL_PREDICT_FALSE(_chsize_s(fd, IntCast<__int64>(new_size)) != 0)) {\n    return FailOperation(\"_chsize_s()\");\n  }\n#endif  // _WIN32\n  if (ABSL_PREDICT_FALSE(\n          cfile_internal::FSeek(dest, IntCast<cfile_internal::Offset>(new_size),\n                                SEEK_SET) != 0)) {\n    return FailOperation(cfile_internal::kFSeekFunctionName);\n  }\n  set_start_pos(new_size);\n  return true;\n}\n\nReader* CFileWriterBase::ReadModeBehindBuffer(Position initial_pos) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::ReadModeBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!CFileWriterBase::SupportsReadMode())) {\n    if (ok()) Fail(read_mode_status_);\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  FILE* const dest = DestFile();\n  // Synchronize the writing aspect and the reading aspect of the `FILE`.\n  if (ABSL_PREDICT_FALSE(cfile_internal::FSeek(dest, 0, SEEK_CUR) != 0)) {\n    FailOperation(cfile_internal::kFSeekFunctionName);\n    return nullptr;\n  }\n  CFileReader<UnownedCFile>* const reader = associated_reader_.ResetReader(\n      UnownedCFile(DestCFileHandle()),\n      CFileReaderBase::Options().set_buffer_options(buffer_options()));\n  reader->Seek(initial_pos);\n  read_mode_ = true;\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/cfile_writer.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_CFILE_WRITER_H_\n#define RIEGELI_BYTES_CFILE_WRITER_H_\n\n#include <stdint.h>\n#include <stdio.h>\n\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/string_ref.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/cfile_handle.h\"\n#include \"riegeli/bytes/file_mode_string.h\"\n#include \"riegeli/bytes/path_ref.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\ntemplate <typename Src>\nclass CFileReader;\nclass Reader;\n\n// Template parameter independent part of `CFileWriter`.\nclass CFileWriterBase : public BufferedWriter {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // If `CFileWriter` opens a `FILE` with a filename, `mode()` is the second\n    // argument of `fopen()` and specifies the open mode, typically \"w\" or \"a\"\n    // (on Windows: \"wb\" or \"ab\").\n    //\n    // `mode()` can also be changed with `set_existing(), `set_read()`,\n    // `set_append()`, `set_exclusive()`, `set_inheritable()`, and `set_text()`.\n    //\n    // Default: \"we\" (on Windows: \"wbN\").\n    Options& set_mode(StringInitializer mode) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      riegeli::Reset(mode_, std::move(mode));\n      return *this;\n    }\n    Options&& set_mode(StringInitializer mode) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_mode(std::move(mode)));\n    }\n    const std::string& mode() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return mode_;\n    }\n\n    // If `false`, the file will be created if it does not exist, or it will be\n    // truncated to empty if it exists. This implies `set_read(false)` and\n    // `set_append(false)` unless overwritten later.\n    //\n    // If `true`, the file must already exist, and its contents will not be\n    // truncated. Writing will start from the beginning, with random access\n    // supported. This implies `set_read(true)`.\n    //\n    // If `CFileWriter` writes to an already open `FILE`, `existing()` has no\n    // effect.\n    //\n    // `set_existing()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_existing(bool existing) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      file_internal::SetExisting(existing, mode_);\n      return *this;\n    }\n    Options&& set_existing(bool existing) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_existing(existing));\n    }\n    bool existing() const { return file_internal::GetExisting(mode_); }\n\n    // If `false`, the `FILE` will be open for writing, except that\n    // `set_read(false)` has no effect after `set_existing(true)`.\n    //\n    // If `true`, the `FILE` will be open for writing and reading (using\n    // `ReadMode()`).\n    //\n    // If `CFileWriter` writes to an already open `FILE`, `read()` has no\n    // effect.\n    //\n    // `set_read()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_read(bool read) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      file_internal::SetRead(read, mode_);\n      return *this;\n    }\n    Options&& set_read(bool read) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_read(read));\n    }\n    bool read() const { return file_internal::GetRead(mode_); }\n\n    // If `false`, the file will be truncated to empty if it exists.\n    //\n    // If `true`, the file will not be truncated if it exists, and writing will\n    // always happen at its end.\n    //\n    // Calling `set_append()` with any argument after `set_existing(true)`\n    // undoes the effect if the file does not exist: it will be created.\n    //\n    // If `CFileWriter` writes to an already open `FILE` and `assumed_pos()` is\n    // not set, `append()` should be `true` if the `FILE` was originally open\n    // in append mode. This allows to determine the effective initial position\n    // and lets `SupportsRandomAccess()` correctly return `false`.\n    //\n    // `set_append()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_append(bool append) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      file_internal::SetAppend(append, mode_);\n      return *this;\n    }\n    Options&& set_append(bool append) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_append(append));\n    }\n    bool append() const { return file_internal::GetAppend(mode_); }\n\n    // If `false`, the file will be created if it does not exist, or it will be\n    // opened if it exists (truncated to empty by default, or left unchanged if\n    // `set_existing(true)` or `set_append(true)` was used).\n    //\n    // If `true`, the file will be created if it does not exist, or opening will\n    // fail if it exists. This is not supported by all systems, but it is\n    // specified by C++17 and supported at least on Linux, Windows, FreeBSD,\n    // OpenBSD, NetBSD, and MacOS X.\n    //\n    // If `CFileWriter` writes to an already open `FILE`, `exclusive()` has no\n    // effect.\n    //\n    // `set_exclusive()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_exclusive(bool exclusive) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      file_internal::SetExclusive(exclusive, mode_);\n      return *this;\n    }\n    Options&& set_exclusive(bool exclusive) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_exclusive(exclusive));\n    }\n    bool exclusive() const { return file_internal::GetExclusive(mode_); }\n\n    // If `false`, `execve()` (`CreateProcess()` on Windows) will close the\n    // file. This is not supported by all systems, but it is supported at least\n    // on Linux, Windows, FreeBSD, OpenBSD, NetBSD, and it is planned for POSIX\n    // (https://www.austingroupbugs.net/view.php?id=411). For MacOS X this is\n    // emulated by `CFileWriter`.\n    //\n    // If `true`, the file will remain open across `execve()` (`CreateProcess()`\n    // on Windows).\n    //\n    // If `CFileWriter` writes to an already open file, `inheritable()` has no\n    // effect.\n    //\n    // `set_inheritable()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_inheritable(bool inheritable) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      file_internal::SetInheritableWriting(inheritable, mode_);\n      return *this;\n    }\n    Options&& set_inheritable(bool inheritable) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_inheritable(inheritable));\n    }\n    bool inheritable() const { return file_internal::GetInheritable(mode_); }\n\n    // If `false`, data will be written directly to the file. This is called the\n    // binary mode.\n    //\n    // If `true`, text mode translation will be applied on Windows:\n    // LF characters are translated to CR-LF.\n    //\n    // It is recommended to use `WriteLine()` or `TextWriter` instead, which\n    // expect a binary mode `Writer`.\n    //\n    // `set_text()` has an effect only on Windows. It is applicable whenever\n    // `CFileWriter` opens a `FILE` with a filename or reads from an already\n    // open `FILE`.\n    //\n    // `set_text()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_text(bool text) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      file_internal::SetTextWriting(text, mode_);\n      return *this;\n    }\n    Options&& set_text(bool text) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_text(text));\n    }\n    // No `text()` getter is provided. On Windows `mode()` can have unspecified\n    // text mode, resolved using `_get_fmode()`. Not on Windows the concept does\n    // not exist.\n\n    // If `std::nullopt`, the current position reported by `pos()` corresponds\n    // to the current `FILE` position if possible, otherwise 0 is assumed as the\n    // initial position. Random access is supported if the `FILE` supports\n    // random access.\n    //\n    // If not `std::nullopt`, this position is assumed initially, to be reported\n    // by `pos()`. It does not need to correspond to the current `FILE`\n    // position. Random access is not supported.\n    //\n    // Default: `std::nullopt`.\n    Options& set_assumed_pos(std::optional<Position> assumed_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      assumed_pos_ = assumed_pos;\n      return *this;\n    }\n    Options&& set_assumed_pos(std::optional<Position> assumed_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_pos(assumed_pos));\n    }\n    std::optional<Position> assumed_pos() const { return assumed_pos_; }\n\n   private:\n#ifndef _WIN32\n    std::string mode_ = \"we\";\n#else\n    std::string mode_ = \"wbN\";\n#endif\n    std::optional<Position> assumed_pos_;\n  };\n\n  // Returns the `CFileHandle` being written to. Unchanged by `Close()`.\n  virtual CFileHandle DestCFileHandle() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the `FILE*` being written to. If the `FILE*` is owned then changed\n  // to `nullptr` by `Close()`, otherwise unchanged.\n  virtual FILE* DestFile() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the filename of the `FILE*` being written to, or \"<none>\" for\n  // closed-constructed or moved-from `CFileWriter`. Unchanged by `Close()`.\n  //\n  // If the constructor from filename was used, this is the filename passed to\n  // the constructor, otherwise a filename is inferred from the `FILE*`. This\n  // can be a placeholder instead of a real filename if the `FILE*` does not\n  // refer to a named file or inferring the filename is not supported.\n  //\n  // If `Dest` does not support `filename()`, returns \"<unsupported>\".\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return DestCFileHandle().filename();\n  }\n\n  bool SupportsRandomAccess() override;\n  bool SupportsTruncate() override;\n  bool SupportsReadMode() override;\n\n protected:\n  explicit CFileWriterBase(Closed) noexcept : BufferedWriter(kClosed) {}\n\n  explicit CFileWriterBase(BufferOptions buffer_options);\n\n  CFileWriterBase(CFileWriterBase&& that) noexcept;\n  CFileWriterBase& operator=(CFileWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options);\n  void Initialize(FILE* dest, Options&& options);\n  void InitializePos(FILE* dest, Options&& options,\n                     bool mode_was_passed_to_fopen);\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation);\n\n  void Done() override;\n  absl::Status AnnotateStatusImpl(absl::Status status) override;\n  bool WriteInternal(absl::string_view src) override;\n  bool FlushBehindBuffer(absl::string_view src, FlushType flush_type) override;\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::optional<Position> SizeBehindBuffer() override;\n  bool TruncateBehindBuffer(Position new_size) override;\n  Reader* ReadModeBehindBuffer(Position initial_pos) override;\n\n private:\n  // Encodes a `bool` or a marker that the value is not resolved yet.\n  enum class LazyBoolState : uint8_t { kUnknown, kTrue, kFalse };\n\n  absl::Status FailedOperationStatus(absl::string_view operation);\n  // Lazily determined condition shared by `SupportsRandomAccess()` and\n  // `SupportsReadMode()`.\n  absl::Status SizeStatus();\n\n  bool WriteMode();\n\n  LazyBoolState supports_random_access_ = LazyBoolState::kUnknown;\n  // If `supports_read_mode_ == LazyBoolState::kUnknown`,\n  // then at least size is known to be supported\n  // when `supports_random_access_ != LazyBoolState::kUnknown`\n  // (no matter whether `LazyBoolState::kTrue` or LazyBoolState::kFalse`).\n  //\n  // Invariant:\n  //   if `supports_random_access_ == LazyBoolState::kUnknown` then\n  //       `supports_read_mode_ != LazyBoolState::kTrue`\n  LazyBoolState supports_read_mode_ = LazyBoolState::kUnknown;\n  absl::Status random_access_status_;\n  absl::Status read_mode_status_;\n#ifdef _WIN32\n  std::optional<int> original_mode_;\n#endif\n\n  AssociatedReader<CFileReader<UnownedCFile>> associated_reader_;\n  bool read_mode_ = false;\n\n  // Invariant:\n  //   `start_pos() <= std::numeric_limits<cfile_internal::Offset>::max()`\n};\n\n// A `Writer` which writes to a `FILE`.\n//\n// `CFileWriter` supports random access if\n// `Options::assumed_pos() == std::nullopt` and the `FILE` supports random\n// access (this is checked by calling `ftell()` and `fseek(SEEK_END)`).\n//\n// `CFileWriter` supports `ReadMode()` if it supports random access and the\n// `FILE` supports reading (this is checked by calling `fread()`).\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `FILE` being written to. `Dest` must support\n// `Dependency<CFileHandle, Dest>`, e.g. `OwnedCFile` (owned, default),\n// `UnownedCFile` (not owned), `AnyCFile` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `OwnedCFile` if\n// the first constructor argument is a filename or a `FILE*`, otherwise as\n// `TargetT` of the type of the first constructor argument.\n//\n// Until the `CFileWriter` is closed or no longer used, the `FILE` must not be\n// closed nor have its position changed, except that if random access is not\n// used, careful interleaving of multiple writers is possible: `Flush()` is\n// needed before switching to another writer, and `pos()` does not take other\n// writers into account.\ntemplate <typename Dest = OwnedCFile>\nclass CFileWriter : public CFileWriterBase {\n public:\n  // Creates a closed `CFileWriter`.\n  explicit CFileWriter(Closed) noexcept : CFileWriterBase(kClosed) {}\n\n  // Will write to the `FILE` provided by `dest`.\n  explicit CFileWriter(Initializer<Dest> dest, Options options = Options());\n\n  // Will write to `dest`.\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<std::is_constructible_v<DependentDest, FILE*>, int> = 0>\n  explicit CFileWriter(FILE* dest ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                       Options options = Options());\n\n  // Opens a file for writing.\n  //\n  // If opening the file fails, `CFileWriter` will be failed and closed.\n  //\n  // This constructor is present only if `Dest` supports `Open()`.\n  template <typename DependentDest = Dest,\n            std::enable_if_t<std::conjunction_v<\n                                 CFileSupportsOpen<DependentDest>,\n                                 std::is_default_constructible<DependentDest>>,\n                             int> = 0>\n  explicit CFileWriter(PathInitializer filename, Options options = Options());\n\n  CFileWriter(CFileWriter&& that) = default;\n  CFileWriter& operator=(CFileWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `CFileWriter`. This avoids\n  // constructing a temporary `CFileWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<std::is_constructible_v<DependentDest, FILE*>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(FILE* dest,\n                                          Options options = Options());\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<std::conjunction_v<CFileSupportsOpen<DependentDest>,\n                                          SupportsReset<DependentDest>>,\n                       int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(PathInitializer filename,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the `FILE` being written\n  // to. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  CFileHandle DestCFileHandle() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n  FILE* DestFile() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get().get();\n  }\n\n  // An optimized implementation in a derived class, avoiding a virtual call.\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.get().filename();\n  }\n\n protected:\n  void Done() override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  template <typename DependentDest = Dest,\n            std::enable_if_t<CFileSupportsOpen<DependentDest>::value, int> = 0>\n  void OpenImpl(PathInitializer filename, Options&& options);\n\n  // The object providing and possibly owning the `FILE` being written to.\n  Dependency<CFileHandle, Dest> dest_;\n};\n\nexplicit CFileWriter(Closed) -> CFileWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit CFileWriter(\n    Dest&& dest, CFileWriterBase::Options options = CFileWriterBase::Options())\n    -> CFileWriter<std::conditional_t<\n        std::disjunction_v<std::is_convertible<Dest&&, FILE*>,\n                           std::is_convertible<Dest&&, PathInitializer>>,\n        OwnedCFile, TargetT<Dest>>>;\n\n// Implementation details follow.\n\ninline CFileWriterBase::CFileWriterBase(BufferOptions buffer_options)\n    : BufferedWriter(buffer_options) {}\n\ninline CFileWriterBase::CFileWriterBase(CFileWriterBase&& that) noexcept\n    : BufferedWriter(static_cast<BufferedWriter&&>(that)),\n      supports_random_access_(\n          std::exchange(that.supports_random_access_, LazyBoolState::kUnknown)),\n      supports_read_mode_(\n          std::exchange(that.supports_read_mode_, LazyBoolState::kUnknown)),\n      random_access_status_(std::move(that.random_access_status_)),\n      read_mode_status_(std::move(that.read_mode_status_)),\n#ifdef _WIN32\n      original_mode_(that.original_mode_),\n#endif\n      associated_reader_(std::move(that.associated_reader_)),\n      read_mode_(that.read_mode_) {\n}\n\ninline CFileWriterBase& CFileWriterBase::operator=(\n    CFileWriterBase&& that) noexcept {\n  BufferedWriter::operator=(static_cast<BufferedWriter&&>(that));\n  supports_random_access_ =\n      std::exchange(that.supports_random_access_, LazyBoolState::kUnknown);\n  supports_read_mode_ =\n      std::exchange(that.supports_read_mode_, LazyBoolState::kUnknown);\n  random_access_status_ = std::move(that.random_access_status_);\n  read_mode_status_ = std::move(that.read_mode_status_);\n#ifdef _WIN32\n  original_mode_ = that.original_mode_;\n#endif\n  associated_reader_ = std::move(that.associated_reader_);\n  read_mode_ = that.read_mode_;\n  return *this;\n}\n\ninline void CFileWriterBase::Reset(Closed) {\n  BufferedWriter::Reset(kClosed);\n  supports_random_access_ = LazyBoolState::kUnknown;\n  supports_read_mode_ = LazyBoolState::kUnknown;\n  random_access_status_ = absl::OkStatus();\n  read_mode_status_ = absl::OkStatus();\n#ifdef _WIN32\n  original_mode_ = std::nullopt;\n#endif\n  associated_reader_.Reset();\n  read_mode_ = false;\n}\n\ninline void CFileWriterBase::Reset(BufferOptions buffer_options) {\n  BufferedWriter::Reset(buffer_options);\n  supports_random_access_ = LazyBoolState::kUnknown;\n  supports_read_mode_ = LazyBoolState::kUnknown;\n  random_access_status_ = absl::OkStatus();\n  read_mode_status_ = absl::OkStatus();\n#ifdef _WIN32\n  original_mode_ = std::nullopt;\n#endif\n  associated_reader_.Reset();\n  read_mode_ = false;\n}\n\ntemplate <typename Dest>\ninline CFileWriter<Dest>::CFileWriter(Initializer<Dest> dest, Options options)\n    : CFileWriterBase(options.buffer_options()), dest_(std::move(dest)) {\n  Initialize(dest_.get().get(), std::move(options));\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_constructible_v<DependentDest, FILE*>, int>>\ninline CFileWriter<Dest>::CFileWriter(FILE* dest ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                                      Options options)\n    : CFileWriter(riegeli::Maker(dest), std::move(options)) {}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<\n              std::conjunction_v<CFileSupportsOpen<DependentDest>,\n                                 std::is_default_constructible<DependentDest>>,\n              int>>\ninline CFileWriter<Dest>::CFileWriter(PathInitializer filename, Options options)\n    : CFileWriterBase(options.buffer_options()), dest_(riegeli::Maker()) {\n  OpenImpl(std::move(filename), std::move(options));\n}\n\ntemplate <typename Dest>\ninline void CFileWriter<Dest>::Reset(Closed) {\n  CFileWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void CFileWriter<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  CFileWriterBase::Reset(options.buffer_options());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get().get(), std::move(options));\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_constructible_v<DependentDest, FILE*>, int>>\ninline void CFileWriter<Dest>::Reset(FILE* dest, Options options) {\n  Reset(riegeli::Maker(dest), std::move(options));\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::conjunction_v<CFileSupportsOpen<DependentDest>,\n                                              SupportsReset<DependentDest>>,\n                           int>>\ninline void CFileWriter<Dest>::Reset(PathInitializer filename,\n                                     Options options) {\n  // In case `filename` is owned by `dest_` and gets invalidated.\n  std::string filename_copy = std::move(filename);\n  riegeli::Reset(dest_.manager());\n  CFileWriterBase::Reset(options.buffer_options());\n  OpenImpl(std::move(filename_copy), std::move(options));\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<CFileSupportsOpen<DependentDest>::value, int>>\nvoid CFileWriter<Dest>::OpenImpl(PathInitializer filename, Options&& options) {\n  absl::Status status =\n      dest_.manager().Open(std::move(filename), options.mode());\n  if (ABSL_PREDICT_FALSE(!status.ok())) {\n    CFileWriterBase::Reset(kClosed);\n    FailWithoutAnnotation(std::move(status));\n    return;\n  }\n  InitializePos(dest_.get().get(), std::move(options),\n                /*mode_was_passed_to_fopen=*/true);\n}\n\ntemplate <typename Dest>\nvoid CFileWriter<Dest>::Done() {\n  CFileWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (absl::Status status = dest_.get().Close();\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      Fail(std::move(status));\n    }\n  }\n}\n\ntemplate <typename Dest>\nbool CFileWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!CFileWriterBase::FlushImpl(flush_type))) return false;\n  switch (flush_type) {\n    case FlushType::kFromObject:\n      if (!dest_.IsOwning() || !dest_.get().IsOwning()) return true;\n      ABSL_FALLTHROUGH_INTENDED;\n    case FlushType::kFromProcess:\n    case FlushType::kFromMachine:\n      if (ABSL_PREDICT_FALSE(fflush(dest_.get().get()) != 0)) {\n        return FailOperation(\"fflush()\");\n      }\n      return true;\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown flush type: \" << static_cast<int>(flush_type);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_CFILE_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/chain_backward_writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/chain_backward_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\nvoid ChainBackwardWriterBase::Done() {\n  ChainBackwardWriterBase::FlushImpl(FlushType::kFromObject);\n  BackwardWriter::Done();\n}\n\ninline void ChainBackwardWriterBase::SyncBuffer(Chain& dest) {\n  set_start_pos(pos());\n  dest.RemovePrefix(available(), options_);\n  set_buffer();\n}\n\ninline void ChainBackwardWriterBase::MakeBuffer(Chain& dest, size_t min_length,\n                                                size_t recommended_length) {\n  const absl::Span<char> buffer = dest.PrependBuffer(\n      min_length, recommended_length, Chain::kAnyLength, options_);\n  set_buffer(buffer.data(), buffer.size());\n}\n\nvoid ChainBackwardWriterBase::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (write_size_hint == std::nullopt) {\n    options_.set_size_hint(std::nullopt);\n  } else {\n    options_.set_size_hint(\n        SaturatingIntCast<size_t>(SaturatingAdd(pos(), *write_size_hint)));\n  }\n}\n\nbool ChainBackwardWriterBase::PushSlow(size_t min_length,\n                                       size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of BackwardWriter::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_EQ(limit_pos(), dest.size())\n      << \"ChainBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(min_length > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  MakeBuffer(dest, min_length, recommended_length);\n  return true;\n}\n\nbool ChainBackwardWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_EQ(limit_pos(), dest.size())\n      << \"ChainBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  dest.Prepend(std::move(src), options_);\n  MakeBuffer(dest);\n  return true;\n}\n\nbool ChainBackwardWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_EQ(limit_pos(), dest.size())\n      << \"ChainBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  dest.Prepend(src, options_);\n  MakeBuffer(dest);\n  return true;\n}\n\nbool ChainBackwardWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_EQ(limit_pos(), dest.size())\n      << \"ChainBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  dest.Prepend(std::move(src), options_);\n  MakeBuffer(dest);\n  return true;\n}\n\nbool ChainBackwardWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_EQ(limit_pos(), dest.size())\n      << \"ChainBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  dest.Prepend(src, options_);\n  MakeBuffer(dest);\n  return true;\n}\n\nbool ChainBackwardWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_EQ(limit_pos(), dest.size())\n      << \"ChainBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  src.PrependTo(dest, options_);\n  MakeBuffer(dest);\n  return true;\n}\n\nbool ChainBackwardWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_EQ(limit_pos(), dest.size())\n      << \"ChainBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  dest.Prepend(std::move(src), options_);\n  MakeBuffer(dest);\n  return true;\n}\n\nbool ChainBackwardWriterBase::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_EQ(limit_pos(), dest.size())\n      << \"ChainBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  return true;\n}\n\nbool ChainBackwardWriterBase::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_EQ(limit_pos(), dest.size())\n      << \"ChainBackwardWriter destination changed unexpectedly\";\n  if (new_size >= start_pos()) {\n    if (ABSL_PREDICT_FALSE(new_size > pos())) return false;\n    set_cursor(start() - (new_size - start_pos()));\n    return true;\n  }\n  set_start_pos(new_size);\n  dest.RemovePrefix(dest.size() - IntCast<size_t>(new_size), options_);\n  set_buffer();\n  return true;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/chain_backward_writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_CHAIN_BACKWARD_WRITER_H_\n#define RIEGELI_BYTES_CHAIN_BACKWARD_WRITER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `ChainBackwardWriter`.\nclass ChainBackwardWriterBase : public BackwardWriter {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // If `false`, replaces existing contents of the destination, clearing it\n    // first.\n    //\n    // If `true`, prepends to existing contents of the destination.\n    //\n    // Default: `false`.\n    Options& set_prepend(bool prepend) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      prepend_ = prepend;\n      return *this;\n    }\n    Options&& set_prepend(bool prepend) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_prepend(prepend));\n    }\n    bool prepend() const { return prepend_; }\n\n    // Minimal size of a block of allocated data.\n    //\n    // This is used initially, while the destination is small.\n    //\n    // Default: `kDefaultMinBlockSize` (512).\n    Options& set_min_block_size(size_t min_block_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      min_block_size_ = UnsignedMin(min_block_size, uint32_t{1} << 31);\n      return *this;\n    }\n    Options&& set_min_block_size(size_t min_block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_min_block_size(min_block_size));\n    }\n    size_t min_block_size() const { return min_block_size_; }\n\n    // Maximal size of a block of allocated data.\n    //\n    // This is for performance tuning, not a guarantee: does not apply to\n    // objects allocated separately and then written to this\n    // `ChainBackwardWriter`.\n    //\n    // Default: `kDefaultMaxBlockSize` (64K).\n    Options& set_max_block_size(size_t max_block_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GT(max_block_size, 0u)\n          << \"Failed precondition of \"\n             \"ChainBackwardWriterBase::Options::set_max_block_size(): \"\n             \"zero block size\";\n      max_block_size_ = UnsignedMin(max_block_size, uint32_t{1} << 31);\n      return *this;\n    }\n    Options&& set_max_block_size(size_t max_block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_block_size(max_block_size));\n    }\n    size_t max_block_size() const { return max_block_size_; }\n\n    // A shortcut for `set_min_block_size(block_size)` with\n    // `set_max_block_size(block_size)`.\n    Options& set_block_size(size_t block_size) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_min_block_size(block_size).set_max_block_size(block_size);\n    }\n    Options&& set_block_size(size_t block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_block_size(block_size));\n    }\n\n   private:\n    bool prepend_ = false;\n    // Use `uint32_t` instead of `size_t` to reduce the object size.\n    uint32_t min_block_size_ = uint32_t{kDefaultMinBlockSize};\n    uint32_t max_block_size_ = uint32_t{kDefaultMaxBlockSize};\n  };\n\n  // Returns the `Chain` being written to. Unchanged by `Close()`.\n  virtual Chain* DestChain() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsTruncate() override { return true; }\n\n protected:\n  explicit ChainBackwardWriterBase(Closed) noexcept : BackwardWriter(kClosed) {}\n\n  explicit ChainBackwardWriterBase(const Options& options);\n\n  ChainBackwardWriterBase(ChainBackwardWriterBase&& that) noexcept;\n  ChainBackwardWriterBase& operator=(ChainBackwardWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(const Options& options);\n  void Initialize(Chain* dest, bool prepend);\n\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using BackwardWriter::WriteSlow;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool FlushImpl(FlushType flush_type) override;\n  bool TruncateImpl(Position new_size) override;\n\n private:\n  // Discards uninitialized space from the beginning of `dest`, so that it\n  // contains only actual data written.\n  void SyncBuffer(Chain& dest);\n\n  // Prepends uninitialized space to `dest`.\n  void MakeBuffer(Chain& dest, size_t min_length = 0,\n                  size_t recommended_length = 0);\n\n  Chain::Options options_;\n\n  // Invariants if `ok()`:\n  //   `limit() == nullptr || limit() == DestChain()->blocks().front().data()`\n  //   `limit_pos() == DestChain()->size()`\n};\n\n// A `BackwardWriter` which writes to a `Chain` backwards.\n// If `Options::prepend()` is `false` (the default), replaces existing contents\n// of the `Chain`, clearing it first. If `Options::prepend()` is `true`,\n// prepends to existing contents of the `Chain`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `Chain` being written to. `Dest` must support\n// `Dependency<Chain*, Dest>`, e.g. `Chain*` (not owned, default),\n// `Chain` (owned), `Any<Chain*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `Chain` if there\n// are no constructor arguments or the only argument is `Options`, otherwise as\n// `TargetT` of the type of the first constructor argument, except that CTAD\n// is deleted if the first constructor argument is a `Chain&` or `const Chain&`\n// (to avoid writing to an unintentionally separate copy of an existing object).\n//\n//\n// The `Chain` must not be accessed until the `ChainBackwardWriter` is closed or\n// no longer used.\ntemplate <typename Dest = Chain*>\nclass ChainBackwardWriter : public ChainBackwardWriterBase {\n public:\n  // Creates a closed `ChainBackwardWriter`.\n  explicit ChainBackwardWriter(Closed) noexcept\n      : ChainBackwardWriterBase(kClosed) {}\n\n  // Will write to the `Chain` provided by `dest`.\n  explicit ChainBackwardWriter(Initializer<Dest> dest,\n                               Options options = Options());\n\n  // Will write to an owned `Chain` which can be accessed by `dest()`.\n  // This constructor is present only if `Dest` is `Chain`.\n  template <typename DependentDest = Dest,\n            std::enable_if_t<std::is_same_v<DependentDest, Chain>, int> = 0>\n  explicit ChainBackwardWriter(Options options = Options());\n\n  ChainBackwardWriter(ChainBackwardWriter&& that) = default;\n  ChainBackwardWriter& operator=(ChainBackwardWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `ChainBackwardWriter`. This\n  // avoids constructing a temporary `ChainBackwardWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n  template <typename DependentDest = Dest,\n            std::enable_if_t<std::is_same_v<DependentDest, Chain>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Options options = Options());\n\n  // Returns the object providing and possibly owning the `Chain` being written\n  // to.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Chain* DestChain() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the `Chain` being written to, with\n  // uninitialized space prepended (possibly empty); `cursor()` points to the\n  // end of the uninitialized space, except that it can be `nullptr` if the\n  // uninitialized space is empty.\n  MovingDependency<Chain*, Dest, Mover> dest_;\n};\n\nexplicit ChainBackwardWriter(Closed) -> ChainBackwardWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit ChainBackwardWriter(Dest&& dest,\n                             ChainBackwardWriterBase::Options options =\n                                 ChainBackwardWriterBase::Options())\n    -> ChainBackwardWriter<std::conditional_t<\n        std::conjunction_v<\n            std::is_lvalue_reference<Dest>,\n            std::is_convertible<std::remove_reference_t<Dest>*, const Chain*>>,\n        DeleteCtad<Dest&&>, TargetT<Dest>>>;\nexplicit ChainBackwardWriter(ChainBackwardWriterBase::Options options =\n                                 ChainBackwardWriterBase::Options())\n    -> ChainBackwardWriter<Chain>;\n\n// Implementation details follow.\n\ninline ChainBackwardWriterBase::ChainBackwardWriterBase(const Options& options)\n    : options_(Chain::Options()\n                   .set_min_block_size(options.min_block_size())\n                   .set_max_block_size(options.max_block_size())) {}\n\ninline ChainBackwardWriterBase::ChainBackwardWriterBase(\n    ChainBackwardWriterBase&& that) noexcept\n    : BackwardWriter(static_cast<BackwardWriter&&>(that)),\n      options_(that.options_) {}\n\ninline ChainBackwardWriterBase& ChainBackwardWriterBase::operator=(\n    ChainBackwardWriterBase&& that) noexcept {\n  BackwardWriter::operator=(static_cast<BackwardWriter&&>(that));\n  options_ = that.options_;\n  return *this;\n}\n\ninline void ChainBackwardWriterBase::Reset(Closed) {\n  BackwardWriter::Reset(kClosed);\n  options_ = Chain::Options();\n}\n\ninline void ChainBackwardWriterBase::Reset(const Options& options) {\n  BackwardWriter::Reset();\n  options_ = Chain::Options()\n                 .set_min_block_size(options.min_block_size())\n                 .set_max_block_size(options.max_block_size());\n}\n\ninline void ChainBackwardWriterBase::Initialize(Chain* dest, bool prepend) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of ChainBackwardWriter: null Chain pointer\";\n  if (prepend) {\n    set_start_pos(dest->size());\n  } else {\n    dest->Clear();\n  }\n}\n\ntemplate <typename Dest>\nclass ChainBackwardWriter<Dest>::Mover {\n public:\n  static auto member() { return &ChainBackwardWriter::dest_; }\n\n  explicit Mover(ChainBackwardWriter& self, ChainBackwardWriter& that)\n      : uses_buffer_(self.start() != nullptr),\n        start_to_cursor_(self.start_to_cursor()) {\n    if (uses_buffer_) {\n      RIEGELI_ASSERT_EQ(that.dest_->blocks().front().data(), self.limit())\n          << \"ChainBackwardWriter destination changed unexpectedly\";\n      RIEGELI_ASSERT_EQ(that.dest_->size(), self.limit_pos())\n          << \"ChainBackwardWriter destination changed unexpectedly\";\n    }\n  }\n\n  void Done(ChainBackwardWriter& self) {\n    if (uses_buffer_) {\n      const size_t buffer_size =\n          self.dest_->size() - IntCast<size_t>(self.start_pos());\n      const absl::string_view first_block = self.dest_->blocks().front();\n      self.set_buffer(const_cast<char*>(first_block.data()), buffer_size,\n                      start_to_cursor_);\n    }\n  }\n\n private:\n  bool uses_buffer_;\n  size_t start_to_cursor_;\n};\n\ntemplate <typename Dest>\ninline ChainBackwardWriter<Dest>::ChainBackwardWriter(Initializer<Dest> dest,\n                                                      Options options)\n    : ChainBackwardWriterBase(options), dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.prepend());\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_same_v<DependentDest, Chain>, int>>\ninline ChainBackwardWriter<Dest>::ChainBackwardWriter(Options options)\n    : ChainBackwardWriter(riegeli::Maker(), std::move(options)) {}\n\ntemplate <typename Dest>\ninline void ChainBackwardWriter<Dest>::Reset(Closed) {\n  ChainBackwardWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void ChainBackwardWriter<Dest>::Reset(Initializer<Dest> dest,\n                                             Options options) {\n  ChainBackwardWriterBase::Reset(options);\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.prepend());\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_same_v<DependentDest, Chain>, int>>\ninline void ChainBackwardWriter<Dest>::Reset(Options options) {\n  Reset(riegeli::Maker(), std::move(options));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_CHAIN_BACKWARD_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/chain_reader.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/chain_reader.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/pullable_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid ChainReaderBase::Done() {\n  PullableReader::Done();\n  iter_ = Chain::BlockIterator();\n}\n\nbool ChainReaderBase::PullBehindScratch(size_t recommended_length) {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"enough data available, use Pull() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const Chain& src = *iter_.chain();\n  RIEGELI_ASSERT_LE(limit_pos(), src.size())\n      << \"ChainReader source changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(iter_ == src.blocks().cend())) return false;\n  while (++iter_ != src.blocks().cend()) {\n    if (ABSL_PREDICT_TRUE(!iter_->empty())) {\n      RIEGELI_ASSERT_LE(iter_->size(), src.size() - limit_pos())\n          << \"ChainReader source changed unexpectedly\";\n      set_buffer(iter_->data(), iter_->size());\n      move_limit_pos(available());\n      return true;\n    }\n  }\n  set_buffer();\n  return false;\n}\n\nbool ChainReaderBase::ReadBehindScratch(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"Chain size overflow\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const Chain& src = *iter_.chain();\n  RIEGELI_ASSERT_LE(limit_pos(), src.size())\n      << \"ChainReader source changed unexpectedly\";\n  if (length <= available()) {\n    dest.Append(ExternalRef(*iter_, absl::string_view(cursor(), length)));\n    move_cursor(length);\n    return true;\n  }\n  if (ABSL_PREDICT_FALSE(iter_ == src.blocks().cend())) return false;\n  dest.Append(ExternalRef(*iter_, absl::string_view(cursor(), available())));\n  length -= available();\n  while (++iter_ != src.blocks().cend()) {\n    RIEGELI_ASSERT_LE(iter_->size(), src.size() - limit_pos())\n        << \"ChainReader source changed unexpectedly\";\n    move_limit_pos(iter_->size());\n    if (length <= iter_->size()) {\n      set_buffer(iter_->data(), iter_->size(), length);\n      dest.Append(ExternalRef(*iter_, absl::string_view(start(), length)));\n      return true;\n    }\n    dest.Append(*iter_);\n    length -= iter_->size();\n  }\n  set_buffer();\n  return false;\n}\n\nbool ChainReaderBase::ReadBehindScratch(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"Cord size overflow\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const Chain& src = *iter_.chain();\n  RIEGELI_ASSERT_LE(limit_pos(), src.size())\n      << \"ChainReader source changed unexpectedly\";\n  if (length <= available()) {\n    ExternalRef(*iter_, absl::string_view(cursor(), length)).AppendTo(dest);\n    move_cursor(length);\n    return true;\n  }\n  if (ABSL_PREDICT_FALSE(iter_ == src.blocks().cend())) return false;\n  ExternalRef(*iter_, absl::string_view(cursor(), available())).AppendTo(dest);\n  length -= available();\n  while (++iter_ != src.blocks().cend()) {\n    RIEGELI_ASSERT_LE(iter_->size(), src.size() - limit_pos())\n        << \"ChainReader source changed unexpectedly\";\n    move_limit_pos(iter_->size());\n    if (length <= iter_->size()) {\n      set_buffer(iter_->data(), iter_->size(), length);\n      ExternalRef(*iter_, absl::string_view(start(), length)).AppendTo(dest);\n      return true;\n    }\n    ExternalRef(*iter_).AppendTo(dest);\n    length -= iter_->size();\n  }\n  set_buffer();\n  return false;\n}\n\nbool ChainReaderBase::CopyBehindScratch(Position length, Writer& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::CopyBehindScratch(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::CopyBehindScratch(Writer&): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const Chain& src = *iter_.chain();\n  RIEGELI_ASSERT_LE(limit_pos(), src.size())\n      << \"ChainReader source changed unexpectedly\";\n  const size_t length_to_copy =\n      UnsignedMin(length, src.size() - IntCast<size_t>(pos()));\n  if (length_to_copy == src.size()) {\n    RIEGELI_EVAL_ASSERT(Skip(length_to_copy));\n    if (ABSL_PREDICT_FALSE(!dest.Write(src))) return false;\n  } else if (length_to_copy <= kMaxBytesToCopy) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(length_to_copy))) return false;\n    RIEGELI_EVAL_ASSERT(Read(length_to_copy, dest.cursor()));\n    dest.move_cursor(length_to_copy);\n  } else {\n    Chain data;\n    RIEGELI_EVAL_ASSERT(Read(length_to_copy, data));\n    if (ABSL_PREDICT_FALSE(!dest.Write(std::move(data)))) return false;\n  }\n  return length_to_copy == length;\n}\n\nbool ChainReaderBase::CopyBehindScratch(size_t length, BackwardWriter& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of \"\n         \"PullableReader::CopyBehindScratch(BackwardWriter&): \"\n         \"enough data available, use Copy(BackwardWriter&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PullableReader::CopyBehindScratch(BackwardWriter&): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const Chain& src = *iter_.chain();\n  RIEGELI_ASSERT_LE(limit_pos(), src.size())\n      << \"ChainReader source changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(length > src.size() - pos())) {\n    RIEGELI_EVAL_ASSERT(Seek(src.size()));\n    return false;\n  }\n  if (length == src.size()) {\n    RIEGELI_EVAL_ASSERT(Skip(length));\n    return dest.Write(src);\n  }\n  if (length <= kMaxBytesToCopy) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(length))) return false;\n    dest.move_cursor(length);\n    if (ABSL_PREDICT_FALSE(!ReadBehindScratch(length, dest.cursor()))) {\n      dest.set_cursor(dest.cursor() + length);\n      return false;\n    }\n    return true;\n  }\n  Chain data;\n  RIEGELI_EVAL_ASSERT(ReadBehindScratch(length, data));\n  return dest.Write(std::move(data));\n}\n\nbool ChainReaderBase::SeekBehindScratch(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const Chain& src = *iter_.chain();\n  RIEGELI_ASSERT_LE(limit_pos(), src.size())\n      << \"ChainReader source changed unexpectedly\";\n  if (new_pos >= src.size()) {\n    // Source ends.\n    iter_ = src.blocks().cend();\n    set_limit_pos(src.size());\n    set_buffer();\n    return new_pos == src.size();\n  }\n  const Chain::BlockAndChar block_and_char =\n      src.BlockAndCharIndex(IntCast<size_t>(new_pos));\n  iter_ = block_and_char.block_iter;\n  set_buffer(iter_->data(), iter_->size(), block_and_char.char_index);\n  set_limit_pos(new_pos + available());\n  return true;\n}\n\nstd::optional<Position> ChainReaderBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  const Chain& src = *iter_.chain();\n  return src.size();\n}\n\nstd::unique_ptr<Reader> ChainReaderBase::NewReaderImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point.\n  const Chain& src = *iter_.chain();\n  std::unique_ptr<Reader> reader = std::make_unique<ChainReader<>>(&src);\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/chain_reader.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_CHAIN_READER_H_\n#define RIEGELI_BYTES_CHAIN_READER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pullable_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nclass BackwardWriter;\nclass Writer;\n\n// Template parameter independent part of `ChainReader`.\nclass ChainReaderBase : public PullableReader {\n public:\n  // Returns the `Chain` being read from. Unchanged by `Close()`.\n  virtual const Chain* SrcChain() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool ToleratesReadingAhead() override { return true; }\n  bool SupportsRandomAccess() override { return true; }\n  bool SupportsNewReader() override { return true; }\n\n protected:\n  using PullableReader::PullableReader;\n\n  ChainReaderBase(ChainReaderBase&& that) noexcept;\n  ChainReaderBase& operator=(ChainReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(const Chain* src);\n  Chain::BlockIterator iter() const { return iter_; }\n  void set_iter(Chain::BlockIterator iter) { iter_ = iter; }\n\n  void Done() override;\n  bool PullBehindScratch(size_t recommended_length) override;\n  using PullableReader::ReadBehindScratch;\n  bool ReadBehindScratch(size_t length, Chain& dest) override;\n  bool ReadBehindScratch(size_t length, absl::Cord& dest) override;\n  using PullableReader::CopyBehindScratch;\n  bool CopyBehindScratch(Position length, Writer& dest) override;\n  bool CopyBehindScratch(size_t length, BackwardWriter& dest) override;\n  bool SeekBehindScratch(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  // Invariant: `iter_.chain() == (is_open() ? SrcChain() : nullptr)`\n  Chain::BlockIterator iter_;\n\n  // Invariants if `is_open()` and scratch is not used:\n  //   `start() ==\n  //       (iter_ == SrcChain()->blocks().cend() ? nullptr : iter_->data())`\n  //   `start_to_limit() ==\n  //       (iter_ == SrcChain()->blocks().cend() ? 0 : iter_->size())`\n  //   `start_pos()` is the position of `iter_` in `*SrcChain()`\n};\n\n// A `Reader` which reads from a `Chain`.\n//\n// It supports random access and `NewReader()`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `Chain` being read from. `Src` must support\n// `Dependency<const Chain*, Src>`, e.g. `const Chain*` (not owned, default),\n// `Chain` (owned), `Any<const Chain*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The `Chain` must not be changed until the `ChainReader` is closed or no\n// longer used.\ntemplate <typename Src = const Chain*>\nclass ChainReader : public ChainReaderBase {\n public:\n  // Creates a closed `ChainReader`.\n  explicit ChainReader(Closed) noexcept : ChainReaderBase(kClosed) {}\n\n  // Will read from the `Chain` provided by `src`.\n  explicit ChainReader(Initializer<Src> src);\n\n  ChainReader(ChainReader&& that) = default;\n  ChainReader& operator=(ChainReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `ChainReader`. This avoids\n  // constructing a temporary `ChainReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src);\n\n  // Returns the object providing and possibly owning the `Chain` being read\n  // from. Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  const Chain* SrcChain() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the `Chain` being read from.\n  MovingDependency<const Chain*, Src, Mover> src_;\n};\n\nexplicit ChainReader(Closed) -> ChainReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit ChainReader(Src&& src) -> ChainReader<TargetT<Src>>;\n\n// Implementation details follow.\n\ninline ChainReaderBase::ChainReaderBase(ChainReaderBase&& that) noexcept\n    : PullableReader(static_cast<PullableReader&&>(that)),\n      iter_(std::exchange(that.iter_, Chain::BlockIterator())) {}\n\ninline ChainReaderBase& ChainReaderBase::operator=(\n    ChainReaderBase&& that) noexcept {\n  PullableReader::operator=(static_cast<PullableReader&&>(that));\n  iter_ = std::exchange(that.iter_, Chain::BlockIterator());\n  return *this;\n}\n\ninline void ChainReaderBase::Reset(Closed) {\n  PullableReader::Reset(kClosed);\n  iter_ = Chain::BlockIterator();\n}\n\ninline void ChainReaderBase::Reset() {\n  PullableReader::Reset();\n  // `iter_` will be set by `Initialize()`.\n}\n\ninline void ChainReaderBase::Initialize(const Chain* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of ChainReader: null Chain pointer\";\n  iter_ = src->blocks().cbegin();\n  if (iter_ != src->blocks().cend()) {\n    set_buffer(iter_->data(), iter_->size());\n    move_limit_pos(available());\n  }\n}\n\ntemplate <typename Src>\nclass ChainReader<Src>::Mover {\n public:\n  static auto member() { return &ChainReader::src_; }\n\n  explicit Mover(ChainReader& self)\n      : behind_scratch_(&self),\n        has_chain_(self.iter().chain() != nullptr),\n        block_index_(self.iter().block_index()),\n        uses_buffer_(self.start() != nullptr),\n        start_to_cursor_(self.start_to_cursor()) {\n    if (uses_buffer_) {\n      RIEGELI_ASSERT_EQ(self.iter()->data(), self.start())\n          << \"ChainReader source changed unexpectedly\";\n      RIEGELI_ASSERT_EQ(self.iter()->size(), self.start_to_limit())\n          << \"ChainReader source changed unexpectedly\";\n    }\n  }\n\n  void Done(ChainReader& self) {\n    if (has_chain_) {\n      self.set_iter(Chain::BlockIterator(self.src_.get(), block_index_));\n      if (uses_buffer_) {\n        self.set_buffer(self.iter()->data(), self.iter()->size(),\n                        start_to_cursor_);\n      }\n    }\n  }\n\n private:\n  BehindScratch behind_scratch_;\n  bool has_chain_;\n  size_t block_index_;\n  bool uses_buffer_;\n  size_t start_to_cursor_;\n};\n\ntemplate <typename Src>\ninline ChainReader<Src>::ChainReader(Initializer<Src> src)\n    : src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void ChainReader<Src>::Reset(Closed) {\n  ChainReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void ChainReader<Src>::Reset(Initializer<Src> src) {\n  ChainReaderBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_CHAIN_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/chain_writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/chain_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid ChainWriterBase::Done() {\n  ChainWriterBase::FlushImpl(FlushType::kFromObject);\n  Writer::Done();\n  tail_.reset();\n  associated_reader_.Reset();\n}\n\ninline void ChainWriterBase::SyncBuffer(Chain& dest) {\n  if (ABSL_PREDICT_FALSE(HasAppendedTail(dest))) {\n    ExtractTail(dest);\n    return;\n  }\n  ShrinkTail(start_to_cursor());\n  set_start_pos(pos());\n  dest.RemoveSuffix(available(), options_);\n  set_buffer();\n}\n\ninline void ChainWriterBase::MakeBuffer(Chain& dest, size_t min_length,\n                                        size_t recommended_length) {\n  const absl::Span<char> buffer = dest.AppendBuffer(\n      min_length, recommended_length, Chain::kAnyLength, options_);\n  set_buffer(buffer.data(), buffer.size());\n}\n\ninline void ChainWriterBase::MoveFromTail(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_NE(tail_, nullptr)\n      << \"Failed precondition of ChainWriterBase::MoveFromTail(): no tail\";\n  RIEGELI_ASSERT_LE(length, tail_->size())\n      << \"Failed precondition of ChainWriterBase::MoveFromTail(): \"\n         \"length longer than the tail\";\n  if (length == tail_->size()) {\n    dest.Append(std::move(*tail_), options_);\n    tail_->Clear();\n    return;\n  }\n  Chain::BlockIterator iter = tail_->blocks().cbegin();\n  size_t remaining = length;\n  while (remaining > iter->size()) {\n    dest.Append(*iter, options_);\n    remaining -= iter->size();\n    ++iter;\n  }\n  dest.Append(ExternalRef(*iter, absl::string_view(iter->data(), remaining)),\n              options_);\n  tail_->RemovePrefix(length, options_);\n}\n\ninline void ChainWriterBase::MoveToTail(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LE(length, dest.size())\n      << \"Failed precondition of ChainWriterBase::MoveToTail(): \"\n         \"length longer than the destination\";\n  if (tail_ == nullptr) tail_ = std::make_unique<Chain>();\n  if (length == dest.size()) {\n    tail_->Prepend(std::move(dest), options_);\n    dest.Clear();\n    return;\n  }\n  Chain::BlockIterator iter = dest.blocks().cend();\n  size_t remaining = length;\n  for (;;) {\n    --iter;\n    if (remaining <= iter->size()) break;\n    tail_->Prepend(*iter, options_);\n    remaining -= iter->size();\n  }\n  tail_->Prepend(ExternalRef(*iter, absl::string_view(\n                                        iter->data() + iter->size() - remaining,\n                                        remaining)),\n                 options_);\n  dest.RemoveSuffix(length, options_);\n}\n\ninline bool ChainWriterBase::HasAppendedTail(const Chain& dest) const {\n  return limit_pos() < dest.size();\n}\n\ninline void ChainWriterBase::ExtractTail(Chain& dest) {\n  RIEGELI_ASSERT(HasAppendedTail(dest))\n      << \"Failed precondition of ChainWriterBase::ExtractTail(): \"\n         \"the tail is not appended\";\n  RIEGELI_ASSERT_EQ(start(), nullptr)\n      << \"Failed invariant of ChainWriterBase: \"\n         \"both a buffer and the appended tail are present\";\n  MoveToTail(dest.size() - IntCast<size_t>(start_pos()), dest);\n}\n\ninline void ChainWriterBase::AppendTail(Chain& dest) {\n  RIEGELI_ASSERT(!HasAppendedTail(dest))\n      << \"Failed precondition of ChainWriterBase::AppendTail(): \"\n         \"the tail is appended\";\n  if (ABSL_PREDICT_FALSE(tail_ != nullptr)) {\n    dest.Append(std::move(*tail_), options_);\n    tail_->Clear();\n  }\n}\n\ninline void ChainWriterBase::ShrinkTail(size_t length) {\n  if (ABSL_PREDICT_FALSE(tail_ != nullptr)) {\n    tail_->RemovePrefix(UnsignedMin(length, tail_->size()), options_);\n  }\n}\n\nvoid ChainWriterBase::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (write_size_hint == std::nullopt) {\n    options_.set_size_hint(std::nullopt);\n  } else {\n    options_.set_size_hint(\n        SaturatingIntCast<size_t>(SaturatingAdd(pos(), *write_size_hint)));\n  }\n}\n\nbool ChainWriterBase::PushSlow(size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_LE(limit_pos(), dest.size())\n      << \"ChainWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(min_length > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  MakeBuffer(dest, min_length, recommended_length);\n  return true;\n}\n\nbool ChainWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_LE(limit_pos(), dest.size())\n      << \"ChainWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  ShrinkTail(src.size());\n  move_start_pos(src.size());\n  dest.Append(std::move(src), options_);\n  MakeBuffer(dest);\n  return true;\n}\n\nbool ChainWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_LE(limit_pos(), dest.size())\n      << \"ChainWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  ShrinkTail(src.size());\n  move_start_pos(src.size());\n  dest.Append(src, options_);\n  MakeBuffer(dest);\n  return true;\n}\n\nbool ChainWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_LE(limit_pos(), dest.size())\n      << \"ChainWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  ShrinkTail(src.size());\n  move_start_pos(src.size());\n  dest.Append(std::move(src), options_);\n  MakeBuffer(dest);\n  return true;\n}\n\nbool ChainWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_LE(limit_pos(), dest.size())\n      << \"ChainWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  ShrinkTail(src.size());\n  move_start_pos(src.size());\n  dest.Append(src, options_);\n  MakeBuffer(dest);\n  return true;\n}\n\nbool ChainWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_LE(limit_pos(), dest.size())\n      << \"ChainWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  ShrinkTail(src.size());\n  move_start_pos(src.size());\n  dest.Append(std::move(src), options_);\n  MakeBuffer(dest);\n  return true;\n}\n\nbool ChainWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_LE(limit_pos(), dest.size())\n      << \"ChainWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  ShrinkTail(IntCast<size_t>(src.size()));\n  move_start_pos(src.size());\n  src.AppendTo(dest, options_);\n  MakeBuffer(dest);\n  return true;\n}\n\nbool ChainWriterBase::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_LE(limit_pos(), dest.size())\n      << \"ChainWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(HasAppendedTail(dest))) {\n    RIEGELI_ASSERT_EQ(start(), nullptr)\n        << \"Failed invariant of ChainWriterBase: \"\n           \"both a buffer and the appended tail are present\";\n    RIEGELI_ASSERT(tail_ == nullptr || tail_->empty())\n        << \"Failed invariant of ChainWriterBase: \"\n           \"the tail is both appended and separated\";\n    return true;\n  }\n  SyncBuffer(dest);\n  AppendTail(dest);\n  return true;\n}\n\nbool ChainWriterBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of Writer::SeekSlow(): \"\n         \"position unchanged, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_LE(limit_pos(), dest.size())\n      << \"ChainWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(HasAppendedTail(dest))) {\n    RIEGELI_ASSERT_EQ(start(), nullptr)\n        << \"Failed invariant of ChainWriterBase: \"\n           \"both a buffer and the appended tail are present\";\n    RIEGELI_ASSERT(tail_ == nullptr || tail_->empty())\n        << \"Failed invariant of ChainWriterBase: \"\n           \"the tail is both appended and separated\";\n    if (ABSL_PREDICT_FALSE(new_pos > dest.size())) {\n      set_start_pos(dest.size());\n      return false;\n    }\n    MoveToTail(dest.size() - IntCast<size_t>(new_pos), dest);\n    set_start_pos(new_pos);\n    return true;\n  }\n  if (new_pos > pos()) {\n    if (ABSL_PREDICT_TRUE(tail_ == nullptr) || tail_->empty()) return false;\n    SyncBuffer(dest);\n    if (ABSL_PREDICT_FALSE(new_pos > dest.size() + tail_->size())) {\n      AppendTail(dest);\n      set_start_pos(dest.size());\n      return false;\n    }\n    MoveFromTail(IntCast<size_t>(new_pos) - dest.size(), dest);\n  } else {\n    SyncBuffer(dest);\n    MoveToTail(dest.size() - IntCast<size_t>(new_pos), dest);\n  }\n  set_start_pos(new_pos);\n  return true;\n}\n\nstd::optional<Position> ChainWriterBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_LE(limit_pos(), dest.size())\n      << \"ChainWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(HasAppendedTail(dest))) {\n    RIEGELI_ASSERT_EQ(start(), nullptr)\n        << \"Failed invariant of ChainWriterBase: \"\n           \"both a buffer and the appended tail are present\";\n    RIEGELI_ASSERT(tail_ == nullptr || tail_->empty())\n        << \"Failed invariant of ChainWriterBase: \"\n           \"the tail is both appended and separated\";\n    return dest.size();\n  }\n  if (ABSL_PREDICT_FALSE(tail_ != nullptr)) {\n    return UnsignedMax(pos(), start_pos() + tail_->size());\n  }\n  return pos();\n}\n\nbool ChainWriterBase::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain& dest = *DestChain();\n  RIEGELI_ASSERT_LE(limit_pos(), dest.size())\n      << \"ChainWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(HasAppendedTail(dest))) {\n    RIEGELI_ASSERT_EQ(start(), nullptr)\n        << \"Failed invariant of ChainWriterBase: \"\n           \"both a buffer and the appended tail are present\";\n    RIEGELI_ASSERT(tail_ == nullptr || tail_->empty())\n        << \"Failed invariant of ChainWriterBase: \"\n           \"the tail is both appended and separated\";\n    if (ABSL_PREDICT_FALSE(new_size > dest.size())) {\n      set_start_pos(dest.size());\n      return false;\n    }\n  } else if (new_size > pos()) {\n    if (ABSL_PREDICT_TRUE(tail_ == nullptr) || tail_->empty()) return false;\n    SyncBuffer(dest);\n    if (ABSL_PREDICT_FALSE(new_size > dest.size() + tail_->size())) {\n      move_start_pos(tail_->size());\n      AppendTail(dest);\n      return false;\n    }\n    set_start_pos(new_size);\n    tail_->RemoveSuffix(dest.size() + tail_->size() - IntCast<size_t>(new_size),\n                        options_);\n    AppendTail(dest);\n    return true;\n  } else {\n    if (ABSL_PREDICT_FALSE(tail_ != nullptr)) tail_->Clear();\n    if (new_size >= start_pos()) {\n      set_cursor(start() + (new_size - start_pos()));\n      return true;\n    }\n  }\n  RIEGELI_ASSERT(tail_ == nullptr || tail_->empty());\n  RIEGELI_ASSERT_LE(new_size, dest.size());\n  set_start_pos(new_size);\n  dest.RemoveSuffix(dest.size() - IntCast<size_t>(new_size), options_);\n  set_buffer();\n  return true;\n}\n\nReader* ChainWriterBase::ReadModeImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ChainWriterBase::FlushImpl(FlushType::kFromObject))) {\n    return nullptr;\n  }\n  Chain& dest = *DestChain();\n  ChainReader<>* const reader = associated_reader_.ResetReader(&dest);\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/chain_writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_CHAIN_WRITER_H_\n#define RIEGELI_BYTES_CHAIN_WRITER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <memory>\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\ntemplate <typename Src>\nclass ChainReader;\nclass Reader;\n\n// Template parameter independent part of `ChainWriter`.\nclass ChainWriterBase : public Writer {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // If `false`, replaces existing contents of the destination, clearing it\n    // first.\n    //\n    // If `true`, appends to existing contents of the destination.\n    //\n    // Default: `false`.\n    Options& set_append(bool append) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      append_ = append;\n      return *this;\n    }\n    Options&& set_append(bool append) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_append(append));\n    }\n    bool append() const { return append_; }\n\n    // Minimal size of a block of allocated data.\n    //\n    // This is used initially, while the destination is small.\n    //\n    // Default: `kDefaultMinBlockSize` (512).\n    Options& set_min_block_size(size_t min_block_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      min_block_size_ = UnsignedMin(min_block_size, uint32_t{1} << 31);\n      return *this;\n    }\n    Options&& set_min_block_size(size_t min_block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_min_block_size(min_block_size));\n    }\n    size_t min_block_size() const { return min_block_size_; }\n\n    // Maximal size of a block of allocated data.\n    //\n    // This is for performance tuning, not a guarantee: does not apply to\n    // objects allocated separately and then written to this `ChainWriter`.\n    //\n    // Default: `kDefaultMaxBlockSize` (64K).\n    Options& set_max_block_size(size_t max_block_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GT(max_block_size, 0u)\n          << \"Failed precondition of \"\n             \"ChainWriterBase::Options::set_max_block_size(): \"\n             \"zero block size\";\n      max_block_size_ = UnsignedMin(max_block_size, uint32_t{1} << 31);\n      return *this;\n    }\n    Options&& set_max_block_size(size_t max_block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_block_size(max_block_size));\n    }\n    size_t max_block_size() const { return max_block_size_; }\n\n    // A shortcut for `set_min_block_size(block_size)` with\n    // `set_max_block_size(block_size)`.\n    Options& set_block_size(size_t block_size) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_min_block_size(block_size).set_max_block_size(block_size);\n    }\n    Options&& set_block_size(size_t block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_block_size(block_size));\n    }\n\n   private:\n    bool append_ = false;\n    // Use `uint32_t` instead of `size_t` to reduce the object size.\n    uint32_t min_block_size_ = uint32_t{kDefaultMinBlockSize};\n    uint32_t max_block_size_ = uint32_t{kDefaultMaxBlockSize};\n  };\n\n  // Returns the `Chain` being written to. Unchanged by `Close()`.\n  virtual Chain* DestChain() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n  Chain& Digest() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    Flush();\n    return *DestChain();\n  }\n\n  bool SupportsRandomAccess() override { return true; }\n  bool SupportsReadMode() override { return true; }\n\n protected:\n  explicit ChainWriterBase(Closed) noexcept : Writer(kClosed) {}\n\n  explicit ChainWriterBase(const Options& options);\n\n  ChainWriterBase(ChainWriterBase&& that) noexcept;\n  ChainWriterBase& operator=(ChainWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(const Options& options);\n  void Initialize(Chain* dest, bool append);\n\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool FlushImpl(FlushType flush_type) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  bool TruncateImpl(Position new_size) override;\n  Reader* ReadModeImpl(Position initial_pos) override;\n\n private:\n  // Discards uninitialized space from the end of `dest`, so that it contains\n  // only actual data written. Ensures that data which follow the current\n  // position are separated in `*tail_`.\n  void SyncBuffer(Chain& dest);\n\n  // Appends uninitialized space to `dest`.\n  void MakeBuffer(Chain& dest, size_t min_length = 0,\n                  size_t recommended_length = 0);\n\n  // Moves `length` of data from the beginning of `*tail_` to the end of `dest`.\n  void MoveFromTail(size_t length, Chain& dest);\n\n  // Moves `length` of data from the end of `dest` to the beginning of `*tail_`.\n  void MoveToTail(size_t length, Chain& dest);\n\n  // Returns `true` if data which follow the current position are appended to\n  // `dest`.\n  bool HasAppendedTail(const Chain& dest) const;\n\n  // Moves data which follow the current position from being appended to `dest`\n  // to being separated in `*tail_`.\n  void ExtractTail(Chain& dest);\n\n  // Moves data which follow the current position from being separated in\n  // `*tail_` to being appended to `dest`.\n  void AppendTail(Chain& dest);\n\n  // Removes a prefix of `*tail_` of the given `length`, staturated at clearing\n  // the whole `*tail_`.\n  void ShrinkTail(size_t length);\n\n  Chain::Options options_;\n\n  // If `limit_pos() < DestChain()->size()`, then data after the current\n  // position are appended to `*DestChain()`, buffer pointers are `nullptr`,\n  // and `tail_ == nullptr || tail_->empty()`.\n  //\n  // Otherwise, if `limit_pos() == DestChain()->size() && tail_ != nullptr`,\n  // data after the current position are separated in `*tail_`, ignoring a\n  // prefix of `*tail_` with length `start_to_cursor()`, saturated at ignoring\n  // the whole `*tail_` (the ignored prefix is being overwritten with buffered\n  // data).\n  //\n  // Otherwise `limit_pos() == DestChain()->size() && tail_ == nullptr`, and\n  // there are no data after the current position.\n  //\n  // `tail_` is stored behind `std::unique_ptr` to reduce the object size in the\n  // common case when random access is not used.\n  std::unique_ptr<Chain> tail_;\n\n  AssociatedReader<ChainReader<const Chain*>> associated_reader_;\n\n  // Invariants if `ok()`:\n  //   `limit() == nullptr || limit() == DestChain()->blocks().back().data() +\n  //                                     DestChain()->blocks().back().size()`\n  //   `limit_pos() <= DestChain()->size()`\n  //   if `limit_pos() < DestChain()->size()` then\n  //       `start() == nullptr && (tail_ == nullptr || tail_->empty())`\n};\n\n// A `Writer` which writes to a `Chain`. If `Options::append()` is `false`\n// (the default), replaces existing contents of the `Chain`, clearing it first.\n// If `Options::append()` is `true`, appends to existing contents of the\n// `Chain`.\n//\n// It supports `Seek()` and `ReadMode()`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `Chain` being written to. `Dest` must support\n// `Dependency<Chain*, Dest>`, e.g. `Chain*` (not owned, default),\n// `Chain` (owned), `Any<Chain*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `Chain` if there\n// are no constructor arguments or the only argument is `Options`, otherwise as\n// `TargetT` of the type of the first constructor argument, except that CTAD\n// is deleted if the first constructor argument is a `Chain&` or `const Chain&`\n// (to avoid writing to an unintentionally separate copy of an existing object).\n//\n//\n// The `Chain` must not be accessed until the `ChainWriter` is closed or no\n// longer used, except that it is allowed to read the `Chain` immediately after\n// `Flush()`.\ntemplate <typename Dest = Chain*>\nclass ChainWriter : public ChainWriterBase {\n public:\n  // Creates a closed `ChainWriter`.\n  explicit ChainWriter(Closed) noexcept : ChainWriterBase(kClosed) {}\n\n  // Will write to the `Chain` provided by `dest`.\n  explicit ChainWriter(Initializer<Dest> dest, Options options = Options());\n\n  // Will write to an owned `Chain` which can be accessed by `dest()`.\n  // This constructor is present only if `Dest` is `Chain`.\n  template <typename DependentDest = Dest,\n            std::enable_if_t<std::is_same_v<DependentDest, Chain>, int> = 0>\n  explicit ChainWriter(Options options = Options());\n\n  ChainWriter(ChainWriter&& that) = default;\n  ChainWriter& operator=(ChainWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `ChainWriter`. This avoids\n  // constructing a temporary `ChainWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n  template <typename DependentDest = Dest,\n            std::enable_if_t<std::is_same_v<DependentDest, Chain>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Options options = Options());\n\n  // Returns the object providing and possibly owning the `Chain` being written\n  // to. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Chain* DestChain() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the `Chain` being written to, with\n  // uninitialized space appended (possibly empty); `cursor()` points to the\n  // uninitialized space, except that it can be `nullptr` if the uninitialized\n  // space is empty.\n  MovingDependency<Chain*, Dest, Mover> dest_;\n};\n\nexplicit ChainWriter(Closed) -> ChainWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit ChainWriter(\n    Dest&& dest, ChainWriterBase::Options options = ChainWriterBase::Options())\n    -> ChainWriter<std::conditional_t<\n        std::conjunction_v<\n            std::is_lvalue_reference<Dest>,\n            std::is_convertible<std::remove_reference_t<Dest>*, const Chain*>>,\n        DeleteCtad<Dest&&>, TargetT<Dest>>>;\nexplicit ChainWriter(ChainWriterBase::Options options =\n                         ChainWriterBase::Options()) -> ChainWriter<Chain>;\n\n// Implementation details follow.\n\ninline ChainWriterBase::ChainWriterBase(const Options& options)\n    : options_(Chain::Options()\n                   .set_min_block_size(options.min_block_size())\n                   .set_max_block_size(options.max_block_size())) {}\n\ninline ChainWriterBase::ChainWriterBase(ChainWriterBase&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)),\n      options_(that.options_),\n      tail_(std::move(that.tail_)),\n      associated_reader_(std::move(that.associated_reader_)) {}\n\ninline ChainWriterBase& ChainWriterBase::operator=(\n    ChainWriterBase&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  options_ = that.options_;\n  tail_ = std::move(that.tail_);\n  associated_reader_ = std::move(that.associated_reader_);\n  return *this;\n}\n\ninline void ChainWriterBase::Reset(Closed) {\n  Writer::Reset(kClosed);\n  options_ = Chain::Options();\n  tail_.reset();\n  associated_reader_.Reset();\n}\n\ninline void ChainWriterBase::Reset(const Options& options) {\n  Writer::Reset();\n  options_ = Chain::Options()\n                 .set_min_block_size(options.min_block_size())\n                 .set_max_block_size(options.max_block_size());\n  if (tail_ != nullptr) tail_->Clear();\n  associated_reader_.Reset();\n}\n\ninline void ChainWriterBase::Initialize(Chain* dest, bool append) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of ChainWriter: null Chain pointer\";\n  if (append) {\n    set_start_pos(dest->size());\n  } else {\n    dest->Clear();\n  }\n}\n\ntemplate <typename Dest>\nclass ChainWriter<Dest>::Mover {\n public:\n  static auto member() { return &ChainWriter::dest_; }\n\n  explicit Mover(ChainWriter& self, ChainWriter& that)\n      : uses_buffer_(self.start() != nullptr),\n        start_to_cursor_(self.start_to_cursor()) {\n    if (uses_buffer_) {\n      RIEGELI_ASSERT(that.dest_->blocks().back().data() +\n                         that.dest_->blocks().back().size() ==\n                     self.limit())\n          << \"ChainWriter destination changed unexpectedly\";\n      RIEGELI_ASSERT_EQ(that.dest_->size(), self.limit_pos())\n          << \"ChainWriter destination changed unexpectedly\";\n    }\n  }\n\n  void Done(ChainWriter& self) {\n    if (uses_buffer_) {\n      const size_t buffer_size =\n          self.dest_->size() - IntCast<size_t>(self.start_pos());\n      const absl::string_view last_block = self.dest_->blocks().back();\n      self.set_buffer(const_cast<char*>(last_block.data() + last_block.size()) -\n                          buffer_size,\n                      buffer_size, start_to_cursor_);\n    }\n  }\n\n private:\n  bool uses_buffer_;\n  size_t start_to_cursor_;\n};\n\ntemplate <typename Dest>\ninline ChainWriter<Dest>::ChainWriter(Initializer<Dest> dest, Options options)\n    : ChainWriterBase(options), dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.append());\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_same_v<DependentDest, Chain>, int>>\ninline ChainWriter<Dest>::ChainWriter(Options options)\n    : ChainWriter(riegeli::Maker(), std::move(options)) {}\n\ntemplate <typename Dest>\ninline void ChainWriter<Dest>::Reset(Closed) {\n  ChainWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void ChainWriter<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  ChainWriterBase::Reset(options);\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.append());\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_same_v<DependentDest, Chain>, int>>\ninline void ChainWriter<Dest>::Reset(Options options) {\n  Reset(riegeli::Maker(), std::move(options));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_CHAIN_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/compact_string_writer.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_COMPACT_STRING_WRITER_H_\n#define RIEGELI_BYTES_COMPACT_STRING_WRITER_H_\n\n#include <stddef.h>\n\n#include <type_traits>\n\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compact_string.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/bytes/resizable_writer.h\"\n\nnamespace riegeli {\n\nnamespace compact_string_writer_internal {\n\n// `ResizableTraits` for `CompactString`.\nstruct CompactStringResizableTraits {\n  using Resizable = CompactString;\n  static char* Data(Resizable& dest) { return dest.data(); }\n  static size_t Size(const Resizable& dest) { return dest.size(); }\n  static constexpr bool kIsStable = false;\n  static bool Resize(Resizable& dest, size_t new_size, size_t used_size) {\n    RIEGELI_ASSERT_LE(used_size, dest.size())\n        << \"Failed precondition of ResizableTraits::Resize(): \"\n           \"used size exceeds old size\";\n    RIEGELI_ASSERT_LE(used_size, new_size)\n        << \"Failed precondition of ResizableTraits::Resize(): \"\n           \"used size exceeds new size\";\n    dest.resize(new_size, used_size);\n    return true;\n  }\n  static bool Grow(Resizable& dest, size_t new_size, size_t used_size) {\n    RIEGELI_ASSERT_GT(new_size, dest.size())\n        << \"Failed precondition of ResizableTraits::Grow(): \"\n           \"no need to grow\";\n    RIEGELI_ASSERT_LE(used_size, dest.size())\n        << \"Failed precondition of ResizableTraits::Grow(): \"\n           \"used size exceeds old size\";\n    RIEGELI_ASSERT_LE(used_size, new_size)\n        << \"Failed precondition of ResizableTraits::Grow(): \"\n           \"used size exceeds new size\";\n    dest.resize(new_size, used_size);\n    dest.set_size(dest.capacity());\n    return true;\n  }\n  static bool GrowUnderCapacity(Resizable& dest, size_t new_size) {\n    RIEGELI_ASSERT_GT(new_size, dest.size())\n        << \"Failed precondition of ResizableTraits::GrowUnderCapacity(): \"\n           \"no need to grow\";\n    if (new_size > dest.capacity()) return false;\n    dest.set_size(dest.capacity());\n    return true;\n  }\n};\n\n}  // namespace compact_string_writer_internal\n\n// Template parameter independent part of `CompactStringWriter`.\nusing CompactStringWriterBase = ResizableWriterBase;\n\n// A `Writer` which writes to a `CompactString`. If `Options::append()` is\n// `false` (the default), replaces existing contents of the `CompactString`,\n// clearing it first. If `Options::append()` is `true`, appends to existing\n// contents of the `CompactString`.\n//\n// It supports `Seek()` and `ReadMode()`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `CompactString` being written to. `Dest` must support\n// `Dependency<CompactString*, Dest>`, e.g.\n// `CompactString*` (not owned, default), `CompactString` (owned),\n// `Any<CompactString*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `CompactString`\n// if there are no constructor arguments or the only argument is `Options`,\n// otherwise as `TargetT` of the type of the first constructor argument, except\n// that CTAD is deleted if the first constructor argument is a `CompactString&`\n// or `const CompactString&` (to avoid writing to an unintentionally separate\n// copy of an existing object).\n//\n// The `CompactString` must not be accessed until the `CompactStringWriter` is\n// closed or no longer used, except that it is allowed to read the\n// `CompactString` immediately after `Flush()`.\n//\n// `CompactStringWriter` is more efficient than `StringWriter`\n// because the destination can be resized with uninitialized space.\n// `VectorWriter` with `UninitializedVector<char>` or\n// `UninitializedInlinedVector<char, inlined_size>` can also be used\n// for this purpose.\ntemplate <typename Dest = CompactString*>\nclass CompactStringWriter\n    : public ResizableWriter<\n          compact_string_writer_internal::CompactStringResizableTraits, Dest> {\n public:\n  using CompactStringWriter::ResizableWriter::ResizableWriter;\n\n  CompactStringWriter(CompactStringWriter&& that) = default;\n  CompactStringWriter& operator=(CompactStringWriter&& that) = default;\n};\n\nexplicit CompactStringWriter(Closed) -> CompactStringWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit CompactStringWriter(Dest&& dest,\n                             CompactStringWriterBase::Options options =\n                                 CompactStringWriterBase::Options())\n    -> CompactStringWriter<std::conditional_t<\n        std::conjunction_v<std::is_lvalue_reference<Dest>,\n                           std::is_convertible<std::remove_reference_t<Dest>*,\n                                               const CompactString*>>,\n        DeleteCtad<Dest&&>, TargetT<Dest>>>;\nexplicit CompactStringWriter(CompactStringWriterBase::Options options =\n                                 CompactStringWriterBase::Options())\n    -> CompactStringWriter<CompactString>;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_COMPACT_STRING_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/copy_all.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/copy_all.h\"\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli::copy_all_internal {\n\nnamespace {\n\nABSL_ATTRIBUTE_COLD absl::Status MaxLengthExceeded(Reader& src,\n                                                   Position max_length) {\n  return src.AnnotateStatus(absl::ResourceExhaustedError(\n      absl::StrCat(\"Maximum length exceeded: \", max_length)));\n}\n\nabsl::Status CopyAllImpl(Reader& src, Writer& dest, Position max_length,\n                         bool set_write_size_hint) {\n  if (src.SupportsSize()) {\n    const std::optional<Position> size = src.Size();\n    if (ABSL_PREDICT_FALSE(size == std::nullopt)) return src.status();\n    const Position remaining = SaturatingSub(*size, src.pos());\n    if (ABSL_PREDICT_FALSE(remaining > max_length)) {\n      if (set_write_size_hint) dest.SetWriteSizeHint(max_length);\n      if (ABSL_PREDICT_FALSE(!src.Copy(max_length, dest))) {\n        if (ABSL_PREDICT_FALSE(!dest.ok())) return dest.status();\n        if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n        return absl::OkStatus();\n      }\n      return MaxLengthExceeded(src, max_length);\n    }\n    if (set_write_size_hint) dest.SetWriteSizeHint(remaining);\n    if (ABSL_PREDICT_FALSE(!src.Copy(remaining, dest))) {\n      if (ABSL_PREDICT_FALSE(!dest.ok())) return dest.status();\n      if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n    }\n  } else {\n    Position remaining_max_length = max_length;\n    do {\n      if (ABSL_PREDICT_FALSE(src.available() > remaining_max_length)) {\n        if (ABSL_PREDICT_FALSE(!src.Copy(remaining_max_length, dest))) {\n          if (ABSL_PREDICT_FALSE(!dest.ok())) return dest.status();\n        }\n        return MaxLengthExceeded(src, max_length);\n      }\n      remaining_max_length -= src.available();\n      if (ABSL_PREDICT_FALSE(!src.Copy(src.available(), dest))) {\n        if (ABSL_PREDICT_FALSE(!dest.ok())) return dest.status();\n      }\n    } while (src.Pull());\n    if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n  }\n  return absl::OkStatus();\n}\n\n}  // namespace\n\nabsl::Status CopyAllImpl(Reader& src, Writer& dest, Position max_length,\n                         Position* length_read, bool set_write_size_hint) {\n  if (length_read == nullptr) {\n    return CopyAllImpl(src, dest, max_length, set_write_size_hint);\n  }\n  const Position pos_before = src.pos();\n  const absl::Status status =\n      CopyAllImpl(src, dest, max_length, set_write_size_hint);\n  RIEGELI_ASSERT_GE(src.pos(), pos_before)\n      << \"CopyAllImpl(Writer&) decreased src.pos()\";\n  RIEGELI_ASSERT_LE(src.pos() - pos_before, max_length)\n      << \"CopyAllImpl(Writer&) read more than requested\";\n  *length_read = src.pos() - pos_before;\n  return status;\n}\n\nabsl::Status CopyAllImpl(Reader& src, BackwardWriter& dest, size_t max_length,\n                         bool set_write_size_hint) {\n  if (src.SupportsSize()) {\n    const std::optional<Position> size = src.Size();\n    if (ABSL_PREDICT_FALSE(size == std::nullopt)) return src.status();\n    const Position remaining = SaturatingSub(*size, src.pos());\n    if (ABSL_PREDICT_FALSE(remaining > max_length)) {\n      if (ABSL_PREDICT_FALSE(!src.Skip(max_length))) {\n        if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n      }\n      return MaxLengthExceeded(src, max_length);\n    }\n    if (set_write_size_hint) dest.SetWriteSizeHint(remaining);\n    if (ABSL_PREDICT_FALSE(!src.Copy(IntCast<size_t>(remaining), dest))) {\n      if (ABSL_PREDICT_FALSE(!dest.ok())) return dest.status();\n      if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n    }\n  } else {\n    size_t remaining_max_length = max_length;\n    Chain data;\n    do {\n      if (ABSL_PREDICT_FALSE(src.available() > remaining_max_length)) {\n        src.move_cursor(remaining_max_length);\n        return MaxLengthExceeded(src, max_length);\n      }\n      remaining_max_length -= src.available();\n      src.ReadAndAppend(src.available(), data);\n    } while (src.Pull());\n    if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n    if (ABSL_PREDICT_FALSE(!dest.Write(std::move(data)))) return dest.status();\n  }\n  return absl::OkStatus();\n}\n\n}  // namespace riegeli::copy_all_internal\n"
  },
  {
    "path": "riegeli/bytes/copy_all.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_COPY_ALL_H_\n#define RIEGELI_BYTES_COPY_ALL_H_\n\n#include <stddef.h>\n\n#include <limits>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// Combines creating a `Reader` and/or `Writer` / `BackwardWriter` (optionally),\n// copying all remaining data, and `VerifyEndAndClose()` and/or `Close()`\n// (if the `Reader` and/or `Writer` / `BackwardWriter` is owned).\n//\n// `CopyAll(Writer&)` writes as much as could be read if reading failed, reads\n// an unspecified length (between what could be written and `max_length`) if\n// writing failed, and reads and writes `max_length` if `max_length` was\n// exceeded.\n//\n// `CopyAll(BackwardWriter&)` writes nothing if reading failed, reads an\n// unspecified length (between what could be written and `max_length`) if\n// writing failed, and reads `max_length` and writes nothing if `max_length` was\n// exceeded.\n//\n// If `length_read != nullptr` then sets `*length_read` to the length read.\n// This is equal to the difference between `src.pos()` after and before the\n// call.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `Reader`. `Src` must support\n// `DependencyRef<Reader*, Src>`, e.g. `Reader&` (not owned),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `AnyRef<Reader*>` (maybe owned).\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `Writer` / `BackwardWriter`. `Dest` must support\n// `DependencyRef<Writer*, Dest>`, e.g. `Writer&` (not owned),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `AnyRef<Writer*>` (maybe owned). Analogously for `BackwardWriter`.\ntemplate <typename Src, typename Dest,\n          std::enable_if_t<\n              std::conjunction_v<TargetRefSupportsDependency<Reader*, Src>,\n                                 TargetRefSupportsDependency<Writer*, Dest>>,\n              int> = 0>\nabsl::Status CopyAll(Src&& src, Dest&& dest,\n                     Position max_length = std::numeric_limits<Position>::max(),\n                     Position* length_read = nullptr);\ntemplate <typename Src, typename Dest,\n          std::enable_if_t<\n              std::conjunction_v<TargetRefSupportsDependency<Reader*, Src>,\n                                 TargetRefSupportsDependency<Writer*, Dest>>,\n              int> = 0>\nabsl::Status CopyAll(Src&& src, Dest&& dest, Position* length_read);\ntemplate <\n    typename Src, typename Dest,\n    std::enable_if_t<\n        std::conjunction_v<TargetRefSupportsDependency<Reader*, Src>,\n                           TargetRefSupportsDependency<BackwardWriter*, Dest>>,\n        int> = 0>\nabsl::Status CopyAll(Src&& src, Dest&& dest,\n                     size_t max_length = std::numeric_limits<size_t>::max());\n\n// Implementation details follow.\n\nnamespace copy_all_internal {\n\nabsl::Status CopyAllImpl(Reader& src, Writer& dest, Position max_length,\n                         Position* length_read, bool set_write_size_hint);\nabsl::Status CopyAllImpl(Reader& src, BackwardWriter& dest, size_t max_length,\n                         bool set_write_size_hint);\n\n}  // namespace copy_all_internal\n\ntemplate <typename Src, typename Dest,\n          std::enable_if_t<\n              std::conjunction_v<TargetRefSupportsDependency<Reader*, Src>,\n                                 TargetRefSupportsDependency<Writer*, Dest>>,\n              int>>\ninline absl::Status CopyAll(Src&& src, Dest&& dest, Position max_length,\n                            Position* length_read) {\n  DependencyRef<Reader*, Src> src_dep(std::forward<Src>(src));\n  DependencyRef<Writer*, Dest> dest_dep(std::forward<Dest>(dest));\n  if (src_dep.IsOwning()) src_dep->SetReadAllHint(true);\n  absl::Status status = copy_all_internal::CopyAllImpl(\n      *src_dep, *dest_dep, max_length, length_read, dest_dep.IsOwning());\n  if (dest_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_dep->Close())) {\n      status.Update(dest_dep->status());\n    }\n  }\n  if (src_dep.IsOwning()) {\n    if (ABSL_PREDICT_TRUE(status.ok())) src_dep->VerifyEnd();\n    if (ABSL_PREDICT_FALSE(!src_dep->Close())) status.Update(src_dep->status());\n  }\n  return status;\n}\n\ntemplate <typename Src, typename Dest,\n          std::enable_if_t<\n              std::conjunction_v<TargetRefSupportsDependency<Reader*, Src>,\n                                 TargetRefSupportsDependency<Writer*, Dest>>,\n              int>>\ninline absl::Status CopyAll(Src&& src, Dest&& dest, Position* length_read) {\n  return CopyAll(std::forward<Src>(src), std::forward<Dest>(dest),\n                 std::numeric_limits<Position>::max(), length_read);\n}\n\ntemplate <\n    typename Src, typename Dest,\n    std::enable_if_t<\n        std::conjunction_v<TargetRefSupportsDependency<Reader*, Src>,\n                           TargetRefSupportsDependency<BackwardWriter*, Dest>>,\n        int>>\ninline absl::Status CopyAll(Src&& src, Dest&& dest, size_t max_length) {\n  DependencyRef<Reader*, Src> src_dep(std::forward<Src>(src));\n  DependencyRef<BackwardWriter*, Dest> dest_dep(std::forward<Dest>(dest));\n  if (src_dep.IsOwning()) src_dep->SetReadAllHint(true);\n  absl::Status status = copy_all_internal::CopyAllImpl(\n      *src_dep, *dest_dep, max_length, dest_dep.IsOwning());\n  if (dest_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_dep->Close())) {\n      status.Update(dest_dep->status());\n    }\n  }\n  if (src_dep.IsOwning()) {\n    if (ABSL_PREDICT_TRUE(status.ok())) src_dep->VerifyEnd();\n    if (ABSL_PREDICT_FALSE(!src_dep->Close())) status.Update(src_dep->status());\n  }\n  return status;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_COPY_ALL_H_\n"
  },
  {
    "path": "riegeli/bytes/cord_backward_writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/cord_backward_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/cord_buffer.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_utils.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\nvoid CordBackwardWriterBase::Done() {\n  CordBackwardWriterBase::FlushImpl(FlushType::kFromObject);\n  BackwardWriter::Done();\n  cord_buffer_ = absl::CordBuffer();\n  buffer_ = Buffer();\n}\n\ninline size_t CordBackwardWriterBase::MaxBytesToCopy() const {\n  if (size_hint_ != std::nullopt && pos() < *size_hint_) {\n    return UnsignedClamp(*size_hint_ - pos() - 1,\n                         cord_internal::kMaxBytesToCopyToEmptyCord,\n                         cord_internal::kMaxBytesToCopyToNonEmptyCord);\n  }\n  if (pos() == 0) return cord_internal::kMaxBytesToCopyToEmptyCord;\n  return cord_internal::kMaxBytesToCopyToNonEmptyCord;\n}\n\ninline void CordBackwardWriterBase::SyncBuffer(absl::Cord& dest) {\n  if (limit() == nullptr) return;\n  set_start_pos(pos());\n  const absl::string_view data(cursor(), start_to_cursor());\n  if (limit() == cord_buffer_.data()) {\n    const size_t prefix_to_remove =\n        PtrDistance(cord_buffer_.data(), data.data());\n    if (prefix_to_remove == 0) {\n      dest.Prepend(std::move(cord_buffer_));\n    } else if (Wasteful(cord_internal::kFlatOverhead + cord_buffer_.capacity(),\n                        data.size()) ||\n               data.size() <= MaxBytesToCopy()) {\n      cord_internal::PrependToBlockyCord(data, dest);\n    } else {\n      dest.Prepend(std::move(cord_buffer_));\n      dest.RemovePrefix(prefix_to_remove);\n    }\n  } else {\n    ExternalRef(std::move(buffer_), data).PrependTo(dest);\n  }\n  set_buffer();\n}\n\nvoid CordBackwardWriterBase::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (write_size_hint == std::nullopt) {\n    size_hint_ = std::nullopt;\n  } else {\n    size_hint_ = SaturatingAdd(pos(), *write_size_hint);\n  }\n}\n\nbool CordBackwardWriterBase::PushSlow(size_t min_length,\n                                      size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of BackwardWriter::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (pos() == 0) {\n    Position needed_length = UnsignedMax(min_length, recommended_length);\n    if (size_hint_ != std::nullopt) {\n      needed_length = UnsignedMax(needed_length, *size_hint_);\n    }\n    if (needed_length <= cord_buffer_.capacity()) {\n      // Use the initial capacity of `cord_buffer_`, even if it is smaller than\n      // `min_block_size_`, because this avoids allocation.\n      cord_buffer_.SetLength(cord_buffer_.capacity());\n      set_buffer(cord_buffer_.data(), cord_buffer_.length());\n      return true;\n    }\n  }\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_EQ(start_pos(), dest.size())\n      << \"CordBackwardWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(min_length > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (start_to_cursor() >= min_block_size_) SyncBuffer(dest);\n  const size_t cursor_index = start_to_cursor();\n  const size_t buffer_length = ApplyBufferConstraints(\n      ApplySizeHint(UnsignedMax(start_pos(), min_block_size_), size_hint_,\n                    start_pos()),\n      cursor_index + min_length,\n      SaturatingAdd(cursor_index, recommended_length), max_block_size_);\n  if (buffer_length <= cord_internal::kCordBufferMaxSize) {\n    RIEGELI_ASSERT(cord_buffer_.capacity() < buffer_length ||\n                   limit() != cord_buffer_.data())\n        << \"Failed invariant of CordBackwardWriter: \"\n           \"cord_buffer_ has enough capacity but was used only partially\";\n    const size_t predicted_cord_buffer_size =\n        cord_internal::CordBufferSizeForCapacity(buffer_length);\n    if (predicted_cord_buffer_size >= cursor_index + min_length) {\n      // Reuse the existing `cord_buffer_` if it has at least the same capacity\n      // as a new one would have.\n      absl::CordBuffer new_cord_buffer =\n          cord_buffer_.capacity() >= predicted_cord_buffer_size\n              ? std::move(cord_buffer_)\n              : absl::CordBuffer::CreateWithCustomLimit(\n                    cord_internal::kCordBufferBlockSize, buffer_length);\n      if (ABSL_PREDICT_FALSE(new_cord_buffer.capacity() <\n                             cursor_index + min_length)) {\n        // The size prediction turned out to be wrong, and the actual size is\n        // insufficient even for what is required. Ignore `new_cord_buffer`.\n      } else {\n        new_cord_buffer.SetLength(\n            UnsignedMin(new_cord_buffer.capacity(),\n                        std::numeric_limits<size_t>::max() - dest.size()));\n        riegeli::null_safe_memcpy(\n            new_cord_buffer.data() + new_cord_buffer.length() - cursor_index,\n            cursor(), cursor_index);\n        cord_buffer_ = std::move(new_cord_buffer);\n        set_buffer(cord_buffer_.data(), cord_buffer_.length(), cursor_index);\n        return true;\n      }\n    }\n  }\n  RIEGELI_ASSERT(buffer_.capacity() < buffer_length ||\n                 limit() != buffer_.data())\n      << \"Failed invariant of CordBackwardWriter: \"\n         \"buffer_ has enough capacity but was used only partially\";\n  Buffer new_buffer = buffer_.capacity() >= buffer_length\n                          ? std::move(buffer_)\n                          : Buffer(buffer_length);\n  const size_t length = UnsignedMin(\n      new_buffer.capacity(), std::numeric_limits<size_t>::max() - dest.size());\n  riegeli::null_safe_memcpy(new_buffer.data() + length - cursor_index, cursor(),\n                            cursor_index);\n  buffer_ = std::move(new_buffer);\n  set_buffer(buffer_.data(), length, cursor_index);\n  return true;\n}\n\nbool CordBackwardWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  if (src.size() <= MaxBytesToCopy()) {\n    return BackwardWriter::WriteSlow(std::move(src));\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_EQ(start_pos(), dest.size())\n      << \"CordBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  std::move(src).PrependTo(dest);\n  return true;\n}\n\nbool CordBackwardWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (src.size() <= MaxBytesToCopy()) return BackwardWriter::WriteSlow(src);\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_EQ(start_pos(), dest.size())\n      << \"CordBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  src.PrependTo(dest);\n  return true;\n}\n\nbool CordBackwardWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  if (src.size() <= MaxBytesToCopy()) {\n    // Not `std::move(src)`: forward to\n    // `BackwardWriter::WriteSlow(const Chain&)`, because\n    // `BackwardWriter::WriteSlow(Chain&&)` would forward to\n    // `CordBackwardWriterBase::WriteSlow(const Chain&)`.\n    return BackwardWriter::WriteSlow(src);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_EQ(start_pos(), dest.size())\n      << \"CordBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  std::move(src).PrependTo(dest);\n  return true;\n}\n\nbool CordBackwardWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (src.size() <= MaxBytesToCopy()) return BackwardWriter::WriteSlow(src);\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_EQ(start_pos(), dest.size())\n      << \"CordBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  dest.Prepend(src);\n  return true;\n}\n\nbool CordBackwardWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  if (src.size() <= MaxBytesToCopy()) {\n    // Not `std::move(src)`: forward to\n    // `BackwardWriter::WriteSlow(const absl::Cord&)`, because\n    // `BackwardWriter::WriteSlow(absl::Cord&&)` would forward to\n    // `CordBackwardWriterBase::WriteSlow(const absl::Cord&)`.\n    return BackwardWriter::WriteSlow(src);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_EQ(start_pos(), dest.size())\n      << \"CordBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  dest.Prepend(std::move(src));\n  return true;\n}\n\nbool CordBackwardWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (src.size() <= MaxBytesToCopy()) return BackwardWriter::WriteSlow(src);\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_EQ(start_pos(), dest.size())\n      << \"CordBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  src.PrependTo(dest);\n  return true;\n}\n\nbool CordBackwardWriterBase::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_EQ(start_pos(), dest.size())\n      << \"CordBackwardWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  return true;\n}\n\nbool CordBackwardWriterBase::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_EQ(start_pos(), dest.size())\n      << \"CordBackwardWriter destination changed unexpectedly\";\n  if (new_size >= start_pos()) {\n    if (ABSL_PREDICT_FALSE(new_size > pos())) return false;\n    set_cursor(start() - (new_size - start_pos()));\n    return true;\n  }\n  set_start_pos(new_size);\n  dest.RemovePrefix(dest.size() - IntCast<size_t>(new_size));\n  set_cursor(start());\n  return true;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/cord_backward_writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_CORD_BACKWARD_WRITER_H_\n#define RIEGELI_BYTES_CORD_BACKWARD_WRITER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/cord_buffer.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `CordBackwardWriter`.\nclass CordBackwardWriterBase : public BackwardWriter {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // If `false`, replaces existing contents of the destination, clearing it\n    // first.\n    //\n    // If `true`, prepends to existing contents of the destination.\n    //\n    // Default: `false`.\n    Options& set_prepend(bool prepend) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      prepend_ = prepend;\n      return *this;\n    }\n    Options&& set_prepend(bool prepend) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_prepend(prepend));\n    }\n    bool prepend() const { return prepend_; }\n\n    // Minimal size of a block of allocated data.\n    //\n    // This is used initially, while the destination is small.\n    //\n    // Default: `kDefaultMinBlockSize` (512).\n    Options& set_min_block_size(size_t min_block_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      min_block_size_ = UnsignedMin(min_block_size, uint32_t{1} << 31);\n      return *this;\n    }\n    Options&& set_min_block_size(size_t min_block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_min_block_size(min_block_size));\n    }\n    size_t min_block_size() const { return min_block_size_; }\n\n    // Maximal size of a block of allocated data.\n    //\n    // This is for performance tuning, not a guarantee: does not apply to\n    // objects allocated separately and then written to this\n    // `CordBackwardWriter`.\n    //\n    // Default: `kDefaultMaxBlockSize - 13` (65523).\n    Options& set_max_block_size(size_t max_block_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GT(max_block_size, 0u)\n          << \"Failed precondition of \"\n             \"CordBackwardWriterBase::Options::set_max_block_size(): \"\n             \"zero block size\";\n      max_block_size_ = UnsignedMin(max_block_size, uint32_t{1} << 31);\n      return *this;\n    }\n    Options&& set_max_block_size(size_t max_block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_block_size(max_block_size));\n    }\n    size_t max_block_size() const { return max_block_size_; }\n\n    // A shortcut for `set_min_block_size(block_size)` with\n    // `set_max_block_size(block_size)`.\n    Options& set_block_size(size_t block_size) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_min_block_size(block_size).set_max_block_size(block_size);\n    }\n    Options&& set_block_size(size_t block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_block_size(block_size));\n    }\n\n   private:\n    bool prepend_ = false;\n    // Use `uint32_t` instead of `size_t` to reduce the object size.\n    uint32_t min_block_size_ = uint32_t{kDefaultMinBlockSize};\n    uint32_t max_block_size_ =\n        uint32_t{absl::CordBuffer::MaximumPayload(kDefaultMaxBlockSize)};\n  };\n\n  // Returns the `absl::Cord` being written to. Unchanged by `Close()`.\n  virtual absl::Cord* DestCord() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsTruncate() override { return true; }\n\n protected:\n  explicit CordBackwardWriterBase(Closed) noexcept : BackwardWriter(kClosed) {}\n\n  explicit CordBackwardWriterBase(const Options& options);\n\n  CordBackwardWriterBase(CordBackwardWriterBase&& that) noexcept;\n  CordBackwardWriterBase& operator=(CordBackwardWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(const Options& options);\n  void Initialize(absl::Cord* dest, bool prepend);\n\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using BackwardWriter::WriteSlow;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool FlushImpl(FlushType flush_type) override;\n  bool TruncateImpl(Position new_size) override;\n\n private:\n  // When deciding whether to copy an array of bytes or share memory, prefer\n  // copying up to this length.\n  size_t MaxBytesToCopy() const;\n\n  // If the buffer is not empty, prepends it to `dest`.\n  void SyncBuffer(absl::Cord& dest);\n\n  // Moves `cord_buffer_`, adjusting buffer pointers if they point to it.\n  void MoveCordBuffer(CordBackwardWriterBase& that);\n\n  std::optional<Position> size_hint_;\n  // Use `uint32_t` instead of `size_t` to reduce the object size.\n  uint32_t min_block_size_ = uint32_t{kDefaultMinBlockSize};\n  uint32_t max_block_size_ =\n      uint32_t{absl::CordBuffer::MaximumPayload(kDefaultMaxBlockSize)};\n\n  // Buffered data to be prepended, in either `cord_buffer_` or `buffer_`.\n  absl::CordBuffer cord_buffer_;\n  Buffer buffer_;\n\n  // Invariants:\n  //   `limit() == nullptr` or\n  //       `limit() == cord_buffer_.data() &&\n  //        start_to_limit() = cord_buffer_.length()` or\n  //       `limit() == buffer_.data()`\n  //   if `ok()` then `start_pos() == DestCord()->size()`\n};\n\n// A `BackwardWriter` which writes to an `absl::Cord` backwards.\n// If `Options::prepend()` is `false` (the default), replaces existing contents\n// of the `absl::Cord`, clearing it first. If `Options::prepend()` is `true`,\n// prepends to existing contents of the `absl::Cord`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `absl::Cord` being written to. `Dest` must support\n// `Dependency<absl::Cord*, Dest>`, e.g. `absl::Cord*` (not owned, default),\n// `absl::Cord` (owned), `Any<absl::Cord*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `absl::Cord`\n// if there are no constructor arguments or the only argument is `Options`,\n// otherwise as `TargetT` of the type of the first constructor argument, except\n// that CTAD is deleted if the first constructor argument is an `absl::Cord&`\n// or `const absl::Cord&` (to avoid writing to an unintentionally separate copy\n// of an existing object).\n//\n// The `absl::Cord` must not be accessed until the `CordBackwardWriter` is\n// closed or no longer used.\ntemplate <typename Dest = absl::Cord*>\nclass CordBackwardWriter : public CordBackwardWriterBase {\n public:\n  // Creates a closed `CordBackwardWriter`.\n  explicit CordBackwardWriter(Closed) noexcept\n      : CordBackwardWriterBase(kClosed) {}\n\n  // Will write to the `absl::Cord` provided by `dest`.\n  explicit CordBackwardWriter(Initializer<Dest> dest,\n                              Options options = Options());\n\n  // Will write to an owned `absl::Cord` which can be accessed by `dest()`.\n  // This constructor is present only if `Dest` is `absl::Cord`.\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<std::is_same_v<DependentDest, absl::Cord>, int> = 0>\n  explicit CordBackwardWriter(Options options = Options());\n\n  CordBackwardWriter(CordBackwardWriter&& that) = default;\n  CordBackwardWriter& operator=(CordBackwardWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `CordBackwardWriter`. This\n  // avoids constructing a temporary `CordBackwardWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<std::is_same_v<DependentDest, absl::Cord>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Options options = Options());\n\n  // Returns the object providing and possibly owning the `absl::Cord` being\n  // written to. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  absl::Cord* DestCord() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n private:\n  // The object providing and possibly owning the `absl::Cord` being written to.\n  Dependency<absl::Cord*, Dest> dest_;\n};\n\nexplicit CordBackwardWriter(Closed) -> CordBackwardWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit CordBackwardWriter(\n    Dest&& dest,\n    CordBackwardWriterBase::Options options = CordBackwardWriterBase::Options())\n    -> CordBackwardWriter<std::conditional_t<\n        std::conjunction_v<std::is_lvalue_reference<Dest>,\n                           std::is_convertible<std::remove_reference_t<Dest>*,\n                                               const absl::Cord*>>,\n        DeleteCtad<Dest&&>, TargetT<Dest>>>;\nexplicit CordBackwardWriter(\n    CordBackwardWriterBase::Options options = CordBackwardWriterBase::Options())\n    -> CordBackwardWriter<absl::Cord>;\n\n// Implementation details follow.\n\ninline CordBackwardWriterBase::CordBackwardWriterBase(const Options& options)\n    : min_block_size_(IntCast<uint32_t>(options.min_block_size())),\n      max_block_size_(IntCast<uint32_t>(options.max_block_size())) {}\n\ninline CordBackwardWriterBase::CordBackwardWriterBase(\n    CordBackwardWriterBase&& that) noexcept\n    : BackwardWriter(static_cast<BackwardWriter&&>(that)),\n      size_hint_(that.size_hint_),\n      min_block_size_(that.min_block_size_),\n      max_block_size_(that.max_block_size_),\n      buffer_(std::move(that.buffer_)) {\n  MoveCordBuffer(that);\n}\n\ninline CordBackwardWriterBase& CordBackwardWriterBase::operator=(\n    CordBackwardWriterBase&& that) noexcept {\n  BackwardWriter::operator=(static_cast<BackwardWriter&&>(that));\n  size_hint_ = that.size_hint_;\n  min_block_size_ = that.min_block_size_;\n  max_block_size_ = that.max_block_size_;\n  buffer_ = std::move(that.buffer_);\n  MoveCordBuffer(that);\n  return *this;\n}\n\ninline void CordBackwardWriterBase::Reset(Closed) {\n  BackwardWriter::Reset(kClosed);\n  size_hint_ = std::nullopt;\n  min_block_size_ = uint32_t{kDefaultMinBlockSize};\n  max_block_size_ =\n      uint32_t{absl::CordBuffer::MaximumPayload(kDefaultMaxBlockSize)};\n  cord_buffer_ = absl::CordBuffer();\n  buffer_ = Buffer();\n}\n\ninline void CordBackwardWriterBase::Reset(const Options& options) {\n  BackwardWriter::Reset();\n  size_hint_ = std::nullopt;\n  min_block_size_ = IntCast<uint32_t>(options.min_block_size());\n  max_block_size_ = IntCast<uint32_t>(options.max_block_size());\n}\n\ninline void CordBackwardWriterBase::Initialize(absl::Cord* dest, bool prepend) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of CordBackwardWriter: null Cord pointer\";\n  if (prepend) {\n    set_start_pos(dest->size());\n  } else {\n    dest->Clear();\n  }\n}\n\ninline void CordBackwardWriterBase::MoveCordBuffer(\n    CordBackwardWriterBase& that) {\n  // Buffer pointers are already moved so `limit()` is taken from `*this`.\n  // `cord_buffer_` is not moved yet so `cord_buffer_` is taken from `that`.\n  const bool uses_cord_buffer = limit() == that.cord_buffer_.data();\n  if (uses_cord_buffer) {\n    RIEGELI_ASSERT_EQ(that.cord_buffer_.length(), start_to_limit())\n        << \"Failed invariant of CordBackwardWriter: \"\n           \"CordBuffer has an unexpected length\";\n  }\n  const size_t saved_start_to_cursor = start_to_cursor();\n  cord_buffer_ = std::move(that.cord_buffer_);\n  if (uses_cord_buffer) {\n    set_buffer(cord_buffer_.data(), cord_buffer_.length(),\n               saved_start_to_cursor);\n  }\n}\n\ntemplate <typename Dest>\ninline CordBackwardWriter<Dest>::CordBackwardWriter(Initializer<Dest> dest,\n                                                    Options options)\n    : CordBackwardWriterBase(options), dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.prepend());\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_same_v<DependentDest, absl::Cord>, int>>\ninline CordBackwardWriter<Dest>::CordBackwardWriter(Options options)\n    : CordBackwardWriter(riegeli::Maker(), std::move(options)) {}\n\ntemplate <typename Dest>\ninline void CordBackwardWriter<Dest>::Reset(Closed) {\n  CordBackwardWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void CordBackwardWriter<Dest>::Reset(Initializer<Dest> dest,\n                                            Options options) {\n  CordBackwardWriterBase::Reset(options);\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.prepend());\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_same_v<DependentDest, absl::Cord>, int>>\ninline void CordBackwardWriter<Dest>::Reset(Options options) {\n  Reset(riegeli::Maker(), std::move(options));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_CORD_BACKWARD_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/cord_reader.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/cord_reader.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/pullable_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid CordReaderBase::Done() {\n  PullableReader::Done();\n  iter_ = std::nullopt;\n}\n\ninline void CordReaderBase::SyncBuffer() {\n  RIEGELI_ASSERT(iter_ != std::nullopt)\n      << \"Failed precondition of CordReaderBase::SyncBuffer(): \"\n         \"no Cord iterator\";\n  set_limit_pos(pos());\n  absl::Cord::Advance(&*iter_, start_to_cursor());\n  set_buffer();\n}\n\nbool CordReaderBase::PullBehindScratch(size_t recommended_length) {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"enough data available, use Pull() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"scratch used\";\n  if (iter_ == std::nullopt) return false;\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const absl::Cord& src = *SrcCord();\n  RIEGELI_ASSERT_LE(limit_pos(), src.size())\n      << \"CordReader source changed unexpectedly\";\n  absl::Cord::Advance(&*iter_, start_to_cursor());\n  if (ABSL_PREDICT_FALSE(*iter_ == src.char_end())) {\n    set_buffer();\n    return false;\n  }\n  const absl::string_view fragment = absl::Cord::ChunkRemaining(*iter_);\n  set_buffer(fragment.data(), fragment.size());\n  move_limit_pos(available());\n  return true;\n}\n\nbool CordReaderBase::ReadBehindScratch(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"Chain size overflow\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"scratch used\";\n  if (iter_ == std::nullopt) {\n    return PullableReader::ReadBehindScratch(length, dest);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const absl::Cord& src = *SrcCord();\n  RIEGELI_ASSERT_LE(limit_pos(), src.size())\n      << \"CordReader source changed unexpectedly\";\n  SyncBuffer();\n  const size_t length_to_read = UnsignedMin(length, src.size() - limit_pos());\n  dest.AppendFrom(*iter_, length_to_read);\n  move_limit_pos(length_to_read);\n  MakeBuffer(src);\n  return length_to_read == length;\n}\n\nbool CordReaderBase::ReadBehindScratch(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"Cord size overflow\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"scratch used\";\n  if (iter_ == std::nullopt) {\n    return PullableReader::ReadBehindScratch(length, dest);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const absl::Cord& src = *SrcCord();\n  RIEGELI_ASSERT_LE(limit_pos(), src.size())\n      << \"CordReader source changed unexpectedly\";\n  SyncBuffer();\n  const size_t length_to_read = UnsignedMin(length, src.size() - limit_pos());\n  if (length_to_read == src.size()) {\n    dest.Append(src);\n    *iter_ = src.char_end();\n  } else {\n    dest.Append(absl::Cord::AdvanceAndRead(&*iter_, length_to_read));\n  }\n  move_limit_pos(length_to_read);\n  MakeBuffer(src);\n  return length_to_read == length;\n}\n\nbool CordReaderBase::CopyBehindScratch(Position length, Writer& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::CopyBehindScratch(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::CopyBehindScratch(Writer&): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const absl::Cord& src = *SrcCord();\n  RIEGELI_ASSERT_LE(limit_pos(), src.size())\n      << \"CordReader source changed unexpectedly\";\n  const size_t length_to_copy =\n      UnsignedMin(length, src.size() - IntCast<size_t>(pos()));\n  if (length_to_copy == src.size()) {\n    RIEGELI_EVAL_ASSERT(Skip(length_to_copy));\n    if (ABSL_PREDICT_FALSE(!dest.Write(src))) return false;\n  } else if (length_to_copy <= kMaxBytesToCopy) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(length_to_copy))) return false;\n    RIEGELI_EVAL_ASSERT(Read(length_to_copy, dest.cursor()));\n    dest.move_cursor(length_to_copy);\n  } else {\n    absl::Cord data;\n    RIEGELI_EVAL_ASSERT(Read(length_to_copy, data));\n    if (ABSL_PREDICT_FALSE(!dest.Write(std::move(data)))) return false;\n  }\n  return length_to_copy == length;\n}\n\nbool CordReaderBase::CopyBehindScratch(size_t length, BackwardWriter& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of \"\n         \"PullableReader::CopyBehindScratch(BackwardWriter&): \"\n         \"enough data available, use Copy(BackwardWriter&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PullableReader::CopyBehindScratch(BackwardWriter&): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const absl::Cord& src = *SrcCord();\n  RIEGELI_ASSERT_LE(limit_pos(), src.size())\n      << \"CordReader source changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(length > src.size() - pos())) {\n    RIEGELI_EVAL_ASSERT(Seek(src.size()));\n    return false;\n  }\n  if (length == src.size()) {\n    RIEGELI_EVAL_ASSERT(Skip(length));\n    return dest.Write(src);\n  }\n  if (length <= kMaxBytesToCopy) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(length))) return false;\n    dest.move_cursor(length);\n    if (ABSL_PREDICT_FALSE(!ReadBehindScratch(length, dest.cursor()))) {\n      dest.set_cursor(dest.cursor() + length);\n      return false;\n    }\n    return true;\n  }\n  absl::Cord data;\n  RIEGELI_EVAL_ASSERT(ReadBehindScratch(length, data));\n  return dest.Write(std::move(data));\n}\n\nbool CordReaderBase::SeekBehindScratch(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (iter_ == std::nullopt) {\n    RIEGELI_ASSERT_EQ(start_pos(), 0u)\n        << \"Failed invariant of CordReaderBase: \"\n           \"no Cord iterator but non-zero position of buffer start\";\n    // Seeking forwards. Source ends.\n    set_cursor(limit());\n    return false;\n  }\n  const absl::Cord& src = *SrcCord();\n  RIEGELI_ASSERT_LE(limit_pos(), src.size())\n      << \"CordReader source changed unexpectedly\";\n  size_t length;\n  if (new_pos > limit_pos()) {\n    // Seeking forwards.\n    if (new_pos >= src.size()) {\n      // Source ends.\n      *iter_ = src.char_end();\n      set_buffer();\n      set_limit_pos(src.size());\n      return new_pos == src.size();\n    }\n    length = IntCast<size_t>(new_pos - start_pos());\n  } else {\n    // Seeking backwards.\n    *iter_ = src.char_begin();\n    length = IntCast<size_t>(new_pos);\n  }\n  absl::Cord::Advance(&*iter_, length);\n  set_limit_pos(new_pos);\n  MakeBuffer(src);\n  return true;\n}\n\nstd::optional<Position> CordReaderBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  const absl::Cord& src = *SrcCord();\n  return src.size();\n}\n\nstd::unique_ptr<Reader> CordReaderBase::NewReaderImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point.\n  const absl::Cord& src = *SrcCord();\n  std::unique_ptr<Reader> reader = std::make_unique<CordReader<>>(&src);\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/cord_reader.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_CORD_READER_H_\n#define RIEGELI_BYTES_CORD_READER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pullable_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nclass BackwardWriter;\nclass Writer;\n\n// Template parameter independent part of `CordReader`.\nclass CordReaderBase : public PullableReader {\n public:\n  // Returns the `absl::Cord` being read from. Unchanged by `Close()`.\n  virtual const absl::Cord* SrcCord() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool ToleratesReadingAhead() override { return true; }\n  bool SupportsRandomAccess() override { return true; }\n  bool SupportsNewReader() override { return true; }\n\n protected:\n  using PullableReader::PullableReader;\n\n  CordReaderBase(CordReaderBase&& that) noexcept;\n  CordReaderBase& operator=(CordReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(const absl::Cord* src);\n\n  void Done() override;\n  bool PullBehindScratch(size_t recommended_length) override;\n  using PullableReader::ReadBehindScratch;\n  bool ReadBehindScratch(size_t length, Chain& dest) override;\n  bool ReadBehindScratch(size_t length, absl::Cord& dest) override;\n  using PullableReader::CopyBehindScratch;\n  bool CopyBehindScratch(Position length, Writer& dest) override;\n  bool CopyBehindScratch(size_t length, BackwardWriter& dest) override;\n  bool SeekBehindScratch(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n  // Invariant:\n  //   if `!is_open()` or\n  //      `*SrcCord()` is flat with size at most `kMaxBytesToCopy`\n  //       then `iter_ == std::nullopt`\n  //       else `iter_ != std::nullopt` and\n  //            `*iter_` reads from `*SrcCord()`\n  std::optional<absl::Cord::CharIterator> iter_;\n\n private:\n  // Moves `*iter_` to account for data which have been read from the buffer.\n  //\n  // Precondition: `iter_ != std::nullopt`\n  void SyncBuffer();\n\n  // Sets buffer pointers to `absl::Cord::ChunkRemaining(*iter_)`,\n  // or to `nullptr` if `*iter_ == src.char_end()`.\n  //\n  // Precondition: `iter_ != std::nullopt`\n  void MakeBuffer(const absl::Cord& src);\n\n  // Invariants if `iter_ == std::nullopt` and `is_open()`:\n  //   scratch is not used\n  //   `start() == SrcCord()->TryFlat()->data()`\n  //   `start_to_limit() == SrcCord()->TryFlat()->size()`\n  //   `start_pos() == 0`\n  //\n  // Invariants if `iter_ != std::nullopt` and scratch is not used:\n  //   `start() == (*iter_ == SrcCord()->char_end()\n  //                    ? nullptr\n  //                    : absl::Cord::ChunkRemaining(*iter_).data())`\n  //   `start_to_limit() == (*iter_ == SrcCord()->char_end()\n  //                          ? 0\n  //                          : absl::Cord::ChunkRemaining(*iter_).size())`\n  //   `start_pos()` is the position of `*iter_` in `*SrcCord()`\n};\n\n// A `Reader` which reads from an `absl::Cord`.\n//\n// It supports random access and `NewReader()`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `absl::Cord` being read from. `Src` must support\n// `Dependency<const absl::Cord*, Src>`, e.g.\n// `const absl::Cord*` (not owned, default), `absl::Cord` (owned),\n// `Any<const absl::Cord*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The `absl::Cord` must not be changed until the `CordReader` is closed or no\n// longer used.\ntemplate <typename Src = const absl::Cord*>\nclass CordReader : public CordReaderBase {\n public:\n  // Creates a closed `CordReader`.\n  explicit CordReader(Closed) noexcept : CordReaderBase(kClosed) {}\n\n  // Will read from the `absl::Cord` provided by `src`.\n  explicit CordReader(Initializer<Src> src);\n\n  CordReader(CordReader&& that) = default;\n  CordReader& operator=(CordReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `CordReader`. This avoids\n  // constructing a temporary `CordReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src);\n\n  // Returns the object providing and possibly owning the `absl::Cord` being\n  // read from. Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  const absl::Cord* SrcCord() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the `absl::Cord` being read from.\n  MovingDependency<const absl::Cord*, Src, Mover> src_;\n};\n\nexplicit CordReader(Closed) -> CordReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit CordReader(Src&& src) -> CordReader<TargetT<Src>>;\n\n// Implementation details follow.\n\ninline CordReaderBase::CordReaderBase(CordReaderBase&& that) noexcept\n    : PullableReader(static_cast<PullableReader&&>(that)),\n      iter_(std::exchange(that.iter_, std::nullopt)) {}\n\ninline CordReaderBase& CordReaderBase::operator=(\n    CordReaderBase&& that) noexcept {\n  PullableReader::operator=(static_cast<PullableReader&&>(that));\n  iter_ = std::exchange(that.iter_, std::nullopt);\n  return *this;\n}\n\ninline void CordReaderBase::Reset(Closed) {\n  PullableReader::Reset(kClosed);\n  iter_ = std::nullopt;\n}\n\ninline void CordReaderBase::Reset() {\n  PullableReader::Reset();\n  iter_ = std::nullopt;\n}\n\ninline void CordReaderBase::Initialize(const absl::Cord* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of CordReader: null Cord pointer\";\n  if (const std::optional<absl::string_view> flat = src->TryFlat()) {\n    if (flat->size() <= kMaxBytesToCopy) {\n      set_buffer(flat->data(), flat->size());\n      move_limit_pos(available());\n      return;\n    }\n  }\n  iter_ = src->char_begin();\n  MakeBuffer(*src);\n}\n\ninline void CordReaderBase::MakeBuffer(const absl::Cord& src) {\n  RIEGELI_ASSERT(iter_ != std::nullopt)\n      << \"Failed precondition of CordReaderBase::MakeBuffer(): \"\n         \"no Cord iterator\";\n  if (*iter_ == src.char_end()) {\n    set_buffer();\n    return;\n  }\n  const absl::string_view fragment = absl::Cord::ChunkRemaining(*iter_);\n  set_buffer(fragment.data(), fragment.size());\n  move_limit_pos(available());\n}\n\ntemplate <typename Src>\nclass CordReader<Src>::Mover {\n public:\n  static auto member() { return &CordReader::src_; }\n\n  explicit Mover(CordReader& self, ABSL_ATTRIBUTE_UNUSED CordReader& that)\n      : behind_scratch_(&self),\n        uses_buffer_(self.start() != nullptr),\n        position_(IntCast<size_t>(self.start_pos())),\n        start_to_cursor_(self.start_to_cursor()) {\n#if RIEGELI_DEBUG\n    if (self.iter_ == std::nullopt) {\n      if (uses_buffer_) {\n        const std::optional<absl::string_view> flat = that.src_->TryFlat();\n        RIEGELI_ASSERT(flat != std::nullopt)\n            << \"CordReader source changed unexpectedly\";\n        RIEGELI_ASSERT_EQ(flat->data(), self.start())\n            << \"CordReader source changed unexpectedly\";\n        RIEGELI_ASSERT_EQ(flat->size(), self.start_to_limit())\n            << \"CordReader source changed unexpectedly\";\n      }\n    } else {\n      if (position_ == that.src_->size()) {\n        RIEGELI_ASSERT(*self.iter_ == that.src_->char_end())\n            << \"CordReader source changed unexpectedly\";\n        RIEGELI_ASSERT(!uses_buffer_)\n            << \"CordReader source changed unexpectedly\";\n      } else {\n        const absl::string_view fragment =\n            absl::Cord::ChunkRemaining(*self.iter_);\n        RIEGELI_ASSERT_EQ(fragment.data(), self.start())\n            << \"CordReader source changed unexpectedly\";\n        RIEGELI_ASSERT_EQ(fragment.size(), self.start_to_limit())\n            << \"CordReader source changed unexpectedly\";\n      }\n    }\n#endif\n  }\n\n  void Done(CordReader& self) {\n    if (self.iter_ == std::nullopt) {\n      if (uses_buffer_) {\n        const std::optional<absl::string_view> flat = self.src_->TryFlat();\n        RIEGELI_ASSERT(flat != std::nullopt)\n            << \"CordReader source changed unexpectedly\";\n        self.set_buffer(flat->data(), flat->size(), start_to_cursor_);\n      }\n    } else {\n      if (position_ == self.src_->size()) {\n        self.iter_ = self.src_->char_end();\n      } else {\n        self.iter_ = self.src_->char_begin();\n        absl::Cord::Advance(&*self.iter_, position_);\n        const absl::string_view fragment =\n            absl::Cord::ChunkRemaining(*self.iter_);\n        self.set_buffer(fragment.data(), fragment.size(), start_to_cursor_);\n      }\n    }\n  }\n\n private:\n  BehindScratch behind_scratch_;\n  bool uses_buffer_;\n  size_t position_;\n  size_t start_to_cursor_;\n};\n\ntemplate <typename Src>\ninline CordReader<Src>::CordReader(Initializer<Src> src)\n    : src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void CordReader<Src>::Reset(Closed) {\n  CordReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void CordReader<Src>::Reset(Initializer<Src> src) {\n  CordReaderBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_CORD_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/cord_writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/cord_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/cord_buffer.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_utils.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/cord_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid CordWriterBase::Done() {\n  CordWriterBase::FlushImpl(FlushType::kFromObject);\n  Writer::Done();\n  cord_buffer_ = absl::CordBuffer();\n  buffer_ = Buffer();\n  tail_.reset();\n  associated_reader_.Reset();\n}\n\ninline size_t CordWriterBase::MaxBytesToCopy() const {\n  if (size_hint_ != std::nullopt && pos() < *size_hint_) {\n    return UnsignedClamp(*size_hint_ - pos() - 1,\n                         cord_internal::kMaxBytesToCopyToEmptyCord,\n                         cord_internal::kMaxBytesToCopyToNonEmptyCord);\n  }\n  if (pos() == 0) return cord_internal::kMaxBytesToCopyToEmptyCord;\n  return cord_internal::kMaxBytesToCopyToNonEmptyCord;\n}\n\ninline void CordWriterBase::SyncBuffer(absl::Cord& dest) {\n  if (ABSL_PREDICT_FALSE(HasAppendedTail(dest))) {\n    ExtractTail(dest);\n    return;\n  }\n  if (start() == nullptr) return;\n  ShrinkTail(start_to_cursor());\n  set_start_pos(pos());\n  const absl::string_view data(start(), start_to_cursor());\n  if (start() == cord_buffer_.data()) {\n    cord_buffer_.SetLength(data.size());\n    if (Wasteful(cord_internal::kFlatOverhead + cord_buffer_.capacity(),\n                 cord_buffer_.length())) {\n      cord_internal::AppendToBlockyCord(data, dest);\n    } else {\n      dest.Append(std::move(cord_buffer_));\n    }\n  } else {\n    ExternalRef(std::move(buffer_), data).AppendTo(dest);\n  }\n  set_buffer();\n}\n\ninline void CordWriterBase::MoveFromTail(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_NE(tail_, nullptr)\n      << \"Failed precondition of CordWriterBase::MoveFromTail(): no tail\";\n  RIEGELI_ASSERT_LE(length, tail_->size())\n      << \"Failed precondition of CordWriterBase::MoveFromTail(): \"\n         \"length longer than the tail\";\n  if (length == tail_->size()) {\n    dest.Append(std::move(*tail_));\n    tail_->Clear();\n    return;\n  }\n  dest.Append(tail_->Subcord(0, length));\n  tail_->RemovePrefix(length);\n}\n\ninline void CordWriterBase::MoveToTail(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LE(length, dest.size())\n      << \"Failed precondition of CordWriterBase::MoveToTail(): \"\n         \"length longer than the destination\";\n  if (tail_ == nullptr) tail_ = std::make_unique<absl::Cord>();\n  if (length == dest.size()) {\n    tail_->Prepend(std::move(dest));\n    dest.Clear();\n    return;\n  }\n  tail_->Prepend(dest.Subcord(dest.size() - length, length));\n  dest.RemoveSuffix(length);\n}\n\ninline bool CordWriterBase::HasAppendedTail(const absl::Cord& dest) const {\n  return start_pos() < dest.size();\n}\n\ninline void CordWriterBase::ExtractTail(absl::Cord& dest) {\n  RIEGELI_ASSERT(HasAppendedTail(dest))\n      << \"Failed precondition of CordWriterBase::ExtractTail(): \"\n         \"the tail is not appended\";\n  RIEGELI_ASSERT_EQ(start(), nullptr)\n      << \"Failed invariant of CordWriterBase: \"\n         \"both a buffer and the appended tail are present\";\n  MoveToTail(dest.size() - IntCast<size_t>(start_pos()), dest);\n}\n\ninline void CordWriterBase::AppendTail(absl::Cord& dest) {\n  RIEGELI_ASSERT(!HasAppendedTail(dest))\n      << \"Failed precondition of CordWriterBase::AppendTail(): \"\n         \"the tail is appended\";\n  if (ABSL_PREDICT_FALSE(tail_ != nullptr)) {\n    dest.Append(std::move(*tail_));\n    tail_->Clear();\n  }\n}\n\ninline void CordWriterBase::ShrinkTail(size_t length) {\n  if (ABSL_PREDICT_FALSE(tail_ != nullptr)) {\n    tail_->RemovePrefix(UnsignedMin(length, tail_->size()));\n  }\n}\n\nvoid CordWriterBase::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (write_size_hint == size_hint_) {\n    size_hint_ = std::nullopt;\n  } else {\n    size_hint_ = SaturatingAdd(pos(), *write_size_hint);\n  }\n}\n\nbool CordWriterBase::PushSlow(size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_LE(start_pos(), dest.size())\n      << \"CordWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(HasAppendedTail(dest))) {\n    ExtractTail(dest);\n  } else if (pos() == 0) {\n    Position needed_length = UnsignedMax(min_length, recommended_length);\n    if (size_hint_ != std::nullopt) {\n      needed_length = UnsignedMax(needed_length, *size_hint_);\n    }\n    if (needed_length <= cord_buffer_.capacity()) {\n      // Use the initial capacity of `cord_buffer_`, even if it is smaller than\n      // `min_block_size_`, because this avoids allocation.\n      cord_buffer_.SetLength(cord_buffer_.capacity());\n      set_buffer(cord_buffer_.data(), cord_buffer_.length());\n      return true;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(min_length > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (start_to_cursor() >= min_block_size_) SyncBuffer(dest);\n  const size_t cursor_index = start_to_cursor();\n  const size_t buffer_length = ApplyBufferConstraints(\n      ApplySizeHint(UnsignedMax(start_pos(), min_block_size_), size_hint_,\n                    start_pos()),\n      cursor_index + min_length,\n      SaturatingAdd(cursor_index, recommended_length), max_block_size_);\n  if (buffer_length <= cord_internal::kCordBufferMaxSize) {\n    RIEGELI_ASSERT(cord_buffer_.capacity() < buffer_length ||\n                   start() != cord_buffer_.data())\n        << \"Failed invariant of CordWriter: \"\n           \"cord_buffer_ has enough capacity but was used only partially\";\n    const size_t predicted_cord_buffer_size =\n        cord_internal::CordBufferSizeForCapacity(buffer_length);\n    if (predicted_cord_buffer_size >= cursor_index + min_length) {\n      // Reuse the existing `cord_buffer_` if it has at least the same capacity\n      // as a new one would have.\n      absl::CordBuffer new_cord_buffer =\n          cord_buffer_.capacity() >= predicted_cord_buffer_size\n              ? std::move(cord_buffer_)\n              : absl::CordBuffer::CreateWithCustomLimit(\n                    cord_internal::kCordBufferBlockSize, buffer_length);\n      if (ABSL_PREDICT_FALSE(new_cord_buffer.capacity() <\n                             cursor_index + min_length)) {\n        // The size prediction turned out to be wrong, and the actual size is\n        // insufficient even for what is required. Ignore `new_cord_buffer`.\n      } else {\n        new_cord_buffer.SetLength(\n            UnsignedMin(new_cord_buffer.capacity(),\n                        std::numeric_limits<size_t>::max() - dest.size()));\n        riegeli::null_safe_memcpy(new_cord_buffer.data(), start(),\n                                  cursor_index);\n        cord_buffer_ = std::move(new_cord_buffer);\n        set_buffer(cord_buffer_.data(), cord_buffer_.length(), cursor_index);\n        return true;\n      }\n    }\n  }\n  RIEGELI_ASSERT(buffer_.capacity() < buffer_length ||\n                 start() != buffer_.data())\n      << \"Failed invariant of CordWriter: \"\n         \"buffer_ has enough capacity but was used only partially\";\n  Buffer new_buffer = buffer_.capacity() >= buffer_length\n                          ? std::move(buffer_)\n                          : Buffer(buffer_length);\n  riegeli::null_safe_memcpy(new_buffer.data(), start(), cursor_index);\n  buffer_ = std::move(new_buffer);\n  set_buffer(buffer_.data(),\n             UnsignedMin(buffer_.capacity(),\n                         std::numeric_limits<size_t>::max() - dest.size()),\n             cursor_index);\n  return true;\n}\n\nbool CordWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  if (src.size() <= MaxBytesToCopy()) return Writer::WriteSlow(std::move(src));\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_LE(start_pos(), dest.size())\n      << \"CordWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  ShrinkTail(src.size());\n  move_start_pos(src.size());\n  std::move(src).AppendTo(dest);\n  return true;\n}\n\nbool CordWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (src.size() <= MaxBytesToCopy()) return Writer::WriteSlow(src);\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_LE(start_pos(), dest.size())\n      << \"CordWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  ShrinkTail(src.size());\n  move_start_pos(src.size());\n  src.AppendTo(dest);\n  return true;\n}\n\nbool CordWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  if (src.size() <= MaxBytesToCopy()) {\n    // Not `std::move(src)`: forward to `Writer::WriteSlow(const Chain&)`,\n    // because `Writer::WriteSlow(Chain&&)` would forward to\n    // `CordWriterBase::WriteSlow(const Chain&)`.\n    return Writer::WriteSlow(src);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_LE(start_pos(), dest.size())\n      << \"CordWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  ShrinkTail(src.size());\n  move_start_pos(src.size());\n  std::move(src).AppendTo(dest);\n  return true;\n}\n\nbool CordWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (src.size() <= MaxBytesToCopy()) return Writer::WriteSlow(src);\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_LE(start_pos(), dest.size())\n      << \"CordWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  ShrinkTail(src.size());\n  move_start_pos(src.size());\n  dest.Append(src);\n  return true;\n}\n\nbool CordWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  if (src.size() <= MaxBytesToCopy()) {\n    // Not `std::move(src)`: forward to `Writer::WriteSlow(const absl::Cord&)`,\n    // because `Writer::WriteSlow(absl::Cord&&)` would forward to\n    // `CordWriterBase::WriteSlow(const absl::Cord&)`.\n    return Writer::WriteSlow(src);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_LE(start_pos(), dest.size())\n      << \"CordWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  ShrinkTail(src.size());\n  move_start_pos(src.size());\n  dest.Append(std::move(src));\n  return true;\n}\n\nbool CordWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (src.size() <= MaxBytesToCopy()) return Writer::WriteSlow(src);\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_LE(start_pos(), dest.size())\n      << \"CordWriter destination changed unexpectedly\";\n  SyncBuffer(dest);\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  ShrinkTail(IntCast<size_t>(src.size()));\n  move_start_pos(src.size());\n  src.AppendTo(dest);\n  return true;\n}\n\nbool CordWriterBase::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_LE(start_pos(), dest.size())\n      << \"CordWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(HasAppendedTail(dest))) {\n    RIEGELI_ASSERT_EQ(start(), nullptr)\n        << \"Failed invariant of CordWriterBase: \"\n           \"both a buffer and the appended tail are present\";\n    RIEGELI_ASSERT(tail_ == nullptr || tail_->empty())\n        << \"Failed invariant of CordWriterBase: \"\n           \"the tail is both appended and separated\";\n    return true;\n  }\n  SyncBuffer(dest);\n  AppendTail(dest);\n  return true;\n}\n\nbool CordWriterBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of Writer::SeekSlow(): \"\n         \"position unchanged, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_LE(start_pos(), dest.size())\n      << \"CordWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(HasAppendedTail(dest))) {\n    RIEGELI_ASSERT_EQ(start(), nullptr)\n        << \"Failed invariant of CordWriterBase: \"\n           \"both a buffer and the appended tail are present\";\n    RIEGELI_ASSERT(tail_ == nullptr || tail_->empty())\n        << \"Failed invariant of CordWriterBase: \"\n           \"the tail is both appended and separated\";\n    if (ABSL_PREDICT_FALSE(new_pos > dest.size())) {\n      set_start_pos(dest.size());\n      return false;\n    }\n    MoveToTail(dest.size() - IntCast<size_t>(new_pos), dest);\n    set_start_pos(new_pos);\n    return true;\n  }\n  if (new_pos > pos()) {\n    if (ABSL_PREDICT_TRUE(tail_ == nullptr) || tail_->empty()) return false;\n    SyncBuffer(dest);\n    if (ABSL_PREDICT_FALSE(new_pos > dest.size() + tail_->size())) {\n      AppendTail(dest);\n      set_start_pos(dest.size());\n      return false;\n    }\n    MoveFromTail(IntCast<size_t>(new_pos) - dest.size(), dest);\n  } else {\n    SyncBuffer(dest);\n    MoveToTail(dest.size() - IntCast<size_t>(new_pos), dest);\n  }\n  set_start_pos(new_pos);\n  return true;\n}\n\nstd::optional<Position> CordWriterBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_LE(start_pos(), dest.size())\n      << \"CordWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(HasAppendedTail(dest))) {\n    RIEGELI_ASSERT_EQ(start(), nullptr)\n        << \"Failed invariant of CordWriterBase: \"\n           \"both a buffer and the appended tail are present\";\n    RIEGELI_ASSERT(tail_ == nullptr || tail_->empty())\n        << \"Failed invariant of CordWriterBase: \"\n           \"the tail is both appended and separated\";\n    return dest.size();\n  }\n  if (ABSL_PREDICT_FALSE(tail_ != nullptr)) {\n    return UnsignedMax(pos(), start_pos() + tail_->size());\n  }\n  return pos();\n}\n\nbool CordWriterBase::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Cord& dest = *DestCord();\n  RIEGELI_ASSERT_LE(start_pos(), dest.size())\n      << \"CordWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(HasAppendedTail(dest))) {\n    RIEGELI_ASSERT_EQ(start(), nullptr)\n        << \"Failed invariant of CordWriterBase: \"\n           \"both a buffer and the appended tail are present\";\n    RIEGELI_ASSERT(tail_ == nullptr || tail_->empty())\n        << \"Failed invariant of CordWriterBase: \"\n           \"the tail is both appended and separated\";\n    if (ABSL_PREDICT_FALSE(new_size > dest.size())) {\n      set_start_pos(dest.size());\n      return false;\n    }\n  } else if (new_size > pos()) {\n    if (ABSL_PREDICT_TRUE(tail_ == nullptr) || tail_->empty()) return false;\n    SyncBuffer(dest);\n    if (ABSL_PREDICT_FALSE(new_size > dest.size() + tail_->size())) {\n      move_start_pos(tail_->size());\n      AppendTail(dest);\n      return false;\n    }\n    set_start_pos(new_size);\n    tail_->RemoveSuffix(dest.size() + tail_->size() -\n                        IntCast<size_t>(new_size));\n    AppendTail(dest);\n    return true;\n  } else {\n    if (ABSL_PREDICT_FALSE(tail_ != nullptr)) tail_->Clear();\n    if (new_size >= start_pos()) {\n      set_cursor(start() + (new_size - start_pos()));\n      return true;\n    }\n  }\n  RIEGELI_ASSERT(tail_ == nullptr || tail_->empty());\n  RIEGELI_ASSERT_LE(new_size, dest.size());\n  set_start_pos(new_size);\n  dest.RemoveSuffix(dest.size() - IntCast<size_t>(new_size));\n  set_cursor(start());\n  return true;\n}\n\nReader* CordWriterBase::ReadModeImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!CordWriterBase::FlushImpl(FlushType::kFromObject))) {\n    return nullptr;\n  }\n  absl::Cord& dest = *DestCord();\n  CordReader<>* const reader = associated_reader_.ResetReader(&dest);\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/cord_writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_CORD_WRITER_H_\n#define RIEGELI_BYTES_CORD_WRITER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/cord_buffer.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\ntemplate <typename Src>\nclass CordReader;\nclass Reader;\n\n// Template parameter independent part of `CordWriter`.\nclass CordWriterBase : public Writer {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // If `false`, replaces existing contents of the destination, clearing it\n    // first.\n    //\n    // If `true`, appends to existing contents of the destination.\n    //\n    // Default: `false`.\n    Options& set_append(bool append) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      append_ = append;\n      return *this;\n    }\n    Options&& set_append(bool append) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_append(append));\n    }\n    bool append() const { return append_; }\n\n    // Minimal size of a block of allocated data.\n    //\n    // This is used initially, while the destination is small.\n    //\n    // Default: `kDefaultMinBlockSize` (512).\n    Options& set_min_block_size(size_t min_block_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      min_block_size_ = UnsignedMin(min_block_size, uint32_t{1} << 31);\n      return *this;\n    }\n    Options&& set_min_block_size(size_t min_block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_min_block_size(min_block_size));\n    }\n    size_t min_block_size() const { return min_block_size_; }\n\n    // Maximal size of a block of allocated data.\n    //\n    // This is for performance tuning, not a guarantee: does not apply to\n    // objects allocated separately and then written to this `CordWriter`.\n    //\n    // Default: `kDefaultMaxBlockSize - 13` (65523).\n    Options& set_max_block_size(size_t max_block_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GT(max_block_size, 0u)\n          << \"Failed precondition of \"\n             \"CordWriterBase::Options::set_max_block_size(): \"\n             \"zero block size\";\n      max_block_size_ = UnsignedMin(max_block_size, uint32_t{1} << 31);\n      return *this;\n    }\n    Options&& set_max_block_size(size_t max_block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_block_size(max_block_size));\n    }\n    size_t max_block_size() const { return max_block_size_; }\n\n    // A shortcut for `set_min_block_size(block_size)` with\n    // `set_max_block_size(block_size)`.\n    Options& set_block_size(size_t block_size) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_min_block_size(block_size).set_max_block_size(block_size);\n    }\n    Options&& set_block_size(size_t block_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_block_size(block_size));\n    }\n\n   private:\n    bool append_ = false;\n    // Use `uint32_t` instead of `size_t` to reduce the object size.\n    uint32_t min_block_size_ = uint32_t{kDefaultMinBlockSize};\n    uint32_t max_block_size_ =\n        uint32_t{absl::CordBuffer::MaximumPayload(kDefaultMaxBlockSize)};\n  };\n\n  // Returns the `absl::Cord` being written to. Unchanged by `Close()`.\n  virtual absl::Cord* DestCord() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n  absl::Cord& Digest() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    Flush();\n    return *DestCord();\n  }\n\n  bool SupportsRandomAccess() override { return true; }\n  bool SupportsReadMode() override { return true; }\n\n protected:\n  explicit CordWriterBase(Closed) noexcept : Writer(kClosed) {}\n\n  explicit CordWriterBase(const Options& options);\n\n  CordWriterBase(CordWriterBase&& that) noexcept;\n  CordWriterBase& operator=(CordWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(const Options& options);\n  void Initialize(absl::Cord* dest, bool append);\n\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool FlushImpl(FlushType flush_type) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  bool TruncateImpl(Position new_size) override;\n  Reader* ReadModeImpl(Position initial_pos) override;\n\n private:\n  // When deciding whether to copy an array of bytes or share memory, prefer\n  // copying up to this length.\n  size_t MaxBytesToCopy() const;\n\n  // If the buffer is not empty, appends it to `dest`. Ensures that data which\n  // follow the current position are separated in `*tail_`.\n  void SyncBuffer(absl::Cord& dest);\n\n  // Moves `cord_buffer_`, adjusting buffer pointers if they point to it.\n  void MoveCordBuffer(CordWriterBase& that);\n\n  // Moves `length` of data from the beginning of `*tail_` to the end of `dest`.\n  void MoveFromTail(size_t length, absl::Cord& dest);\n\n  // Moves `length` of data from the end of `dest` to the beginning of `*tail_`.\n  void MoveToTail(size_t length, absl::Cord& dest);\n\n  // Returns `true` if data which follow the current position are appended to\n  // `dest`.\n  bool HasAppendedTail(const absl::Cord& dest) const;\n\n  // Moves data which follow the current position from being appended to `dest`\n  // to being separated in `*tail_`.\n  void ExtractTail(absl::Cord& dest);\n\n  // Moves data which follow the current position from being separated in\n  // `*tail_` to being appended to `dest`.\n  void AppendTail(absl::Cord& dest);\n\n  // Removes a prefix of `*tail_` of the given `length`, staturated at clearing\n  // the whole `*tail_`.\n  void ShrinkTail(size_t length);\n\n  std::optional<Position> size_hint_;\n  // Use `uint32_t` instead of `size_t` to reduce the object size.\n  uint32_t min_block_size_ = uint32_t{kDefaultMinBlockSize};\n  uint32_t max_block_size_ =\n      uint32_t{absl::CordBuffer::MaximumPayload(kDefaultMaxBlockSize)};\n\n  // Buffered data to be appended, in either `cord_buffer_` or `buffer_`.\n  absl::CordBuffer cord_buffer_;\n  Buffer buffer_;\n\n  // If `start_pos() < DestCord()->size()`, then data after the current\n  // position are appended to `*DestCord()`, buffer pointers are `nullptr`,\n  // and `tail_ == nullptr || tail_->empty()`.\n  //\n  // Otherwise, if `start_pos() == DestCord()->size() && tail_ != nullptr`,\n  // data after the current position are separated in `*tail_`, ignoring a\n  // prefix of `*tail_` with length `start_to_cursor()`, saturated at ignoring\n  // the whole `*tail_` (the ignored prefix is being overwritten with buffered\n  // data).\n  //\n  // Otherwise `start_pos() == DestCord()->size() && tail_ == nullptr`, and\n  // there are no data after the current position.\n  //\n  // `tail_` is stored behind `std::unique_ptr` to reduce the object size in the\n  // common case when random access is not used.\n  std::unique_ptr<absl::Cord> tail_;\n\n  AssociatedReader<CordReader<const absl::Cord*>> associated_reader_;\n\n  // Invariants:\n  //   `start() == nullptr` or\n  //       `start() == cord_buffer_.data() &&\n  //        start_to_limit() == cord_buffer_.length()` or\n  //       `start() == buffer_.data()`\n  //   if `ok()` then `start_pos() <= DestCord()->size()`\n  //   if `ok() && start_pos() < DestCord()->size()` then\n  //       `start() == nullptr && (tail_ == nullptr || tail_->empty())`\n};\n\n// A `Writer` which writes to an `absl::Cord`. If `Options::append()` is `false`\n// (the default), replaces existing contents of the `absl::Cord`, clearing it\n// first. If `Options::append()` is `true`, appends to existing contents of the\n// `absl::Cord`.\n//\n// It supports `Seek()` and `ReadMode()`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `absl::Cord` being written to. `Dest` must support\n// `Dependency<absl::Cord*, Dest>`, e.g. `absl::Cord*` (not owned, default),\n// `absl::Cord` (owned), `Any<absl::Cord*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `absl::Cord`\n// if there are no constructor arguments or the only argument is `Options`,\n// otherwise as `TargetT` of the type of the first constructor argument, except\n// that CTAD is deleted if the first constructor argument is an `absl::Cord&`\n// or `const absl::Cord&` (to avoid writing to an unintentionally separate copy\n// of an existing object).\n//\n// The `absl::Cord` must not be accessed until the `CordWriter` is closed or no\n// longer used, except that it is allowed to read the `absl::Cord` immediately\n// after `Flush()`.\ntemplate <typename Dest = absl::Cord*>\nclass CordWriter : public CordWriterBase {\n public:\n  // Creates a closed `CordWriter`.\n  explicit CordWriter(Closed) noexcept : CordWriterBase(kClosed) {}\n\n  // Will write to the `absl::Cord` provided by `dest`.\n  explicit CordWriter(Initializer<Dest> dest, Options options = Options());\n\n  // Will write to an owned `absl::Cord` which can be accessed by `dest()`.\n  // This constructor is present only if `Dest` is `absl::Cord`.\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<std::is_same_v<DependentDest, absl::Cord>, int> = 0>\n  explicit CordWriter(Options options = Options());\n\n  CordWriter(CordWriter&& that) = default;\n  CordWriter& operator=(CordWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `CordWriter`. This avoids\n  // constructing a temporary `CordWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<std::is_same_v<DependentDest, absl::Cord>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Options options = Options());\n\n  // Returns the object providing and possibly owning the `absl::Cord` being\n  // written to. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  absl::Cord* DestCord() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n private:\n  // The object providing and possibly owning the `absl::Cord` being written to.\n  Dependency<absl::Cord*, Dest> dest_;\n};\n\nexplicit CordWriter(Closed) -> CordWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit CordWriter(Dest&& dest,\n                    CordWriterBase::Options options = CordWriterBase::Options())\n    -> CordWriter<std::conditional_t<\n        std::conjunction_v<std::is_lvalue_reference<Dest>,\n                           std::is_convertible<std::remove_reference_t<Dest>*,\n                                               const absl::Cord*>>,\n        DeleteCtad<Dest&&>, TargetT<Dest>>>;\nexplicit CordWriter(CordWriterBase::Options options = CordWriterBase::Options())\n    -> CordWriter<absl::Cord>;\n\n// Implementation details follow.\n\ninline CordWriterBase::CordWriterBase(const Options& options)\n    : min_block_size_(IntCast<uint32_t>(options.min_block_size())),\n      max_block_size_(IntCast<uint32_t>(options.max_block_size())) {}\n\ninline CordWriterBase::CordWriterBase(CordWriterBase&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)),\n      size_hint_(that.size_hint_),\n      min_block_size_(that.min_block_size_),\n      max_block_size_(that.max_block_size_),\n      buffer_(std::move(that.buffer_)),\n      tail_(std::move(that.tail_)),\n      associated_reader_(std::move(that.associated_reader_)) {\n  MoveCordBuffer(that);\n}\n\ninline CordWriterBase& CordWriterBase::operator=(\n    CordWriterBase&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  size_hint_ = that.size_hint_;\n  min_block_size_ = that.min_block_size_;\n  max_block_size_ = that.max_block_size_;\n  buffer_ = std::move(that.buffer_);\n  tail_ = std::move(that.tail_);\n  associated_reader_ = std::move(that.associated_reader_);\n  MoveCordBuffer(that);\n  return *this;\n}\n\ninline void CordWriterBase::Reset(Closed) {\n  Writer::Reset(kClosed);\n  size_hint_ = std::nullopt;\n  min_block_size_ = uint32_t{kDefaultMinBlockSize};\n  max_block_size_ =\n      uint32_t{absl::CordBuffer::MaximumPayload(kDefaultMaxBlockSize)};\n  cord_buffer_ = absl::CordBuffer();\n  buffer_ = Buffer();\n  tail_.reset();\n  associated_reader_.Reset();\n}\n\ninline void CordWriterBase::Reset(const Options& options) {\n  Writer::Reset();\n  size_hint_ = std::nullopt;\n  min_block_size_ = IntCast<uint32_t>(options.min_block_size());\n  max_block_size_ = IntCast<uint32_t>(options.max_block_size());\n  if (tail_ != nullptr) tail_->Clear();\n  associated_reader_.Reset();\n}\n\ninline void CordWriterBase::Initialize(absl::Cord* dest, bool append) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of CordWriter: null Cord pointer\";\n  if (append) {\n    cord_buffer_ = dest->GetAppendBuffer(0, 1);\n    set_start_pos(dest->size());\n    const size_t existing_length = cord_buffer_.length();\n    if (existing_length > 0) {\n      cord_buffer_.SetLength(\n          UnsignedMin(cord_buffer_.capacity(),\n                      std::numeric_limits<size_t>::max() - dest->size()));\n      set_buffer(cord_buffer_.data(), cord_buffer_.length(), existing_length);\n    }\n  } else {\n    cord_buffer_ = dest->GetAppendBuffer(0, 0);\n    dest->Clear();\n    cord_buffer_.SetLength(0);\n  }\n}\n\ninline void CordWriterBase::MoveCordBuffer(CordWriterBase& that) {\n  // Buffer pointers are already moved so `start()` is taken from `*this`.\n  // `cord_buffer_` is not moved yet so `cord_buffer_` is taken from `that`.\n  const bool uses_cord_buffer = start() == that.cord_buffer_.data();\n  if (uses_cord_buffer) {\n    RIEGELI_ASSERT_EQ(that.cord_buffer_.length(), start_to_limit())\n        << \"Failed invariant of CordWriter: \"\n           \"CordBuffer has an unexpected length\";\n  }\n  const size_t saved_start_to_cursor = start_to_cursor();\n  cord_buffer_ = std::move(that.cord_buffer_);\n  if (uses_cord_buffer) {\n    set_buffer(cord_buffer_.data(), cord_buffer_.length(),\n               saved_start_to_cursor);\n  }\n}\n\ntemplate <typename Dest>\ninline CordWriter<Dest>::CordWriter(Initializer<Dest> dest, Options options)\n    : CordWriterBase(options), dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.append());\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_same_v<DependentDest, absl::Cord>, int>>\ninline CordWriter<Dest>::CordWriter(Options options)\n    : CordWriter(riegeli::Maker(), std::move(options)) {}\n\ntemplate <typename Dest>\ninline void CordWriter<Dest>::Reset(Closed) {\n  CordWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void CordWriter<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  CordWriterBase::Reset(options);\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.append());\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_same_v<DependentDest, absl::Cord>, int>>\ninline void CordWriter<Dest>::Reset(Options options) {\n  Reset(riegeli::Maker(), std::move(options));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_CORD_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/fd_handle.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// Make `openat()` available.\n#if !defined(_POSIX_C_SOURCE) || _POSIX_C_SOURCE < 200809\n#undef _POSIX_C_SOURCE\n#define _POSIX_C_SOURCE 200809\n#endif\n\n#include \"riegeli/bytes/fd_handle.h\"\n\n#include <fcntl.h>\n#ifdef _WIN32\n#include <io.h>\n#include <share.h>\n#else\n#include <stddef.h>\n#include <unistd.h>\n#endif\n\n#include <cerrno>\n#ifndef _WIN32\n#include <utility>\n#endif\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/type_erased_ref.h\"\n#ifdef _WIN32\n#include \"riegeli/base/unicode.h\"\n#endif\n#include \"riegeli/bytes/path_ref.h\"\n\nnamespace riegeli {\n\nnamespace fd_internal {\n\ntemplate class FdBase<UnownedFdDeleter>;\ntemplate class FdBase<OwnedFdDeleter>;\n\n}  // namespace fd_internal\n\nint FdHandle::GetMethodDefault(ABSL_ATTRIBUTE_UNUSED TypeErasedRef target) {\n  return -1;\n}\n\nbool FdHandle::IsOwningMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target) {\n  return false;\n}\n\nabsl::string_view FdHandle::FilenameMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target) {\n  return kDefaultFilename;\n}\n\nabsl::Status FdHandle::CloseMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target) {\n  return absl::OkStatus();\n}\n\nabsl::Status OwnedFd::Open(PathInitializer filename, int mode,\n                           Permissions permissions) {\n  Reset(-1, std::move(filename));\n#ifndef _WIN32\nagain:\n  const int fd = open(c_filename(), mode, permissions);\n  if (ABSL_PREDICT_FALSE(fd < 0)) {\n    const int error_number = errno;\n    if (error_number == EINTR) goto again;\n    return Annotate(absl::ErrnoToStatus(error_number, \"open() failed\"),\n                    absl::StrCat(\"opening \", this->filename()));\n  }\n#else   // _WIN32\n  std::wstring filename_wide;\n  if (ABSL_PREDICT_FALSE(!Utf8ToWide(this->filename(), filename_wide))) {\n    return absl::InvalidArgumentError(\n        absl::StrCat(\"Filename not valid UTF-8: \", this->filename()));\n  }\n  int fd;\n  if (ABSL_PREDICT_FALSE(_wsopen_s(&fd, filename_wide.c_str(), mode, _SH_DENYNO,\n                                   permissions) != 0)) {\n    const int error_number = errno;\n    return Annotate(absl::ErrnoToStatus(error_number, \"_wsopen_s() failed\"),\n                    absl::StrCat(\"opening \", this->filename()));\n  }\n#endif  // _WIN32\n  SetFdKeepFilename(fd);\n  return absl::OkStatus();\n}\n\n#ifndef _WIN32\nabsl::Status OwnedFd::OpenAt(UnownedFd dir_fd, PathRef filename, int mode,\n                             Permissions permissions) {\n  absl::string_view dir_filename;\n  absl::string_view separator;\n  if (dir_fd != AT_FDCWD && (filename.empty() || filename.front() != '/')) {\n    dir_filename = dir_fd.filename();\n    if (!dir_filename.empty() && dir_filename.back() != '/') separator = \"/\";\n  }\n  Reset(-1, absl::StrCat(dir_filename, separator, filename));\n\nagain:\n  const int fd = openat(dir_fd.get(),\n                        c_filename() + dir_filename.size() + separator.size(),\n                        mode, permissions);\n  if (ABSL_PREDICT_FALSE(fd < 0)) {\n    const int error_number = errno;\n    if (error_number == EINTR) goto again;\n    return Annotate(absl::ErrnoToStatus(error_number, \"openat() failed\"),\n                    absl::StrCat(\"opening \", this->filename()));\n  }\n  SetFdKeepFilename(fd);\n  return absl::OkStatus();\n}\n#endif  // !_WIN32\n\nabsl::Status OwnedFd::Close() {\n  const int fd = Release();\n  if (fd < 0) return absl::OkStatus();\n#ifndef _WIN32\n  // http://austingroupbugs.net/view.php?id=529 explains this mess.\n#ifdef POSIX_CLOSE_RESTART\n  // Avoid `EINTR` by using `posix_close(_, 0)` if available.\n  if (ABSL_PREDICT_FALSE(posix_close(fd, 0) < 0)) {\n    const int error_number = errno;\n    if (error_number != EINPROGRESS) {\n      return Annotate(absl::ErrnoToStatus(error_number, \"posix_close() failed\"),\n                      absl::StrCat(\"closing \", filename()));\n    }\n  }\n#else   // !POSIX_CLOSE_RESTART\n  if (ABSL_PREDICT_FALSE(close(fd) < 0)) {\n    const int error_number = errno;\n    // After `EINTR` it is unspecified whether `fd` has been closed or not.\n    // Assume that it is closed, which is the case e.g. on Linux.\n    if (error_number != EINPROGRESS && error_number != EINTR) {\n      return Annotate(absl::ErrnoToStatus(error_number, \"close() failed\"),\n                      absl::StrCat(\"closing \", filename()));\n    }\n  }\n#endif  // !POSIX_CLOSE_RESTART\n#else   // _WIN32\n  if (ABSL_PREDICT_FALSE(_close(fd) < 0)) {\n    const int error_number = errno;\n    return Annotate(absl::ErrnoToStatus(error_number, \"_close() failed\"),\n                    absl::StrCat(\"closing \", filename()));\n  }\n#endif  // _WIN32\n  return absl::OkStatus();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/fd_handle.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_FD_HANDLE_H_\n#define RIEGELI_BYTES_FD_HANDLE_H_\n\n#ifdef _WIN32\n#include <sys/stat.h>\n#else\n#include <sys/types.h>\n#endif\n\n#ifdef _WIN32\n#include <io.h>\n#else\n#include <unistd.h>\n#endif\n\n#include <cstddef>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/any.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/shared_ptr.h\"\n#include \"riegeli/base/type_erased_ref.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/bytes/fd_internal.h\"\n#include \"riegeli/bytes/path_ref.h\"\n\nnamespace riegeli {\n\nnamespace fd_internal {\n\n#ifndef _WIN32\nusing Permissions = mode_t;\n#else\nusing Permissions = int;\n#endif\n\n}  // namespace fd_internal\n\nclass UnownedFd;\n\n// `SupportsFdHandle<T>::value` is `true` if `T&` is a valid constructor\n// argument for `FdHandle`.\n\ntemplate <typename T, typename Enable = void>\nstruct SupportsFdHandle : std::false_type {};\n\ntemplate <typename T>\nstruct SupportsFdHandle<\n    T, std::enable_if_t<std::conjunction_v<\n           std::negation<std::is_const<T>>,\n           std::is_convertible<decltype(std::declval<const T&>().get()), int>>>>\n    : std::true_type {};\n\n// `FdSupportsOpen<T>::value` is `true` if `T` supports `Open()` with the\n// signature like in `OwnedFd`, but `permissions` can be required.\n\ntemplate <typename T, typename Enable = void>\nstruct FdSupportsOpen : std::false_type {};\n\ntemplate <typename T>\nstruct FdSupportsOpen<\n    T, std::enable_if_t<std::is_convertible_v<\n           decltype(std::declval<T&>().Open(\n               std::declval<PathInitializer>(), std::declval<int>(),\n               std::declval<fd_internal::Permissions>())),\n           absl::Status>>> : std::true_type {};\n\n// `FdSupportsOpenAt<T>::value` is `true` if `T` supports `OpenAt()` with the\n// signature like in `OwnedFd` (with `permissions` present).\n\ntemplate <typename T, typename Enable = void>\nstruct FdSupportsOpenAt : std::false_type {};\n\ntemplate <typename T>\nstruct FdSupportsOpenAt<\n    T, std::enable_if_t<std::is_convertible_v<\n           decltype(std::declval<T&>().OpenAt(\n               std::declval<UnownedFd>(), std::declval<PathRef>(),\n               std::declval<int>(), std::declval<fd_internal::Permissions>())),\n           absl::Status>>> : std::true_type {};\n\n// Type-erased pointer to a target object like `UnownedFd` or `OwnedFd` which\n// stores and possibly owns a fd.\n//\n// The target should support:\n//\n// ```\n//   // Returns the fd.\n//   int get() const;\n//\n//   // Returns `true` if the target owns the fd, i.e. is responsible for\n//   // closing it and the fd is present.\n//   //\n//   // Optional. If absent, the presence of `Close()` determines whether the\n//   // target is considered to own the fd.\n//   bool IsOwning() const;\n//\n//   // Opens a new fd, like with `open()`, but taking\n//   // `PathInitializer filename` instead of `const char* filename` and\n//   // returning `absl::Status` instead of `int`.\n//   //\n//   // Optional. Not used by `FdHandle` itself. Used by `FdReader` and\n//   // `FdWriter` constructors from the filename.\n//   absl::Status Open(PathInitializer filename, int mode,\n//                     OwnedFd::Permissions permissions);\n//\n//   // Returns the filename of the fd, or \"<none>\" for\n//   // default-constructed or moved-from target. Unchanged by `Close()`.\n//   //\n//   // If `Open()` was used, this is the filename passed to `Open()`, otherwise\n//   // a filename is inferred from the fd. This can be a placeholder instead of\n//   // a real filename if the fd does not refer to a named file or inferring\n//   // the filename is not supported.\n//   //\n//   // Optional. If absent, \"<unsupported>\" is assumed.\n//   absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n//\n//   // If `IsOwning()`, closes the fd.\n//   //\n//   // If `!IsOwning()`, does nothing and returns `absl::OkStatus()`.\n//   //\n//   // Optional. If absent, `absl::OkStatus()` is assumed.\n//   absl::Status Close();\n// ```\nclass FdHandle : public WithEqual<FdHandle> {\n public:\n  // Creates an `FdHandle` which does not refer to a target.\n  FdHandle() = default;\n  /*implicit*/ FdHandle(std::nullptr_t) {}\n\n  // Creates an `FdHandle` which refers to `target`.\n  template <typename T,\n            std::enable_if_t<std::conjunction_v<NotSameRef<FdHandle, T&>,\n                                                SupportsFdHandle<T>>,\n                             int> = 0>\n  /*implicit*/ FdHandle(T& target ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : methods_(&kMethods<T>), target_(target) {}\n\n  FdHandle(const FdHandle& that) = default;\n  FdHandle& operator=(const FdHandle& that) = default;\n\n  // Returns `true` if the fd is present.\n  bool is_open() const { return *this != nullptr; }\n\n  // Returns the fd.\n  int get() const { return methods_->get(target_); }\n\n  // Returns `true` if the `FdHandle` owns the fd, i.e. is responsible for\n  // closing it and the fd is present.\n  bool IsOwning() const { return methods_->is_owning(target_); }\n\n  // Returns the filename of the fd, or \"<none>\" for default-constructed or\n  // moved-from target. Unchanged by `Close()`.\n  //\n  // If `Open()` was used, this is the filename passed to `Open()`, otherwise a\n  // filename is inferred from the fd. This can be a placeholder instead of a\n  // real filename if the fd does not refer to a named file or inferring the\n  // filename is not supported.\n  //\n  // If the target does not support `filename()`, returns \"<unsupported>\".\n  absl::string_view filename() const { return methods_->filename(target_); }\n\n  // If `IsOwning()`, closes the fd.\n  //\n  // If `!IsOwning()`, does nothing and returns `absl::OkStatus()`.\n  absl::Status Close() { return methods_->close(target_); }\n\n  friend bool operator==(FdHandle a, FdHandle b) { return a.get() == b.get(); }\n  friend bool operator==(FdHandle a, int b) { return a.get() == b; }\n  friend bool operator==(FdHandle a, std::nullptr_t) {\n    return a.target_.empty() || a.get() < 0;\n  }\n\n private:\n  struct Methods {\n    int (*get)(TypeErasedRef target);\n    bool (*is_owning)(TypeErasedRef target);\n    absl::string_view (*filename)(TypeErasedRef target);\n    absl::Status (*close)(TypeErasedRef target);\n  };\n\n  template <typename T, typename Enable = void>\n  struct HasIsOwning : std::false_type {};\n  template <typename T>\n  struct HasIsOwning<T,\n                     std::enable_if_t<std::is_convertible_v<\n                         decltype(std::declval<const T&>().IsOwning()), bool>>>\n      : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct HasFilename : std::false_type {};\n  template <typename T>\n  struct HasFilename<\n      T, std::enable_if_t<std::is_convertible_v<\n             decltype(std::declval<const T&>().filename()), absl::string_view>>>\n      : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct HasClose : std::false_type {};\n  template <typename T>\n  struct HasClose<T, std::enable_if_t<std::is_convertible_v<\n                         decltype(std::declval<T&>().Close()), absl::Status>>>\n      : std::true_type {};\n\n  static int GetMethodDefault(TypeErasedRef target);\n  static bool IsOwningMethodDefault(TypeErasedRef target);\n  static absl::string_view FilenameMethodDefault(TypeErasedRef target);\n  static absl::Status CloseMethodDefault(TypeErasedRef target);\n\n  static constexpr Methods kMethodsDefault = {\n      GetMethodDefault, IsOwningMethodDefault, FilenameMethodDefault,\n      CloseMethodDefault};\n\n  template <typename T>\n  static int GetMethod(TypeErasedRef target) {\n    return target.Cast<const T&>().get();\n  }\n\n  template <typename T>\n  static bool IsOwningMethod(TypeErasedRef target) {\n    if constexpr (HasIsOwning<T>::value) {\n      return target.Cast<const T&>().IsOwning();\n    } else if constexpr (HasClose<T>::value) {\n      return target.Cast<const T&>().get() >= 0;\n    } else {\n      return false;\n    }\n  }\n\n  template <typename T>\n  static absl::string_view FilenameMethod(TypeErasedRef target) {\n    if constexpr (HasFilename<T>::value) {\n      return target.Cast<const T&>().filename();\n    } else {\n      return \"<unsupported>\";\n    }\n  }\n\n  template <typename T>\n  static absl::Status CloseMethod(TypeErasedRef target) {\n    if constexpr (HasClose<T>::value) {\n      return target.Cast<T&>().Close();\n    } else {\n      return absl::OkStatus();\n    }\n  }\n\n  template <typename T>\n  static constexpr Methods kMethods = {GetMethod<T>, IsOwningMethod<T>,\n                                       FilenameMethod<T>, CloseMethod<T>};\n\n  const Methods* methods_ = &kMethodsDefault;\n  TypeErasedRef target_;\n};\n\nnamespace fd_internal {\n\nclass UnownedFdDeleter;\n\n// Common parts of `UnownedFdDeleter` and `OwnedFdDeleter`.\nclass ABSL_ATTRIBUTE_TRIVIAL_ABI FdDeleterBase {\n public:\n  FdDeleterBase() = default;\n\n  explicit FdDeleterBase(PathInitializer filename)\n      : metadata_(riegeli::Maker<Metadata>(std::move(filename))) {}\n\n  // Supports creating a `FdBase` converted from `UnownedFd`.\n  explicit FdDeleterBase(const UnownedFdDeleter& that);\n  explicit FdDeleterBase(UnownedFdDeleter&& that);\n\n  void Reset() { metadata_ = nullptr; }\n\n  void Reset(PathInitializer filename) {\n    if (!metadata_.IsUnique()) {\n      metadata_.Reset(riegeli::Maker<Metadata>(std::move(filename)));\n    } else {\n      riegeli::Reset(metadata_->filename, std::move(filename));\n    }\n  }\n\n  // Supports creating a `FdBase` converted from `UnownedFd`.\n  void Reset(const UnownedFdDeleter& that);\n\n  // Supports creating a `FdBase` converted from `UnownedFd`, and resetting\n  // `FdBase` from the same `FdBase`.\n  void Reset(FdDeleterBase&& that);\n\n  absl::string_view filename() const {\n    if (ABSL_PREDICT_FALSE(metadata_ == nullptr)) return kDefaultFilename;\n    return metadata_->filename;\n  }\n\n  const char* c_filename() const {\n    if (ABSL_PREDICT_FALSE(metadata_ == nullptr)) return kDefaultFilenameCStr;\n    return metadata_->filename.c_str();\n  }\n\n protected:\n  FdDeleterBase(const FdDeleterBase& that) = default;\n  FdDeleterBase& operator=(const FdDeleterBase& that) = default;\n\n  FdDeleterBase(FdDeleterBase&& that) = default;\n  FdDeleterBase& operator=(FdDeleterBase&& that) = default;\n\n private:\n  struct Metadata {\n    explicit Metadata(PathInitializer filename)\n        : filename(std::move(filename)) {}\n\n    std::string filename;\n  };\n\n  // `nullptr` means `filename = kDefaultFilename`.\n  SharedPtr<Metadata> metadata_;\n};\n\nclass UnownedFdDeleter : public FdDeleterBase {\n public:\n  using FdDeleterBase::FdDeleterBase;\n\n  // Supports creating an `UnownedFd` converted from any `FdBase`.\n  explicit UnownedFdDeleter(const FdDeleterBase& that) : FdDeleterBase(that) {}\n\n  UnownedFdDeleter(const UnownedFdDeleter& that) = default;\n  UnownedFdDeleter& operator=(const UnownedFdDeleter& that) = default;\n\n  UnownedFdDeleter(UnownedFdDeleter&& that) = default;\n  UnownedFdDeleter& operator=(UnownedFdDeleter&& that) = default;\n\n  using FdDeleterBase::Reset;\n  // Supports creating an `UnownedFd` converted from any `FdBase`.\n  void Reset(const FdDeleterBase& that) { FdDeleterBase::operator=(that); }\n\n  static void Destroy(ABSL_ATTRIBUTE_UNUSED int fd) {}\n};\n\nclass OwnedFdDeleter : public FdDeleterBase {\n public:\n  using FdDeleterBase::FdDeleterBase;\n\n  OwnedFdDeleter(OwnedFdDeleter&& that) = default;\n  OwnedFdDeleter& operator=(OwnedFdDeleter&& that) = default;\n\n  static void Destroy(int fd) {\n#ifndef _WIN32\n    // http://austingroupbugs.net/view.php?id=529 explains this mess.\n#ifdef POSIX_CLOSE_RESTART\n    // Avoid `EINTR` by using `posix_close(_, 0)` if available.\n    posix_close(fd, 0);\n#else   // !POSIX_CLOSE_RESTART\n    close(fd);\n#endif  // !POSIX_CLOSE_RESTART\n#else   // _WIN32\n    _close(fd);\n#endif  // _WIN32\n  }\n};\n\ninline FdDeleterBase::FdDeleterBase(const UnownedFdDeleter& that)\n    : metadata_(that.metadata_) {}\n\ninline FdDeleterBase::FdDeleterBase(UnownedFdDeleter&& that)\n    : metadata_(std::move(that.metadata_)) {}\n\ninline void FdDeleterBase::Reset(const UnownedFdDeleter& that) {\n  metadata_ = that.metadata_;\n}\n\ninline void FdDeleterBase::Reset(FdDeleterBase&& that) {\n  metadata_ = std::move(that.metadata_);\n}\n\n// Common parts of `UnownedFd` and `OwnedFd`.\ntemplate <typename Deleter>\nclass ABSL_ATTRIBUTE_TRIVIAL_ABI FdBase {\n public:\n  // Creates an `FdBase` which does not store a fd and stores \"<none>\" as the\n  // filename.\n  FdBase() = default;\n  /*implicit*/ FdBase(std::nullptr_t) {}\n\n  // Creates an `FdBase` which stores `fd` with the filename inferred from the\n  // fd (or \"<none>\" if `fd < 0`).\n  explicit FdBase(int fd ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : fd_(fd), deleter_(fd_ < 0 ? Deleter() : Deleter(FilenameForFd(fd_))) {}\n\n  // Creates an `FdBase` which stores `fd` with `filename`.\n  explicit FdBase(int fd ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                  PathInitializer filename)\n      : fd_(fd), deleter_(std::move(filename)) {}\n\n  // Creates a `FdBase` converted from `UnownedFd`.\n  template <typename DependentDeleter = Deleter,\n            std::enable_if_t<\n                !std::is_same_v<DependentDeleter, UnownedFdDeleter>, int> = 0>\n  explicit FdBase(const FdBase<UnownedFdDeleter>& that)\n      : fd_(that.fd_), deleter_(that.deleter_) {}\n  template <typename DependentDeleter = Deleter,\n            std::enable_if_t<\n                !std::is_same_v<DependentDeleter, UnownedFdDeleter>, int> = 0>\n  explicit FdBase(FdBase<UnownedFdDeleter>&& that)\n      : fd_(that.Release()), deleter_(std::move(that.deleter_)) {}\n\n  // Creates an `UnownedFd` converted from any `FdBase`.\n  template <\n      typename OtherDeleter,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::is_same<Deleter, UnownedFdDeleter>,\n              std::negation<std::is_same<OtherDeleter, UnownedFdDeleter>>>,\n          int> = 0>\n  /*implicit*/ FdBase(\n      const FdBase<OtherDeleter>& that ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : fd_(that.fd_), deleter_(that.deleter_) {}\n\n  // Makes `*this` equivalent to a newly constructed `FdBase`.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(std::nullptr_t = nullptr) {\n    SetFdKeepFilename();\n    deleter_.Reset();\n  }\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(int fd) {\n    SetFdKeepFilename(fd);\n    if (fd < 0) {\n      deleter_.Reset();\n    } else {\n      deleter_.Reset(FilenameForFd(fd));\n    }\n  }\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(int fd, PathInitializer filename) {\n    SetFdKeepFilename(fd);\n    deleter_.Reset(std::move(filename));\n  }\n  template <typename OtherDeleter,\n            std::enable_if_t<std::disjunction_v<\n                                 std::is_same<Deleter, UnownedFdDeleter>,\n                                 std::is_same<OtherDeleter, UnownedFdDeleter>>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(const FdBase<OtherDeleter>& that) {\n    SetFdKeepFilename(that.fd_);\n    deleter_.Reset(that.deleter_);\n  }\n  template <typename OtherDeleter,\n            std::enable_if_t<\n                std::disjunction_v<std::is_same<OtherDeleter, UnownedFdDeleter>,\n                                   std::is_same<OtherDeleter, Deleter>>,\n                int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(FdBase<OtherDeleter>&& that) {\n    SetFdKeepFilename(that.Release());\n    deleter_.Reset(std::move(that.deleter_));\n  }\n\n  // Sets the fd, keeping `filename()` unchanged.\n  void SetFdKeepFilename(int fd = -1) {\n    Destroy();\n    fd_ = fd;\n  }\n\n  // Returns `true` if the fd is present.\n  bool is_open() const { return fd_ >= 0; }\n\n  // Returns the fd.\n  int get() const { return fd_; }\n\n  // Returns the filename of the fd, or \"<none>\" for default-constructed or\n  // moved-from `FdBase`. Unchanged by `Close()` and `Release()`.\n  //\n  // If `Open()` was used, this is the filename passed to `Open()`, otherwise\n  // a filename is inferred from the fd. This can be a placeholder instead of\n  // a real filename if the fd does not refer to a named file or inferring the\n  // filename is not supported.\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return deleter_.filename();\n  }\n\n  // Returns `filename()` as a NUL-terminated string.\n  const char* c_filename() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return deleter_.c_filename();\n  }\n\n protected:\n  FdBase(const FdBase& that) = default;\n  FdBase& operator=(const FdBase& that) = default;\n\n  FdBase(FdBase&& that) noexcept\n      : fd_(that.Release()), deleter_(std::move(that.deleter_)) {}\n  FdBase& operator=(FdBase&& that) noexcept {\n    const int fd = that.Release();\n    Destroy();\n    fd_ = fd;\n    deleter_ = std::move(that.deleter_);\n    return *this;\n  }\n\n  ~FdBase() { Destroy(); }\n\n  // Returns the fd. The stored fd is left absent, without modifying\n  // `filename()`.\n  int Release() { return std::exchange(fd_, -1); }\n\n private:\n  template <typename OtherDeleter>\n  friend class FdBase;  // For conversions.\n\n  void Destroy() {\n    if (is_open()) deleter_.Destroy(fd_);\n  }\n\n  int fd_ = -1;\n  Deleter deleter_;\n};\n\nextern template class FdBase<UnownedFdDeleter>;\nextern template class FdBase<OwnedFdDeleter>;\n\n}  // namespace fd_internal\n\n// Stores a file descriptor but does not own it, i.e. is not responsible for\n// closing it.\n//\n// The fd can be negative which means absent.\nclass UnownedFd : public fd_internal::FdBase<fd_internal::UnownedFdDeleter>,\n                  public WithEqual<UnownedFd> {\n public:\n  using FdBase::FdBase;\n\n  // Overridden to make implicit.\n  /*implicit*/ UnownedFd(int fd ABSL_ATTRIBUTE_LIFETIME_BOUND) : FdBase(fd) {}\n\n  // Creates an `UnownedFd` which stores `fd.get()` with `fd.filename()`.\n  explicit UnownedFd(FdHandle fd) : FdBase(fd.get(), fd.filename()) {}\n\n  UnownedFd(const UnownedFd& that) = default;\n  UnownedFd& operator=(const UnownedFd& that) = default;\n\n  // The moved-from fd is left absent.\n  UnownedFd(UnownedFd&& that) = default;\n  UnownedFd& operator=(UnownedFd&& that) = default;\n\n  using FdBase::Reset;\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(FdHandle fd) {\n    Reset(fd.get(), fd.filename());\n  }\n\n  friend bool operator==(const UnownedFd& a, const UnownedFd& b) {\n    return a.get() == b.get();\n  }\n  friend bool operator==(const UnownedFd& a, int b) { return a.get() == b; }\n  friend bool operator==(const UnownedFd& a, std::nullptr_t) {\n    return a.get() < 0;\n  }\n};\n\n// Owns a file descriptor, i.e. stores it and is responsible for closing it.\n//\n// The fd can be negative which means absent.\nclass OwnedFd : public fd_internal::FdBase<fd_internal::OwnedFdDeleter>,\n                public WithEqual<OwnedFd> {\n public:\n  using Permissions = fd_internal::Permissions;\n#ifndef _WIN32\n  static constexpr Permissions kDefaultPermissions = 0666;\n#else\n  static constexpr Permissions kDefaultPermissions = _S_IREAD | _S_IWRITE;\n#endif\n\n  using FdBase::FdBase;\n\n  // The moved-from fd is left absent.\n  OwnedFd(OwnedFd&& that) = default;\n  OwnedFd& operator=(OwnedFd&& that) = default;\n\n  // Overridden to apply `ABSL_ATTRIBUTE_LIFETIME_BOUND`.\n  int get() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return FdBase::get(); }\n\n  using FdBase::Release;\n\n  // Opens a new fd, like with `open()`, but taking `PathInitializer filename`\n  // instead of `const char* filename` and returning `absl::Status` instead of\n  // `int`.\n  ABSL_ATTRIBUTE_REINITIALIZES absl::Status Open(\n      PathInitializer filename, int mode,\n      Permissions permissions = kDefaultPermissions);\n\n#ifndef _WIN32\n  // Opens a new fd with the filename interpreted relatively to the directory\n  // specified by an existing fd, like with `openat()`, but taking\n  // `PathRef filename` instead of `const char* filename` and returning\n  // `absl::Status` instead of `int`.\n  ABSL_ATTRIBUTE_REINITIALIZES absl::Status OpenAt(\n      UnownedFd dir_fd, PathRef filename, int mode,\n      Permissions permissions = kDefaultPermissions);\n#endif  // !_WIN32\n\n  // Closes the fd if present.\n  //\n  // Returns `absl::OkStatus()` if absent.\n  absl::Status Close();\n\n  friend bool operator==(const OwnedFd& a, int b) { return a.get() == b; }\n  friend bool operator==(const OwnedFd& a, std::nullptr_t) {\n    return a.get() < 0;\n  }\n};\n\n// Type-erased object like `UnownedFd` or `OwnedFd` which stores and possibly\n// owns a fd.\nusing AnyFd = Any<FdHandle>::Inlining<UnownedFd, OwnedFd>;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_FD_HANDLE_H_\n"
  },
  {
    "path": "riegeli/bytes/fd_internal.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef _WIN32\n\n// Make `readlink()` available, and make `O_CLOEXEC` available on Darwin.\n#if !defined(_XOPEN_SOURCE) || _XOPEN_SOURCE < 700\n#undef _XOPEN_SOURCE\n#define _XOPEN_SOURCE 700\n#endif\n\n#endif\n\n#include \"riegeli/bytes/fd_internal.h\"\n\n#ifdef __APPLE__\n#include <fcntl.h>\n#endif\n#ifndef _WIN32\n#include <stddef.h>\n#include <unistd.h>\n#endif\n\n#include <string>\n\n#ifndef _WIN32\n#include \"absl/base/optimization.h\"\n#endif\n#include \"absl/strings/str_cat.h\"\n#ifndef _WIN32\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/buffer.h\"\n#endif\n\nnamespace riegeli::fd_internal {\n\nstd::string FilenameForFd(int fd) {\n#ifndef _WIN32\n  std::string filename = absl::StrCat(\"/proc/self/fd/\", fd);\n  Buffer buffer(PATH_MAX);\n  const ssize_t length = readlink(filename.c_str(), buffer.data(), PATH_MAX);\n  if (ABSL_PREDICT_TRUE(length >= 0)) {\n    filename.assign(buffer.data(), IntCast<size_t>(length));\n  }\n  return filename;\n#else   // _WIN32\n  return absl::StrCat(\"<fd \", fd, \">\");\n#endif  // _WIN32\n}\n\n#ifdef __APPLE__\n// On Darwin `O_CLOEXEC` is available conditionally, so `kCloseOnExec` is\n// defined out of line.\nextern const int kCloseOnExec = O_CLOEXEC;\n#endif  // __APPLE__\n\n}  // namespace riegeli::fd_internal\n"
  },
  {
    "path": "riegeli/bytes/fd_internal.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_FD_INTERNAL_H_\n#define RIEGELI_BYTES_FD_INTERNAL_H_\n\n#ifndef __APPLE__\n#include <fcntl.h>\n#endif\n\n#include <string>\n\nnamespace riegeli::fd_internal {\n\n// Infers a filename from fd by reading the symlink target for\n// `absl::StrCat(\"/proc/self/fd/\", fd)` (on Windows returns a\n// `absl::StrCat(\"<fd \", fd, \">\")` placeholder instead).\nstd::string FilenameForFd(int fd);\n\n#ifndef _WIN32\n#ifndef __APPLE__\ninline constexpr int kCloseOnExec = O_CLOEXEC;\n#else   // __APPLE__\n// On Darwin `O_CLOEXEC` is available conditionally, so `kCloseOnExec` is\n// defined out of line.\nextern const int kCloseOnExec;\n#endif  // __APPLE__\n#else   // _WIN32\ninline constexpr int kCloseOnExec = _O_NOINHERIT;\n#endif  // _WIN32\n\n}  // namespace riegeli::fd_internal\n\n#endif  // RIEGELI_BYTES_FD_INTERNAL_H_\n"
  },
  {
    "path": "riegeli/bytes/fd_internal_for_cc.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_FD_INTERNAL_FOR_CC_H_\n#define RIEGELI_BYTES_FD_INTERNAL_FOR_CC_H_\n\n// Warning: Do not include this header in other headers, because the definition\n// of `off_t` depends on `_FILE_OFFSET_BITS` which can reliably be set only\n// in a standalone compilation unit.\n\n#ifdef _WIN32\n#include <io.h>\n#endif\n#include <sys/stat.h>\n#include <sys/types.h>\n#ifndef _WIN32\n#include <unistd.h>\n#endif\n\n#include \"absl/strings/string_view.h\"\n\nnamespace riegeli::fd_internal {\n\n#ifndef _WIN32\n\nusing Offset = off_t;\n\ninline Offset LSeek(int fd, Offset offset, int whence) {\n  return lseek(fd, offset, whence);\n}\n\ninline constexpr absl::string_view kLSeekFunctionName = \"lseek()\";\n\nusing StatInfo = struct stat;\n\ninline int FStat(int fd, StatInfo* stat_info) { return fstat(fd, stat_info); }\n\ninline constexpr absl::string_view kFStatFunctionName = \"fstat()\";\n\n#else  // _WIN32\n\nusing Offset = __int64;\n\ninline Offset LSeek(int fd, Offset offset, int whence) {\n  return _lseeki64(fd, offset, whence);\n}\n\ninline constexpr absl::string_view kLSeekFunctionName = \"_lseeki64()\";\n\n// `struct __stat64` in a namespace does not work in MSVC due to a bug regarding\n// https://en.cppreference.com/w/cpp/language/elaborated_type_specifier.\nusing StatInfo = struct ::__stat64;\n\ninline int FStat(int fd, StatInfo* stat_info) {\n  return _fstat64(fd, stat_info);\n}\n\ninline constexpr absl::string_view kFStatFunctionName = \"_fstat64()\";\n\n#endif  // _WIN32\n\n}  // namespace riegeli::fd_internal\n\n#endif  // RIEGELI_BYTES_FD_INTERNAL_FOR_CC_H_\n"
  },
  {
    "path": "riegeli/bytes/fd_mmap_reader.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef _WIN32\n\n// Make `posix_fadvise()` available.\n#if !defined(_XOPEN_SOURCE) || _XOPEN_SOURCE < 600\n#undef _XOPEN_SOURCE\n#define _XOPEN_SOURCE 600\n#endif\n\n// Make `off_t` 64-bit even on 32-bit systems.\n#undef _FILE_OFFSET_BITS\n#define _FILE_OFFSET_BITS 64\n\n#else\n\n#define WIN32_LEAN_AND_MEAN\n\n#endif\n\n#include \"riegeli/bytes/fd_mmap_reader.h\"\n\n#include <fcntl.h>\n#ifdef _WIN32\n#include <io.h>\n#endif\n#include <stddef.h>\n#ifdef _WIN32\n#include <stdint.h>\n#endif\n#include <stdio.h>\n#ifndef _WIN32\n#include <sys/mman.h>\n#endif\n#include <sys/types.h>\n#ifndef _WIN32\n#include <unistd.h>\n#else\n#include <windows.h>\n#endif\n\n#include <cerrno>\n#include <limits>\n#include <memory>\n#include <optional>\n#include <ostream>\n#ifndef _WIN32\n#include <type_traits>\n#endif\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#ifndef _WIN32\n#include \"absl/status/statusor.h\"\n#endif\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#ifdef _WIN32\n#include \"riegeli/base/errno_mapping.h\"\n#endif\n#include \"riegeli/base/external_ref.h\"\n#ifndef _WIN32\n#include \"riegeli/base/global.h\"\n#endif\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/fd_handle.h\"\n#include \"riegeli/bytes/fd_internal_for_cc.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nnamespace {\n\n#ifdef _WIN32\n\nstruct HandleDeleter {\n  void operator()(void* handle) const {\n    RIEGELI_CHECK(CloseHandle(reinterpret_cast<HANDLE>(handle)))\n        << WindowsErrorToStatus(IntCast<uint32_t>(GetLastError()),\n                                \"CloseHandle() failed\")\n               .message();\n  }\n};\n\nusing UniqueHandle = std::unique_ptr<void, HandleDeleter>;\n\n#endif  // _WIN32\n\n#ifndef _WIN32\n\ninline absl::StatusOr<Position> GetPageSize() {\n  const long page_size = sysconf(_SC_PAGE_SIZE);\n  if (ABSL_PREDICT_FALSE(page_size < 0)) {\n    return absl::ErrnoToStatus(errno, \"sysconf() failed\");\n  }\n  return IntCast<Position>(page_size);\n}\n\n#else  // _WIN32\n\ninline Position GetPageSize() {\n  SYSTEM_INFO system_info;\n  GetSystemInfo(&system_info);\n  return IntCast<Position>(system_info.dwAllocationGranularity);\n}\n\n#endif  // _WIN32\n\n#ifndef _WIN32\n\n// `posix_fadvise()` is supported by POSIX systems but not MacOS.\n\ntemplate <typename DependentInt, typename Enable = void>\nstruct HavePosixFadvise : std::false_type {};\n\ntemplate <typename DependentInt>\nstruct HavePosixFadvise<\n    DependentInt,\n    std::void_t<decltype(posix_fadvise(\n        std::declval<DependentInt>(), std::declval<fd_internal::Offset>(),\n        std::declval<fd_internal::Offset>(), std::declval<int>()))>>\n    : std::true_type {};\n\ntemplate <typename DependentInt>\ninline void FdSetReadAllHint(ABSL_ATTRIBUTE_UNUSED DependentInt src,\n                             ABSL_ATTRIBUTE_UNUSED bool read_all_hint) {\n  if constexpr (HavePosixFadvise<DependentInt>::value) {\n#ifdef POSIX_FADV_SEQUENTIAL\n    posix_fadvise(src, 0, 0,\n                  read_all_hint ? POSIX_FADV_SEQUENTIAL : POSIX_FADV_NORMAL);\n#endif\n  }\n}\n\n#endif  // !_WIN32\n\nclass MMapBlock {\n public:\n  explicit MMapBlock(const char* addr) : addr_(addr) {}\n\n  MMapBlock(MMapBlock&& that) = default;\n  MMapBlock& operator=(MMapBlock&& that) = default;\n\n  // Indicates support for `ExternalRef(MMapBlock&&, substr)`.\n  friend void RiegeliSupportsExternalRef(MMapBlock*) {}\n\n  // Supports `ExternalRef`, `Chain::Block`, and `absl::MakeCordFromExternal()`.\n  void operator()(absl::string_view data) const;\n\n  // Supports `ExternalRef` and `Chain::Block`.\n  friend void RiegeliDumpStructure(ABSL_ATTRIBUTE_UNUSED const MMapBlock* self,\n                                   std::ostream& dest) {\n    dest << \"[mmap] { }\";\n  }\n\n private:\n  const char* addr_;\n};\n\nvoid MMapBlock::operator()(absl::string_view data) const {\n#ifndef _WIN32\n  RIEGELI_CHECK_EQ(munmap(const_cast<char*>(addr_),\n                          data.size() + PtrDistance(addr_, data.data())),\n                   0)\n      << absl::ErrnoToStatus(errno, \"munmap() failed\").message();\n#else   // _WIN32\n  RIEGELI_CHECK(UnmapViewOfFile(addr_))\n      << WindowsErrorToStatus(IntCast<uint32_t>(GetLastError()),\n                              \"UnmapViewOfFile() failed\")\n             .message();\n#endif  // _WIN32\n}\n\n}  // namespace\n\nvoid FdMMapReaderBase::Initialize(int src, Options&& options) {\n  RIEGELI_ASSERT_GE(src, 0)\n      << \"Failed precondition of FdMMapReader: negative file descriptor\";\n  InitializePos(src, std::move(options));\n}\n\nvoid FdMMapReaderBase::InitializePos(int src, Options&& options) {\n  Position initial_pos;\n  if (options.independent_pos() != std::nullopt) {\n    initial_pos = *options.independent_pos();\n  } else {\n    const fd_internal::Offset file_pos = fd_internal::LSeek(src, 0, SEEK_CUR);\n    if (ABSL_PREDICT_FALSE(file_pos < 0)) {\n      FailOperation(fd_internal::kLSeekFunctionName);\n      return;\n    }\n    initial_pos = IntCast<Position>(file_pos);\n  }\n\n  fd_internal::StatInfo stat_info;\n  if (ABSL_PREDICT_FALSE(fd_internal::FStat(src, &stat_info) < 0)) {\n    FailOperation(fd_internal::kFStatFunctionName);\n    return;\n  }\n  Position base_pos = 0;\n  Position length = IntCast<Position>(stat_info.st_size);\n  if (options.max_length() != std::nullopt) {\n    base_pos = initial_pos;\n    length =\n        UnsignedMin(SaturatingSub(length, initial_pos), *options.max_length());\n  }\n  if (options.independent_pos() == std::nullopt) base_pos_to_sync_ = base_pos;\n  if (length == 0) {\n    // The `Chain` to read from was not known in `FdMMapReaderBase` constructor.\n    // Set it now to empty.\n    ChainReader::Reset(riegeli::Maker());\n    return;\n  }\n\n  Position rounded_base_pos = base_pos;\n  if (rounded_base_pos > 0) {\n#ifndef _WIN32\n    const absl::StatusOr<Position>& page_size =\n        Global([] { return GetPageSize(); });\n    if (ABSL_PREDICT_FALSE(!page_size.ok())) {\n      Fail(page_size.status());\n      return;\n    }\n    rounded_base_pos &= ~(*page_size - 1);\n#else   // _WIN32\n    static const Position kPageSize = GetPageSize();\n    rounded_base_pos &= ~(kPageSize - 1);\n#endif  // _WIN32\n  }\n  const Position rounding = base_pos - rounded_base_pos;\n  const Position rounded_length = length + rounding;\n  if (ABSL_PREDICT_FALSE(rounded_length > std::numeric_limits<size_t>::max())) {\n    Fail(absl::OutOfRangeError(\"File too large for memory mapping\"));\n    return;\n  }\n#ifndef _WIN32\n  void* const addr = mmap(nullptr, IntCast<size_t>(rounded_length), PROT_READ,\n                          MAP_SHARED, src, IntCast<off_t>(rounded_base_pos));\n  if (ABSL_PREDICT_FALSE(addr == MAP_FAILED)) {\n    FailOperation(\"mmap()\");\n    return;\n  }\n#else   // _WIN32\n  const HANDLE file_handle = reinterpret_cast<HANDLE>(_get_osfhandle(src));\n  if (ABSL_PREDICT_FALSE(file_handle == INVALID_HANDLE_VALUE ||\n                         file_handle == reinterpret_cast<HANDLE>(-2))) {\n    FailWindowsOperation(\"_get_osfhandle()\");\n    return;\n  }\n  UniqueHandle memory_handle(reinterpret_cast<void*>(\n      CreateFileMappingW(file_handle, nullptr, PAGE_READONLY, 0, 0, nullptr)));\n  if (ABSL_PREDICT_FALSE(memory_handle == nullptr)) {\n    FailWindowsOperation(\"CreateFileMappingW()\");\n    return;\n  }\n  void* const addr =\n      MapViewOfFile(reinterpret_cast<HANDLE>(memory_handle.get()),\n                    FILE_MAP_READ, IntCast<DWORD>(rounded_base_pos >> 32),\n                    IntCast<DWORD>(rounded_base_pos & 0xffffffff),\n                    IntCast<size_t>(rounded_length));\n  if (ABSL_PREDICT_FALSE(addr == nullptr)) {\n    FailWindowsOperation(\"MapViewOfFile()\");\n    return;\n  }\n#endif  // _WIN32\n\n  // The `Chain` to read from was not known in `FdMMapReaderBase` constructor.\n  // Set it now.\n  ChainReader::Reset(riegeli::Maker(\n      ExternalRef(riegeli::Maker<MMapBlock>(static_cast<const char*>(addr)),\n                  absl::string_view(static_cast<const char*>(addr) + rounding,\n                                    IntCast<size_t>(length)))));\n  if (options.max_length() == std::nullopt) Seek(initial_pos);\n}\n\nvoid FdMMapReaderBase::Done() {\n  FdMMapReaderBase::SyncImpl(SyncType::kFromObject);\n  ChainReader::Done();\n  ChainReader::src().Clear();\n}\n\nbool FdMMapReaderBase::FailOperation(absl::string_view operation) {\n  const int error_number = errno;\n  RIEGELI_ASSERT_NE(error_number, 0)\n      << \"Failed precondition of FdMMapReaderBase::FailOperation(): \"\n         \"zero errno\";\n  return Fail(\n      absl::ErrnoToStatus(error_number, absl::StrCat(operation, \" failed\")));\n}\n\n#ifdef _WIN32\n\nbool FdMMapReaderBase::FailWindowsOperation(absl::string_view operation) {\n  const DWORD error_number = GetLastError();\n  RIEGELI_ASSERT_NE(error_number, 0)\n      << \"Failed precondition of FdMMapReaderBase::FailWindowsOperation(): \"\n         \"zero error code\";\n  return Fail(WindowsErrorToStatus(IntCast<uint32_t>(error_number),\n                                   absl::StrCat(operation, \" failed\")));\n}\n\n#endif  // _WIN32\n\nabsl::Status FdMMapReaderBase::AnnotateStatusImpl(absl::Status status) {\n  return ChainReader::AnnotateStatusImpl(\n      Annotate(status, absl::StrCat(\"reading \", filename())));\n}\n\n#ifndef _WIN32\n\nvoid FdMMapReaderBase::SetReadAllHintImpl(bool read_all_hint) {\n  ChainReader::SetReadAllHintImpl(read_all_hint);\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  const int src = SrcFd();\n  FdSetReadAllHint(src, read_all_hint);\n}\n\n#endif  // !_WIN32\n\nbool FdMMapReaderBase::SyncImpl(SyncType sync_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const int src = SrcFd();\n  if (base_pos_to_sync_ != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(\n            fd_internal::LSeek(\n                src, IntCast<fd_internal::Offset>(*base_pos_to_sync_ + pos()),\n                SEEK_SET) < 0)) {\n      return FailOperation(fd_internal::kLSeekFunctionName);\n    }\n  }\n  return true;\n}\n\nstd::unique_ptr<Reader> FdMMapReaderBase::NewReaderImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point.\n  std::unique_ptr<FdMMapReader<UnownedFd>> reader =\n      std::make_unique<FdMMapReader<UnownedFd>>(kClosed);\n  reader->InitializeWithExistingData(UnownedFd(SrcFdHandle()),\n                                     ChainReader::src());\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/fd_mmap_reader.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_FD_MMAP_READER_H_\n#define RIEGELI_BYTES_FD_MMAP_READER_H_\n\n#include <fcntl.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/fd_handle.h\"\n#include \"riegeli/bytes/fd_internal.h\"\n#include \"riegeli/bytes/path_ref.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `FdMMapReader`.\nclass FdMMapReaderBase : public ChainReader<Chain> {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // If `FdMMapReader` opens a fd with a filename, `mode()` is the second\n    // argument of `open()` (on Windows: `_open()`) and specifies the open mode\n    // and flags, typically `O_RDONLY` (on Windows: `_O_RDONLY | _O_BINARY`).\n    // It must include either `O_RDONLY` or `O_RDWR` (on Windows: `_O_RDONLY` or\n    // `_O_RDWR`).\n    //\n    // If `FdMMapReader` reads from an already open fd, `mode()` has no effect.\n    //\n    // `mode()` can also be changed with `set_inheritable()`.\n    //\n    // Default: `O_RDONLY | O_CLOEXEC`\n    // (on Windows: `_O_RDONLY | _O_BINARY | _O_NOINHERIT`).\n    Options& set_mode(int mode) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      mode_ = mode;\n      return *this;\n    }\n    Options&& set_mode(int mode) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_mode(mode));\n    }\n    int mode() const { return mode_; }\n\n    // If `false`, `execve()` (`CreateProcess()` on Windows) will close the fd.\n    //\n    // If `true`, the fd will remain open across `execve()` (`CreateProcess()`\n    // on Windows).\n    //\n    // If `FdMMapReader` reads from an already open fd, `inheritable()` has no\n    // effect.\n    //\n    // `set_inheritable()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_inheritable(bool inheritable) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      mode_ = (mode_ & ~fd_internal::kCloseOnExec) |\n              (inheritable ? 0 : fd_internal::kCloseOnExec);\n      return *this;\n    }\n    Options&& set_inheritable(bool inheritable) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_inheritable(inheritable));\n    }\n    bool inheritable() const {\n      return (mode_ & fd_internal::kCloseOnExec) == 0;\n    }\n\n    // If `std::nullopt`, `FdMMapReader` reads starting from the current fd\n    // position. The `FdMMapReader` position is synchronized back to the fd by\n    // `Close()` and `Sync()`.\n    //\n    // If not `std::nullopt`, `FdMMapReader` reads starting from this position,\n    // without disturbing the current fd position. This is useful for multiple\n    // readers concurrently reading from the same fd.\n    //\n    // Default: `std::nullopt`.\n    Options& set_independent_pos(std::optional<Position> independent_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      independent_pos_ = independent_pos;\n      return *this;\n    }\n    Options&& set_independent_pos(std::optional<Position> independent_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_independent_pos(independent_pos));\n    }\n    std::optional<Position> independent_pos() const { return independent_pos_; }\n\n    // If `std::nullopt`, the whole file is mapped into memory. `pos()`\n    // corresponds to original file positions.\n    //\n    // If not `std::nullopt`, only the range of this length starting from the\n    // current position or `independent_pos()` is mapped into memory, or the\n    // remaining part of the file if that is shorter. `pos()` starts from 0.\n    //\n    // Default: `std::nullopt`.\n    Options& set_max_length(std::optional<Position> max_length) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      max_length_ = max_length;\n      return *this;\n    }\n    Options&& set_max_length(std::optional<Position> max_length) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_length(max_length));\n    }\n    std::optional<Position> max_length() const { return max_length_; }\n\n    // Sets `max_length()` to the remaining part of the file.\n    Options& set_remaining_length() & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      set_max_length(std::numeric_limits<Position>::max());\n      return *this;\n    }\n    Options&& set_remaining_length() && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_remaining_length());\n    }\n\n   private:\n#ifndef _WIN32\n    int mode_ = O_RDONLY | fd_internal::kCloseOnExec;\n#else\n    int mode_ = _O_RDONLY | _O_BINARY | fd_internal::kCloseOnExec;\n#endif\n    std::optional<Position> independent_pos_;\n    std::optional<Position> max_length_;\n  };\n\n  // Returns the `FdHandle` being read from. Unchanged by `Close()`.\n  virtual FdHandle SrcFdHandle() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the fd being read from. If the fd is owned then changed to -1 by\n  // `Close()`, otherwise unchanged.\n  virtual int SrcFd() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the filename of the fd being read from, or \"<none>\" for\n  // closed-constructed or moved-from `FdMMapReader`. Unchanged by `Close()`.\n  //\n  // If the constructor from filename was used, this is the filename passed to\n  // the constructor, otherwise a filename is inferred from the fd. This can be\n  // a placeholder instead of a real filename if the fd does not refer to a\n  // named file or inferring the filename is not supported.\n  //\n  // If `Src` does not support `filename()`, returns \"<unsupported>\".\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return SrcFdHandle().filename();\n  }\n\n  bool SupportsNewReader() override { return true; }\n\n protected:\n  explicit FdMMapReaderBase(Closed) noexcept : ChainReader(kClosed) {}\n\n  explicit FdMMapReaderBase();\n\n  FdMMapReaderBase(FdMMapReaderBase&& that) noexcept;\n  FdMMapReaderBase& operator=(FdMMapReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(int src, Options&& options);\n  void InitializePos(int src, Options&& options);\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation);\n#ifdef _WIN32\n  ABSL_ATTRIBUTE_COLD bool FailWindowsOperation(absl::string_view operation);\n#endif\n\n  void Done() override;\n  absl::Status AnnotateStatusImpl(absl::Status status) override;\n#ifndef _WIN32\n  void SetReadAllHintImpl(bool read_all_hint) override;\n#endif\n  bool SyncImpl(SyncType sync_type) override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  std::optional<Position> base_pos_to_sync_;\n};\n\n// A `Reader` which reads from a file descriptor by mapping the whole file to\n// memory.\n//\n// The fd must support:\n#ifndef _WIN32\n//  * `close()` - if the fd is owned\n//  * `fstat()`\n//  * `mmap()`\n//  * `lseek()` - if `Options::independent_pos() == std::nullopt`\n#else\n//  * `_close()`    - if the fd is owned\n//  * `_fstat64()`\n//  * `_get_osfhandle()`, `CreateFileMappingW()`, `MapViewOfFile()`\n//  * `_lseeki64()` - if `Options::independent_pos() == std::nullopt`\n#endif\n//\n// `FdMMapReader` supports random access and `NewReader()`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the fd being read from. `Src` must support\n// `Dependency<FdHandle, Src>`, e.g. `OwnedFd` (owned, default),\n// `UnownedFd` (not owned), `AnyFd` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `OwnedFd` if the\n// first constructor argument is a filename or an `int`, otherwise as `TargetT`\n// of the type of the first constructor argument.\n//\n// The fd must not be closed until the `FdMMapReader` is closed or no longer\n// used. File contents must not be changed while data read from the file is\n// accessed without a memory copy.\ntemplate <typename Src = OwnedFd>\nclass FdMMapReader : public FdMMapReaderBase {\n public:\n  // Creates a closed `FdMMapReader`.\n  explicit FdMMapReader(Closed) noexcept : FdMMapReaderBase(kClosed) {}\n\n  // Will read from the fd provided by `src`.\n  explicit FdMMapReader(Initializer<Src> src, Options options = Options());\n\n  // Will read from `src`.\n  template <\n      typename DependentSrc = Src,\n      std::enable_if_t<std::is_constructible_v<DependentSrc, int>, int> = 0>\n  explicit FdMMapReader(int src ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                        Options options = Options());\n\n  // Opens a file for reading.\n  //\n  // If opening the file fails, `FdMMapReader` will be failed and closed.\n  //\n  // This constructor is present only if `Src` supports `Open()`.\n  template <typename DependentSrc = Src,\n            std::enable_if_t<\n                std::conjunction_v<FdSupportsOpen<DependentSrc>,\n                                   std::is_default_constructible<DependentSrc>>,\n                int> = 0>\n  explicit FdMMapReader(PathInitializer filename, Options options = Options());\n\n  // Opens a file for reading, with the filename interpreted relatively to the\n  // directory specified by an existing fd.\n  //\n  // If opening the file fails, `FdMMapReader` will be failed and closed.\n  //\n  // This constructor is present only if `Src` supports `Open()`.\n  template <typename DependentSrc = Src,\n            std::enable_if_t<\n                std::conjunction_v<FdSupportsOpenAt<DependentSrc>,\n                                   std::is_default_constructible<DependentSrc>>,\n                int> = 0>\n  explicit FdMMapReader(UnownedFd dir_fd, PathRef filename,\n                        Options options = Options());\n\n  FdMMapReader(FdMMapReader&& that) = default;\n  FdMMapReader& operator=(FdMMapReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `FdMMapReader`. This avoids\n  // constructing a temporary `FdMMapReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n  template <\n      typename DependentSrc = Src,\n      std::enable_if_t<std::is_constructible_v<DependentSrc, int>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(int src, Options options = Options());\n  template <typename DependentSrc = Src,\n            std::enable_if_t<std::conjunction_v<FdSupportsOpen<DependentSrc>,\n                                                SupportsReset<DependentSrc>>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(PathInitializer filename,\n                                          Options options = Options());\n  template <typename DependentSrc = Src,\n            std::enable_if_t<std::conjunction_v<FdSupportsOpenAt<DependentSrc>,\n                                                SupportsReset<DependentSrc>>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(UnownedFd dir_fd,\n                                          PathInitializer filename,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the fd being read from.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  FdHandle SrcFdHandle() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n  int SrcFd() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get().get();\n  }\n\n  // An optimized implementation in a derived class, avoiding a virtual call.\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.get().filename();\n  }\n\n protected:\n  void Done() override;\n\n private:\n  friend class FdMMapReaderBase;  // For `InitializeWithExistingData()`.\n\n  template <typename DependentSrc = Src,\n            std::enable_if_t<FdSupportsOpen<DependentSrc>::value, int> = 0>\n  void OpenImpl(PathInitializer filename, Options&& options);\n  template <typename DependentSrc = Src,\n            std::enable_if_t<FdSupportsOpenAt<DependentSrc>::value, int> = 0>\n  void OpenAtImpl(UnownedFd dir_fd, PathRef filename, Options&& options);\n\n  template <typename DependentSrc = Src,\n            std::enable_if_t<std::is_same_v<DependentSrc, UnownedFd>, int> = 0>\n  void InitializeWithExistingData(UnownedFd src, const Chain& data);\n\n  // The object providing and possibly owning the fd being read from.\n  Dependency<FdHandle, Src> src_;\n};\n\nexplicit FdMMapReader(Closed) -> FdMMapReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit FdMMapReader(\n    Src&& src, FdMMapReaderBase::Options options = FdMMapReaderBase::Options())\n    -> FdMMapReader<std::conditional_t<\n        std::disjunction_v<std::is_convertible<Src&&, int>,\n                           std::is_convertible<Src&&, absl::string_view>>,\n        OwnedFd, TargetT<Src>>>;\nexplicit FdMMapReader(UnownedFd dir_fd, PathRef filename,\n                      FdMMapReaderBase::Options options =\n                          FdMMapReaderBase::Options()) -> FdMMapReader<OwnedFd>;\n\n// Implementation details follow.\n\ninline FdMMapReaderBase::FdMMapReaderBase()\n    // The `Chain` to read from is not known yet. `ChainReader` will be reset in\n    // `Initialize()` to read from the `Chain` when it is known.\n    : ChainReader(kClosed) {}\n\ninline FdMMapReaderBase::FdMMapReaderBase(FdMMapReaderBase&& that) noexcept\n    : ChainReader(static_cast<ChainReader&&>(that)),\n      base_pos_to_sync_(that.base_pos_to_sync_) {}\n\ninline FdMMapReaderBase& FdMMapReaderBase::operator=(\n    FdMMapReaderBase&& that) noexcept {\n  ChainReader::operator=(static_cast<ChainReader&&>(that));\n  base_pos_to_sync_ = that.base_pos_to_sync_;\n  return *this;\n}\n\ninline void FdMMapReaderBase::Reset(Closed) {\n  ChainReader::Reset(kClosed);\n  base_pos_to_sync_ = std::nullopt;\n}\n\ninline void FdMMapReaderBase::Reset() {\n  // The `Chain` to read from is not known yet. `ChainReader` will be reset in\n  // `Initialize()` to read from the `Chain` when it is known.\n  ChainReader::Reset(kClosed);\n  base_pos_to_sync_ = std::nullopt;\n}\n\ntemplate <typename Src>\ninline FdMMapReader<Src>::FdMMapReader(Initializer<Src> src, Options options)\n    : src_(std::move(src)) {\n  Initialize(src_.get().get(), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<std::is_constructible_v<DependentSrc, int>, int>>\ninline FdMMapReader<Src>::FdMMapReader(int src ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                                       Options options)\n    : FdMMapReader(riegeli::Maker(src), std::move(options)) {}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<\n              std::conjunction_v<FdSupportsOpen<DependentSrc>,\n                                 std::is_default_constructible<DependentSrc>>,\n              int>>\ninline FdMMapReader<Src>::FdMMapReader(PathInitializer filename,\n                                       Options options)\n    : src_(riegeli::Maker()) {\n  OpenImpl(std::move(filename), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<\n              std::conjunction_v<FdSupportsOpenAt<DependentSrc>,\n                                 std::is_default_constructible<DependentSrc>>,\n              int>>\ninline FdMMapReader<Src>::FdMMapReader(UnownedFd dir_fd, PathRef filename,\n                                       Options options)\n    : src_(riegeli::Maker()) {\n  OpenAtImpl(std::move(dir_fd), filename, std::move(options));\n}\n\ntemplate <typename Src>\ninline void FdMMapReader<Src>::Reset(Closed) {\n  FdMMapReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void FdMMapReader<Src>::Reset(Initializer<Src> src, Options options) {\n  FdMMapReaderBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(src_.get().get(), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<std::is_constructible_v<DependentSrc, int>, int>>\ninline void FdMMapReader<Src>::Reset(int src, Options options) {\n  Reset(riegeli::Maker(src), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<std::conjunction_v<FdSupportsOpen<DependentSrc>,\n                                              SupportsReset<DependentSrc>>,\n                           int>>\ninline void FdMMapReader<Src>::Reset(PathInitializer filename,\n                                     Options options) {\n  // In case `filename` is owned by `src_` and gets invalidated.\n  std::string filename_copy = std::move(filename);\n  riegeli::Reset(src_.manager());\n  FdMMapReaderBase::Reset();\n  OpenImpl(std::move(filename_copy), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<std::conjunction_v<FdSupportsOpenAt<DependentSrc>,\n                                              SupportsReset<DependentSrc>>,\n                           int>>\ninline void FdMMapReader<Src>::Reset(UnownedFd dir_fd, PathInitializer filename,\n                                     Options options) {\n  // In case `filename` is owned by `src_` and gets invalidated.\n  std::string filename_copy = std::move(filename);\n  riegeli::Reset(src_.manager());\n  FdMMapReaderBase::Reset();\n  OpenAtImpl(dir_fd, filename_copy, std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<FdSupportsOpen<DependentSrc>::value, int>>\nvoid FdMMapReader<Src>::OpenImpl(PathInitializer filename, Options&& options) {\n  absl::Status status = src_.manager().Open(std::move(filename), options.mode(),\n                                            OwnedFd::kDefaultPermissions);\n  if (ABSL_PREDICT_FALSE(!status.ok())) {\n    FdMMapReaderBase::Reset(kClosed);\n    FailWithoutAnnotation(std::move(status));\n    return;\n  }\n  InitializePos(src_.get().get(), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<FdSupportsOpenAt<DependentSrc>::value, int>>\nvoid FdMMapReader<Src>::OpenAtImpl(UnownedFd dir_fd, PathRef filename,\n                                   Options&& options) {\n  absl::Status status =\n      src_.manager().OpenAt(std::move(dir_fd), filename, options.mode(),\n                            OwnedFd::kDefaultPermissions);\n  if (ABSL_PREDICT_FALSE(!status.ok())) {\n    FdMMapReaderBase::Reset(kClosed);\n    FailWithoutAnnotation(std::move(status));\n    return;\n  }\n  InitializePos(src_.get().get(), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<std::is_same_v<DependentSrc, UnownedFd>, int>>\nvoid FdMMapReader<Src>::InitializeWithExistingData(UnownedFd src,\n                                                   const Chain& data) {\n  FdMMapReaderBase::Reset();\n  src_.Reset(std::move(src));\n  ChainReader::Reset(data);\n}\n\ntemplate <typename Src>\nvoid FdMMapReader<Src>::Done() {\n  FdMMapReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (absl::Status status = src_.get().Close();\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      Fail(std::move(status));\n    }\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_FD_MMAP_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/fd_reader.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef _WIN32\n\n// Make `pread()` and `posix_fadvise()` available.\n#if !defined(_XOPEN_SOURCE) || _XOPEN_SOURCE < 600\n#undef _XOPEN_SOURCE\n#define _XOPEN_SOURCE 600\n#endif\n\n// Make `off_t` 64-bit even on 32-bit systems.\n#undef _FILE_OFFSET_BITS\n#define _FILE_OFFSET_BITS 64\n\n// Make `copy_file_range()` available on Linux.\n#ifndef _GNU_SOURCE\n#define _GNU_SOURCE\n#endif\n\n#else\n\n#define WIN32_LEAN_AND_MEAN\n\n#endif\n\n#include \"riegeli/bytes/fd_reader.h\"\n\n#include <fcntl.h>\n#ifdef _WIN32\n#include <io.h>\n#endif\n#include <stddef.h>\n#include <stdio.h>\n#include <sys/types.h>\n#ifndef _WIN32\n#include <unistd.h>\n#else\n#include <windows.h>\n#endif\n\n#include <cerrno>\n#include <limits>\n#include <memory>\n#include <optional>\n#ifndef _WIN32\n#include <type_traits>\n#endif\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/bits.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#ifdef _WIN32\n#include \"riegeli/base/errno_mapping.h\"\n#endif\n#include \"riegeli/base/global.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/fd_handle.h\"\n#include \"riegeli/bytes/fd_internal_for_cc.h\"\n#ifndef _WIN32\n#include \"riegeli/bytes/fd_writer.h\"\n#endif\n#include \"riegeli/bytes/reader.h\"\n#ifndef _WIN32\n#include \"riegeli/bytes/writer.h\"\n#endif\n\nnamespace riegeli {\n\n#ifndef _WIN32\n\nnamespace {\n\n// If `copy_file_range()` is available, it is used to make copying from\n// `FdReader` to `FdWriter` more efficient.\n//\n// By default its availability is autodetected.\n// Define `RIEGELI_DISABLE_COPY_FILE_RANGE` to disable using it even if it\n// appears to be available.\n\n#if (defined(__EMSCRIPTEN__) ||                         \\\n     (defined(__ANDROID__) && __ANDROID_API__ < 34)) && \\\n    !defined(RIEGELI_DISABLE_COPY_FILE_RANGE)\n#define RIEGELI_DISABLE_COPY_FILE_RANGE 1\n#endif\n\n#if !RIEGELI_DISABLE_COPY_FILE_RANGE\n\n// `copy_file_range()` is supported by Linux and FreeBSD.\n\ntemplate <typename DependentInt, typename Enable = void>\nstruct HaveCopyFileRange : std::false_type {};\n\ntemplate <typename DependentInt>\nstruct HaveCopyFileRange<\n    DependentInt,\n    std::void_t<decltype(copy_file_range(\n        std::declval<DependentInt>(), std::declval<fd_internal::Offset*>(),\n        std::declval<int>(), std::declval<fd_internal::Offset*>(),\n        std::declval<size_t>(), std::declval<unsigned>()))>> : std::true_type {\n};\n\ntemplate <typename DependentInt>\ninline ssize_t CopyFileRange(DependentInt src, fd_internal::Offset* src_offset,\n                             int dest, fd_internal::Offset* dest_offset,\n                             size_t length, unsigned flags) {\n  if constexpr (HaveCopyFileRange<DependentInt>::value) {\n    return copy_file_range(src, src_offset, dest, dest_offset, length, flags);\n  } else {\n    errno = EOPNOTSUPP;\n    return -1;\n  }\n}\n\n#endif  // !RIEGELI_DISABLE_COPY_FILE_RANGE\n\n// `posix_fadvise()` is supported by POSIX systems but not MacOS.\n\ntemplate <typename DependentInt, typename Enable = void>\nstruct HavePosixFadvise : std::false_type {};\n\ntemplate <typename DependentInt>\nstruct HavePosixFadvise<\n    DependentInt,\n    std::void_t<decltype(posix_fadvise(\n        std::declval<DependentInt>(), std::declval<fd_internal::Offset>(),\n        std::declval<fd_internal::Offset>(), std::declval<int>()))>>\n    : std::true_type {};\n\ntemplate <typename DependentInt>\ninline void FdSetReadAllHint(ABSL_ATTRIBUTE_UNUSED DependentInt src,\n                             ABSL_ATTRIBUTE_UNUSED bool read_all_hint) {\n  if constexpr (HavePosixFadvise<DependentInt>::value) {\n#ifdef POSIX_FADV_SEQUENTIAL\n    posix_fadvise(src, 0, 0,\n                  read_all_hint ? POSIX_FADV_SEQUENTIAL : POSIX_FADV_NORMAL);\n#endif\n  }\n}\n\n}  // namespace\n\n#endif\n\nvoid FdReaderBase::Initialize(int src, Options&& options) {\n  RIEGELI_ASSERT_GE(src, 0)\n      << \"Failed precondition of FdReader: negative file descriptor\";\n  InitializePos(src, std::move(options)\n#ifdef _WIN32\n                         ,\n                /*mode_was_passed_to_open=*/false\n#endif\n  );\n}\n\nvoid FdReaderBase::InitializePos(int src, Options&& options\n#ifdef _WIN32\n                                 ,\n                                 bool mode_was_passed_to_open\n#endif\n) {\n  RIEGELI_ASSERT(!has_independent_pos_)\n      << \"Failed precondition of FdReaderBase::InitializePos(): \"\n         \"has_independent_pos_ not reset\";\n  RIEGELI_ASSERT(!supports_random_access_)\n      << \"Failed precondition of FdReaderBase::InitializePos(): \"\n         \"supports_random_access_ not reset\";\n  RIEGELI_ASSERT_OK(random_access_status_)\n      << \"Failed precondition of FdReaderBase::InitializePos(): \"\n         \"random_access_status_ not reset\";\n#ifdef _WIN32\n  RIEGELI_ASSERT_EQ(original_mode_, std::nullopt)\n      << \"Failed precondition of FdWriterBase::InitializePos(): \"\n         \"original_mode_ not reset\";\n  int text_mode = options.mode() &\n                  (_O_BINARY | _O_TEXT | _O_WTEXT | _O_U16TEXT | _O_U8TEXT);\n  if (!mode_was_passed_to_open && text_mode != 0) {\n    const int original_mode = _setmode(src, text_mode);\n    if (ABSL_PREDICT_FALSE(original_mode < 0)) {\n      FailOperation(\"_setmode()\");\n      return;\n    }\n    original_mode_ = original_mode;\n  }\n  if (options.assumed_pos() == std::nullopt) {\n    if (text_mode == 0) {\n      // There is no `_getmode()`, but `_setmode()` returns the previous mode.\n      text_mode = _setmode(src, _O_BINARY);\n      if (ABSL_PREDICT_FALSE(text_mode < 0)) {\n        FailOperation(\"_setmode()\");\n        return;\n      }\n      if (ABSL_PREDICT_FALSE(_setmode(src, text_mode) < 0)) {\n        FailOperation(\"_setmode()\");\n        return;\n      }\n    }\n    if (text_mode != _O_BINARY) {\n      if (ABSL_PREDICT_FALSE(options.independent_pos() != std::nullopt)) {\n        Fail(absl::InvalidArgumentError(\n            \"FdReaderBase::Options::independent_pos() requires binary mode\"));\n        return;\n      }\n      options.set_assumed_pos(0);\n    }\n  }\n#endif  // _WIN32\n  if (options.assumed_pos() != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(options.independent_pos() != std::nullopt)) {\n      Fail(absl::InvalidArgumentError(\n          \"FdReaderBase::Options::assumed_pos() and independent_pos() \"\n          \"must not be both set\"));\n      return;\n    }\n    if (ABSL_PREDICT_FALSE(\n            *options.assumed_pos() >\n            Position{std::numeric_limits<fd_internal::Offset>::max()})) {\n      FailOverflow();\n      return;\n    }\n    set_limit_pos(*options.assumed_pos());\n    // `supports_random_access_` is left as `false`.\n    random_access_status_ = Global([] {\n      return absl::UnimplementedError(\n          \"FdReaderBase::Options::assumed_pos() excludes random access\");\n    });\n  } else if (options.independent_pos() != std::nullopt) {\n    has_independent_pos_ = true;\n    if (ABSL_PREDICT_FALSE(\n            *options.independent_pos() >\n            Position{std::numeric_limits<fd_internal::Offset>::max()})) {\n      FailOverflow();\n      return;\n    }\n    set_limit_pos(*options.independent_pos());\n    supports_random_access_ = true;\n  } else {\n    const fd_internal::Offset file_pos = fd_internal::LSeek(src, 0, SEEK_CUR);\n    if (file_pos < 0) {\n      // Random access is not supported. Assume 0 as the initial position.\n      // `supports_random_access_` is left as `false`.\n      random_access_status_ =\n          FailedOperationStatus(fd_internal::kLSeekFunctionName);\n      return;\n    }\n    set_limit_pos(IntCast<Position>(file_pos));\n\n    // Check the size, and whether random access is supported.\n    fd_internal::Offset file_size = fd_internal::LSeek(src, 0, SEEK_END);\n    if (file_size < 0) {\n      // Random access is not supported. `supports_random_access_` is left as\n      // `false`.\n      //\n      // This covers some \"/proc\" files for which `fd_internal::LSeek(SEEK_CUR)`\n      // succeeds but `fd_internal::LSeek(SEEK_END)` fails.\n      random_access_status_ =\n          FailedOperationStatus(fd_internal::kLSeekFunctionName);\n      return;\n    }\n    if (limit_pos() != IntCast<Position>(file_size)) {\n      if (ABSL_PREDICT_FALSE(\n              fd_internal::LSeek(src, IntCast<fd_internal::Offset>(limit_pos()),\n                                 SEEK_SET) < 0)) {\n        FailOperation(fd_internal::kLSeekFunctionName);\n        return;\n      }\n    }\n#ifndef _WIN32\n    if (file_size == 0 && limit_pos() == 0) {\n      // Some \"/sys\" files claim to have zero size but have non-empty contents\n      // when read. Some \"/proc\" files too, but they have been recognized with\n      // a failing `fd_internal::LSeek(SEEK_END)`.\n      if (BufferedReader::PullSlow(1, 0)) {\n        if (growing_source_) {\n          // Check the size again. Maybe the file has grown.\n          file_size = fd_internal::LSeek(src, 0, SEEK_END);\n          if (ABSL_PREDICT_FALSE(file_size < 0)) {\n            FailOperation(fd_internal::kLSeekFunctionName);\n            return;\n          }\n          if (limit_pos() != IntCast<Position>(file_size)) {\n            if (ABSL_PREDICT_FALSE(\n                    fd_internal::LSeek(\n                        src, IntCast<fd_internal::Offset>(limit_pos()),\n                        SEEK_SET) < 0)) {\n              FailOperation(fd_internal::kLSeekFunctionName);\n              return;\n            }\n          }\n          if (file_size > 0) goto regular;\n        }\n        // This is one of \"/sys\" files which claim to have zero size but have\n        // non-empty contents when read. Random access is not supported.\n        // `supports_random_access_` is left as `false`.\n        random_access_status_ = Global([] {\n          return absl::UnimplementedError(\n              \"Random access is not supported because \"\n              \"the file claims zero size but has non-empty contents when read\");\n        });\n        return;\n      }\n      if (ABSL_PREDICT_FALSE(!ok())) return;\n      // This is a regular empty file.\n    }\n  regular:\n#endif\n    // Random access is supported.\n    supports_random_access_ = true;\n    if (!growing_source_) set_exact_size(IntCast<Position>(file_size));\n  }\n  BeginRun();\n}\n\nvoid FdReaderBase::Done() {\n  BufferedReader::Done();\n#ifdef _WIN32\n  if (original_mode_ != std::nullopt) {\n    const int src = SrcFd();\n    if (ABSL_PREDICT_FALSE(_setmode(src, *original_mode_) < 0)) {\n      FailOperation(\"_setmode()\");\n    }\n  }\n#endif  // _WIN32\n  random_access_status_ = absl::OkStatus();\n}\n\ninline absl::Status FdReaderBase::FailedOperationStatus(\n    absl::string_view operation) {\n  const int error_number = errno;\n  RIEGELI_ASSERT_NE(error_number, 0)\n      << \"Failed precondition of FdReaderBase::FailedOperationStatus(): \"\n         \"zero errno\";\n  return absl::ErrnoToStatus(error_number, absl::StrCat(operation, \" failed\"));\n}\n\nbool FdReaderBase::FailOperation(absl::string_view operation) {\n  return Fail(FailedOperationStatus(operation));\n}\n\n#ifdef _WIN32\n\nbool FdReaderBase::FailWindowsOperation(absl::string_view operation) {\n  const DWORD error_number = GetLastError();\n  RIEGELI_ASSERT_NE(error_number, 0)\n      << \"Failed precondition of FdReaderBase::FailWindowsOperation(): \"\n         \"zero error code\";\n  return Fail(WindowsErrorToStatus(IntCast<uint32_t>(error_number),\n                                   absl::StrCat(operation, \" failed\")));\n}\n\n#endif  // _WIN32\n\nabsl::Status FdReaderBase::AnnotateStatusImpl(absl::Status status) {\n  return BufferedReader::AnnotateStatusImpl(\n      Annotate(status, absl::StrCat(\"reading \", filename())));\n}\n\n#ifndef _WIN32\n\nvoid FdReaderBase::SetReadAllHintImpl(bool read_all_hint) {\n  BufferedReader::SetReadAllHintImpl(read_all_hint);\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  const int src = SrcFd();\n  FdSetReadAllHint(src, read_all_hint);\n}\n\n#endif  // !_WIN32\n\nbool FdReaderBase::ReadInternal(size_t min_length, size_t max_length,\n                                char* dest) {\n  RIEGELI_ASSERT_GT(min_length, 0u)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"nothing to read\";\n  RIEGELI_ASSERT_GE(max_length, min_length)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"max_length < min_length\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::ReadInternal()\";\n  const int src = SrcFd();\n  for (;;) {\n    if (ABSL_PREDICT_FALSE(\n            limit_pos() >=\n            Position{std::numeric_limits<fd_internal::Offset>::max()})) {\n      return FailOverflow();\n    }\n#ifndef _WIN32\n    const size_t length_to_read = UnsignedMin(\n        max_length,\n        Position{std::numeric_limits<fd_internal::Offset>::max()} - limit_pos(),\n        absl::bit_floor(size_t{std::numeric_limits<ssize_t>::max()}),\n        // Darwin and FreeBSD cannot read more than 2 GB - 1 at a time.\n        // Limit to 1 GB for better alignment of reads.\n        // https://codereview.appspot.com/89900044#msg9\n        size_t{1} << 30);\n  again:\n    const ssize_t length_read =\n        has_independent_pos_ ? pread(src, dest, length_to_read,\n                                     IntCast<fd_internal::Offset>(limit_pos()))\n                             : read(src, dest, length_to_read);\n    if (ABSL_PREDICT_FALSE(length_read < 0)) {\n      if (errno == EINTR) goto again;\n      return FailOperation(has_independent_pos_ ? \"pread()\" : \"read()\");\n    }\n#else   // _WIN32\n    DWORD length_to_read;\n    DWORD length_read;\n    if (has_independent_pos_) {\n      const HANDLE file_handle = reinterpret_cast<HANDLE>(_get_osfhandle(src));\n      if (ABSL_PREDICT_FALSE(file_handle == INVALID_HANDLE_VALUE ||\n                             file_handle == reinterpret_cast<HANDLE>(-2))) {\n        return FailWindowsOperation(\"_get_osfhandle()\");\n      }\n      length_to_read = UnsignedMin(\n          max_length,\n          Position{std::numeric_limits<fd_internal::Offset>::max()} -\n              limit_pos(),\n          absl::bit_floor(std::numeric_limits<DWORD>::max()));\n      OVERLAPPED overlapped{};\n      overlapped.Offset = IntCast<DWORD>(limit_pos() & 0xffffffff);\n      overlapped.OffsetHigh = IntCast<DWORD>(limit_pos() >> 32);\n      if (ABSL_PREDICT_FALSE(!ReadFile(file_handle, dest, length_to_read,\n                                       &length_read, &overlapped)) &&\n          ABSL_PREDICT_FALSE(GetLastError() != ERROR_HANDLE_EOF)) {\n        return FailWindowsOperation(\"ReadFile()\");\n      }\n    } else {\n      length_to_read = UnsignedMin(\n          max_length,\n          Position{std::numeric_limits<fd_internal::Offset>::max()} -\n              limit_pos(),\n          absl::bit_floor(unsigned{std::numeric_limits<int>::max()}));\n      const int length_read_int =\n          _read(src, dest, IntCast<unsigned>(length_to_read));\n      if (ABSL_PREDICT_FALSE(length_read_int < 0)) {\n        return FailOperation(\"_read()\");\n      }\n      length_read = IntCast<DWORD>(length_read_int);\n    }\n#endif  // _WIN32\n    if (ABSL_PREDICT_FALSE(length_read == 0)) {\n      if (!growing_source_) set_exact_size(limit_pos());\n      return false;\n    }\n    RIEGELI_ASSERT_LE(UnsignedCast(length_read), length_to_read)\n#ifndef _WIN32\n        << (has_independent_pos_ ? \"pread()\" : \"read()\")\n#else\n        << (has_independent_pos_ ? \"ReadFile()\" : \"_read()\")\n#endif\n        << \" read more than requested\";\n    move_limit_pos(IntCast<size_t>(length_read));\n    if (IntCast<size_t>(length_read) >= min_length) return true;\n    dest += length_read;\n    min_length -= IntCast<size_t>(length_read);\n    max_length -= IntCast<size_t>(length_read);\n  }\n}\n\n#ifndef _WIN32\n\nbool FdReaderBase::CopyInternal(Position length, Writer& dest) {\n  RIEGELI_ASSERT_GT(length, 0u)\n      << \"Failed precondition of BufferedReader::CopyInternal(): \"\n         \"nothing to copy\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::CopyInternal()\";\n#if !RIEGELI_DISABLE_COPY_FILE_RANGE\n  if (HaveCopyFileRange<int>::value) {\n    if (FdWriterBase* const fd_writer = dest.GetIf<FdWriterBase>()) {\n      const int src = SrcFd();\n      for (;;) {\n        if (ABSL_PREDICT_FALSE(!fd_writer->Flush(FlushType::kFromObject))) {\n          return false;\n        }\n        const int dest_fd = fd_writer->DestFd();\n        fd_internal::Offset src_offset = limit_pos();\n        fd_internal::Offset dest_offset = fd_writer->start_pos();\n        if (ABSL_PREDICT_FALSE(\n                limit_pos() >=\n                Position{std::numeric_limits<fd_internal::Offset>::max()})) {\n          return FailOverflow();\n        }\n        const size_t length_to_copy = UnsignedMin(\n            length,\n            Position{std::numeric_limits<fd_internal::Offset>::max()} -\n                limit_pos(),\n            absl::bit_floor(size_t{std::numeric_limits<ssize_t>::max()}));\n        if (ABSL_PREDICT_FALSE(\n                length_to_copy >\n                Position{std::numeric_limits<fd_internal::Offset>::max()} -\n                    fd_writer->start_pos())) {\n          return fd_writer->FailOverflow();\n        }\n      again:\n        const ssize_t length_copied = CopyFileRange(\n            src, has_independent_pos_ ? &src_offset : nullptr, dest_fd,\n            fd_writer->has_independent_pos_ ? &dest_offset : nullptr,\n            length_to_copy, 0);\n        if (ABSL_PREDICT_FALSE(length_copied < 0)) {\n          if (errno == EINTR) goto again;\n          // File descriptors might not support `copy_file_range()` for a\n          // variety of reasons, e.g. append mode, not regular files,\n          // unsupported filesystem, or cross filesystem copy. Fall back to\n          // `read()` and `write()`.\n          break;\n        }\n        if (ABSL_PREDICT_FALSE(length_copied == 0)) {\n          if (!growing_source_) set_exact_size(limit_pos());\n          return false;\n        }\n        RIEGELI_ASSERT_LE(IntCast<size_t>(length_copied), length_to_copy)\n            << \"copy_file_range() copied more than requested\";\n        move_limit_pos(IntCast<size_t>(length_copied));\n        fd_writer->move_start_pos(IntCast<size_t>(length_copied));\n        length -= IntCast<size_t>(length_copied);\n        if (length == 0) return true;\n      }\n    }\n  }\n#endif  // !RIEGELI_DISABLE_COPY_FILE_RANGE\n  return BufferedReader::CopyInternal(length, dest);\n}\n\n#endif  // !_WIN32\n\ninline bool FdReaderBase::SeekInternal(int src, Position new_pos) {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of FdReaderBase::SeekInternal(): \"\n         \"buffer not empty\";\n  RIEGELI_ASSERT(FdReaderBase::SupportsRandomAccess())\n      << \"Failed precondition of FdReaderBase::SeekInternal(): \"\n         \"random access not supported\";\n  if (!has_independent_pos_) {\n    if (ABSL_PREDICT_FALSE(\n            fd_internal::LSeek(src, IntCast<fd_internal::Offset>(new_pos),\n                               SEEK_SET) < 0)) {\n      return FailOperation(fd_internal::kLSeekFunctionName);\n    }\n  }\n  set_limit_pos(new_pos);\n  return true;\n}\n\nbool FdReaderBase::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!FdReaderBase::SupportsRandomAccess())) {\n    if (ABSL_PREDICT_FALSE(new_pos < start_pos())) {\n      if (ok()) Fail(random_access_status_);\n      return false;\n    }\n    return BufferedReader::SeekBehindBuffer(new_pos);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const int src = SrcFd();\n  if (new_pos > limit_pos()) {\n    // Seeking forwards.\n    Position file_size;\n    if (exact_size() != std::nullopt) {\n      file_size = *exact_size();\n    } else {\n      fd_internal::StatInfo stat_info;\n      if (ABSL_PREDICT_FALSE(fd_internal::FStat(src, &stat_info) < 0)) {\n        return FailOperation(fd_internal::kFStatFunctionName);\n      }\n      file_size = IntCast<Position>(stat_info.st_size);\n      if (!growing_source_) set_exact_size(file_size);\n    }\n    if (ABSL_PREDICT_FALSE(new_pos > file_size)) {\n      // File ends.\n      SeekInternal(src, file_size);\n      return false;\n    }\n  }\n  return SeekInternal(src, new_pos);\n}\n\nstd::optional<Position> FdReaderBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  if (exact_size() != std::nullopt) return *exact_size();\n  if (ABSL_PREDICT_FALSE(!FdReaderBase::SupportsRandomAccess())) {\n    Fail(random_access_status_);\n    return std::nullopt;\n  }\n  const int src = SrcFd();\n  fd_internal::StatInfo stat_info;\n  if (ABSL_PREDICT_FALSE(fd_internal::FStat(src, &stat_info) < 0)) {\n    FailOperation(fd_internal::kFStatFunctionName);\n    return std::nullopt;\n  }\n  if (!growing_source_) set_exact_size(IntCast<Position>(stat_info.st_size));\n  return IntCast<Position>(stat_info.st_size);\n}\n\nstd::unique_ptr<Reader> FdReaderBase::NewReaderImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!FdReaderBase::SupportsNewReader())) {\n    if (ok()) {\n      Fail(\n#ifdef _WIN32\n          !has_independent_pos_\n              ? absl::UnimplementedError(\n                    \"NewReader() requires \"\n                    \"FdReaderBase::Options::independent_pos()\")\n              :\n#endif\n              random_access_status_);\n    }\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point.\n  std::unique_ptr<FdReader<UnownedFd>> reader =\n      std::make_unique<FdReader<UnownedFd>>(\n          UnownedFd(SrcFdHandle()), FdReaderBase::Options()\n                                        .set_independent_pos(initial_pos)\n                                        .set_growing_source(growing_source_)\n                                        .set_buffer_options(buffer_options()));\n  reader->set_exact_size(exact_size());\n  return reader;\n}\n\nstd::unique_ptr<Reader> FdReaderBase::NewReaderCurrentPosImpl() {\n  std::unique_ptr<Reader> reader = FdReaderBase::NewReaderImpl(pos());\n  if (ABSL_PREDICT_TRUE(reader != nullptr)) {\n    ShareBufferTo(*static_cast<FdReaderBase*>(reader.get()));\n  }\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/fd_reader.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_FD_READER_H_\n#define RIEGELI_BYTES_FD_READER_H_\n\n#include <fcntl.h>\n#include <stddef.h>\n\n#include <memory>\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/fd_handle.h\"\n#include \"riegeli/bytes/fd_internal.h\"\n#include \"riegeli/bytes/path_ref.h\"\n#include \"riegeli/bytes/reader.h\"\n#ifndef _WIN32\n#include \"riegeli/bytes/writer.h\"\n#endif\n\nnamespace riegeli {\n\n// Template parameter independent part of `FdReader`.\nclass FdReaderBase : public BufferedReader {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // If `FdReader` opens a fd with a filename, `mode()` is the second argument\n    // of `open()` (on Windows: `_open()`) and specifies the open mode and\n    // flags, typically `O_RDONLY` (on Windows: `_O_RDONLY | _O_BINARY`).\n    // It must include either `O_RDONLY` or `O_RDWR` (on Windows: `_O_RDONLY` or\n    // `_O_RDWR`).\n    //\n    // `mode()` can also be changed with `set_inheritable()` and `set_text()`.\n    //\n    // Default: `O_RDONLY | O_CLOEXEC`\n    // (on Windows: `_O_RDONLY | _O_BINARY | _O_NOINHERIT`).\n    Options& set_mode(int mode) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      mode_ = mode;\n      return *this;\n    }\n    Options&& set_mode(int mode) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_mode(mode));\n    }\n    int mode() const { return mode_; }\n\n    // If `false`, `execve()` (`CreateProcess()` on Windows) will close the fd.\n    //\n    // If `true`, the fd will remain open across `execve()` (`CreateProcess()`\n    // on Windows).\n    //\n    // If `FdReader` reads from an already open fd, `inheritable()` has no\n    // effect.\n    //\n    // `set_inheritable()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_inheritable(bool inheritable) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      mode_ = (mode_ & ~fd_internal::kCloseOnExec) |\n              (inheritable ? 0 : fd_internal::kCloseOnExec);\n      return *this;\n    }\n    Options&& set_inheritable(bool inheritable) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_inheritable(inheritable));\n    }\n    bool inheritable() const {\n      return (mode_ & fd_internal::kCloseOnExec) == 0;\n    }\n\n    // If `false`, data will be read directly from the file. This is called the\n    // binary mode.\n    //\n    // If `true`, text mode translation will be applied on Windows:\n    // CR-LF character pairs are translated to LF, and a ^Z character is\n    // interpreted as end of file.\n    //\n    // It is recommended to use `ReadLine()` or `TextReader` instead, which\n    // expect a binary mode `Reader`.\n    //\n    // `set_text()` has an effect only on Windows. It is applicable whenever\n    // `FdReader` opens a fd with a filename or reads from an already open fd.\n    //\n    // `set_text()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_text(ABSL_ATTRIBUTE_UNUSED bool text) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n#ifdef _WIN32\n      mode_ =\n          (mode_ & ~(_O_BINARY | _O_TEXT | _O_WTEXT | _O_U16TEXT | _O_U8TEXT)) |\n          (text ? _O_TEXT : _O_BINARY);\n#endif\n      return *this;\n    }\n    Options&& set_text(bool text) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_text(text));\n    }\n    // No `text()` getter is provided. On Windows `mode()` can have unspecified\n    // text mode, resolved using `_get_fmode()`. Not on Windows the concept does\n    // not exist.\n\n    // If `std::nullopt`, the current position reported by `pos()` corresponds\n    // to the current fd position if possible, otherwise 0 is assumed as the\n    // initial position. Random access is supported if the fd supports random\n    // access. On Windows binary mode is also required.\n    //\n    // If not `std::nullopt`, this position is assumed initially, to be reported\n    // by `pos()`. It does not need to correspond to the current fd position.\n    // Random access is not supported.\n    //\n    // `assumed_pos()` and `independent_pos()` must not be both set.\n    //\n    // Default: `std::nullopt`.\n    Options& set_assumed_pos(std::optional<Position> assumed_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      assumed_pos_ = assumed_pos;\n      return *this;\n    }\n    Options&& set_assumed_pos(std::optional<Position> assumed_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_pos(assumed_pos));\n    }\n    std::optional<Position> assumed_pos() const { return assumed_pos_; }\n\n    // If `std::nullopt`, `FdReader` reads at the current fd position.\n    //\n    // If not `std::nullopt`, `FdReader` reads starting from this position.\n    // The current fd position is not disturbed except on Windows, where seeking\n    // and reading is nevertheless atomic. This is useful for multiple readers\n    // concurrently reading from the same fd. The fd must support `pread()`\n    // (`_get_osfhandle()` and `ReadFile()` with `OVERLAPPED*` on Windows).\n    // On Windows binary mode is also required.\n    //\n    // `assumed_pos()` and `independent_pos()` must not be both set.\n    //\n    // Default: `std::nullopt`.\n    Options& set_independent_pos(std::optional<Position> independent_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      independent_pos_ = independent_pos;\n      return *this;\n    }\n    Options&& set_independent_pos(std::optional<Position> independent_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_independent_pos(independent_pos));\n    }\n    std::optional<Position> independent_pos() const { return independent_pos_; }\n\n    // If `true`, supports reading up to the end of the file, then retrying when\n    // the file has grown. This disables caching the file size.\n    //\n    // Default: `false`.\n    Options& set_growing_source(bool growing_source) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      growing_source_ = growing_source;\n      return *this;\n    }\n    Options&& set_growing_source(bool growing_source) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_growing_source(growing_source));\n    }\n    bool growing_source() const { return growing_source_; }\n\n   private:\n#ifndef _WIN32\n    int mode_ = O_RDONLY | fd_internal::kCloseOnExec;\n#else\n    int mode_ = _O_RDONLY | _O_BINARY | fd_internal::kCloseOnExec;\n#endif\n    std::optional<Position> assumed_pos_;\n    std::optional<Position> independent_pos_;\n    bool growing_source_ = false;\n  };\n\n  // Returns the `FdHandle` being read from. Unchanged by `Close()`.\n  virtual FdHandle SrcFdHandle() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the fd being read from. If the fd is owned then changed to -1 by\n  // `Close()`, otherwise unchanged.\n  virtual int SrcFd() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the filename of the fd being read from, or \"<none>\" for\n  // closed-constructed or moved-from `FdReader`. Unchanged by `Close()`.\n  //\n  // If the constructor from filename was used, this is the filename passed to\n  // the constructor, otherwise a filename is inferred from the fd. This can be\n  // a placeholder instead of a real filename if the fd does not refer to a\n  // named file or inferring the filename is not supported.\n  //\n  // If `Src` does not support `filename()`, returns \"<unsupported>\".\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return SrcFdHandle().filename();\n  }\n\n  bool ToleratesReadingAhead() override {\n    return BufferedReader::ToleratesReadingAhead() ||\n           FdReaderBase::SupportsRandomAccess();\n  }\n  bool SupportsRandomAccess() override { return supports_random_access_; }\n  bool SupportsNewReader() override {\n#ifndef _WIN32\n    return FdReaderBase::SupportsRandomAccess();\n#else\n    return has_independent_pos_ && FdReaderBase::SupportsRandomAccess();\n#endif\n  }\n\n protected:\n  explicit FdReaderBase(Closed) noexcept : BufferedReader(kClosed) {}\n\n  explicit FdReaderBase(BufferOptions buffer_options, bool growing_source);\n\n  FdReaderBase(FdReaderBase&& that) noexcept;\n  FdReaderBase& operator=(FdReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, bool growing_source);\n  void Initialize(int src, Options&& options);\n  void InitializePos(int src, Options&& options\n#ifdef _WIN32\n                     ,\n                     bool mode_was_passed_to_open\n#endif\n  );\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation);\n#ifdef _WIN32\n  ABSL_ATTRIBUTE_COLD bool FailWindowsOperation(absl::string_view operation);\n#endif\n\n  void Done() override;\n  absl::Status AnnotateStatusImpl(absl::Status status) override;\n#ifndef _WIN32\n  void SetReadAllHintImpl(bool read_all_hint) override;\n#endif\n  bool ReadInternal(size_t min_length, size_t max_length, char* dest) override;\n#ifndef _WIN32\n  bool CopyInternal(Position length, Writer& dest) override;\n#endif\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n  std::unique_ptr<Reader> NewReaderCurrentPosImpl() override;\n\n private:\n  absl::Status FailedOperationStatus(absl::string_view operation);\n\n  bool SeekInternal(int src, Position new_pos);\n\n  bool has_independent_pos_ = false;\n  bool growing_source_ = false;\n  bool supports_random_access_ = false;\n  absl::Status random_access_status_;\n#ifdef _WIN32\n  std::optional<int> original_mode_;\n#endif\n\n  // Invariant: `limit_pos() <= std::numeric_limits<fd_internal::Offset>::max()`\n};\n\n// A `Reader` which reads from a file descriptor.\n//\n// The fd must support:\n#ifndef _WIN32\n//  * `close()` - if the fd is owned\n//  * `read()`  - if `Options::independent_pos() == std::nullopt`\n//  * `pread()` - if `Options::independent_pos() != std::nullopt`,\n//                or for `NewReader()`\n//  * `lseek()` - for `Seek()` or `Size()`\n//                if `Options::independent_pos() == std::nullopt`\n//  * `fstat()` - for `Seek()` or `Size()`\n#else\n//  * `_close()`    - if the fd is owned\n//  * `_read()`     - if `Options::independent_pos() == std::nullopt`\n//  * `_get_osfhandle()`, `ReadFile()` with `OVERLAPPED*`\n//                  - if `Options::independent_pos() != std::nullopt`\n//  * `_lseeki64()` - for `Seek()` or `Size()`\n//                    if `Options::independent_pos() == std::nullopt`\n//  * `_fstat64()`  - for `Seek()` or `Size()`\n#endif\n//\n// `FdReader` supports random access if\n// `Options::assumed_pos() == std::nullopt` and the fd supports random access\n// (this is assumed if `Options::independent_pos() != std::nullopt`, otherwise\n// this is checked by calling `lseek(SEEK_END)`, or `_lseeki64()` on Windows).\n// On Windows binary mode is also required.\n//\n// `FdReader` supports `NewReader()` if it supports random access. On Windows\n// `independent_pos() != std::nullopt` is also required.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the fd being read from. `Src` must support\n// `Dependency<FdHandle, Src>`, e.g. `OwnedFd` (owned, default),\n// `UnownedFd` (not owned), `AnyFd` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `OwnedFd` if the\n// first constructor argument is a filename or an `int`, otherwise as `TargetT`\n// of the type of the first constructor argument.\n//\n// Warning: if random access is not supported and the fd is not owned, it will\n// have an unpredictable amount of extra data consumed because of buffering.\n//\n// Until the `FdReader` is closed or no longer used, the fd must not be closed.\n// Additionally, if `Options::independent_pos() == std::nullopt`\n// (or unconditionally on Windows), the fd must not have its position changed.\ntemplate <typename Src = OwnedFd>\nclass FdReader : public FdReaderBase {\n public:\n  // Creates a closed `FdReader`.\n  explicit FdReader(Closed) noexcept : FdReaderBase(kClosed) {}\n\n  // Will read from the fd provided by `src`.\n  explicit FdReader(Initializer<Src> src, Options options = Options());\n\n  // Will read from `src`.\n  template <\n      typename DependentSrc = Src,\n      std::enable_if_t<std::is_constructible_v<DependentSrc, int>, int> = 0>\n  explicit FdReader(int src ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                    Options options = Options());\n\n  // Opens a file for reading.\n  //\n  // If opening the file fails, `FdReader` will be failed and closed.\n  //\n  // This constructor is present only if `Src` supports `Open()`.\n  template <typename DependentSrc = Src,\n            std::enable_if_t<\n                std::conjunction_v<FdSupportsOpen<DependentSrc>,\n                                   std::is_default_constructible<DependentSrc>>,\n                int> = 0>\n  explicit FdReader(PathInitializer filename, Options options = Options());\n\n  // Opens a file for reading, with the filename interpreted relatively to the\n  // directory specified by an existing fd.\n  //\n  // If opening the file fails, `FdReader` will be failed and closed.\n  //\n  // This constructor is present only if `Src` supports `OpenAt()`.\n  template <typename DependentSrc = Src,\n            std::enable_if_t<\n                std::conjunction_v<FdSupportsOpenAt<DependentSrc>,\n                                   std::is_default_constructible<DependentSrc>>,\n                int> = 0>\n  explicit FdReader(UnownedFd dir_fd, PathRef filename,\n                    Options options = Options());\n\n  FdReader(FdReader&& that) = default;\n  FdReader& operator=(FdReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `FdReader`. This avoids\n  // constructing a temporary `FdReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n  template <\n      typename DependentSrc = Src,\n      std::enable_if_t<std::is_constructible_v<DependentSrc, int>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(int src, Options options = Options());\n  template <typename DependentSrc = Src,\n            std::enable_if_t<std::conjunction_v<FdSupportsOpen<DependentSrc>,\n                                                SupportsReset<DependentSrc>>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(PathInitializer filename,\n                                          Options options = Options());\n  template <typename DependentSrc = Src,\n            std::enable_if_t<std::conjunction_v<FdSupportsOpenAt<DependentSrc>,\n                                                SupportsReset<DependentSrc>>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(UnownedFd dir_fd,\n                                          PathInitializer filename,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the fd being read from.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  FdHandle SrcFdHandle() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n  int SrcFd() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get().get();\n  }\n\n  // An optimized implementation in a derived class, avoiding a virtual call.\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.get().filename();\n  }\n\n protected:\n  void Done() override;\n\n private:\n  template <typename DependentSrc = Src,\n            std::enable_if_t<FdSupportsOpen<DependentSrc>::value, int> = 0>\n  void OpenImpl(PathInitializer filename, Options&& options);\n  template <typename DependentSrc = Src,\n            std::enable_if_t<FdSupportsOpenAt<DependentSrc>::value, int> = 0>\n  void OpenAtImpl(UnownedFd dir_fd, PathRef filename, Options&& options);\n\n  // The object providing and possibly owning the fd being read from.\n  Dependency<FdHandle, Src> src_;\n};\n\nexplicit FdReader(Closed) -> FdReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit FdReader(Src&& src,\n                  FdReaderBase::Options options = FdReaderBase::Options())\n    -> FdReader<std::conditional_t<\n        std::disjunction_v<std::is_convertible<Src&&, int>,\n                           std::is_convertible<Src&&, PathInitializer>>,\n        OwnedFd, TargetT<Src>>>;\nexplicit FdReader(UnownedFd dir_fd, PathRef filename,\n                  FdReaderBase::Options options = FdReaderBase::Options())\n    -> FdReader<OwnedFd>;\n\n// Implementation details follow.\n\ninline FdReaderBase::FdReaderBase(BufferOptions buffer_options,\n                                  bool growing_source)\n    : BufferedReader(buffer_options), growing_source_(growing_source) {}\n\ninline FdReaderBase::FdReaderBase(FdReaderBase&& that) noexcept\n    : BufferedReader(static_cast<BufferedReader&&>(that)),\n      has_independent_pos_(that.has_independent_pos_),\n      growing_source_(that.growing_source_),\n      supports_random_access_(\n          std::exchange(that.supports_random_access_, false)),\n      random_access_status_(std::move(that.random_access_status_))\n#ifdef _WIN32\n      ,\n      original_mode_(that.original_mode_)\n#endif\n{\n}\n\ninline FdReaderBase& FdReaderBase::operator=(FdReaderBase&& that) noexcept {\n  BufferedReader::operator=(static_cast<BufferedReader&&>(that));\n  has_independent_pos_ = that.has_independent_pos_;\n  growing_source_ = that.growing_source_;\n  supports_random_access_ = std::exchange(that.supports_random_access_, false);\n  random_access_status_ = std::move(that.random_access_status_);\n#ifdef _WIN32\n  original_mode_ = that.original_mode_;\n#endif\n  return *this;\n}\n\ninline void FdReaderBase::Reset(Closed) {\n  BufferedReader::Reset(kClosed);\n  has_independent_pos_ = false;\n  growing_source_ = false;\n  supports_random_access_ = false;\n  random_access_status_ = absl::OkStatus();\n#ifdef _WIN32\n  original_mode_ = std::nullopt;\n#endif\n}\n\ninline void FdReaderBase::Reset(BufferOptions buffer_options,\n                                bool growing_source) {\n  BufferedReader::Reset(buffer_options);\n  has_independent_pos_ = false;\n  growing_source_ = growing_source;\n  supports_random_access_ = false;\n  random_access_status_ = absl::OkStatus();\n#ifdef _WIN32\n  original_mode_ = std::nullopt;\n#endif\n}\n\ntemplate <typename Src>\ninline FdReader<Src>::FdReader(Initializer<Src> src, Options options)\n    : FdReaderBase(options.buffer_options(), options.growing_source()),\n      src_(std::move(src)) {\n  Initialize(src_.get().get(), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<std::is_constructible_v<DependentSrc, int>, int>>\ninline FdReader<Src>::FdReader(int src ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                               Options options)\n    : FdReader(riegeli::Maker(src), std::move(options)) {}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<\n              std::conjunction_v<FdSupportsOpen<DependentSrc>,\n                                 std::is_default_constructible<DependentSrc>>,\n              int>>\ninline FdReader<Src>::FdReader(PathInitializer filename, Options options)\n    : FdReaderBase(options.buffer_options(), options.growing_source()),\n      src_(riegeli::Maker()) {\n  OpenImpl(std::move(filename), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<\n              std::conjunction_v<FdSupportsOpenAt<DependentSrc>,\n                                 std::is_default_constructible<DependentSrc>>,\n              int>>\ninline FdReader<Src>::FdReader(UnownedFd dir_fd, PathRef filename,\n                               Options options)\n    : FdReaderBase(options.buffer_options(), options.growing_source()),\n      src_(riegeli::Maker()) {\n  OpenAtImpl(std::move(dir_fd), filename, std::move(options));\n}\n\ntemplate <typename Src>\ninline void FdReader<Src>::Reset(Closed) {\n  FdReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void FdReader<Src>::Reset(Initializer<Src> src, Options options) {\n  FdReaderBase::Reset(options.buffer_options(), options.growing_source());\n  src_.Reset(std::move(src));\n  Initialize(src_.get().get(), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<std::is_constructible_v<DependentSrc, int>, int>>\ninline void FdReader<Src>::Reset(int src, Options options) {\n  Reset(riegeli::Maker(src), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<std::conjunction_v<FdSupportsOpen<DependentSrc>,\n                                              SupportsReset<DependentSrc>>,\n                           int>>\ninline void FdReader<Src>::Reset(PathInitializer filename, Options options) {\n  // In case `filename` is owned by `src_` and gets invalidated.\n  std::string filename_copy = std::move(filename);\n  riegeli::Reset(src_.manager());\n  FdReaderBase::Reset(options.buffer_options(), options.growing_source());\n  OpenImpl(std::move(filename_copy), std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<std::conjunction_v<FdSupportsOpenAt<DependentSrc>,\n                                              SupportsReset<DependentSrc>>,\n                           int>>\ninline void FdReader<Src>::Reset(UnownedFd dir_fd, PathInitializer filename,\n                                 Options options) {\n  // In case `filename` is owned by `src_` and gets invalidated.\n  std::string filename_copy = std::move(filename);\n  riegeli::Reset(src_.manager());\n  FdReaderBase::Reset(options.buffer_options(), options.growing_source());\n  OpenAtImpl(dir_fd, filename_copy, std::move(options));\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<FdSupportsOpen<DependentSrc>::value, int>>\nvoid FdReader<Src>::OpenImpl(PathInitializer filename, Options&& options) {\n  absl::Status status = src_.manager().Open(std::move(filename), options.mode(),\n                                            OwnedFd::kDefaultPermissions);\n  if (ABSL_PREDICT_FALSE(!status.ok())) {\n    FdReaderBase::Reset(kClosed);\n    FailWithoutAnnotation(std::move(status));\n    return;\n  }\n  InitializePos(src_.get().get(), std::move(options)\n#ifdef _WIN32\n                                      ,\n                /*mode_was_passed_to_open=*/true\n#endif\n  );\n}\n\ntemplate <typename Src>\ntemplate <typename DependentSrc,\n          std::enable_if_t<FdSupportsOpenAt<DependentSrc>::value, int>>\nvoid FdReader<Src>::OpenAtImpl(UnownedFd dir_fd, PathRef filename,\n                               Options&& options) {\n  absl::Status status =\n      src_.manager().OpenAt(std::move(dir_fd), filename, options.mode(),\n                            OwnedFd::kDefaultPermissions);\n  if (ABSL_PREDICT_FALSE(!status.ok())) {\n    FdReaderBase::Reset(kClosed);\n    FailWithoutAnnotation(std::move(status));\n    return;\n  }\n  InitializePos(src_.get().get(), std::move(options)\n#ifdef _WIN32\n                                      ,\n                /*mode_was_passed_to_open=*/true\n#endif\n  );\n}\n\ntemplate <typename Src>\nvoid FdReader<Src>::Done() {\n  FdReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (absl::Status status = src_.get().Close();\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      Fail(std::move(status));\n    }\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_FD_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/fd_writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef _WIN32\n\n// Make `pwrite()` and `ftruncate()` available.\n#if !defined(_XOPEN_SOURCE) || _XOPEN_SOURCE < 500\n#undef _XOPEN_SOURCE\n#define _XOPEN_SOURCE 500\n#endif\n\n// Make `off_t` 64-bit even on 32-bit systems.\n#undef _FILE_OFFSET_BITS\n#define _FILE_OFFSET_BITS 64\n\n#else\n\n#define WIN32_LEAN_AND_MEAN\n\n#endif\n\n#include \"riegeli/bytes/fd_writer.h\"\n\n#include <fcntl.h>\n#ifdef _WIN32\n#include <io.h>\n#endif\n#include <stddef.h>\n#include <stdio.h>\n#include <sys/types.h>\n#ifndef _WIN32\n#include <unistd.h>\n#else\n#include <windows.h>\n#endif\n\n#include <cerrno>\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/bits.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#ifdef _WIN32\n#include \"riegeli/base/errno_mapping.h\"\n#endif\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/global.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/type_id.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/fd_handle.h\"\n#include \"riegeli/bytes/fd_internal_for_cc.h\"\n#include \"riegeli/bytes/fd_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nTypeId FdWriterBase::GetTypeId() const { return TypeId::For<FdWriterBase>(); }\n\nvoid FdWriterBase::Initialize(int dest, Options&& options) {\n  RIEGELI_ASSERT_GE(dest, 0)\n      << \"Failed precondition of FdWriter: negative file descriptor\";\n  InitializePos(dest, std::move(options), /*mode_was_passed_to_open=*/false);\n}\n\nvoid FdWriterBase::InitializePos(int dest, Options&& options,\n                                 bool mode_was_passed_to_open) {\n  RIEGELI_ASSERT(!has_independent_pos_)\n      << \"Failed precondition of FdWriterBase::InitializePos(): \"\n         \"has_independent_pos_ not reset\";\n  RIEGELI_ASSERT_EQ(supports_random_access_, LazyBoolState::kUnknown)\n      << \"Failed precondition of FdWriterBase::InitializePos(): \"\n         \"supports_random_access_ not reset\";\n  RIEGELI_ASSERT_EQ(supports_read_mode_, LazyBoolState::kUnknown)\n      << \"Failed precondition of FdWriterBase::InitializePos(): \"\n         \"supports_read_mode_ not reset\";\n  RIEGELI_ASSERT_OK(random_access_status_)\n      << \"Failed precondition of FdWriterBase::InitializePos(): \"\n         \"random_access_status_ not reset\";\n  RIEGELI_ASSERT_OK(read_mode_status_)\n      << \"Failed precondition of FdWriterBase::InitializePos(): \"\n         \"read_mode_status_ not reset\";\n#ifndef _WIN32\n  if (!mode_was_passed_to_open) {\n    const int mode = fcntl(dest, F_GETFL);\n    if (ABSL_PREDICT_FALSE(mode < 0)) {\n      FailOperation(\"fcntl()\");\n      return;\n    }\n    options.set_mode(mode);\n  }\n  if ((options.mode() & O_ACCMODE) != O_RDWR) {\n    supports_read_mode_ = LazyBoolState::kFalse;\n    read_mode_status_ = Global([] {\n      return absl::UnimplementedError(\"Mode does not include O_RDWR\");\n    });\n  }\n#else   // _WIN32\n  RIEGELI_ASSERT_EQ(original_mode_, std::nullopt)\n      << \"Failed precondition of FdWriterBase::InitializePos(): \"\n         \"original_mode_ not reset\";\n  int text_mode = options.mode() &\n                  (_O_BINARY | _O_TEXT | _O_WTEXT | _O_U16TEXT | _O_U8TEXT);\n  if (mode_was_passed_to_open) {\n    if ((options.mode() & (_O_RDONLY | _O_WRONLY | _O_RDWR)) != _O_RDWR) {\n      supports_read_mode_ = LazyBoolState::kFalse;\n      read_mode_status_ = Global([] {\n        return absl::UnimplementedError(\"Mode does not include _O_RDWR\");\n      });\n    }\n  } else if (text_mode != 0) {\n    const int original_mode = _setmode(dest, text_mode);\n    if (ABSL_PREDICT_FALSE(original_mode < 0)) {\n      FailOperation(\"_setmode()\");\n      return;\n    }\n    original_mode_ = original_mode;\n  }\n  if (options.assumed_pos() == std::nullopt) {\n    if (text_mode == 0) {\n      // There is no `_getmode()`, but `_setmode()` returns the previous mode.\n      text_mode = _setmode(dest, _O_BINARY);\n      if (ABSL_PREDICT_FALSE(text_mode < 0)) {\n        FailOperation(\"_setmode()\");\n        return;\n      }\n      if (ABSL_PREDICT_FALSE(_setmode(dest, text_mode) < 0)) {\n        FailOperation(\"_setmode()\");\n        return;\n      }\n    }\n    if (text_mode != _O_BINARY) {\n      if (ABSL_PREDICT_FALSE(options.independent_pos() != std::nullopt)) {\n        Fail(absl::InvalidArgumentError(\n            \"FdWriterBase::Options::independent_pos() requires binary mode\"));\n        return;\n      }\n      options.set_assumed_pos(0);\n    }\n  }\n#endif  // _WIN32\n  if (options.assumed_pos() != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(options.independent_pos() != std::nullopt)) {\n      Fail(absl::InvalidArgumentError(\n          \"FdWriterBase::Options::assumed_pos() and independent_pos() \"\n          \"must not be both set\"));\n      return;\n    }\n    if (ABSL_PREDICT_FALSE(\n            *options.assumed_pos() >\n            Position{std::numeric_limits<fd_internal::Offset>::max()})) {\n      FailOverflow();\n      return;\n    }\n    set_start_pos(*options.assumed_pos());\n    supports_random_access_ = LazyBoolState::kFalse;\n    supports_read_mode_ = LazyBoolState::kFalse;\n    random_access_status_ = Global([] {\n      return absl::UnimplementedError(\n          \"FileWriterBase::Options::assumed_pos() excludes random access\");\n    });\n    read_mode_status_.Update(random_access_status_);\n  } else if (options.independent_pos() != std::nullopt) {\n    if (ABSL_PREDICT_FALSE((options.mode() & O_APPEND) != 0)) {\n      Fail(\n          absl::InvalidArgumentError(\"FdWriterBase::Options::independent_pos() \"\n                                     \"is incompatible with append mode\"));\n      return;\n    }\n    has_independent_pos_ = true;\n    if (ABSL_PREDICT_FALSE(\n            *options.independent_pos() >\n            Position{std::numeric_limits<fd_internal::Offset>::max()})) {\n      FailOverflow();\n      return;\n    }\n    set_start_pos(*options.independent_pos());\n    supports_random_access_ = LazyBoolState::kTrue;\n    if (\n#ifdef _WIN32\n        mode_was_passed_to_open &&\n#endif\n        supports_read_mode_ == LazyBoolState::kUnknown) {\n      supports_read_mode_ = LazyBoolState::kTrue;\n    }\n  } else {\n    const fd_internal::Offset file_pos = fd_internal::LSeek(\n        dest, 0, (options.mode() & O_APPEND) != 0 ? SEEK_END : SEEK_CUR);\n    if (file_pos < 0) {\n      // Random access is not supported. Assume 0 as the initial position.\n      supports_random_access_ = LazyBoolState::kFalse;\n      supports_read_mode_ = LazyBoolState::kFalse;\n      random_access_status_ =\n          FailedOperationStatus(fd_internal::kLSeekFunctionName);\n      read_mode_status_.Update(random_access_status_);\n      return;\n    }\n    set_start_pos(IntCast<Position>(file_pos));\n    if ((options.mode() & O_APPEND) != 0) {\n      // `fd_internal::LSeek(SEEK_END)` succeeded.\n      supports_random_access_ = LazyBoolState::kFalse;\n      if (\n#ifdef _WIN32\n          mode_was_passed_to_open &&\n#endif\n          supports_read_mode_ == LazyBoolState::kUnknown) {\n        supports_read_mode_ = LazyBoolState::kTrue;\n      }\n      random_access_status_ = Global([] {\n        return absl::UnimplementedError(\"Append mode excludes random access\");\n      });\n    } else {\n      // `fd_internal::LSeek(SEEK_CUR)` succeeded, and\n      // `fd_internal::LSeek(SEEK_END)` will be checked later.\n      //  `supports_random_access_` and `supports_read_mode_` are left as\n      // `LazyBoolState::kUnknown`.\n    }\n  }\n  BeginRun();\n}\n\nvoid FdWriterBase::Done() {\n  BufferedWriter::Done();\n#ifdef _WIN32\n  if (original_mode_ != std::nullopt) {\n    const int dest = DestFd();\n    if (ABSL_PREDICT_FALSE(_setmode(dest, *original_mode_) < 0)) {\n      FailOperation(\"_setmode()\");\n    }\n  }\n#endif  // _WIN32\n  random_access_status_ = absl::OkStatus();\n  read_mode_status_ = absl::OkStatus();\n  associated_reader_.Reset();\n}\n\ninline absl::Status FdWriterBase::FailedOperationStatus(\n    absl::string_view operation) {\n  const int error_number = errno;\n  RIEGELI_ASSERT_NE(error_number, 0)\n      << \"Failed precondition of FdWriterBase::FailedOperationStatus(): \"\n         \"zero errno\";\n  return absl::ErrnoToStatus(error_number, absl::StrCat(operation, \" failed\"));\n}\n\nbool FdWriterBase::FailOperation(absl::string_view operation) {\n  return Fail(FailedOperationStatus(operation));\n}\n\n#ifdef _WIN32\n\nabsl::Status FdWriterBase::FailedWindowsOperationStatus(\n    absl::string_view operation) {\n  const DWORD error_number = GetLastError();\n  RIEGELI_ASSERT_NE(error_number, 0)\n      << \"Failed precondition of FdWriterBase::FailedWindowsOperationStatus(): \"\n         \"zero error code\";\n  return WindowsErrorToStatus(IntCast<uint32_t>(error_number),\n                              absl::StrCat(operation, \" failed\"));\n}\n\nbool FdWriterBase::FailWindowsOperation(absl::string_view operation) {\n  return Fail(FailedWindowsOperationStatus(operation));\n}\n\n#endif  // _WIN32\n\nabsl::Status FdWriterBase::AnnotateStatusImpl(absl::Status status) {\n  return BufferedWriter::AnnotateStatusImpl(\n      Annotate(status, absl::StrCat(\"writing \", filename())));\n}\n\ninline absl::Status FdWriterBase::SizeStatus() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of FdWriterBase::SizeStatus()\";\n  const int dest = DestFd();\n  if (fd_internal::LSeek(dest, 0, SEEK_END) < 0) {\n    // Not supported.\n    return FailedOperationStatus(fd_internal::kLSeekFunctionName);\n  }\n  // Supported.\n  if (ABSL_PREDICT_FALSE(\n          fd_internal::LSeek(dest, IntCast<fd_internal::Offset>(start_pos()),\n                             SEEK_SET) < 0)) {\n    FailOperation(fd_internal::kLSeekFunctionName);\n    return status();\n  }\n  return absl::OkStatus();\n}\n\nbool FdWriterBase::SupportsRandomAccess() {\n  if (ABSL_PREDICT_TRUE(supports_random_access_ != LazyBoolState::kUnknown)) {\n    return supports_random_access_ == LazyBoolState::kTrue;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Status status = SizeStatus();\n  if (!status.ok()) {\n    // Not supported.\n    supports_random_access_ = LazyBoolState::kFalse;\n    supports_read_mode_ = LazyBoolState::kFalse;\n    random_access_status_ = std::move(status);\n    read_mode_status_.Update(random_access_status_);\n    return false;\n  }\n  // Supported.\n  supports_random_access_ = LazyBoolState::kTrue;\n#ifndef _WIN32\n  if (supports_read_mode_ == LazyBoolState::kUnknown) {\n    supports_read_mode_ = LazyBoolState::kTrue;\n  }\n#endif\n  return true;\n}\n\nbool FdWriterBase::SupportsReadMode() {\n  if (ABSL_PREDICT_TRUE(supports_read_mode_ != LazyBoolState::kUnknown)) {\n    return supports_read_mode_ == LazyBoolState::kTrue;\n  }\n#ifndef _WIN32\n  RIEGELI_ASSERT_EQ(supports_random_access_, LazyBoolState::kUnknown)\n      << \"Failed invariant of FdWriterBase: \"\n         \"supports_random_access_ is resolved but supports_read_mode_ is not\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::Status status = SizeStatus();\n  if (!status.ok()) {\n    // Not supported.\n    supports_random_access_ = LazyBoolState::kFalse;\n    supports_read_mode_ = LazyBoolState::kFalse;\n    random_access_status_ = std::move(status);\n    read_mode_status_ = random_access_status_;\n    return false;\n  }\n  // Supported.\n  supports_random_access_ = LazyBoolState::kTrue;\n  supports_read_mode_ = LazyBoolState::kTrue;\n  return true;\n#else   // _WIN32\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (supports_random_access_ == LazyBoolState::kUnknown) {\n    // It is unknown whether even size is supported.\n    absl::Status status = SizeStatus();\n    if (!status.ok()) {\n      // Not supported.\n      supports_random_access_ = LazyBoolState::kFalse;\n      supports_read_mode_ = LazyBoolState::kFalse;\n      random_access_status_ = std::move(status);\n      read_mode_status_ = random_access_status_;\n      return false;\n    }\n    // Size is supported.\n    supports_random_access_ = LazyBoolState::kTrue;\n  }\n\n  const int dest = DestFd();\n  if (has_independent_pos_) {\n    const HANDLE file_handle = reinterpret_cast<HANDLE>(_get_osfhandle(dest));\n    if (file_handle == INVALID_HANDLE_VALUE ||\n        file_handle == reinterpret_cast<HANDLE>(-2)) {\n      // Not supported.\n      supports_read_mode_ = LazyBoolState::kFalse;\n      read_mode_status_ = absl::UnimplementedError(\"Invalid _get_osfhandle()\");\n      return false;\n    }\n    char buf[1];\n    DWORD length_read;\n    OVERLAPPED overlapped{};\n    overlapped.Offset = IntCast<DWORD>(start_pos() & 0xffffffff);\n    overlapped.OffsetHigh = IntCast<DWORD>(start_pos() >> 32);\n    if (!ReadFile(file_handle, &buf, 1, &length_read, &overlapped) &&\n        GetLastError() != ERROR_HANDLE_EOF) {\n      supports_read_mode_ = LazyBoolState::kFalse;\n      read_mode_status_ = FailedWindowsOperationStatus(\"ReadFile()\");\n      return false;\n    }\n    // Supported.\n    supports_read_mode_ = LazyBoolState::kTrue;\n    return true;\n  } else {\n    if (fd_internal::LSeek(dest, 0, SEEK_END) < 0) {\n      // Not supported.\n      supports_read_mode_ = LazyBoolState::kFalse;\n      read_mode_status_ =\n          FailedOperationStatus(fd_internal::kLSeekFunctionName);\n      return false;\n    }\n    char buf[1];\n    if (_read(dest, buf, 1) < 0) {\n      // Not supported.\n      supports_read_mode_ = LazyBoolState::kFalse;\n      read_mode_status_ = FailedOperationStatus(\"_read()\");\n    } else {\n      // Supported.\n      supports_read_mode_ = LazyBoolState::kTrue;\n    }\n    if (ABSL_PREDICT_FALSE(\n            fd_internal::LSeek(dest, IntCast<fd_internal::Offset>(start_pos()),\n                               SEEK_SET) < 0)) {\n      return FailOperation(fd_internal::kLSeekFunctionName);\n    }\n    return supports_read_mode_ == LazyBoolState::kTrue;\n  }\n#endif  // _WIN32\n}\n\ninline bool FdWriterBase::WriteMode() {\n  if (ABSL_PREDICT_TRUE(!read_mode_)) return true;\n  read_mode_ = false;\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const int dest = DestFd();\n  return SeekInternal(dest, start_pos());\n}\n\nbool FdWriterBase::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of BufferedWriter::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedWriter::WriteInternal()\";\n  if (ABSL_PREDICT_FALSE(!WriteMode())) return false;\n  const int dest = DestFd();\n  if (ABSL_PREDICT_FALSE(\n          src.size() >\n          Position{std::numeric_limits<fd_internal::Offset>::max()} -\n              start_pos())) {\n    return FailOverflow();\n  }\n  do {\n#ifndef _WIN32\n  again:\n    const size_t length_to_write = UnsignedMin(\n        src.size(),\n        absl::bit_floor(size_t{std::numeric_limits<ssize_t>::max()}),\n        // Darwin and FreeBSD cannot write more than 2 GB - 1 at a time.\n        // Limit to 1 GB for better alignment of writes.\n        // https://codereview.appspot.com/89900044#msg9\n        size_t{1} << 30);\n    const ssize_t length_written =\n        has_independent_pos_ ? pwrite(dest, src.data(), length_to_write,\n                                      IntCast<fd_internal::Offset>(start_pos()))\n                             : write(dest, src.data(), length_to_write);\n    if (ABSL_PREDICT_FALSE(length_written < 0)) {\n      if (errno == EINTR) goto again;\n      return FailOperation(has_independent_pos_ ? \"pwrite()\" : \"write()\");\n    }\n#else   // _WIN32\n    DWORD length_to_write;\n    DWORD length_written;\n    if (has_independent_pos_) {\n      const HANDLE file_handle = reinterpret_cast<HANDLE>(_get_osfhandle(dest));\n      if (ABSL_PREDICT_FALSE(file_handle == INVALID_HANDLE_VALUE ||\n                             file_handle == reinterpret_cast<HANDLE>(-2))) {\n        return FailWindowsOperation(\"_get_osfhandle()\");\n      }\n      length_to_write = UnsignedMin(\n          src.size(), absl::bit_floor(std::numeric_limits<DWORD>::max()));\n      OVERLAPPED overlapped{};\n      overlapped.Offset = IntCast<DWORD>(start_pos() & 0xffffffff);\n      overlapped.OffsetHigh = IntCast<DWORD>(start_pos() >> 32);\n      if (ABSL_PREDICT_FALSE(!WriteFile(file_handle, src.data(),\n                                        length_to_write, &length_written,\n                                        &overlapped))) {\n        return FailWindowsOperation(\"WriteFile()\");\n      }\n    } else {\n      length_to_write = UnsignedMin(\n          src.size(),\n          absl::bit_floor(unsigned{std::numeric_limits<int>::max()}));\n      const int length_written_int =\n          _write(dest, src.data(), IntCast<unsigned>(length_to_write));\n      if (ABSL_PREDICT_FALSE(length_written_int < 0)) {\n        return FailOperation(\"_write()\");\n      }\n      length_written = IntCast<DWORD>(length_written_int);\n    }\n#endif  // _WIN32\n    RIEGELI_ASSERT_GT(length_written, 0)\n#ifndef _WIN32\n        << (has_independent_pos_ ? \"pwrite()\" : \"write()\")\n#else\n        << (has_independent_pos_ ? \"WriteFile()\" : \"_write()\")\n#endif\n        << \" returned 0\";\n    RIEGELI_ASSERT_LE(UnsignedCast(length_written), length_to_write)\n#ifndef _WIN32\n        << (has_independent_pos_ ? \"pwrite()\" : \"write()\")\n#else\n        << (has_independent_pos_ ? \"WriteFile()\" : \"_write()\")\n#endif\n        << \" wrote more than requested\";\n    move_start_pos(IntCast<size_t>(length_written));\n    src.remove_prefix(IntCast<size_t>(length_written));\n  } while (!src.empty());\n  return true;\n}\n\nbool FdWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (src.fill() != '\\0' || !FdWriterBase::SupportsRandomAccess()) {\n    return BufferedWriter::WriteSlow(src);\n  }\n  const std::optional<Position> size = SizeImpl();\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) return false;\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"BufferedWriter::SizeImpl() flushes the buffer\";\n  if (ABSL_PREDICT_FALSE(\n          src.size() >\n          Position{std::numeric_limits<fd_internal::Offset>::max()} -\n              start_pos())) {\n    return FailOverflow();\n  }\n  const Position new_pos = start_pos() + src.size();\n  if (new_pos < *size) {\n    // Existing data after zeros must be preserved. Optimization below is not\n    // feasible.\n    return BufferedWriter::WriteSlow(src);\n  }\n\n  // Optimize extending with zeros by calling `ftruncate()` (`_chsize_s()` on\n  // Windows).\n  const int dest = DestFd();\n  if (start_pos() < *size) {\n    // Remove the part to be overwritten with zeros.\n    if (ABSL_PREDICT_FALSE(!TruncateInternal(dest, start_pos()))) return false;\n  }\n  // Extend with zeros.\n  if (ABSL_PREDICT_FALSE(!TruncateInternal(dest, new_pos))) return false;\n  return SeekInternal(dest, new_pos);\n}\n\nbool FdWriterBase::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!BufferedWriter::FlushImpl(flush_type))) return false;\n  switch (flush_type) {\n    case FlushType::kFromObject:\n    case FlushType::kFromProcess:\n      return true;\n    case FlushType::kFromMachine: {\n      const int dest = DestFd();\n#ifndef _WIN32\n      if (ABSL_PREDICT_FALSE(fsync(dest) < 0)) {\n        return FailOperation(\"fsync()\");\n      }\n#else   // _WIN32\n      if (ABSL_PREDICT_FALSE(_commit(dest) < 0)) {\n        return FailOperation(\"_commit()\");\n      }\n#endif  // _WIN32\n      return true;\n    }\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown flush type: \" << static_cast<int>(flush_type);\n}\n\nbool FdWriterBase::FlushBehindBuffer(absl::string_view src,\n                                     FlushType flush_type) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::FlushBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!WriteMode())) return false;\n  return BufferedWriter::FlushBehindBuffer(src, flush_type);\n}\n\ninline bool FdWriterBase::SeekInternal(int dest, Position new_pos) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of FdWriterBase::SeekInternal(): \"\n         \"buffer not empty\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of FdWriterBase::SeekInternal()\";\n  if (!has_independent_pos_) {\n    if (ABSL_PREDICT_FALSE(\n            fd_internal::LSeek(dest, IntCast<fd_internal::Offset>(new_pos),\n                               SEEK_SET) < 0)) {\n      return FailOperation(fd_internal::kLSeekFunctionName);\n    }\n  }\n  set_start_pos(new_pos);\n  return true;\n}\n\nbool FdWriterBase::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of BufferedWriter::SeekBehindBuffer(): \"\n         \"position unchanged, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!FdWriterBase::SupportsRandomAccess())) {\n    if (ok()) Fail(random_access_status_);\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  read_mode_ = false;\n  const int dest = DestFd();\n  if (new_pos > start_pos()) {\n    // Seeking forwards.\n    fd_internal::StatInfo stat_info;\n    if (ABSL_PREDICT_FALSE(fd_internal::FStat(dest, &stat_info) < 0)) {\n      return FailOperation(fd_internal::kFStatFunctionName);\n    }\n    if (ABSL_PREDICT_FALSE(new_pos > IntCast<Position>(stat_info.st_size))) {\n      // File ends.\n      SeekInternal(dest, IntCast<Position>(stat_info.st_size));\n      return false;\n    }\n  }\n  return SeekInternal(dest, new_pos);\n}\n\nstd::optional<Position> FdWriterBase::SizeBehindBuffer() {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::SizeBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!FdWriterBase::SupportsRandomAccess())) {\n    if (ok()) Fail(random_access_status_);\n    return std::nullopt;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  const int dest = DestFd();\n  fd_internal::StatInfo stat_info;\n  if (ABSL_PREDICT_FALSE(fd_internal::FStat(dest, &stat_info) < 0)) {\n    FailOperation(fd_internal::kFStatFunctionName);\n    return std::nullopt;\n  }\n  return IntCast<Position>(stat_info.st_size);\n}\n\ninline bool FdWriterBase::TruncateInternal(int dest, Position new_size) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of FdWriterBase::TruncateInternal(): \"\n         \"buffer not empty\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of FdWriterBase::TruncateInternal()\";\n#ifndef _WIN32\nagain:\n  if (ABSL_PREDICT_FALSE(\n          ftruncate(dest, IntCast<fd_internal::Offset>(new_size)) < 0)) {\n    if (errno == EINTR) goto again;\n    return FailOperation(\"ftruncate()\");\n  }\n#else   // _WIN32\n  if (ABSL_PREDICT_FALSE(\n          _chsize_s(dest, IntCast<fd_internal::Offset>(new_size)) != 0)) {\n    return FailOperation(\"_chsize_s()\");\n  }\n#endif  // _WIN32\n  return true;\n}\n\nbool FdWriterBase::TruncateBehindBuffer(Position new_size) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::TruncateBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  read_mode_ = false;\n  const int dest = DestFd();\n  if (new_size >= start_pos()) {\n    // Seeking forwards.\n    fd_internal::StatInfo stat_info;\n    if (ABSL_PREDICT_FALSE(fd_internal::FStat(dest, &stat_info) < 0)) {\n      return FailOperation(fd_internal::kFStatFunctionName);\n    }\n    if (ABSL_PREDICT_FALSE(new_size > IntCast<Position>(stat_info.st_size))) {\n      // File ends.\n      SeekInternal(dest, IntCast<Position>(stat_info.st_size));\n      return false;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!TruncateInternal(dest, new_size))) return false;\n  return SeekInternal(dest, new_size);\n}\n\nReader* FdWriterBase::ReadModeBehindBuffer(Position initial_pos) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::ReadModeBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!FdWriterBase::SupportsReadMode())) {\n    if (ok()) Fail(read_mode_status_);\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  FdReader<UnownedFd>* const reader = associated_reader_.ResetReader(\n      UnownedFd(DestFdHandle()),\n      FdReaderBase::Options()\n          .set_independent_pos(has_independent_pos_\n                                   ? std::make_optional(initial_pos)\n                                   : std::nullopt)\n          .set_buffer_options(buffer_options()));\n  if (!has_independent_pos_) reader->Seek(initial_pos);\n  read_mode_ = true;\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/fd_writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_FD_WRITER_H_\n#define RIEGELI_BYTES_FD_WRITER_H_\n\n#include <fcntl.h>\n#include <stdint.h>\n#ifdef _WIN32\n#include <sys/stat.h>\n#endif\n\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/type_id.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/fd_handle.h\"\n#include \"riegeli/bytes/fd_internal.h\"\n#include \"riegeli/bytes/path_ref.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\ntemplate <typename Src>\nclass FdReader;\nclass Reader;\n\n// Template parameter independent part of `FdWriter`.\nclass FdWriterBase : public BufferedWriter {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // If `FdWriter` opens a fd with a filename, `mode()` is the second argument\n    // of `open()` (on Windows: `_open()`) and specifies the open mode and\n    // flags, typically one of:\n    //  * `O_WRONLY | O_CREAT | O_TRUNC`\n    //    (on Windows: `_O_WRONLY | _O_CREAT | _O_TRUNC | _O_BINARY`)\n    //  * `O_WRONLY | O_CREAT | O_APPEND`\n    //    (on Windows: `_O_WRONLY | _O_CREAT | _O_APPEND | _O_BINARY`)\n    //\n    // It must include either `O_WRONLY` or `O_RDWR` (on Windows: `_O_WRONLY` or\n    // `_O_RDWR`).\n    //\n    // `mode()` can also be changed with `set_existing()`, `set_read()`,\n    // `set_append()`, `set_exclusive()`, `set_inheritable()`, and `set_text()`.\n    //\n    // Default: `O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC`\n    // (on Windows: `_O_WRONLY | _O_CREAT | _O_TRUNC | _O_BINARY |\n    //               _O_NOINHERIT`).\n    Options& set_mode(int mode) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      mode_ = mode;\n      return *this;\n    }\n    Options&& set_mode(int mode) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_mode(mode));\n    }\n    int mode() const { return mode_; }\n\n    // If `false`, the file will be created if it does not exist, or it will be\n    // truncated to empty if it exists. This implies `set_read(false)` and\n    // `set_append(false)` unless overwritten later.\n    //\n    // If `true`, the file must already exist, and its contents will not be\n    // truncated. Writing will start from the beginning, with random access\n    // supported. This implies `set_read(true)` unless overwritten later.\n    //\n    // If `FdWriter` writes to an already open fd, `existing()` has no effect.\n    //\n    // `set_existing()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_existing(bool existing) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n#ifndef _WIN32\n      mode_ = (mode_ & ~(O_ACCMODE | O_CREAT | O_TRUNC | O_APPEND)) |\n              (existing ? O_RDWR : O_WRONLY | O_CREAT | O_TRUNC);\n#else\n      mode_ = (mode_ & ~(_O_RDONLY | _O_WRONLY | _O_RDWR | _O_CREAT | _O_TRUNC |\n                         _O_APPEND)) |\n              (existing ? _O_RDWR : _O_WRONLY | _O_CREAT | _O_TRUNC);\n#endif\n      return *this;\n    }\n    Options&& set_existing(bool existing) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_existing(existing));\n    }\n    bool existing() const {\n#ifndef _WIN32\n      return (mode_ & O_CREAT) == 0;\n#else\n      return (mode_ & _O_CREAT) == 0;\n#endif\n    }\n\n    // If `false`, the fd will be open for writing.\n    //\n    // If `true`, the fd will be open for writing and reading (using\n    // `ReadMode()`).\n    //\n    // If `FdWriter` writes to an already open fd, `read()` has no effect.\n    //\n    // `set_read()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_read(bool read) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n#ifndef _WIN32\n      mode_ = (mode_ & ~O_ACCMODE) | (read ? O_RDWR : O_WRONLY);\n#else\n      mode_ = (mode_ & ~(_O_RDONLY | _O_WRONLY | _O_RDWR)) |\n              (read ? _O_RDWR : _O_WRONLY);\n#endif\n      return *this;\n    }\n    Options&& set_read(bool read) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_read(read));\n    }\n    bool read() const {\n#ifndef _WIN32\n      return (mode_ & O_ACCMODE) == O_RDWR;\n#else\n      return (mode_ & (_O_RDONLY | _O_WRONLY | _O_RDWR)) == _O_RDWR;\n#endif\n    }\n\n    // If `false`, the file will be truncated to empty if it exists.\n    //\n    // If `true`, the file will not be truncated if it exists, and writing will\n    // always happen at its end.\n    //\n    // If `FdWriter` writes to an already open fd, `append()` has effect only on\n    // Windows. If `assumed_pos()` is not set, `append()` should be `true` if\n    // the fd was originally open in append mode. This allows to determine the\n    // effective initial position and lets `SupportsRandomAccess()` correctly\n    // return `false`.\n    //\n    // `set_append()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_append(bool append) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n#ifndef _WIN32\n      mode_ = (mode_ & ~(O_TRUNC | O_APPEND)) | (append ? O_APPEND : O_TRUNC);\n#else\n      mode_ =\n          (mode_ & ~(_O_TRUNC | _O_APPEND)) | (append ? _O_APPEND : _O_TRUNC);\n#endif\n      return *this;\n    }\n    Options&& set_append(bool append) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_append(append));\n    }\n    bool append() const {\n#ifndef _WIN32\n      return (mode_ & O_APPEND) != 0;\n#else\n      return (mode_ & _O_APPEND) != 0;\n#endif\n    }\n\n    // If `false`, the file will be created if it does not exist, or it will be\n    // opened if it exists (truncated to empty by default, or left unchanged if\n    // `set_existing(true)` or `set_append(true)` was used).\n    //\n    // If `true`, the file will be created if it does not exist, or opening will\n    // fail if it exists.\n    //\n    // If `FdWriter` writes to an already open fd, `exclusive()` has no effect.\n    //\n    // `set_exclusive()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_exclusive(bool exclusive) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n#ifndef _WIN32\n      mode_ = (mode_ & ~O_EXCL) | (exclusive ? O_EXCL : 0);\n#else\n      mode_ = (mode_ & ~_O_EXCL) | (exclusive ? _O_EXCL : 0);\n#endif\n      return *this;\n    }\n    Options&& set_exclusive(bool exclusive) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_exclusive(exclusive));\n    }\n    bool exclusive() const {\n#ifndef _WIN32\n      return (mode_ & O_EXCL) != 0;\n#else\n      return (mode_ & _O_EXCL) != 0;\n#endif\n    }\n\n    // If `false`, `execve()` (`CreateProcess()` on Windows) will close the fd.\n    //\n    // If `true`, the fd will remain open across `execve()` (`CreateProcess()`\n    // on Windows).\n    //\n    // If `FdWriter` writes to an already open fd, `inheritable()` has no\n    // effect.\n    //\n    // `set_inheritable()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_inheritable(bool inheritable) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      mode_ = (mode_ & ~fd_internal::kCloseOnExec) |\n              (inheritable ? 0 : fd_internal::kCloseOnExec);\n      return *this;\n    }\n    Options&& set_inheritable(bool inheritable) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_inheritable(inheritable));\n    }\n    bool inheritable() const {\n      return (mode_ & fd_internal::kCloseOnExec) == 0;\n    }\n\n    // If `false`, data will be written directly to the file. This is called the\n    // binary mode.\n    //\n    // If `true`, text mode translation will be applied on Windows:\n    // LF characters are translated to CR-LF.\n    //\n    // It is recommended to use `WriteLine()` or `TextWriter` instead, which\n    // expect a binary mode `Writer`.\n    //\n    // `set_text()` has an effect only on Windows. It is applicable whenever\n    // `FdWriter` opens a fd with a filename or writes to an already open fd.\n    //\n    // `set_text()` affects `mode()`.\n    //\n    // Default: `false`.\n    Options& set_text(ABSL_ATTRIBUTE_UNUSED bool text) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n#ifdef _WIN32\n      mode_ =\n          (mode_ & ~(_O_BINARY | _O_TEXT | _O_WTEXT | _O_U16TEXT | _O_U8TEXT)) |\n          (text ? _O_TEXT : _O_BINARY);\n#endif\n      return *this;\n    }\n    Options&& set_text(bool text) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_text(text));\n    }\n    // No `text()` getter is provided. On Windows `mode()` can have unspecified\n    // text mode, resolved using `_get_fmode()`. Not on Windows the concept does\n    // not exist.\n\n    // Permissions to use in case a new file is created (9 bits, except on\n    // Windows: `_S_IREAD`, `_S_IWRITE`, or `_S_IREAD | _S_IWRITE`). The\n    // effective permissions are modified by the process' umask.\n    //\n    // Default: `0666` (on Windows: `_S_IREAD | _S_IWRITE`).\n    Options& set_permissions(OwnedFd::Permissions permissions) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      permissions_ = permissions;\n      return *this;\n    }\n    Options&& set_permissions(OwnedFd::Permissions permissions) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_permissions(permissions));\n    }\n    OwnedFd::Permissions permissions() const { return permissions_; }\n\n    // If `std::nullopt`, the current position reported by `pos()` corresponds\n    // to the current fd position if possible, otherwise 0 is assumed as the\n    // initial position. Random access is supported if the fd supports random\n    // access. On Windows binary mode is also required.\n    //\n    // If not `std::nullopt`, this position is assumed initially, to be reported\n    // by `pos()`. It does not need to correspond to the current fd position.\n    // Random access is not supported.\n    //\n    // `assumed_pos()` and `independent_pos()` must not be both set.\n    //\n    // Default: `std::nullopt`.\n    Options& set_assumed_pos(std::optional<Position> assumed_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      assumed_pos_ = assumed_pos;\n      return *this;\n    }\n    Options&& set_assumed_pos(std::optional<Position> assumed_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_pos(assumed_pos));\n    }\n    std::optional<Position> assumed_pos() const { return assumed_pos_; }\n\n    // If `std::nullopt`, `FdWriter` writes at the current fd position.\n    //\n    // If not `std::nullopt`, `FdWriter` writes starting from this position.\n    // The current fd position is not disturbed except on Windows, where seeking\n    // and writing is nevertheless atomic. This is useful for multiple writers\n    // concurrently writing to disjoint regions of the same file. The fd must\n    // support `pwrite()` (`_get_osfhandle()` and `WriteFile()` with\n    // `OVERLAPPED*` on Windows). On Windows binary mode is also required.\n    //\n    // `assumed_pos()` and `independent_pos()` must not be both set.\n    //\n    // If the original open mode of the fd includes `O_APPEND` then\n    // `independent_pos()` must not be set.\n    //\n    // Default: `std::nullopt`.\n    Options& set_independent_pos(std::optional<Position> independent_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      independent_pos_ = independent_pos;\n      return *this;\n    }\n    Options&& set_independent_pos(std::optional<Position> independent_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_independent_pos(independent_pos));\n    }\n    std::optional<Position> independent_pos() const { return independent_pos_; }\n\n   private:\n#ifndef _WIN32\n    int mode_ = O_WRONLY | O_CREAT | O_TRUNC | fd_internal::kCloseOnExec;\n#else\n    int mode_ =\n        _O_WRONLY | _O_CREAT | _O_TRUNC | _O_BINARY | fd_internal::kCloseOnExec;\n#endif\n    OwnedFd::Permissions permissions_ = OwnedFd::kDefaultPermissions;\n    std::optional<Position> assumed_pos_;\n    std::optional<Position> independent_pos_;\n  };\n\n  // Returns the `FdHandle` being written to. Unchanged by `Close()`.\n  virtual FdHandle DestFdHandle() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the fd being written to. If the fd is owned then changed to -1 by\n  // `Close()`, otherwise unchanged.\n  virtual int DestFd() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  TypeId GetTypeId() const override;\n\n  // Returns the filename of the fd being written to, or \"<none>\" for\n  // closed-constructed or moved-from `FdWriter`. Unchanged by `Close()`.\n  //\n  // If the constructor from filename was used, this is the filename passed to\n  // the constructor, otherwise a filename is inferred from the fd. This can be\n  // a placeholder instead of a real filename if the fd does not refer to a\n  // named file or inferring the filename is not supported.\n  //\n  // If `Dest` does not support `filename()`, returns \"<unsupported>\".\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return DestFdHandle().filename();\n  }\n\n  bool SupportsRandomAccess() override;\n  bool SupportsReadMode() override;\n\n protected:\n  explicit FdWriterBase(Closed) noexcept : BufferedWriter(kClosed) {}\n\n  explicit FdWriterBase(BufferOptions buffer_options);\n\n  FdWriterBase(FdWriterBase&& that) noexcept;\n  FdWriterBase& operator=(FdWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options);\n  void Initialize(int dest, Options&& options);\n  void InitializePos(int dest, Options&& options, bool mode_was_passed_to_open);\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation);\n#ifdef _WIN32\n  ABSL_ATTRIBUTE_COLD bool FailWindowsOperation(absl::string_view operation);\n#endif\n\n  void Done() override;\n  absl::Status AnnotateStatusImpl(absl::Status status) override;\n  bool WriteInternal(absl::string_view src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool FlushImpl(FlushType flush_type) override;\n  bool FlushBehindBuffer(absl::string_view src, FlushType flush_type) override;\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::optional<Position> SizeBehindBuffer() override;\n  bool TruncateBehindBuffer(Position new_size) override;\n  Reader* ReadModeBehindBuffer(Position initial_pos) override;\n\n private:\n  friend class FdReaderBase;  // For `has_independent_pos_`.\n\n  // Encodes a `bool` or a marker that the value is not resolved yet.\n  enum class LazyBoolState : uint8_t { kUnknown, kTrue, kFalse };\n\n  absl::Status FailedOperationStatus(absl::string_view operation);\n#ifdef _WIN32\n  absl::Status FailedWindowsOperationStatus(absl::string_view operation);\n#endif\n  // Lazily determined condition shared by `SupportsRandomAccess()` and\n  // `SupportsReadMode()`.\n  absl::Status SizeStatus();\n\n  bool WriteMode();\n  bool SeekInternal(int dest, Position new_pos);\n  bool TruncateInternal(int dest, Position new_size);\n\n  bool has_independent_pos_ = false;\n  // Invariant except on Windows:\n  //   if `supports_read_mode_ == LazyBoolState::kUnknown` then\n  //       `supports_random_access_ == LazyBoolState::kUnknown`\n  LazyBoolState supports_random_access_ = LazyBoolState::kUnknown;\n  // If `supports_read_mode_ == LazyBoolState::kUnknown`,\n  // then at least size is known to be supported\n  // when `supports_random_access_ != LazyBoolState::kUnknown`\n  // (no matter whether `LazyBoolState::kTrue` or LazyBoolState::kFalse`).\n  //\n  // This is useful on Windows, otherwise this is trivially true\n  // (`supports_random_access_ == LazyBoolState::kUnknown`).\n  LazyBoolState supports_read_mode_ = LazyBoolState::kUnknown;\n  absl::Status random_access_status_;\n  absl::Status read_mode_status_;\n#ifdef _WIN32\n  std::optional<int> original_mode_;\n#endif\n\n  AssociatedReader<FdReader<UnownedFd>> associated_reader_;\n  bool read_mode_ = false;\n\n  // Invariant: `start_pos() <= std::numeric_limits<fd_internal::Offset>::max()`\n};\n\n// A `Writer` which writes to a file descriptor.\n//\n// The fd must support:\n#ifndef _WIN32\n//  * `fcntl()`     - for the constructor from fd\n//  * `close()`     - if the fd is owned\n//  * `write()`     - if `Options::independent_pos() == std::nullopt`\n//  * `pwrite()`    - if `Options::independent_pos() != std::nullopt`\n//  * `lseek()`     - for `Seek()`, `Size()`, or `Truncate()`,\n//                    if `Options::independent_pos() == std::nullopt`\n//  * `fstat()`     - for `Seek()`, `Size()`, or `Truncate()`\n//  * `fsync()`     - for `Flush(FlushType::kFromMachine)`\n//  * `ftruncate()` - for `Truncate()`\n//  * `read()`      - for `ReadMode()`\n//                    if `Options::independent_pos() == std::nullopt`\n//                    (fd must be opened with `O_RDWR`)\n//  * `pread()`     - for `ReadMode()`\n//                    if `Options::independent_pos() != std::nullopt`\n//                    (fd must be opened with `O_RDWR`)\n#else\n//  * `_close()`    - if the fd is owned\n//  * `_write()`    - if `Options::independent_pos() == std::nullopt`\n//  * `_get_osfhandle()`, `WriteFile()` with `OVERLAPPED*`\n//                  - if `Options::independent_pos() != std::nullopt`\n//  * `_lseeki64()` - for `Seek()`, `Size()`, or `Truncate(),\n//                    if `Options::independent_pos() == std::nullopt`\n//  * `_fstat64()`  - for `Seek()`, `Size()`, or `Truncate(),\n//  * `_commit()`   - for `Flush(FlushType::kFromMachine)`\n//  * `_chsize_s()` - for `Truncate()`\n//  * `_read()`     - for `ReadMode()`\n//                    if `Options::independent_pos() == std::nullopt`\n//                    (fd must be opened with `_O_RDWR`)\n//  * `_get_osfhandle()`, `ReadFile()` with `OVERLAPPED*`\n//                  - for `ReadMode()`\n//                    if `Options::independent_pos() != std::nullopt`\n//                    (fd must be opened with `_O_RDWR`)\n#endif\n//\n// `FdWriter` supports random access if\n// `Options::assumed_pos() == std::nullopt` and the fd supports random access\n// (this is assumed if `Options::independent_pos() != std::nullopt`, otherwise\n// this is checked by calling `lseek(SEEK_END)`, or `_lseeki64()` on Windows).\n// On Windows binary mode is also required.\n//\n// `FdWriter` supports `ReadMode()` if it supports random access and the fd was\n// opened with `O_RDWR` (`_O_RDWR` on Windows).\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the fd being written to. `Dest` must support\n// `Dependency<FdHandle, Dest>`, e.g. `OwnedFd` (owned, default),\n// `UnownedFd` (not owned), `AnyFd` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `OwnedFd` if the\n// first constructor argument is a filename or an `int`, otherwise as `TargetT`\n// of the type of the first constructor argument.\n//\n// Until the `FdWriter` is closed or no longer used, the fd must not be closed.\n// Additionally, if `Options::independent_pos() == std::nullopt`\n// (or unconditionally on Windows), the fd should not have its position changed,\n// except that if random access is not used, careful interleaving of multiple\n// writers is possible: `Flush()` is needed before switching to another writer,\n// and `pos()` does not take other writers into account.\ntemplate <typename Dest = OwnedFd>\nclass FdWriter : public FdWriterBase {\n public:\n  // Creates a closed `FdWriter`.\n  explicit FdWriter(Closed) noexcept : FdWriterBase(kClosed) {}\n\n  // Will write to the fd provided by `dest`.\n  explicit FdWriter(Initializer<Dest> dest, Options options = Options());\n\n  // Will write to `dest`.\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<std::is_constructible_v<DependentDest, int>, int> = 0>\n  explicit FdWriter(int dest ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                    Options options = Options());\n\n  // Opens a file for writing.\n  //\n  // If opening the file fails, `FdWriter` will be failed and closed.\n  //\n  // This constructor is present only if `Dest` supports `Open()`.\n  template <typename DependentDest = Dest,\n            std::enable_if_t<std::conjunction_v<\n                                 FdSupportsOpen<DependentDest>,\n                                 std::is_default_constructible<DependentDest>>,\n                             int> = 0>\n  explicit FdWriter(PathInitializer filename, Options options = Options());\n\n  // Opens a file for writing, with the filename interpreted relatively to the\n  // directory specified by an existing fd.\n  //\n  // If opening the file fails, `FdWriter` will be failed and closed.\n  //\n  // This constructor is present only if `Dest` supports `OpenAt()`.\n  template <typename DependentDest = Dest,\n            std::enable_if_t<std::conjunction_v<\n                                 FdSupportsOpenAt<DependentDest>,\n                                 std::is_default_constructible<DependentDest>>,\n                             int> = 0>\n  explicit FdWriter(UnownedFd dir_fd, PathRef filename,\n                    Options options = Options());\n\n  FdWriter(FdWriter&& that) = default;\n  FdWriter& operator=(FdWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `FdWriter`. This avoids\n  // constructing a temporary `FdWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<std::is_constructible_v<DependentDest, int>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(int dest,\n                                          Options options = Options());\n  template <typename DependentDest = Dest,\n            std::enable_if_t<std::conjunction_v<FdSupportsOpen<DependentDest>,\n                                                SupportsReset<DependentDest>>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(PathInitializer filename,\n                                          Options options = Options());\n  template <typename DependentDest = Dest,\n            std::enable_if_t<std::conjunction_v<FdSupportsOpenAt<DependentDest>,\n                                                SupportsReset<DependentDest>>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(UnownedFd dir_fd,\n                                          PathInitializer filename,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the fd being written to.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  FdHandle DestFdHandle() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n  int DestFd() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get().get();\n  }\n\n  // An optimized implementation in a derived class, avoiding a virtual call.\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.get().filename();\n  }\n\n protected:\n  void Done() override;\n\n private:\n  template <typename DependentDest = Dest,\n            std::enable_if_t<FdSupportsOpen<DependentDest>::value, int> = 0>\n  void OpenImpl(PathInitializer filename, Options&& options);\n  template <typename DependentDest = Dest,\n            std::enable_if_t<FdSupportsOpenAt<DependentDest>::value, int> = 0>\n  void OpenAtImpl(UnownedFd dir_fd, PathRef filename, Options&& options);\n\n  // The object providing and possibly owning the fd being written to.\n  Dependency<FdHandle, Dest> dest_;\n};\n\nexplicit FdWriter(Closed) -> FdWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit FdWriter(Dest&& dest,\n                  FdWriterBase::Options options = FdWriterBase::Options())\n    -> FdWriter<std::conditional_t<\n        std::disjunction_v<std::is_convertible<Dest&&, int>,\n                           std::is_convertible<Dest&&, PathInitializer>>,\n        OwnedFd, TargetT<Dest>>>;\nexplicit FdWriter(UnownedFd dir_fd, PathRef filename,\n                  FdWriterBase::Options options = FdWriterBase::Options())\n    -> FdWriter<OwnedFd>;\n\n// Implementation details follow.\n\ninline FdWriterBase::FdWriterBase(BufferOptions buffer_options)\n    : BufferedWriter(buffer_options) {}\n\ninline FdWriterBase::FdWriterBase(FdWriterBase&& that) noexcept\n    : BufferedWriter(static_cast<BufferedWriter&&>(that)),\n      has_independent_pos_(that.has_independent_pos_),\n      supports_random_access_(\n          std::exchange(that.supports_random_access_, LazyBoolState::kUnknown)),\n      supports_read_mode_(\n          std::exchange(that.supports_read_mode_, LazyBoolState::kUnknown)),\n      random_access_status_(std::move(that.random_access_status_)),\n      read_mode_status_(std::move(that.read_mode_status_)),\n#ifdef _WIN32\n      original_mode_(that.original_mode_),\n#endif\n      associated_reader_(std::move(that.associated_reader_)),\n      read_mode_(that.read_mode_) {\n}\n\ninline FdWriterBase& FdWriterBase::operator=(FdWriterBase&& that) noexcept {\n  BufferedWriter::operator=(static_cast<BufferedWriter&&>(that));\n  has_independent_pos_ = that.has_independent_pos_;\n  supports_random_access_ =\n      std::exchange(that.supports_random_access_, LazyBoolState::kUnknown),\n  supports_read_mode_ =\n      std::exchange(that.supports_read_mode_, LazyBoolState::kUnknown),\n  random_access_status_ = std::move(that.random_access_status_);\n  read_mode_status_ = std::move(that.read_mode_status_);\n#ifdef _WIN32\n  original_mode_ = that.original_mode_;\n#endif\n  associated_reader_ = std::move(that.associated_reader_);\n  read_mode_ = that.read_mode_;\n  return *this;\n}\n\ninline void FdWriterBase::Reset(Closed) {\n  BufferedWriter::Reset(kClosed);\n  has_independent_pos_ = false;\n  supports_random_access_ = LazyBoolState::kUnknown;\n  supports_read_mode_ = LazyBoolState::kUnknown;\n  random_access_status_ = absl::OkStatus();\n  read_mode_status_ = absl::OkStatus();\n#ifdef _WIN32\n  original_mode_ = std::nullopt;\n#endif\n  associated_reader_.Reset();\n  read_mode_ = false;\n}\n\ninline void FdWriterBase::Reset(BufferOptions buffer_options) {\n  BufferedWriter::Reset(buffer_options);\n  has_independent_pos_ = false;\n  supports_random_access_ = LazyBoolState::kUnknown;\n  supports_read_mode_ = LazyBoolState::kUnknown;\n  random_access_status_ = absl::OkStatus();\n  read_mode_status_ = absl::OkStatus();\n#ifdef _WIN32\n  original_mode_ = std::nullopt;\n#endif\n  associated_reader_.Reset();\n  read_mode_ = false;\n}\n\ntemplate <typename Dest>\ninline FdWriter<Dest>::FdWriter(Initializer<Dest> dest, Options options)\n    : FdWriterBase(options.buffer_options()), dest_(std::move(dest)) {\n  Initialize(dest_.get().get(), std::move(options));\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_constructible_v<DependentDest, int>, int>>\ninline FdWriter<Dest>::FdWriter(int dest ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                                Options options)\n    : FdWriter(riegeli::Maker(dest), std::move(options)) {}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<\n              std::conjunction_v<FdSupportsOpen<DependentDest>,\n                                 std::is_default_constructible<DependentDest>>,\n              int>>\ninline FdWriter<Dest>::FdWriter(PathInitializer filename, Options options)\n    : FdWriterBase(options.buffer_options()), dest_(riegeli::Maker()) {\n  OpenImpl(std::move(filename), std::move(options));\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<\n              std::conjunction_v<FdSupportsOpenAt<DependentDest>,\n                                 std::is_default_constructible<DependentDest>>,\n              int>>\ninline FdWriter<Dest>::FdWriter(UnownedFd dir_fd, PathRef filename,\n                                Options options)\n    : FdWriterBase(options.buffer_options()), dest_(riegeli::Maker()) {\n  OpenAtImpl(std::move(dir_fd), filename, std::move(options));\n}\n\ntemplate <typename Dest>\ninline void FdWriter<Dest>::Reset(Closed) {\n  FdWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void FdWriter<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  FdWriterBase::Reset(options.buffer_options());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get().get(), std::move(options));\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_constructible_v<DependentDest, int>, int>>\ninline void FdWriter<Dest>::Reset(int dest, Options options) {\n  Reset(riegeli::Maker(dest), std::move(options));\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::conjunction_v<FdSupportsOpen<DependentDest>,\n                                              SupportsReset<DependentDest>>,\n                           int>>\ninline void FdWriter<Dest>::Reset(PathInitializer filename, Options options) {\n  // In case `filename` is owned by `dest_` and gets invalidated.\n  std::string filename_copy = std::move(filename);\n  riegeli::Reset(dest_.manager());\n  FdWriterBase::Reset(options.buffer_options());\n  OpenImpl(std::move(filename_copy), std::move(options));\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::conjunction_v<FdSupportsOpenAt<DependentDest>,\n                                              SupportsReset<DependentDest>>,\n                           int>>\ninline void FdWriter<Dest>::Reset(UnownedFd dir_fd, PathInitializer filename,\n                                  Options options) {\n  // In case `filename` is owned by `dest_` and gets invalidated.\n  std::string filename_copy = std::move(filename);\n  riegeli::Reset(dest_.manager());\n  FdWriterBase::Reset(options.buffer_options());\n  OpenAtImpl(dir_fd, filename_copy, std::move(options));\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<FdSupportsOpen<DependentDest>::value, int>>\nvoid FdWriter<Dest>::OpenImpl(PathInitializer filename, Options&& options) {\n  absl::Status status = dest_.manager().Open(\n      std::move(filename), options.mode(), options.permissions());\n  if (ABSL_PREDICT_FALSE(!status.ok())) {\n    FdWriterBase::Reset(kClosed);\n    FailWithoutAnnotation(std::move(status));\n    return;\n  }\n  InitializePos(dest_.get().get(), std::move(options),\n                /*mode_was_passed_to_open=*/true);\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<FdSupportsOpenAt<DependentDest>::value, int>>\nvoid FdWriter<Dest>::OpenAtImpl(UnownedFd dir_fd, PathRef filename,\n                                Options&& options) {\n  absl::Status status = dest_.manager().OpenAt(\n      std::move(dir_fd), filename, options.mode(), options.permissions());\n  if (ABSL_PREDICT_FALSE(!status.ok())) {\n    FdWriterBase::Reset(kClosed);\n    FailWithoutAnnotation(std::move(status));\n    return;\n  }\n  InitializePos(dest_.get().get(), std::move(options),\n                /*mode_was_passed_to_open=*/true);\n}\n\ntemplate <typename Dest>\nvoid FdWriter<Dest>::Done() {\n  FdWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (absl::Status status = dest_.get().Close();\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      Fail(std::move(status));\n    }\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_FD_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/file_mode_string.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/file_mode_string.h\"\n\n#ifdef _WIN32\n#include <fcntl.h>\n#endif\n#include <stddef.h>\n\n#include <string>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/string_view.h\"\n\n// Syntax of a file mode for `fopen()`:\n//  * 'r', 'w', or 'a'.\n//  * Single character modifiers, in any order. Some modifiers are standard:\n//    '+', 'b', 'x' (since C++17 / C11), while others are OS-specific.\n//  * ',ccs=<encoding>'. This is not standard but it is understood by glibc and\n//    on Windows. To avoid breaking the encoding name which may use characters\n//    ordinarily used as modifiers, functions below stop parsing at ','.\n\nnamespace riegeli::file_internal {\n\nvoid SetExisting(bool existing, std::string& mode) {\n  if (ABSL_PREDICT_FALSE(mode.empty())) mode = \"w\";\n  if (existing) {\n    mode[0] = 'r';\n    // Add '+' to modifiers unless it already exists there.\n    for (size_t i = 1; i < mode.size(); ++i) {\n      if (mode[i] == '+') return;\n      if (mode[i] == ',') break;\n    }\n    mode.insert(1, \"+\");\n  } else {\n    mode[0] = 'w';\n    // Remove '+' from modifiers.\n    for (size_t i = 1; i < mode.size(); ++i) {\n      if (mode[i] == '+') {\n        mode.erase(i, 1);\n        --i;\n        continue;\n      }\n      if (mode[i] == ',') break;\n    }\n  }\n}\n\nvoid SetRead(bool read, std::string& mode) {\n  if (ABSL_PREDICT_FALSE(mode.empty())) mode = \"w\";\n  if (read) {\n    // Add '+' to modifiers unless it already exists there.\n    for (size_t i = 1; i < mode.size(); ++i) {\n      if (mode[i] == '+') return;\n      if (mode[i] == ',') break;\n    }\n    mode.insert(1, \"+\");\n  } else {\n    if (mode[0] == 'r') return;\n    // Remove '+' from modifiers.\n    for (size_t i = 1; i < mode.size(); ++i) {\n      if (mode[i] == '+') {\n        mode.erase(i, 1);\n        --i;\n        continue;\n      }\n      if (mode[i] == ',') break;\n    }\n  }\n}\n\nbool GetRead(absl::string_view mode) {\n  if (ABSL_PREDICT_FALSE(mode.empty())) return false;\n  if (mode[0] == 'r') return true;\n  for (size_t i = 1; i < mode.size(); ++i) {\n    if (mode[i] == '+') return true;\n    if (mode[i] == ',') break;\n  }\n  return false;\n}\n\nvoid SetExclusive(bool exclusive, std::string& mode) {\n  if (ABSL_PREDICT_FALSE(mode.empty())) mode = \"w\";\n  if (exclusive) {\n    // Add 'x' to modifiers unless it already exists there.\n    for (size_t i = 1; i < mode.size(); ++i) {\n      if (mode[i] == 'x') return;\n      if (mode[i] == ',') break;\n    }\n    size_t position = 1;\n    while (mode.size() > position &&\n           (mode[position] == '+' || mode[position] == 'b' ||\n            mode[position] == 't')) {\n      ++position;\n    }\n    mode.insert(position, \"x\");\n  } else {\n    // Remove 'x' from modifiers.\n    for (size_t i = 1; i < mode.size(); ++i) {\n      if (mode[i] == 'x') {\n        mode.erase(i, 1);\n        --i;\n        continue;\n      }\n      if (mode[i] == ',') break;\n    }\n  }\n}\n\nbool GetExclusive(absl::string_view mode) {\n  if (ABSL_PREDICT_FALSE(mode.empty())) return false;\n  for (size_t i = 1; i < mode.size(); ++i) {\n    if (mode[i] == 'x') return true;\n    if (mode[i] == ',') break;\n  }\n  return false;\n}\n\nnamespace {\n\nvoid SetInheritableImpl(bool inheritable, std::string& mode) {\n#ifndef _WIN32\n  static constexpr char kModifier = 'e';\n#else\n  static constexpr char kModifier = 'N';\n#endif\n  static constexpr const char kModifierStr[2] = {kModifier, '\\0'};\n  if (inheritable) {\n    // Remove `kModifier` from modifiers.\n    for (size_t i = 1; i < mode.size(); ++i) {\n      if (mode[i] == kModifier) {\n        mode.erase(i, 1);\n        --i;\n        continue;\n      }\n      if (mode[i] == ',') break;\n    }\n  } else {\n    // Add `kModifier` to modifiers unless it already exists there.\n    for (size_t i = 1; i < mode.size(); ++i) {\n      if (mode[i] == kModifier) return;\n      if (mode[i] == ',') break;\n    }\n    size_t position = 1;\n    while (mode.size() > position &&\n           (mode[position] == '+' || mode[position] == 'b' ||\n            mode[position] == 't' || mode[position] == 'x')) {\n      ++position;\n    }\n    mode.insert(position, kModifierStr);\n  }\n}\n\n}  // namespace\n\nvoid SetInheritableReading(bool inheritable, std::string& mode) {\n  if (ABSL_PREDICT_FALSE(mode.empty())) mode = \"r\";\n  SetInheritableImpl(inheritable, mode);\n}\n\nvoid SetInheritableWriting(bool inheritable, std::string& mode) {\n  if (ABSL_PREDICT_FALSE(mode.empty())) mode = \"w\";\n  SetInheritableImpl(inheritable, mode);\n}\n\nbool GetInheritable(absl::string_view mode) {\n#ifndef _WIN32\n  static constexpr char kModifier = 'e';\n#else\n  static constexpr char kModifier = 'N';\n#endif\n  if (ABSL_PREDICT_FALSE(mode.empty())) return true;\n  for (size_t i = 1; i < mode.size(); ++i) {\n    if (mode[i] == kModifier) return false;\n    if (mode[i] == ',') break;\n  }\n  return true;\n}\n\nnamespace {\n\ninline void SetTextImpl(bool text, std::string& mode) {\n#ifdef _WIN32\n  const char to_remove = text ? 'b' : 't';\n  const char to_add[2] = {text ? 't' : 'b', '\\0'};\n  bool need_to_add = true;\n  for (size_t i = 1; i < mode.size(); ++i) {\n    if (mode[i] == to_remove) {\n      if (need_to_add) {\n        mode[i] = to_add[0];\n        need_to_add = false;\n      } else {\n        mode.erase(i, 1);\n        --i;\n      }\n      continue;\n    }\n    if (mode[i] == to_add[0]) {\n      need_to_add = false;\n      continue;\n    }\n    if (mode[i] == ',') break;\n  }\n  if (need_to_add) {\n    size_t position = 1;\n    while (mode.size() > position && mode[position] == '+') ++position;\n    mode.insert(position, to_add);\n  }\n#endif  // _WIN32\n}\n\n}  // namespace\n\nvoid SetTextReading(bool text, std::string& mode) {\n  if (ABSL_PREDICT_FALSE(mode.empty())) mode = \"r\";\n  SetTextImpl(text, mode);\n}\n\nvoid SetTextWriting(bool text, std::string& mode) {\n  if (ABSL_PREDICT_FALSE(mode.empty())) mode = \"w\";\n  SetTextImpl(text, mode);\n}\n\n#ifdef _WIN32\n\nint GetTextAsFlags(absl::string_view mode) {\n  for (size_t i = 1; i < mode.size(); ++i) {\n    if (mode[i] == 'b') return _O_BINARY;\n    if (mode[i] == 't') return _O_TEXT;\n    if (mode[i] == ',') break;\n  }\n  return 0;\n}\n\n#endif  // _WIN32\n\n}  // namespace riegeli::file_internal\n"
  },
  {
    "path": "riegeli/bytes/file_mode_string.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_FILE_MODE_STRING_H_\n#define RIEGELI_BYTES_FILE_MODE_STRING_H_\n\n#include <string>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/string_view.h\"\n\nnamespace riegeli::file_internal {\n\n// If `false`, the file will be created if it does not exist, or it will be\n// truncated to empty if it exists. This implies `SetRead(false)` and\n// `set_append(false)` unless overwritten later.\n//\n// If `true`, the file must already exist, and its contents will not be\n// truncated. Writing will start from the beginning, with random access\n// supported. This implies `SetRead(true)`.\nvoid SetExisting(bool existing, std::string& mode);\nbool GetExisting(absl::string_view mode);\n\n// If `false`, the file will be open for writing, except that `SetRead(false)`\n// has no effect after `SetExisting(true)`.\n//\n// If `true`, the file will be open for writing and reading (using\n// `ReadMode()`).\nvoid SetRead(bool read, std::string& mode);\nbool GetRead(absl::string_view mode);\n\n// If `false`, the file will be truncated to empty if it exists.\n//\n// If `true`, the file will not be truncated if it exists, and writing will\n// always happen at its end.\n//\n// Calling `SetAppend()` with any argument after `SetExisting(true)` undoes the\n// effect if the file does not exist: it will be created.\nvoid SetAppend(bool append, std::string& mode);\nbool GetAppend(absl::string_view mode);\n\n// If `false`, the file will be created if it does not exist, or it will be\n// opened if it exists (truncated to empty by default, or left unchanged if\n// `SetExisting(true)` or `SetAppend(true)` was used).\n//\n// If `true`, the file will be created if it does not exist, or opening will\n// fail if it exists.\nvoid SetExclusive(bool exclusive, std::string& mode);\nbool GetExclusive(absl::string_view mode);\n\n// If `false`, `execve()` (`CreateProcess()` on Windows) will close the file.\n//\n// If `true`, the file will remain open across `execve()` (`CreateProcess()` on\n// Windows).\nvoid SetInheritableReading(bool inheritable, std::string& mode);\nvoid SetInheritableWriting(bool inheritable, std::string& mode);\nbool GetInheritable(absl::string_view mode);\n\n// If `false`, data will be read/written directly from/to the file. This is\n// called the binary mode.\n//\n// If `true`, text mode translation will be applied on Windows: for reading\n// CR-LF character pairs are translated to LF, and a ^Z character is interpreted\n// as end of file; for writing LF characters are translated to CR-LF.\nvoid SetTextReading(bool text, std::string& mode);\nvoid SetTextWriting(bool text, std::string& mode);\n\n#ifdef _WIN32\n\n// Translates the text mode marker from 'b' / 't' / nothing to `_O_BINARY` /\n// `_O_TEXT` / 0.\nint GetTextAsFlags(absl::string_view mode);\n\n#endif  // _WIN32\n\n// Implementation details follow.\n\ninline bool GetExisting(absl::string_view mode) {\n  return ABSL_PREDICT_TRUE(!mode.empty()) && mode[0] == 'r';\n}\n\ninline void SetAppend(bool append, std::string& mode) {\n  if (ABSL_PREDICT_FALSE(mode.empty())) mode = \"w\";\n  mode[0] = append ? 'a' : 'w';\n}\n\ninline bool GetAppend(absl::string_view mode) {\n  return ABSL_PREDICT_TRUE(!mode.empty()) && mode[0] == 'a';\n}\n\n}  // namespace riegeli::file_internal\n\n#endif  // RIEGELI_BYTES_FILE_MODE_STRING_H_\n"
  },
  {
    "path": "riegeli/bytes/iostream_internal.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_IOSTREAM_INTERNAL_H_\n#define RIEGELI_BYTES_IOSTREAM_INTERNAL_H_\n\n#include <istream>\n#include <type_traits>\n#include <utility>\n\nnamespace riegeli::iostream_internal {\n\n// There is no `std::istream::close()` nor `std::ostream::close()`, but some\n// subclasses have `close()`, e.g. `std::ifstream`, `std::ofstream`,\n// `std::fstream`. It is important to call `close()` before their destructor\n// to detect errors.\n//\n// `iostream_internal::Close(stream)` calls `stream->close()` if that is\n// defined, otherwise does nothing.\n\ntemplate <typename T, typename Enable = void>\nstruct HasClose : std::false_type {};\n\ntemplate <typename T>\nstruct HasClose<T, std::void_t<decltype(std::declval<T>().close())>>\n    : std::true_type {};\n\ntemplate <typename Stream>\ninline void Close(Stream& stream) {\n  if constexpr (HasClose<Stream>::value) {\n    stream.close();\n  }\n}\n\ntemplate <typename T>\ninline std::istream* DetectIStream(T* stream) {\n  if constexpr (std::is_base_of_v<std::istream, T>) {\n    return stream;\n  } else {\n    return nullptr;\n  }\n}\n\n}  // namespace riegeli::iostream_internal\n\n#endif  // RIEGELI_BYTES_IOSTREAM_INTERNAL_H_\n"
  },
  {
    "path": "riegeli/bytes/istream_reader.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/istream_reader.h\"\n\n#include <stddef.h>\n\n#include <cerrno>\n#include <ios>\n#include <istream>\n#include <limits>\n#include <optional>\n#include <string>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/global.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n\nnamespace riegeli {\n\nvoid IStreamReaderBase::Initialize(std::istream* src,\n                                   std::optional<Position> assumed_pos) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of IStreamReader: null stream pointer\";\n  RIEGELI_ASSERT(!supports_random_access_)\n      << \"Failed precondition of IStreamReaderBase::Initialize(): \"\n         \"supports_random_access_ not reset\";\n  RIEGELI_ASSERT_OK(random_access_status_)\n      << \"Failed precondition of IStreamReaderBase::Initialize(): \"\n         \"random_access_status_ not reset\";\n  if (ABSL_PREDICT_FALSE(src->fail())) {\n    // Either constructing the stream failed or the stream was already in a\n    // failed state. In any case `IStreamReaderBase` should fail.\n    FailOperation(\"istream::istream()\");\n    return;\n  }\n  // A sticky `std::ios_base::eofbit` breaks future operations like\n  // `std::istream::peek()` and `std::istream::tellg()`.\n  src->clear(src->rdstate() & ~std::ios_base::eofbit);\n  if (assumed_pos != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(\n            *assumed_pos >\n            Position{std::numeric_limits<std::streamoff>::max()})) {\n      FailOverflow();\n      return;\n    }\n    set_limit_pos(*assumed_pos);\n    // `supports_random_access_` is left as `false`.\n    random_access_status_ = Global([] {\n      return absl::UnimplementedError(\n          \"IStreamReaderBase::Options::assumed_pos() excludes random access\");\n    });\n  } else {\n    errno = 0;\n    const std::streamoff stream_pos = src->tellg();\n    if (stream_pos < 0) {\n      // Random access is not supported. Assume 0 as the initial position.\n      // `supports_random_access_` is left as `false`.\n      random_access_status_ = FailedOperationStatus(\"istream::tellg()\");\n      return;\n    }\n    set_limit_pos(IntCast<Position>(stream_pos));\n\n    // Check the size, and whether random access is supported.\n    src->seekg(0, std::ios_base::end);\n    if (src->fail()) {\n      // Random access is not supported. `supports_random_access_` is left as\n      // `false`.\n      random_access_status_ = FailedOperationStatus(\"istream::seekg()\");\n      src->clear(src->rdstate() & ~std::ios_base::failbit);\n      return;\n    }\n    std::streamoff stream_size = src->tellg();\n    if (ABSL_PREDICT_FALSE(stream_size < 0)) {\n      FailOperation(\"istream::tellg()\");\n      return;\n    }\n    if (limit_pos() != IntCast<Position>(stream_size)) {\n      src->seekg(IntCast<std::streamoff>(limit_pos()), std::ios_base::beg);\n      if (ABSL_PREDICT_FALSE(src->fail())) {\n        FailOperation(\"istream::seekg()\");\n        return;\n      }\n    }\n#ifndef _WIN32\n    if (stream_size == 0 && limit_pos() == 0) {\n      // Some \"/proc\" and \"/sys\" files claim to have zero size but have\n      // non-empty contents when read.\n      if (BufferedReader::PullSlow(1, 0)) {\n        if (growing_source_) {\n          // Check the size again. Maybe the stream has grown.\n          src->seekg(0, std::ios_base::end);\n          if (ABSL_PREDICT_FALSE(src->fail())) {\n            FailOperation(\"istream::seekg()\");\n            return;\n          }\n          stream_size = src->tellg();\n          if (ABSL_PREDICT_FALSE(stream_size < 0)) {\n            FailOperation(\"istream::tellg()\");\n            return;\n          }\n          if (limit_pos() != IntCast<Position>(stream_size)) {\n            src->seekg(IntCast<std::streamoff>(limit_pos()),\n                       std::ios_base::beg);\n            if (ABSL_PREDICT_FALSE(src->fail())) {\n              FailOperation(\"istream::seekg()\");\n              return;\n            }\n          }\n          if (stream_size > 0) goto regular;\n        }\n        // This is one of \"/proc\" or \"/sys\" files which claim to have zero size\n        // but have non-empty contents when read. Random access is not\n        // supported. `supports_random_access_` is left as `false`.\n        random_access_status_ = Global([] {\n          return absl::UnimplementedError(\n              \"Random access is not supported because \"\n              \"the file claims zero size but has non-empty contents when read\");\n        });\n        return;\n      }\n      if (ABSL_PREDICT_FALSE(!ok())) return;\n      // This is a regular empty stream.\n    }\n  regular:\n#endif\n    // Random access is supported.\n    supports_random_access_ = true;\n    if (!growing_source_) set_exact_size(IntCast<Position>(stream_size));\n  }\n  BeginRun();\n}\n\nvoid IStreamReaderBase::Done() {\n  BufferedReader::Done();\n  random_access_status_ = absl::OkStatus();\n}\n\ninline absl::Status IStreamReaderBase::FailedOperationStatus(\n    absl::string_view operation) {\n  // There is no way to get details why a stream operation failed without\n  // letting the stream throw exceptions. Hopefully low level failures have set\n  // `errno` as a side effect.\n  //\n  // This requires resetting `errno` to 0 before the stream operation because\n  // the operation may fail without setting `errno`.\n  const int error_number = errno;\n  const std::string message = absl::StrCat(operation, \" failed\");\n  return error_number == 0 ? absl::UnknownError(message)\n                           : absl::ErrnoToStatus(error_number, message);\n}\n\nbool IStreamReaderBase::FailOperation(absl::string_view operation) {\n  return Fail(FailedOperationStatus(operation));\n}\n\nbool IStreamReaderBase::ReadInternal(size_t min_length, size_t max_length,\n                                     char* dest) {\n  RIEGELI_ASSERT_GT(min_length, 0u)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"nothing to read\";\n  RIEGELI_ASSERT_GE(max_length, min_length)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"max_length < min_length\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::ReadInternal()\";\n  std::istream& src = *SrcStream();\n  errno = 0;\n  for (;;) {\n    if (ABSL_PREDICT_FALSE(\n            limit_pos() >=\n            Position{std::numeric_limits<std::streamoff>::max()})) {\n      return FailOverflow();\n    }\n    std::streamsize length_to_read = IntCast<std::streamsize>(UnsignedMin(\n        min_length,\n        Position{std::numeric_limits<std::streamoff>::max()} - limit_pos()));\n    const std::streamsize max_length_to_read =\n        IntCast<std::streamsize>(UnsignedMin(\n            max_length, Position{std::numeric_limits<std::streamoff>::max()} -\n                            limit_pos()));\n    std::streamsize length_read;\n    if (length_to_read < max_length_to_read) {\n      // Use `std::istream::readsome()` to read as much data as is available,\n      // up to `max_length_to_read`.\n      //\n      // `std::istream::peek()` asks to read some characters into the buffer,\n      // otherwise `std::istream::readsome()` may return 0.\n      if (ABSL_PREDICT_FALSE(src.peek() == std::char_traits<char>::eof())) {\n        if (ABSL_PREDICT_FALSE(src.fail())) {\n          return FailOperation(\"istream::peek()\");\n        }\n        // A sticky `std::ios_base::eofbit` breaks future operations like\n        // `std::istream::peek()` and `std::istream::tellg()`.\n        src.clear(src.rdstate() & ~std::ios_base::eofbit);\n        if (!growing_source_) set_exact_size(limit_pos());\n        return false;\n      }\n      length_read = src.readsome(dest, max_length_to_read);\n      RIEGELI_ASSERT_GE(length_read, 0) << \"negative istream::readsome()\";\n      RIEGELI_ASSERT_LE(length_read, max_length_to_read)\n          << \"istream::readsome() read more than requested\";\n      if (ABSL_PREDICT_TRUE(length_read > 0)) goto fragment_read;\n      // `std::istream::peek()` returned non-`eof()` but\n      // `std::istream::readsome()` returned 0. This might happen if\n      // `src.rdbuf()->sgetc()` does not use the get area but leaves the next\n      // character buffered elsewhere, e.g. for `std::cin` synchronized to\n      // stdio. Fall back to `std::istream::read()`.\n    }\n    // Use `std::istream::read()` to read a fixed length of `length_to_read`.\n    src.read(dest, length_to_read);\n    length_read = src.gcount();\n    RIEGELI_ASSERT_GE(length_read, 0) << \"negative istream::gcount()\";\n    RIEGELI_ASSERT_LE(length_read, length_to_read)\n        << \"istream::read() read more than requested\";\n  fragment_read:\n    move_limit_pos(IntCast<size_t>(length_read));\n    if (ABSL_PREDICT_FALSE(src.fail())) {\n      if (ABSL_PREDICT_FALSE(src.bad())) {\n        FailOperation(\"istream::read()\");\n      } else {\n        // End of stream is not a failure.\n        //\n        // A sticky `std::ios_base::eofbit` breaks future operations like\n        // `std::istream::peek()` and `std::istream::tellg()`.\n        src.clear(src.rdstate() &\n                  ~(std::ios_base::eofbit | std::ios_base::failbit));\n        if (!growing_source_) set_exact_size(limit_pos());\n      }\n      return IntCast<size_t>(length_read) >= min_length;\n    }\n    if (IntCast<size_t>(length_read) >= min_length) return true;\n    dest += length_read;\n    min_length -= IntCast<size_t>(length_read);\n    max_length -= IntCast<size_t>(length_read);\n  }\n}\n\nbool IStreamReaderBase::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!IStreamReaderBase::SupportsRandomAccess())) {\n    if (ABSL_PREDICT_FALSE(new_pos < start_pos())) {\n      if (ok()) Fail(random_access_status_);\n      return false;\n    }\n    return BufferedReader::SeekBehindBuffer(new_pos);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::istream& src = *SrcStream();\n  errno = 0;\n  if (new_pos > limit_pos()) {\n    // Seeking forwards.\n    if (exact_size() != std::nullopt) {\n      if (ABSL_PREDICT_FALSE(new_pos > *exact_size())) {\n        // Stream ends.\n        src.seekg(IntCast<std::streamoff>(*exact_size()), std::ios_base::beg);\n        if (ABSL_PREDICT_FALSE(src.fail())) {\n          return FailOperation(\"istream::seekg()\");\n        }\n        set_limit_pos(*exact_size());\n        return false;\n      }\n    } else {\n      src.seekg(0, std::ios_base::end);\n      if (ABSL_PREDICT_FALSE(src.fail())) {\n        return FailOperation(\"istream::seekg()\");\n      }\n      const std::streamoff stream_size = src.tellg();\n      if (ABSL_PREDICT_FALSE(stream_size < 0)) {\n        return FailOperation(\"istream::tellg()\");\n      }\n      if (!growing_source_) set_exact_size(IntCast<Position>(stream_size));\n      if (ABSL_PREDICT_FALSE(new_pos > IntCast<Position>(stream_size))) {\n        // Stream ends.\n        set_limit_pos(IntCast<Position>(stream_size));\n        return false;\n      }\n    }\n  }\n  src.seekg(IntCast<std::streamoff>(new_pos), std::ios_base::beg);\n  if (ABSL_PREDICT_FALSE(src.fail())) return FailOperation(\"istream::seekg()\");\n  set_limit_pos(new_pos);\n  return true;\n}\n\nstd::optional<Position> IStreamReaderBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  if (exact_size() != std::nullopt) return *exact_size();\n  if (ABSL_PREDICT_FALSE(!IStreamReaderBase::SupportsRandomAccess())) {\n    Fail(random_access_status_);\n    return std::nullopt;\n  }\n  std::istream& src = *SrcStream();\n  errno = 0;\n  src.seekg(0, std::ios_base::end);\n  if (ABSL_PREDICT_FALSE(src.fail())) {\n    FailOperation(\"istream::seekg()\");\n    return std::nullopt;\n  }\n  const std::streamoff stream_size = src.tellg();\n  if (ABSL_PREDICT_FALSE(stream_size < 0)) {\n    FailOperation(\"istream::tellg()\");\n    return std::nullopt;\n  }\n  src.seekg(IntCast<std::streamoff>(limit_pos()), std::ios_base::beg);\n  if (ABSL_PREDICT_FALSE(src.fail())) {\n    FailOperation(\"istream::seekg()\");\n    return std::nullopt;\n  }\n  if (!growing_source_) set_exact_size(IntCast<Position>(stream_size));\n  return IntCast<Position>(stream_size);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/istream_reader.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_ISTREAM_READER_H_\n#define RIEGELI_BYTES_ISTREAM_READER_H_\n\n#include <stddef.h>\n\n#include <cerrno>\n#include <istream>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/iostream_internal.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `IStreamReader`.\nclass IStreamReaderBase : public BufferedReader {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // If `std::nullopt`, the current position reported by `pos()` corresponds\n    // to the current stream position if possible, otherwise 0 is assumed as the\n    // initial position. Random access is supported if the stream supports\n    // random access.\n    //\n    // If not `std::nullopt`, this position is assumed initially, to be reported\n    // by `pos()`. It does not need to correspond to the current stream\n    // position. Random access is not supported.\n    //\n    // Warning: On Windows this must not be `std::nullopt` if the stream is a\n    // `std::ifstream` or `std::fstream` opened in text mode.\n    //\n    // Default: `std::nullopt`.\n    Options& set_assumed_pos(std::optional<Position> assumed_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      assumed_pos_ = assumed_pos;\n      return *this;\n    }\n    Options&& set_assumed_pos(std::optional<Position> assumed_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_pos(assumed_pos));\n    }\n    std::optional<Position> assumed_pos() const { return assumed_pos_; }\n\n    // If `true`, supports reading up to the end of the stream, then retrying\n    // when the stream has grown. This disables caching the stream size.\n    //\n    // Default: `false`.\n    Options& set_growing_source(bool growing_source) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      growing_source_ = growing_source;\n      return *this;\n    }\n    Options&& set_growing_source(bool growing_source) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_growing_source(growing_source));\n    }\n    bool growing_source() const { return growing_source_; }\n\n   private:\n    std::optional<Position> assumed_pos_;\n    bool growing_source_ = false;\n  };\n\n  // Returns the stream being read from. Unchanged by `Close()`.\n  virtual std::istream* SrcStream() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool ToleratesReadingAhead() override {\n    return BufferedReader::ToleratesReadingAhead() ||\n           IStreamReaderBase::SupportsRandomAccess();\n  }\n  bool SupportsRandomAccess() override { return supports_random_access_; }\n\n protected:\n  explicit IStreamReaderBase(Closed) noexcept : BufferedReader(kClosed) {}\n\n  explicit IStreamReaderBase(BufferOptions buffer_options, bool growing_source);\n\n  IStreamReaderBase(IStreamReaderBase&& that) noexcept;\n  IStreamReaderBase& operator=(IStreamReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, bool growing_source);\n  void Initialize(std::istream* src, std::optional<Position> assumed_pos);\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation);\n\n  void Done() override;\n  bool ReadInternal(size_t min_length, size_t max_length, char* dest) override;\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n\n private:\n  absl::Status FailedOperationStatus(absl::string_view operation);\n\n  bool growing_source_ = false;\n  bool supports_random_access_ = false;\n  absl::Status random_access_status_;\n\n  // Invariant: `limit_pos() <= std::numeric_limits<std::streamoff>::max()`\n};\n\n// A `Reader` which reads from a `std::istream`.\n//\n// `IStreamReader` supports random access if\n// `Options::assumed_pos() == std::nullopt` and the stream supports random\n// access (this is checked by calling `std::istream::tellg()` and\n// `std::istream::seekg()` to the end and back).\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the stream being read from. `Src` must support\n// `Dependency<std::istream*, Src>`, e.g. `std::istream*` (not owned, default),\n// `std::ifstream` (owned), `std::unique_ptr<std::istream>` (owned),\n// `Any<std::istream*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// Warning: if random access is not supported and the stream is not owned,\n// it will have an unpredictable amount of extra data consumed because of\n// buffering.\n//\n// Until the `IStreamReader` is closed or no longer used, the stream must not be\n// closed nor have its position changed.\ntemplate <typename Src = std::istream*>\nclass IStreamReader : public IStreamReaderBase {\n public:\n  // Creates a closed `IStreamReader`.\n  explicit IStreamReader(Closed) noexcept : IStreamReaderBase(kClosed) {}\n\n  // Will read from the stream provided by `src`.\n  explicit IStreamReader(Initializer<Src> src, Options options = Options());\n\n  IStreamReader(IStreamReader&& that) = default;\n  IStreamReader& operator=(IStreamReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `IStreamReader`. This\n  // avoids constructing a temporary `IStreamReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the stream being read\n  // from. Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  std::istream* SrcStream() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  bool SyncImpl(SyncType sync_type) override;\n\n  // The object providing and possibly owning the stream being read from.\n  Dependency<std::istream*, Src> src_;\n};\n\nexplicit IStreamReader(Closed) -> IStreamReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit IStreamReader(Src&& src, IStreamReaderBase::Options options =\n                                      IStreamReaderBase::Options())\n    -> IStreamReader<TargetT<Src>>;\n\n// Implementation details follow.\n\ninline IStreamReaderBase::IStreamReaderBase(BufferOptions buffer_options,\n                                            bool growing_source)\n    : BufferedReader(buffer_options), growing_source_(growing_source) {\n  // Clear `errno` so that `Initialize()` can attribute failures to opening the\n  // stream.\n  errno = 0;\n}\n\ninline IStreamReaderBase::IStreamReaderBase(IStreamReaderBase&& that) noexcept\n    : BufferedReader(static_cast<BufferedReader&&>(that)),\n      growing_source_(that.growing_source_),\n      supports_random_access_(\n          std::exchange(that.supports_random_access_, false)),\n      random_access_status_(std::move(that.random_access_status_)) {}\n\ninline IStreamReaderBase& IStreamReaderBase::operator=(\n    IStreamReaderBase&& that) noexcept {\n  BufferedReader::operator=(static_cast<BufferedReader&&>(that));\n  growing_source_ = that.growing_source_;\n  supports_random_access_ = std::exchange(that.supports_random_access_, false);\n  random_access_status_ = std::move(that.random_access_status_);\n  return *this;\n}\n\ninline void IStreamReaderBase::Reset(Closed) {\n  BufferedReader::Reset(kClosed);\n  growing_source_ = false;\n  supports_random_access_ = false;\n  random_access_status_ = absl::OkStatus();\n}\n\ninline void IStreamReaderBase::Reset(BufferOptions buffer_options,\n                                     bool growing_source) {\n  BufferedReader::Reset(buffer_options);\n  growing_source_ = growing_source;\n  supports_random_access_ = false;\n  random_access_status_ = absl::OkStatus();\n  // Clear `errno` so that `Initialize()` can attribute failures to opening the\n  // stream.\n  errno = 0;\n}\n\ntemplate <typename Src>\ninline IStreamReader<Src>::IStreamReader(Initializer<Src> src, Options options)\n    : IStreamReaderBase(options.buffer_options(), options.growing_source()),\n      src_(std::move(src)) {\n  Initialize(src_.get(), options.assumed_pos());\n}\n\ntemplate <typename Src>\ninline void IStreamReader<Src>::Reset(Closed) {\n  IStreamReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void IStreamReader<Src>::Reset(Initializer<Src> src, Options options) {\n  IStreamReaderBase::Reset(options.buffer_options(), options.growing_source());\n  src_.Reset(std::move(src));\n  Initialize(src_.get(), options.assumed_pos());\n}\n\ntemplate <typename Src>\nvoid IStreamReader<Src>::Done() {\n  IStreamReaderBase::Done();\n  if (src_.IsOwning()) {\n    errno = 0;\n    iostream_internal::Close(*src_);\n    if (ABSL_PREDICT_FALSE(src_->fail()) && ABSL_PREDICT_TRUE(ok())) {\n      FailOperation(\"istream::close()\");\n    }\n  }\n}\n\ntemplate <typename Src>\nbool IStreamReader<Src>::SyncImpl(SyncType sync_type) {\n  if (ABSL_PREDICT_FALSE(!IStreamReaderBase::SyncImpl(sync_type))) return false;\n  if ((sync_type != SyncType::kFromObject || src_.IsOwning()) &&\n      IStreamReaderBase::SupportsRandomAccess()) {\n    if (ABSL_PREDICT_FALSE(src_->sync() != 0)) {\n      return FailOperation(\"istream::sync()\");\n    }\n  }\n  return true;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_ISTREAM_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/joining_reader.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/joining_reader.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pullable_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid JoiningReaderBase::Done() {\n  PullableReader::Done();\n  if (ABSL_PREDICT_TRUE(ok())) {\n    Reader* shard = ShardReader();\n    if (shard_is_open(shard)) CloseShardInternal();\n  }\n}\n\nbool JoiningReaderBase::CloseShardImpl() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of JoiningReaderBase::CloseShardImpl()\";\n  RIEGELI_ASSERT(shard_is_open())\n      << \"Failed precondition of JoiningReaderBase::CloseShardImpl(): \"\n         \"shard already closed\";\n  Reader* shard = ShardReader();\n  if (ABSL_PREDICT_FALSE(!shard->Close())) {\n    return FailWithoutAnnotation(AnnotateOverShard(shard->status()));\n  }\n  return true;\n}\n\ninline bool JoiningReaderBase::OpenShardInternal() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of JoiningReaderBase::OpenShardInternal()\";\n  RIEGELI_ASSERT(!shard_is_open())\n      << \"Failed precondition of JoiningReaderBase::OpenShardInternal(): \"\n         \"shard already opened\";\n  if (ABSL_PREDICT_FALSE(!OpenShardImpl())) return false;\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed postcondition of JoiningReaderBase::OpenShardImpl()\";\n  RIEGELI_ASSERT(shard_is_open())\n      << \"Failed postcondition of JoiningReaderBase::OpenShardImpl(): \"\n         \"shard not opened\";\n  if (read_all_hint_) {\n    Reader* shard = ShardReader();\n    shard->SetReadAllHint(true);\n  }\n  return true;\n}\n\ninline bool JoiningReaderBase::CloseShardInternal() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of JoiningReaderBase::CloseShardInternal()\";\n  RIEGELI_ASSERT(shard_is_open())\n      << \"Failed precondition of JoiningReaderBase::CloseShardInternal(): \"\n         \"shard already closed\";\n  const bool close_shard_ok = CloseShardImpl();\n  RIEGELI_ASSERT(!shard_is_open())\n      << \"Failed postcondition of JoiningReaderBase::CloseShardImpl(): \"\n         \"shard not closed\";\n  if (ABSL_PREDICT_FALSE(!close_shard_ok)) {\n    RIEGELI_ASSERT(!ok())\n        << \"Failed postcondition of JoiningReaderBase::CloseShardImpl(): \"\n           \"false returned but JoiningReaderBase OK\";\n    return false;\n  }\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed postcondition of JoiningReaderBase::CloseShardImpl()\";\n  return true;\n}\n\nbool JoiningReaderBase::OpenShard() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of JoiningReaderBase::OpenShard()\";\n  RIEGELI_ASSERT(!shard_is_open())\n      << \"Failed precondition of JoiningReaderBase::OpenShard(): \"\n         \"shard already opened\";\n  if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n  Reader* shard = ShardReader();\n  MakeBuffer(*shard);\n  return true;\n}\n\nbool JoiningReaderBase::CloseShard() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of JoiningReaderBase::CloseShard()\";\n  RIEGELI_ASSERT(shard_is_open())\n      << \"Failed precondition of JoiningReaderBase::CloseShard(): \"\n         \"shard already closed\";\n  Reader* shard = ShardReader();\n  SyncBuffer(*shard);\n  return CloseShardInternal();\n}\n\nabsl::Status JoiningReaderBase::AnnotateStatusImpl(absl::Status status) {\n  Reader* shard = ShardReader();\n  if (shard_is_open(shard)) status = shard->AnnotateStatus(std::move(status));\n  // The status might have been annotated by `*shard` with the position within\n  // the shard. Clarify that the current position is the position across shards\n  // instead of delegating to `PullableReader::AnnotateStatusImpl()`.\n  return AnnotateOverShard(std::move(status));\n}\n\nabsl::Status JoiningReaderBase::AnnotateOverShard(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"across shards at byte \", pos()));\n  }\n  return status;\n}\n\nvoid JoiningReaderBase::SetReadAllHintImpl(bool read_all_hint) {\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  read_all_hint_ = read_all_hint;\n  Reader* shard = ShardReader();\n  if (!shard_is_open(shard)) return;\n  BehindScratch behind_scratch(this);\n  SyncBuffer(*shard);\n  shard->SetReadAllHint(read_all_hint_);\n  MakeBuffer(*shard);\n}\n\nbool JoiningReaderBase::PullBehindScratch(size_t recommended_length) {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"enough data available, use Pull() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader* shard = ShardReader();\n  if (shard_is_open(shard)) {\n    SyncBuffer(*shard);\n  } else {\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardReader();\n  }\n  while (ABSL_PREDICT_FALSE(!shard->Pull(1, recommended_length))) {\n    if (ABSL_PREDICT_FALSE(!shard->ok())) {\n      return FailWithoutAnnotation(AnnotateOverShard(shard->status()));\n    }\n    if (ABSL_PREDICT_FALSE(!CloseShardInternal())) return false;\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardReader();\n  }\n  if (ABSL_PREDICT_FALSE(limit_pos() == std::numeric_limits<Position>::max())) {\n    return FailOverflow();\n  }\n  MakeBuffer(*shard);\n  return true;\n}\n\nbool JoiningReaderBase::ReadBehindScratch(size_t length, char* dest) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of PullableReader::ReadBehindScratch(char*): \"\n         \"enough data available, use Read(char*) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(char*): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader* shard = ShardReader();\n  if (shard_is_open(shard)) {\n    SyncBuffer(*shard);\n  } else {\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardReader();\n  }\n  for (;;) {\n    const size_t length_to_read =\n        UnsignedMin(length, std::numeric_limits<Position>::max() - limit_pos());\n    size_t length_read;\n    const bool read_ok = shard->Read(length_to_read, dest, &length_read);\n    move_limit_pos(length_read);\n    if (ABSL_PREDICT_TRUE(read_ok)) {\n      if (ABSL_PREDICT_FALSE(length_to_read < length)) return FailOverflow();\n      break;\n    }\n    if (ABSL_PREDICT_FALSE(!shard->ok())) {\n      return FailWithoutAnnotation(AnnotateOverShard(shard->status()));\n    }\n    dest += length_read;\n    length -= length_read;\n    if (ABSL_PREDICT_FALSE(!CloseShardInternal())) return false;\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardReader();\n  }\n  MakeBuffer(*shard);\n  return true;\n}\n\nbool JoiningReaderBase::ReadBehindScratch(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"Chain size overflow\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"scratch used\";\n  return ReadInternal(length, dest);\n}\n\nbool JoiningReaderBase::ReadBehindScratch(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"Cord size overflow\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"scratch used\";\n  return ReadInternal(length, dest);\n}\n\ntemplate <typename Dest>\ninline bool JoiningReaderBase::ReadInternal(size_t length, Dest& dest) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader* shard = ShardReader();\n  if (shard_is_open(shard)) {\n    SyncBuffer(*shard);\n  } else {\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardReader();\n  }\n  for (;;) {\n    const size_t length_to_read =\n        UnsignedMin(length, std::numeric_limits<Position>::max() - limit_pos());\n    size_t length_read;\n    const bool read_ok =\n        shard->ReadAndAppend(length_to_read, dest, &length_read);\n    move_limit_pos(length_read);\n    if (ABSL_PREDICT_TRUE(read_ok)) {\n      if (ABSL_PREDICT_FALSE(length_to_read < length)) return FailOverflow();\n      break;\n    }\n    if (ABSL_PREDICT_FALSE(!shard->ok())) {\n      return FailWithoutAnnotation(AnnotateOverShard(shard->status()));\n    }\n    length -= length_read;\n    if (ABSL_PREDICT_FALSE(!CloseShardInternal())) return false;\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardReader();\n  }\n  MakeBuffer(*shard);\n  return true;\n}\n\nbool JoiningReaderBase::CopyBehindScratch(Position length, Writer& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::CopyBehindScratch(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::CopyBehindScratch(Writer&): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader* shard = ShardReader();\n  if (shard_is_open(shard)) {\n    SyncBuffer(*shard);\n  } else {\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardReader();\n  }\n  for (;;) {\n    const size_t length_to_read =\n        UnsignedMin(length, std::numeric_limits<Position>::max() - limit_pos());\n    Position length_read;\n    const bool copy_ok = shard->Copy(length_to_read, dest, &length_read);\n    move_limit_pos(length_read);\n    if (ABSL_PREDICT_TRUE(copy_ok)) {\n      if (ABSL_PREDICT_FALSE(length_to_read < length)) return FailOverflow();\n      break;\n    }\n    if (ABSL_PREDICT_FALSE(!shard->ok())) {\n      return FailWithoutAnnotation(AnnotateOverShard(shard->status()));\n    }\n    if (ABSL_PREDICT_FALSE(!dest.ok())) {\n      MakeBuffer(*shard);\n      return false;\n    }\n    length -= length_read;\n    if (ABSL_PREDICT_FALSE(!CloseShardInternal())) return false;\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardReader();\n  }\n  MakeBuffer(*shard);\n  return true;\n}\n\nbool JoiningReaderBase::ReadSomeBehindScratch(size_t max_length, char* dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of PullableReader::ReadSomeBehindScratch(char*): \"\n         \"nothing to read, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PullableReader::ReadSomeBehindScratch(char*): \"\n         \"some data available, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadSomeBehindScratch(char*): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader* shard = ShardReader();\n  if (shard_is_open(shard)) {\n    SyncBuffer(*shard);\n  } else {\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardReader();\n  }\n  const Position remaining = std::numeric_limits<Position>::max() - limit_pos();\n  if (ABSL_PREDICT_FALSE(remaining == 0)) return FailOverflow();\n  max_length = UnsignedMin(max_length, remaining);\n  for (;;) {\n    size_t length_read;\n    if (ABSL_PREDICT_TRUE(shard->ReadSome(max_length, dest, &length_read))) {\n      move_limit_pos(length_read);\n      break;\n    }\n    if (ABSL_PREDICT_FALSE(!shard->ok())) {\n      return FailWithoutAnnotation(AnnotateOverShard(shard->status()));\n    }\n    if (ABSL_PREDICT_FALSE(!CloseShardInternal())) return false;\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardReader();\n  }\n  MakeBuffer(*shard);\n  return true;\n}\n\nbool JoiningReaderBase::CopySomeBehindScratch(size_t max_length, Writer& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of \"\n         \"PullableReader::CopySomeBehindScratch(Writer&): \"\n         \"nothing to read, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of \"\n         \"PullableReader::CopySomeBehindScratch(Writer&): \"\n         \"some data available, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PullableReader::CopySomeBehindScratch(Writer&): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader* shard = ShardReader();\n  if (shard_is_open(shard)) {\n    SyncBuffer(*shard);\n  } else {\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardReader();\n  }\n  const Position remaining = std::numeric_limits<Position>::max() - limit_pos();\n  if (ABSL_PREDICT_FALSE(remaining == 0)) return FailOverflow();\n  max_length = UnsignedMin(max_length, remaining);\n  for (;;) {\n    size_t length_read;\n    const bool copy_ok = shard->CopySome(max_length, dest, &length_read);\n    move_limit_pos(length_read);\n    if (ABSL_PREDICT_TRUE(copy_ok)) break;\n    if (ABSL_PREDICT_FALSE(!shard->ok())) {\n      return FailWithoutAnnotation(AnnotateOverShard(shard->status()));\n    }\n    if (ABSL_PREDICT_FALSE(!dest.ok())) {\n      MakeBuffer(*shard);\n      return false;\n    }\n    if (ABSL_PREDICT_FALSE(!CloseShardInternal())) return false;\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardReader();\n  }\n  MakeBuffer(*shard);\n  return true;\n}\n\nvoid JoiningReaderBase::ReadHintBehindScratch(size_t min_length,\n                                              size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of PullableReader::ReadHintBehindScratch(): \"\n         \"enough data available, use ReadHint() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadHintBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  Reader* shard = ShardReader();\n  if (shard_is_open(shard)) {\n    SyncBuffer(*shard);\n  } else {\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return;\n    shard = ShardReader();\n  }\n  shard->ReadHint(min_length, recommended_length);\n  MakeBuffer(*shard);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/joining_reader.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_JOINING_READER_H_\n#define RIEGELI_BYTES_JOINING_READER_H_\n\n#include <stddef.h>\n\n#include <limits>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pullable_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nclass Writer;\n\n// Template parameter independent part of `JoiningReader`.\nclass JoiningReaderBase : public PullableReader {\n public:\n  bool ToleratesReadingAhead() override { return read_all_hint_; }\n\n protected:\n  using PullableReader::PullableReader;\n\n  JoiningReaderBase(JoiningReaderBase&& that) noexcept;\n  JoiningReaderBase& operator=(JoiningReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n\n  void Done() override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n\n  // Returns the shard `Reader`.\n  virtual Reader* ShardReader() ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n  virtual const Reader* ShardReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Opens the next shard as `shard()` if it exists.\n  //\n  // Preconditions:\n  //   `ok()`\n  //   `!shard_is_open()`\n  //\n  // Return values:\n  //  * `true`                 - success (`ok()`, `shard_is_open()`)\n  //  * `false` (when `ok()`)  - there is no next shard\n  //  * `false` (when `!ok()`) - failure\n  //\n  // `OpenShardImpl()` must be overridden but should not be called directly\n  // because it does not synchronize buffer pointers of `*ShardReader()` with\n  // `*this`. See `OpenShard()` for that.\n  virtual bool OpenShardImpl() = 0;\n\n  // Closes `shard()`.\n  //\n  // Preconditions:\n  //   `ok()`\n  //   `shard_is_open()`\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`, `!shard_is_open()`)\n  //  * `false` - failure (`!ok()`, `!shard_is_open()`)\n  //\n  // The default implementation calls `ShardReader()->Close()` and propagates\n  // failures from that.\n  //\n  // `CloseShardImpl()` can be overridden but should not be called directly\n  // because it does not synchronize buffer pointers of `*this` with\n  // `*ShardReader()`. See `CloseShard()` for that.\n  virtual bool CloseShardImpl();\n\n  // Calls `OpenShardImpl()` and synchronizes buffer pointers of\n  // `*ShardReader()` with `*this`.\n  //\n  // Preconditions:\n  //   `ok()`\n  //   `!shard_is_open()`\n  //\n  // Return values:\n  //  * `true`                 - success (`ok()`, `shard_is_open()`)\n  //  * `false` (when `ok()`)  - there is no next shard\n  //  * `false` (when `!ok()`) - failure\n  bool OpenShard();\n\n  // Synchronizes buffer pointers of `*this` with `*ShardReader()` and calls\n  // `CloseShardImpl()`.\n  //\n  // Preconditions:\n  //   `ok()`\n  //   `shard_is_open()`\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`, `!shard_is_open()`)\n  //  * `false` - failure (`!ok()`, `!shard_is_open()`)\n  bool CloseShard();\n\n  // Returns `true` if a shard is open.\n  //\n  // Same as `shard != nullptr && shard->is_open()`, with the default `shard` of\n  // `ShardReader()`.\n  bool shard_is_open() const;\n  bool shard_is_open(const Reader* shard) const;\n\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverShard(absl::Status status);\n\n  // Sets cursor of `shard` to cursor of `*this`. Sets buffer pointers of\n  // `*this` to `nullptr`.\n  void SyncBuffer(Reader& shard);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `shard`. Fails\n  // `*this` if `shard` failed.\n  void MakeBuffer(Reader& shard);\n\n  bool PullBehindScratch(size_t recommended_length) override;\n  using PullableReader::ReadBehindScratch;\n  bool ReadBehindScratch(size_t length, char* dest) override;\n  bool ReadBehindScratch(size_t length, Chain& dest) override;\n  bool ReadBehindScratch(size_t length, absl::Cord& dest) override;\n  using PullableReader::CopyBehindScratch;\n  bool CopyBehindScratch(Position length, Writer& dest) override;\n  using PullableReader::ReadSomeBehindScratch;\n  bool ReadSomeBehindScratch(size_t max_length, char* dest) override;\n  using PullableReader::CopySomeBehindScratch;\n  bool CopySomeBehindScratch(size_t max_length, Writer& dest) override;\n  void ReadHintBehindScratch(size_t min_length,\n                             size_t recommended_length) override;\n\n private:\n  bool OpenShardInternal();\n  bool CloseShardInternal();\n\n  // This template is defined and used only in joining_reader.cc.\n  template <typename Dest>\n  bool ReadInternal(size_t length, Dest& dest);\n\n  bool read_all_hint_ = false;\n\n  // Invariants if `ok()` and scratch is not used:\n  //   `start() == (shard_is_open() ? ShardReader()->cursor() : nullptr)`\n  //   `limit() <= (shard_is_open() ? ShardReader()->limit() : nullptr)`\n};\n\n// Abstract class of a `Reader` which joins data from multiple shards.\n//\n// The `Shard` template parameter specifies the type of the object providing and\n// possibly owning the shard `Reader`. `Shard` must support\n// `Dependency<Reader*, Shard>`, e.g. `Reader*` (not owned),\n// `std::unique_ptr<Reader>` (owned), `ChainReader<>` (owned),\n// `Any<Reader*>` (maybe owned).\ntemplate <typename Shard>\nclass JoiningReader : public JoiningReaderBase {\n protected:\n  using JoiningReaderBase::JoiningReaderBase;\n\n  JoiningReader(JoiningReader&& that) = default;\n  JoiningReader& operator=(JoiningReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `JoiningReader`. This\n  // avoids constructing a temporary `JoiningReader` and moving from it.\n  // Derived classes which override `Reset()` should include a call to\n  // `JoiningReader::Reset()`.\n  void Reset(Closed);\n  void Reset();\n\n  void Done() override;\n\n  // Returns the object providing and possibly owning the shard `Reader`.\n  Shard& shard() ABSL_ATTRIBUTE_LIFETIME_BOUND { return shard_.manager(); }\n  const Shard& shard() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return shard_.manager();\n  }\n  Reader* ShardReader() ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return shard_.get();\n  }\n  const Reader* ShardReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return shard_.get();\n  }\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the shard `Reader`.\n  MovingDependency<Reader*, Shard, Mover> shard_;\n};\n\n// Implementation details follow.\n\ninline JoiningReaderBase::JoiningReaderBase(JoiningReaderBase&& that) noexcept\n    : PullableReader(static_cast<PullableReader&&>(that)),\n      read_all_hint_(that.read_all_hint_) {}\n\ninline JoiningReaderBase& JoiningReaderBase::operator=(\n    JoiningReaderBase&& that) noexcept {\n  PullableReader::operator=(static_cast<PullableReader&&>(that));\n  read_all_hint_ = that.read_all_hint_;\n  return *this;\n}\n\ninline void JoiningReaderBase::Reset(Closed) {\n  PullableReader::Reset(kClosed);\n  read_all_hint_ = false;\n}\n\ninline void JoiningReaderBase::Reset() {\n  PullableReader::Reset();\n  read_all_hint_ = false;\n}\n\ninline bool JoiningReaderBase::shard_is_open() const {\n  return shard_is_open(ShardReader());\n}\n\ninline bool JoiningReaderBase::shard_is_open(const Reader* shard) const {\n  return shard != nullptr && shard->is_open();\n}\n\ninline void JoiningReaderBase::SyncBuffer(Reader& shard) {\n  RIEGELI_ASSERT(shard_is_open(&shard))\n      << \"Failed precondition of JoiningReaderBase::SyncBuffer(): \"\n         \"shard is closed\";\n  shard.set_cursor(cursor());\n  set_limit_pos(pos());\n  set_buffer();\n}\n\ninline void JoiningReaderBase::MakeBuffer(Reader& shard) {\n  RIEGELI_ASSERT(shard_is_open(&shard))\n      << \"Failed precondition of JoiningReaderBase::MakeBuffer(): \"\n         \"shard is closed\";\n  set_buffer(shard.cursor(),\n             UnsignedMin(shard.available(),\n                         std::numeric_limits<Position>::max() - limit_pos()));\n  move_limit_pos(available());\n  if (ABSL_PREDICT_FALSE(!shard.ok())) {\n    FailWithoutAnnotation(AnnotateOverShard(shard.status()));\n  }\n}\n\ntemplate <typename Shard>\nclass JoiningReader<Shard>::Mover {\n public:\n  static auto member() { return &JoiningReader::shard_; }\n\n  explicit Mover(JoiningReader& self, JoiningReader& that)\n      : behind_scratch_(&self), uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `shard_` is not moved yet so `shard_` is taken from `that`.\n    if (uses_buffer_) self.SyncBuffer(*that.shard_);\n  }\n\n  void Done(JoiningReader& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.shard_);\n  }\n\n private:\n  BehindScratch behind_scratch_;\n  bool uses_buffer_;\n};\n\ntemplate <typename Shard>\ninline void JoiningReader<Shard>::Reset(Closed) {\n  JoiningReaderBase::Reset(kClosed);\n  shard_.Reset();\n}\n\ntemplate <typename Shard>\ninline void JoiningReader<Shard>::Reset() {\n  JoiningReaderBase::Reset();\n  shard_.Reset();\n}\n\ntemplate <typename Shard>\nvoid JoiningReader<Shard>::Done() {\n  JoiningReaderBase::Done();\n  shard_.Reset();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_JOINING_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/limiting_backward_writer.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/limiting_backward_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\nvoid LimitingBackwardWriterBase::Initialize(BackwardWriter* dest,\n                                            const Options& options,\n                                            bool is_owning) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of LimitingBackwardWriter: \"\n         \"null BackwardWriter pointer\";\n  if (is_owning && exact()) {\n    if (options.max_pos() != std::nullopt) {\n      dest->SetWriteSizeHint(SaturatingSub(*options.max_pos(), dest->pos()));\n    } else if (options.max_length() != std::nullopt) {\n      dest->SetWriteSizeHint(*options.max_length());\n    }\n  }\n  set_buffer(dest->limit(), dest->start_to_limit(), dest->start_to_cursor());\n  set_start_pos(dest->start_pos());\n  if (ABSL_PREDICT_FALSE(!dest->ok())) FailWithoutAnnotation(dest->status());\n  if (options.max_pos() != std::nullopt) {\n    set_max_pos(*options.max_pos());\n  } else if (options.max_length() != std::nullopt) {\n    set_max_length(*options.max_length());\n  }\n}\n\nvoid LimitingBackwardWriterBase::set_max_pos(Position max_pos) {\n  max_pos_ = max_pos;\n  if (ABSL_PREDICT_FALSE(start_pos() > max_pos_)) {\n    set_buffer(cursor());\n    set_start_pos(max_pos_);\n    FailLimitExceeded();\n  }\n}\n\nvoid LimitingBackwardWriterBase::set_max_length(Position max_length) {\n  if (ABSL_PREDICT_FALSE(max_length >\n                         std::numeric_limits<Position>::max() - pos())) {\n    if (exact_) FailLengthOverflow(max_length);\n    max_pos_ = std::numeric_limits<Position>::max();\n    return;\n  }\n  set_max_pos(pos() + max_length);\n}\n\nvoid LimitingBackwardWriterBase::Done() {\n  BackwardWriter& dest = *DestWriter();\n  if (ABSL_PREDICT_TRUE(ok())) SyncBuffer(dest);\n  if (exact_ && ABSL_PREDICT_FALSE(pos() < max_pos_)) {\n    // Do not call `Fail()` because `AnnotateStatusImpl()` synchronizes the\n    // buffer again.\n    FailWithoutAnnotation(dest.AnnotateStatus(absl::InvalidArgumentError(\n        absl::StrCat(\"Not enough data: expected \", max_pos(), \" or \",\n                     max_length(), \" more\"))));\n  }\n  BackwardWriter::Done();\n}\n\nbool LimitingBackwardWriterBase::FailLimitExceeded() {\n  BackwardWriter& dest = *DestWriter();\n  return FailLimitExceeded(dest);\n}\n\nbool LimitingBackwardWriterBase::FailLimitExceeded(BackwardWriter& dest) {\n  set_start_pos(max_pos_);\n  set_buffer();\n  // Do not call `Fail()` because `AnnotateStatusImpl()` synchronizes the buffer\n  // again.\n  return FailWithoutAnnotation(dest.AnnotateStatus(\n      absl::ResourceExhaustedError(\"Position limit exceeded\")));\n}\n\ninline void LimitingBackwardWriterBase::FailLengthOverflow(\n    Position max_length) {\n  Fail(absl::InvalidArgumentError(absl::StrCat(\n      \"Not enough data: expected \", max_length,\n      \" more, which overflows the BackwardWriter position range\")));\n}\n\nabsl::Status LimitingBackwardWriterBase::AnnotateStatusImpl(\n    absl::Status status) {\n  // Fully delegate annotations to `*DestWriter()`.\n  if (is_open()) {\n    BackwardWriter& dest = *DestWriter();\n    const bool sync_buffer_ok = SyncBuffer(dest);\n    status = dest.AnnotateStatus(std::move(status));\n    if (ABSL_PREDICT_TRUE(sync_buffer_ok)) MakeBuffer(dest);\n  }\n  return status;\n}\n\nbool LimitingBackwardWriterBase::PushSlow(size_t min_length,\n                                          size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of BackwardWriter::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  RIEGELI_ASSERT_LE(start_pos(), max_pos_)\n      << \"Failed invariant of LimitingBackwardWriterBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  BackwardWriter& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return false;\n  const bool push_ok = dest.Push(min_length, recommended_length);\n  MakeBuffer(dest);\n  return push_ok;\n}\n\nbool LimitingBackwardWriterBase::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  return WriteInternal(src, [](absl::string_view src, size_t length) {\n    src.remove_prefix(length);\n    return src;\n  });\n}\n\nbool LimitingBackwardWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  return WriteInternal(std::move(src), [](ExternalRef src, size_t length) {\n    Chain result(std::move(src));\n    result.RemovePrefix(length);\n    return result;\n  });\n}\n\nbool LimitingBackwardWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  return WriteInternal(src, [](const Chain& src, size_t length) {\n    Chain result = src;\n    result.RemovePrefix(length);\n    return result;\n  });\n}\n\nbool LimitingBackwardWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  return WriteInternal(std::move(src),\n                       [](Chain&& src, size_t length) -> Chain&& {\n                         src.RemovePrefix(length);\n                         return std::move(src);\n                       });\n}\n\nbool LimitingBackwardWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  return WriteInternal(src, [](const absl::Cord& src, size_t length) {\n    absl::Cord result = src;\n    result.RemovePrefix(length);\n    return result;\n  });\n}\n\nbool LimitingBackwardWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  return WriteInternal(std::move(src),\n                       [](absl::Cord&& src, size_t length) -> absl::Cord&& {\n                         src.RemovePrefix(length);\n                         return std::move(src);\n                       });\n}\n\ntemplate <typename Src, typename RemovePrefix>\ninline bool LimitingBackwardWriterBase::WriteInternal(\n    Src&& src, RemovePrefix&& remove_prefix) {\n  RIEGELI_ASSERT_LE(start_pos(), max_pos_)\n      << \"Failed invariant of LimitingBackwardWriterBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  BackwardWriter& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return false;\n  const Position max_length = max_pos_ - pos();\n  if (ABSL_PREDICT_TRUE(src.size() <= max_pos_ - pos())) {\n    const bool write_ok = dest.Write(std::forward<Src>(src));\n    MakeBuffer(dest);\n    return write_ok;\n  }\n  if (ABSL_PREDICT_FALSE(!dest.Write(std::forward<RemovePrefix>(remove_prefix)(\n          std::forward<Src>(src), src.size() - max_length)))) {\n    MakeBuffer(dest);\n    return false;\n  }\n  return FailLimitExceeded(dest);\n}\n\nbool LimitingBackwardWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  RIEGELI_ASSERT_LE(start_pos(), max_pos_)\n      << \"Failed invariant of LimitingBackwardWriterBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  BackwardWriter& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return false;\n  const Position max_length = max_pos_ - pos();\n  if (ABSL_PREDICT_TRUE(src.size() <= max_length)) {\n    const bool write_ok = dest.Write(src);\n    MakeBuffer(dest);\n    return write_ok;\n  }\n  if (ABSL_PREDICT_FALSE(!dest.Write(src.Extract(max_length)))) {\n    MakeBuffer(dest);\n    return false;\n  }\n  return FailLimitExceeded(dest);\n}\n\nbool LimitingBackwardWriterBase::SupportsTruncate() {\n  BackwardWriter* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsTruncate();\n}\n\nbool LimitingBackwardWriterBase::TruncateImpl(Position new_size) {\n  RIEGELI_ASSERT_LE(start_pos(), max_pos_)\n      << \"Failed invariant of LimitingBackwardWriterBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  BackwardWriter& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(pos() > max_pos_) && new_size <= max_pos_) {\n    set_cursor(cursor() + IntCast<size_t>(pos() - max_pos_));\n  }\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return false;\n  const bool truncate_ok = dest.Truncate(new_size);\n  MakeBuffer(dest);\n  return truncate_ok;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/limiting_backward_writer.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_LIMITING_BACKWARD_WRITER_H_\n#define RIEGELI_BYTES_LIMITING_BACKWARD_WRITER_H_\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `LimitingBackwardWriter`.\nclass LimitingBackwardWriterBase : public BackwardWriter {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // The limit expressed as an absolute position.\n    //\n    // `std::nullopt` means no limit, unless `max_length()` is set.\n    //\n    // Default: `std::nullopt`.\n    Options& set_max_pos(std::optional<Position> max_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      max_pos_ = max_pos;\n      max_length_ = std::nullopt;\n      return *this;\n    }\n    Options&& set_max_pos(std::optional<Position> max_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_pos(max_pos));\n    }\n    std::optional<Position> max_pos() const { return max_pos_; }\n\n    // A shortcut for `set_max_pos(pos)` with `set_exact(true)`.\n    Options& set_exact_pos(Position exact_pos) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_max_pos(exact_pos).set_exact(true);\n    }\n    Options&& set_exact_pos(Position exact_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_exact_pos(exact_pos));\n    }\n\n    // The limit expressed as a length relative to the current position.\n    //\n    // `std::nullopt` means no limit, unless `max_pos()` is set.\n    //\n    // Default: `std::nullopt`.\n    Options& set_max_length(std::optional<Position> max_length) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      max_length_ = max_length;\n      max_pos_ = std::nullopt;\n      return *this;\n    }\n    Options&& set_max_length(std::optional<Position> max_length) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_length(max_length));\n    }\n    std::optional<Position> max_length() const { return max_length_; }\n\n    // A shortcut for `set_max_length(length)` with `set_exact(true)`.\n    Options& set_exact_length(Position exact_length) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_max_length(exact_length).set_exact(true);\n    }\n    Options&& set_exact_length(Position exact_length) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_exact_length(exact_length));\n    }\n\n    // If `false`, `LimitingBackwardWriter` will write data at most up to the\n    // limit. Writing will fail if the limit is exceeded.\n    //\n    // If `true`, `LimitingBackwardWriter` will write data exactly up to the\n    // limit. Writing will fail if the limit is exceeded, and `Close()` will\n    // fail if the current position at that time is before the limit.\n    //\n    // Default: `false`.\n    Options& set_exact(bool exact) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      exact_ = exact;\n      return *this;\n    }\n    Options&& set_exact(bool exact) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_exact(exact));\n    }\n    bool exact() const { return exact_; }\n\n   private:\n    std::optional<Position> max_pos_;\n    std::optional<Position> max_length_;\n    bool exact_ = false;\n  };\n\n  // Returns the original `BackwardWriter`. Unchanged by `Close()`.\n  virtual BackwardWriter* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Accesses the limit expressed as an absolute position.\n  //\n  // If `set_max_length()` was used, `max_pos()` returns the same limit\n  // translated to an absolute position.\n  //\n  // If no limit is set, returns `std::numeric_limits<Position>::max()`.\n  void set_max_pos(Position max_pos);\n  Position max_pos() const { return max_pos_; }\n\n  // Accesses the limit expressed as a length relative to the current position,\n  // i.e. the length remaining to the limit.\n  //\n  // If `set_max_pos()` was used, `max_length()` returns the same limit\n  // translated to a length relative to the current position.\n  //\n  // If no limit is set, returns `std::numeric_limits<Position>::max() - pos()`.\n  void set_max_length(Position max_length);\n  Position max_length() const { return SaturatingSub(max_pos_, pos()); }\n\n  // Clears the limit.\n  void clear_limit() { max_pos_ = std::numeric_limits<Position>::max(); }\n\n  bool SupportsTruncate() override;\n\n protected:\n  explicit LimitingBackwardWriterBase(Closed) noexcept\n      : BackwardWriter(kClosed) {}\n\n  explicit LimitingBackwardWriterBase(bool exact);\n\n  LimitingBackwardWriterBase(LimitingBackwardWriterBase&& that) noexcept;\n  LimitingBackwardWriterBase& operator=(\n      LimitingBackwardWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(bool exact);\n  void Initialize(BackwardWriter* dest, const Options& options, bool is_owning);\n  bool exact() const { return exact_; }\n\n  // Sets cursor of `dest` to cursor of `*this`. Fails `*this` if the limit is\n  // exceeded.\n  bool SyncBuffer(BackwardWriter& dest);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `dest`. Fails `*this`\n  // if `dest` failed.\n  void MakeBuffer(BackwardWriter& dest);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using BackwardWriter::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool TruncateImpl(Position new_size) override;\n\n private:\n  ABSL_ATTRIBUTE_COLD bool FailLimitExceeded();\n  ABSL_ATTRIBUTE_COLD bool FailLimitExceeded(BackwardWriter& dest);\n  ABSL_ATTRIBUTE_COLD void FailLengthOverflow(Position max_length);\n\n  // This template is defined and used only in limiting_backward_writer.cc.\n  template <typename Src, typename RemovePrefix>\n  bool WriteInternal(Src&& src, RemovePrefix&& remove_prefix);\n\n  // Invariant: `start_pos() <= max_pos_`\n  Position max_pos_ = std::numeric_limits<Position>::max();\n\n  bool exact_ = false;\n\n  // Invariants if `ok()`:\n  //   `start() == DestWriter()->start()`\n  //   `limit() == DestWriter()->limit()`\n  //   `start_pos() == DestWriter()->start_pos()`\n};\n\n// A `BackwardWriter` which writes to another `BackwardWriter` up to the\n// specified size limit. An attempt to write more fails, after writing to the\n// destination everything up to the limit.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the original `BackwardWriter`. `Dest` must support\n// `Dependency<BackwardWriter*, Dest>`, e.g.\n// `BackwardWriter*` (not owned, default),\n// `ChainBackwardWriter<>` (owned), `std::unique_ptr<BackwardWriter>` (owned),\n// `Any<BackwardWriter*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The original `BackwardWriter` must not be accessed until the\n// `LimitingBackwardWriter` is closed or no longer used, except that it is\n// allowed to read the destination of the original `BackwardWriter` immediately\n// after `Flush()`.\ntemplate <typename Dest = BackwardWriter*>\nclass LimitingBackwardWriter : public LimitingBackwardWriterBase {\n public:\n  // Creates a closed `LimitingBackwardWriter`.\n  explicit LimitingBackwardWriter(Closed) noexcept\n      : LimitingBackwardWriterBase(kClosed) {}\n\n  // Will write to the original `BackwardWriter` provided by `dest`.\n  explicit LimitingBackwardWriter(Initializer<Dest> dest,\n                                  Options options = Options());\n\n  LimitingBackwardWriter(LimitingBackwardWriter&& that) = default;\n  LimitingBackwardWriter& operator=(LimitingBackwardWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `LimitingBackwardWriter`.\n  // This avoids constructing a temporary `LimitingBackwardWriter` and moving\n  // from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the original\n  // `BackwardWriter`. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  BackwardWriter* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the original `BackwardWriter`.\n  MovingDependency<BackwardWriter*, Dest, Mover> dest_;\n};\n\nexplicit LimitingBackwardWriter(Closed)\n    -> LimitingBackwardWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit LimitingBackwardWriter(Dest&& dest,\n                                LimitingBackwardWriterBase::Options options =\n                                    LimitingBackwardWriterBase::Options())\n    -> LimitingBackwardWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline LimitingBackwardWriterBase::LimitingBackwardWriterBase(bool exact)\n    : exact_(exact) {}\n\ninline LimitingBackwardWriterBase::LimitingBackwardWriterBase(\n    LimitingBackwardWriterBase&& that) noexcept\n    : BackwardWriter(static_cast<BackwardWriter&&>(that)),\n      max_pos_(that.max_pos_),\n      exact_(that.exact_) {}\n\ninline LimitingBackwardWriterBase& LimitingBackwardWriterBase::operator=(\n    LimitingBackwardWriterBase&& that) noexcept {\n  BackwardWriter::operator=(static_cast<BackwardWriter&&>(that));\n  max_pos_ = that.max_pos_;\n  exact_ = that.exact_;\n  return *this;\n}\n\ninline void LimitingBackwardWriterBase::Reset(Closed) {\n  BackwardWriter::Reset(kClosed);\n  max_pos_ = std::numeric_limits<Position>::max();\n  exact_ = false;\n}\n\ninline void LimitingBackwardWriterBase::Reset(bool exact) {\n  BackwardWriter::Reset();\n  max_pos_ = std::numeric_limits<Position>::max();\n  exact_ = exact;\n}\n\ninline bool LimitingBackwardWriterBase::SyncBuffer(BackwardWriter& dest) {\n  if (ABSL_PREDICT_FALSE(pos() > max_pos_)) {\n    dest.set_cursor(cursor() + IntCast<size_t>(pos() - max_pos_));\n    return FailLimitExceeded(dest);\n  }\n  dest.set_cursor(cursor());\n  return true;\n}\n\ninline void LimitingBackwardWriterBase::MakeBuffer(BackwardWriter& dest) {\n  set_buffer(dest.limit(), dest.start_to_limit(), dest.start_to_cursor());\n  set_start_pos(dest.start_pos());\n  if (ABSL_PREDICT_FALSE(start_pos() > max_pos_)) {\n    set_buffer(cursor());\n    set_start_pos(max_pos_);\n    FailLimitExceeded();\n  }\n  if (ABSL_PREDICT_FALSE(!dest.ok())) FailWithoutAnnotation(dest.status());\n}\n\ntemplate <typename Dest>\nclass LimitingBackwardWriter<Dest>::Mover {\n public:\n  static auto member() { return &LimitingBackwardWriter::dest_; }\n\n  explicit Mover(LimitingBackwardWriter& self, LimitingBackwardWriter& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `dest_` is not moved yet so `dest_` is taken from `that`.\n    if (uses_buffer_) {\n      if (ABSL_PREDICT_FALSE(!self.SyncBuffer(*that.dest_))) {\n        uses_buffer_ = false;\n      }\n    }\n  }\n\n  void Done(LimitingBackwardWriter& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.dest_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Dest>\ninline LimitingBackwardWriter<Dest>::LimitingBackwardWriter(\n    Initializer<Dest> dest, Options options)\n    : LimitingBackwardWriterBase(options.exact()), dest_(std::move(dest)) {\n  Initialize(dest_.get(), options, dest_.IsOwning());\n}\n\ntemplate <typename Dest>\ninline void LimitingBackwardWriter<Dest>::Reset(Closed) {\n  LimitingBackwardWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void LimitingBackwardWriter<Dest>::Reset(Initializer<Dest> dest,\n                                                Options options) {\n  LimitingBackwardWriterBase::Reset(options.exact());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options, dest_.IsOwning());\n}\n\ntemplate <typename Dest>\nvoid LimitingBackwardWriter<Dest>::Done() {\n  LimitingBackwardWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(dest_->status());\n    }\n  }\n}\n\ntemplate <typename Dest>\nvoid LimitingBackwardWriter<Dest>::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (dest_.IsOwning() && !exact()) {\n    if (ABSL_PREDICT_FALSE(!SyncBuffer(*dest_))) return;\n    dest_->SetWriteSizeHint(\n        write_size_hint == std::nullopt\n            ? std::nullopt\n            : std::make_optional(UnsignedMin(*write_size_hint, max_length())));\n    MakeBuffer(*dest_);\n  }\n}\n\ntemplate <typename Dest>\nbool LimitingBackwardWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(*dest_))) return false;\n  bool flush_ok = true;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    flush_ok = dest_->Flush(flush_type);\n  }\n  MakeBuffer(*dest_);\n  return flush_ok;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_LIMITING_BACKWARD_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/limiting_reader.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/limiting_reader.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid LimitingReaderBase::Initialize(Reader* src, const Options& options) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of LimitingReader: null Reader pointer\";\n  set_buffer(src->start(), src->start_to_limit(), src->start_to_cursor());\n  set_limit_pos(src->limit_pos());\n  if (ABSL_PREDICT_FALSE(!src->ok())) FailWithoutAnnotation(src->status());\n  if (options.max_pos() != std::nullopt) {\n    set_max_pos(*options.max_pos());\n  } else if (options.max_length() != std::nullopt) {\n    set_max_length(*options.max_length());\n  }\n}\n\nvoid LimitingReaderBase::MakeBufferSlow() {\n  if (pos() > max_pos_) {\n    set_buffer(cursor());\n  } else {\n    set_buffer(start(),\n               start_to_limit() - IntCast<size_t>(limit_pos() - max_pos_),\n               start_to_cursor());\n  }\n  set_limit_pos(max_pos_);\n}\n\nvoid LimitingReaderBase::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    Reader& src = *SrcReader();\n    SyncBuffer(src);\n    if (fail_if_longer_ && pos() == max_pos_ &&\n        ABSL_PREDICT_FALSE(src.Pull())) {\n      // Do not call `Fail()` because `AnnotateStatusImpl()` synchronizes the\n      // buffer again.\n      FailWithoutAnnotation(src.AnnotateStatus(\n          absl::ResourceExhaustedError(\"Position limit exceeded\")));\n    }\n  }\n  Reader::Done();\n}\n\ninline bool LimitingReaderBase::CheckEnough() {\n  if (ABSL_PREDICT_FALSE(exact_)) return FailNotEnough();\n  return false;\n}\n\ninline bool LimitingReaderBase::FailNotEnough() {\n  return Fail(absl::InvalidArgumentError(\n      max_pos() == std::numeric_limits<Position>::max()\n          ? \"Not enough data: expected impossibly much\"\n          : absl::StrCat(\"Not enough data: expected at least \", max_pos(),\n                         \" or \", max_length(), \" more\")));\n}\n\nvoid LimitingReaderBase::FailNotEnoughAtPos(Position expected_pos) {\n  Fail(absl::InvalidArgumentError(\n      absl::StrCat(\"Not enough data: expected at least \", expected_pos,\n                   \", will have at most \", max_pos())));\n}\n\nvoid LimitingReaderBase::FailNotEnoughAtLength(Position expected_length) {\n  Fail(absl::InvalidArgumentError(\n      absl::StrCat(\"Not enough data: expected at least \", expected_length,\n                   \" more, will have at most \", max_length(), \" more\")));\n}\n\nvoid LimitingReaderBase::FailNotEnoughAtEnd() {\n  Fail(absl::InvalidArgumentError(absl::StrCat(\n      \"Not enough data: expected impossibly much, will have at most \",\n      max_pos())));\n}\n\nvoid LimitingReaderBase::FailLengthOverflow(Position max_length) {\n  Fail(absl::InvalidArgumentError(\n      absl::StrCat(\"Not enough data: expected at least \", max_length,\n                   \" more, which overflows the Reader position range\")));\n}\n\nvoid LimitingReaderBase::FailPositionLimitExceeded() {\n  Fail(absl::ResourceExhaustedError(\"Position limit exceeded\"));\n}\n\nabsl::Status LimitingReaderBase::AnnotateStatusImpl(absl::Status status) {\n  // Fully delegate annotations to `*SrcReader()`.\n  if (is_open()) {\n    Reader& src = *SrcReader();\n    SyncBuffer(src);\n    status = src.AnnotateStatus(std::move(status));\n    MakeBuffer(src);\n  }\n  return status;\n}\n\nbool LimitingReaderBase::PullSlow(size_t min_length,\n                                  size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::PullSlow(): \"\n         \"enough data available, use Pull() instead\";\n  RIEGELI_ASSERT_LE(pos(), max_pos_)\n      << \"Failed invariant of LimitingReaderBase: position exceeds the limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const size_t min_length_to_pull = UnsignedMin(min_length, max_pos_ - pos());\n  const bool pull_ok = src.Pull(min_length_to_pull, recommended_length);\n  MakeBuffer(src);\n  if (ABSL_PREDICT_FALSE(!pull_ok)) return CheckEnough();\n  return min_length_to_pull == min_length;\n}\n\nbool LimitingReaderBase::ReadSlow(size_t length, char* dest) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of Reader::ReadSlow(char*): \"\n         \"enough data available, use Read(char&) instead\";\n  RIEGELI_ASSERT_LE(pos(), max_pos_)\n      << \"Failed invariant of LimitingReaderBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const size_t length_to_read = UnsignedMin(length, max_pos_ - pos());\n  const bool read_ok = src.Read(length_to_read, dest);\n  MakeBuffer(src);\n  if (ABSL_PREDICT_FALSE(!read_ok)) return CheckEnough();\n  return length_to_read == length;\n}\n\nbool LimitingReaderBase::ReadSlow(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"Chain size overflow\";\n  return ReadInternal(length, dest);\n}\n\nbool LimitingReaderBase::ReadSlow(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"Cord size overflow\";\n  return ReadInternal(length, dest);\n}\n\ntemplate <typename Dest>\ninline bool LimitingReaderBase::ReadInternal(size_t length, Dest& dest) {\n  RIEGELI_ASSERT_LE(pos(), max_pos_)\n      << \"Failed invariant of LimitingReaderBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const size_t length_to_read = UnsignedMin(length, max_pos_ - pos());\n  const bool read_ok = src.ReadAndAppend(length_to_read, dest);\n  MakeBuffer(src);\n  if (ABSL_PREDICT_FALSE(!read_ok)) return CheckEnough();\n  return length_to_read == length;\n}\n\nbool LimitingReaderBase::CopySlow(Position length, Writer& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  RIEGELI_ASSERT_LE(pos(), max_pos_)\n      << \"Failed invariant of LimitingReaderBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const Position length_to_copy = UnsignedMin(length, max_pos_ - pos());\n  const bool copy_ok = src.Copy(length_to_copy, dest);\n  MakeBuffer(src);\n  if (ABSL_PREDICT_FALSE(!copy_ok)) {\n    if (dest.ok()) return CheckEnough();\n    return false;\n  }\n  return length_to_copy == length;\n}\n\nbool LimitingReaderBase::CopySlow(size_t length, BackwardWriter& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(BackwardWriter&): \"\n         \"enough data available, use Copy(BackwardWriter&) instead\";\n  RIEGELI_ASSERT_LE(pos(), max_pos_)\n      << \"Failed invariant of LimitingReaderBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  if (ABSL_PREDICT_FALSE(length > max_pos_ - pos())) {\n    const bool seek_ok = src.Seek(max_pos_);\n    MakeBuffer(src);\n    if (ABSL_PREDICT_FALSE(!seek_ok)) return CheckEnough();\n    return false;\n  }\n  const bool copy_ok = src.Copy(length, dest);\n  MakeBuffer(src);\n  if (ABSL_PREDICT_FALSE(!copy_ok)) {\n    if (dest.ok()) return CheckEnough();\n    return false;\n  }\n  return true;\n}\n\nbool LimitingReaderBase::ReadSomeSlow(size_t max_length, char* dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"nothing to read, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"some data available, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_LE(pos(), max_pos_)\n      << \"Failed invariant of LimitingReaderBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  max_length = UnsignedMin(max_length, max_pos_ - pos());\n  const bool read_ok = src.ReadSome(max_length, dest);\n  MakeBuffer(src);\n  if (ABSL_PREDICT_FALSE(!read_ok)) return CheckEnough();\n  return max_length > 0;\n}\n\nbool LimitingReaderBase::CopySomeSlow(size_t max_length, Writer& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"nothing to read, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"some data available, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT_LE(pos(), max_pos_)\n      << \"Failed invariant of LimitingReaderBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  max_length = UnsignedMin(max_length, max_pos_ - pos());\n  const bool copy_ok = src.CopySome(max_length, dest);\n  MakeBuffer(src);\n  if (ABSL_PREDICT_FALSE(!copy_ok)) {\n    if (dest.ok()) return CheckEnough();\n    return false;\n  }\n  return max_length > 0;\n}\n\nvoid LimitingReaderBase::ReadHintSlow(size_t min_length,\n                                      size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::ReadHintSlow(): \"\n         \"enough data available, use ReadHint() instead\";\n  RIEGELI_ASSERT_LE(pos(), max_pos_)\n      << \"Failed invariant of LimitingReaderBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const Position remaining = max_pos_ - pos();\n  src.ReadHint(UnsignedMin(min_length, remaining),\n               UnsignedMin(recommended_length, remaining));\n  MakeBuffer(src);\n}\n\nbool LimitingReaderBase::ToleratesReadingAhead() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->ToleratesReadingAhead();\n}\n\nbool LimitingReaderBase::SupportsRandomAccess() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRandomAccess();\n}\n\nbool LimitingReaderBase::SupportsRewind() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRewind();\n}\n\nbool LimitingReaderBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of Reader::SeekSlow(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_LE(pos(), max_pos_)\n      << \"Failed invariant of LimitingReaderBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const Position pos_to_seek = UnsignedMin(new_pos, max_pos_);\n  const bool seek_ok = src.Seek(pos_to_seek);\n  MakeBuffer(src);\n  if (ABSL_PREDICT_FALSE(!seek_ok)) return CheckEnough();\n  return pos_to_seek == new_pos;\n}\n\nbool LimitingReaderBase::SupportsSize() {\n  if (exact_) return true;\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsSize();\n}\n\nstd::optional<Position> LimitingReaderBase::SizeImpl() {\n  RIEGELI_ASSERT_LE(pos(), max_pos_)\n      << \"Failed invariant of LimitingReaderBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  if (exact_) return max_pos_;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const std::optional<Position> size = src.Size();\n  MakeBuffer(src);\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) return std::nullopt;\n  return UnsignedMin(*size, max_pos_);\n}\n\nbool LimitingReaderBase::SupportsNewReader() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsNewReader();\n}\n\nstd::unique_ptr<Reader> LimitingReaderBase::NewReaderImpl(\n    Position initial_pos) {\n  RIEGELI_ASSERT_LE(pos(), max_pos_)\n      << \"Failed invariant of LimitingReaderBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point\n  // if `SrcReader()->SupportsNewReader()`.\n  Reader& src = *SrcReader();\n  std::unique_ptr<Reader> reader =\n      src.NewReader(UnsignedMin(initial_pos, max_pos_));\n  if (ABSL_PREDICT_FALSE(reader == nullptr)) {\n    FailWithoutAnnotation(src.status());\n    return nullptr;\n  }\n  return std::make_unique<LimitingReader<std::unique_ptr<Reader>>>(\n      std::move(reader),\n      LimitingReaderBase::Options().set_max_pos(max_pos_).set_exact(exact_));\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/limiting_reader.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_LIMITING_READER_H_\n#define RIEGELI_BYTES_LIMITING_READER_H_\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nclass BackwardWriter;\nclass ScopedLimiter;\nclass Writer;\n\n// Template parameter independent part of `LimitingReader`.\nclass LimitingReaderBase : public Reader {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // The limit expressed as an absolute position.\n    //\n    // `std::nullopt` means no limit, unless `max_length()` is set.\n    //\n    // Default: `std::nullopt`.\n    Options& set_max_pos(std::optional<Position> max_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      max_pos_ = max_pos;\n      max_length_ = std::nullopt;\n      return *this;\n    }\n    Options&& set_max_pos(std::optional<Position> max_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_pos(max_pos));\n    }\n    std::optional<Position> max_pos() const { return max_pos_; }\n\n    // A shortcut for `set_max_pos(pos)` with `set_exact(true)`.\n    Options& set_exact_pos(Position exact_pos) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_max_pos(exact_pos).set_exact(true);\n    }\n    Options&& set_exact_pos(Position exact_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_exact_pos(exact_pos));\n    }\n\n    // The limit expressed as a length relative to the current position.\n    //\n    // `std::nullopt` means no limit, unless `max_pos()` is set.\n    //\n    // Default: `std::nullopt`.\n    Options& set_max_length(std::optional<Position> max_length) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      max_length_ = max_length;\n      max_pos_ = std::nullopt;\n      return *this;\n    }\n    Options&& set_max_length(std::optional<Position> max_length) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_length(max_length));\n    }\n    std::optional<Position> max_length() const { return max_length_; }\n\n    // A shortcut for `set_max_length(length)` with `set_exact(true)`.\n    Options& set_exact_length(Position exact_length) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_max_length(exact_length).set_exact(true);\n    }\n    Options&& set_exact_length(Position exact_length) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_exact_length(exact_length));\n    }\n\n    // If `false`, `LimitingReader` will read data at most up to the limit.\n    // Reading will end cleanly when either the limit is reached or the source\n    // ends.\n    //\n    // If `true`, `LimitingReader` will read data exactly up to the limit.\n    // Reading will end cleanly when the limit is reached, but will fail if the\n    // source ends before the limit.\n    //\n    // Default: `false`.\n    Options& set_exact(bool exact) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      exact_ = exact;\n      return *this;\n    }\n    Options&& set_exact(bool exact) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_exact(exact));\n    }\n    bool exact() const { return exact_; }\n\n    // If `false`, `LimitingReader` will allow the original source to exceed the\n    // limit.\n    //\n    // If `true`, `LimitingReader` will require the original source to end\n    // before or at the limit (depending on `exact()`), but will fail if the\n    // original source exceeds the limit. This is checked when `LimitingReader`\n    // is closed while positioned at its end.\n    //\n    // Default: `false`.\n    Options& set_fail_if_longer(bool fail_if_longer) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      fail_if_longer_ = fail_if_longer;\n      return *this;\n    }\n    Options&& set_fail_if_longer(bool fail_if_longer) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_fail_if_longer(fail_if_longer));\n    }\n    bool fail_if_longer() const { return fail_if_longer_; }\n\n   private:\n    std::optional<Position> max_pos_;\n    std::optional<Position> max_length_;\n    bool exact_ = false;\n    bool fail_if_longer_ = false;\n  };\n\n  // Returns the original `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Accesses the limit expressed as an absolute position.\n  //\n  // If `set_max_length()` was used, `max_pos()` returns the same limit\n  // translated to an absolute position.\n  //\n  // If no limit is set, returns `std::numeric_limits<Position>::max()`.\n  //\n  // `ScopedLimiter` is often easier to use correctly than calling\n  // `set_max_pos()` directly.\n  void set_max_pos(Position max_pos);\n  Position max_pos() const { return max_pos_; }\n\n  // Accesses the limit expressed as a length relative to the current position,\n  // i.e. the length remaining to the limit.\n  //\n  // If `set_max_pos()` was used, `max_length()` returns the same limit\n  // translated to a length relative to the current position.\n  //\n  // If no limit is set, returns `std::numeric_limits<Position>::max() - pos()`.\n  //\n  // `ScopedLimiter` is often easier to use correctly than calling\n  // `set_max_length()` directly.\n  void set_max_length(Position max_length);\n  Position max_length() const;\n\n  // Clears the limit.\n  void clear_limit() { max_pos_ = std::numeric_limits<Position>::max(); }\n\n  // Accesses the exactness setting.\n  //\n  // If `false`, `LimitingReader` will read data at most up to the limit.\n  // Reading will end cleanly when either the limit is reached or the source\n  // ends.\n  //\n  // If `true`, `LimitingReader` will read data exactly up to the limit.\n  // Reading will end cleanly when the limit is reached, but will fail if the\n  // source ends before the limit.\n  //\n  // `ScopedLimiter` is often easier to use correctly than calling `set_exact()`\n  // directly.\n  void set_exact(bool exact) { exact_ = exact; }\n  bool exact() const { return exact_; }\n\n  // Accesses the failure if larger setting.\n  //\n  // If `false`, `LimitingReader` will allow the original source to exceed the\n  // limit.\n  //\n  // If `true`, `LimitingReader` will require the original source to end before\n  // or at the limit (depending on `exact()`), but will fail if the original\n  // source exceeds the limit. This is checked when `LimitingReader` is closed\n  // while positioned at its end.\n  void set_fail_if_longer(bool fail_if_longer) {\n    fail_if_longer_ = fail_if_longer;\n  }\n  bool fail_if_longer() const { return fail_if_longer_; }\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRandomAccess() override;\n  bool SupportsRewind() override;\n  bool SupportsSize() override;\n  bool SupportsNewReader() override;\n\n protected:\n  explicit LimitingReaderBase(Closed) noexcept : Reader(kClosed) {}\n\n  explicit LimitingReaderBase(bool exact, bool fail_if_longer);\n\n  LimitingReaderBase(LimitingReaderBase&& that) noexcept;\n  LimitingReaderBase& operator=(LimitingReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(bool exact, bool fail_if_longer);\n  void Initialize(Reader* src, const Options& options);\n\n  // Sets cursor of `src` to cursor of `*this`.\n  void SyncBuffer(Reader& src);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `src`, adjusting\n  // them for `max_pos_`. Fails `*this` if `src` failed.\n  void MakeBuffer(Reader& src);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PullSlow(size_t min_length, size_t recommended_length) override;\n  using Reader::ReadSlow;\n  bool ReadSlow(size_t length, char* dest) override;\n  bool ReadSlow(size_t length, Chain& dest) override;\n  bool ReadSlow(size_t length, absl::Cord& dest) override;\n  using Reader::CopySlow;\n  bool CopySlow(Position length, Writer& dest) override;\n  bool CopySlow(size_t length, BackwardWriter& dest) override;\n  using Reader::ReadSomeSlow;\n  bool ReadSomeSlow(size_t max_length, char* dest) override;\n  using Reader::CopySomeSlow;\n  bool CopySomeSlow(size_t max_length, Writer& dest) override;\n  void ReadHintSlow(size_t min_length, size_t recommended_length) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  // For `FailNotEnoughAtPos()`, `FailNotEnoughAtLength()`,\n  // `FailNotEnoughAtEnd()`, and `FailPositionLimitExceeded()`.\n  friend class ScopedLimiter;\n\n  bool CheckEnough();\n  ABSL_ATTRIBUTE_COLD bool FailNotEnough();\n  ABSL_ATTRIBUTE_COLD void FailNotEnoughAtPos(Position expected_pos);\n  ABSL_ATTRIBUTE_COLD void FailNotEnoughAtLength(Position expected_length);\n  ABSL_ATTRIBUTE_COLD void FailNotEnoughAtEnd();\n  ABSL_ATTRIBUTE_COLD void FailLengthOverflow(Position max_length);\n  ABSL_ATTRIBUTE_COLD void FailPositionLimitExceeded();\n\n  void restore_max_pos(Position max_pos);\n\n  void MakeBufferSlow();\n\n  // This template is defined and used only in limiting_reader.cc.\n  template <typename Dest>\n  bool ReadInternal(size_t length, Dest& dest);\n\n  // Invariant: `pos() <= max_pos_`\n  Position max_pos_ = std::numeric_limits<Position>::max();\n\n  bool exact_ = false;\n  bool fail_if_longer_ = false;\n\n  // Invariants if `is_open()`:\n  //   `start() >= SrcReader()->start()`\n  //   `limit() <= SrcReader()->limit()`\n  //   `start_pos() >= SrcReader()->start_pos()`\n  //   `limit_pos() <= max_pos_`\n};\n\n// A `Reader` which reads from another `Reader` up to the specified limit, then\n// pretends that the source ends, or fails if configured to fail and the source\n// is longer.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the original `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The original `Reader` must not be accessed until the `LimitingReader` is\n// closed or no longer used.\n//\n// For reading multiple delimited fragments, two techniques can be used:\n//\n//  * Create a `LimitingReader` without a limit. For each delimited fragment\n//    create a `ScopedLimiter`.\n//\n//  * Create a `LimitingReader` without a limit. For each delimited fragment\n//    use `set_max_length()` or `set_max_pos()`, and also possibly\n//    `clear_limit()` to read data between fragments.\ntemplate <typename Src = Reader*>\nclass LimitingReader : public LimitingReaderBase {\n public:\n  // Creates a closed `LimitingReader`.\n  explicit LimitingReader(Closed) noexcept : LimitingReaderBase(kClosed) {}\n\n  // Will read from the original `Reader` provided by `src`.\n  explicit LimitingReader(Initializer<Src> src, Options options = Options());\n\n  LimitingReader(LimitingReader&& that) = default;\n  LimitingReader& operator=(LimitingReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `LimitingReader`. This\n  // avoids constructing a temporary `LimitingReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the original `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  bool SyncImpl(SyncType sync_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the original `Reader`.\n  MovingDependency<Reader*, Src, Mover> src_;\n};\n\nexplicit LimitingReader(Closed) -> LimitingReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit LimitingReader(Src&& src, LimitingReaderBase::Options options =\n                                       LimitingReaderBase::Options())\n    -> LimitingReader<TargetT<Src>>;\n\n// `ReaderSpan` specifies a span of `Reader` contents from the current position\n// with the given length. The type of the `Reader` is specified as a template\n// parameter so that `LimitingReaderBase` can be treated specially.\n//\n// This can express the span as a single object, which is sometimes convenient.\n//\n// `ReaderSpan` supports `Dependency<Reader*, ReaderSpan<ReaderType>>`,\n// which internally applies a `ScopedLimiterOrLimitingReader`. Some functions\n// treat a parameter of type `ReaderSpan` specially to enable a more efficient\n// implementation.\ntemplate <typename ReaderType = LimitingReaderBase>\nclass ReaderSpan {\n public:\n  // Specifies the span from the current position of `*reader` with `length`.\n  explicit ReaderSpan(ReaderType* reader, Position length)\n      : reader_(reader), length_(length) {}\n\n  template <typename OtherReaderType,\n            std::enable_if_t<\n                std::conjunction_v<\n                    std::negation<std::is_same<OtherReaderType, ReaderType>>,\n                    std::is_convertible<OtherReaderType*, ReaderType*>>,\n                int> = 0>\n  /*implicit*/ ReaderSpan(ReaderSpan<OtherReaderType> src)\n      : ReaderSpan(&src.reader(), src.length()) {}\n\n  ReaderSpan(ReaderSpan&& that) = default;\n  ReaderSpan& operator=(ReaderSpan&& that) = default;\n\n  ReaderType& reader() const { return *reader_; }\n  Position length() const { return length_; }\n\n private:\n  ReaderType* reader_;\n  Position length_;\n};\n\ntemplate <typename ReaderType>\nexplicit ReaderSpan(ReaderType* reader, Position length)\n    -> ReaderSpan<ReaderType>;\n\n// Changes the options of a `LimitingReader` in the constructor, and restores\n// them in the destructor.\nclass ScopedLimiter {\n public:\n  using Options = LimitingReaderBase::Options;\n\n  // Changes the effective options of `*reader` to be more strict than either\n  // the provided options or the previous options. The limit can become only\n  // smaller, and `exact()` can change only from `false` to `true`.\n  //\n  // This is similar to making a new `LimitingReader` reading from the previous\n  // `LimitingReader` but more efficient. A difference is when `options.exact()`\n  // is `true` and the new limit exceeds the old limit. In this case the\n  // `LimitingReader` fails immediately rather than when the old limit is\n  // reached, because it is already known that it cannot eventually succeed.\n  explicit ScopedLimiter(LimitingReaderBase* reader\n                             ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                         Options options);\n\n  explicit ScopedLimiter(const ReaderSpan<>& src)\n      : ScopedLimiter(\n            &src.reader(),\n            LimitingReaderBase::Options().set_exact_length(src.length())) {}\n\n  ScopedLimiter(const ScopedLimiter&) = delete;\n  ScopedLimiter& operator=(const ScopedLimiter&) = delete;\n\n  // Returns the underlying `LimitingReaderBase`.\n  LimitingReaderBase& reader() const { return *reader_; }\n\n  // Restores the options.\n  //\n  // Precondition:\n  //   `reader->max_pos()` is not smaller than it was\n  //       when the `ScopedLimiter` was constructed.\n  ~ScopedLimiter();\n\n private:\n  LimitingReaderBase* reader_;\n  Position old_max_pos_;\n  bool old_exact_;\n  bool fail_if_longer_;\n};\n\n// If `src` is a `LimitingReaderBase`, creates a `ScopedLimiter`, otherwise\n// creates a new `LimitingReader`.\n//\n// The primary template applies if `Src` is a `Reader` other than\n// `LimitingReaderBase`.\ntemplate <typename Src, typename Enable = void>\nclass ScopedLimiterOrLimitingReader {\n public:\n  using Options = LimitingReaderBase::Options;\n\n  explicit ScopedLimiterOrLimitingReader(Src* src ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                                         Options options)\n      : reader_(src, options) {}\n\n  explicit ScopedLimiterOrLimitingReader(const ReaderSpan<Src>& src)\n      : reader_(&src.reader(),\n                LimitingReaderBase::Options().set_exact_length(src.length())) {}\n\n  ScopedLimiterOrLimitingReader(const ScopedLimiterOrLimitingReader&) = delete;\n  ScopedLimiterOrLimitingReader& operator=(\n      const ScopedLimiterOrLimitingReader&) = delete;\n\n  // Returns the `LimitingReaderBase` from which data should be read.\n  LimitingReaderBase& reader() { return reader_; }\n\n  // Closes the `LimitingReaderBase`. If this fails, `reader().status()` can be\n  // used.\n  //\n  // For consistency with the specialization if `Src` is a `LimitingReaderBase`,\n  // the original `Reader` must not be changed between `Close()` and destroying\n  // the `ScopedLimiterOrLimitingReader`.\n  bool Close() { return reader_.Close(); }\n\n private:\n  LimitingReader<> reader_;\n};\n\n// Specialization of `ScopedLimiterOrLimitingReader` if `Src` is a\n// `LimitingReaderBase`.\n//\n// In this specialization the original `LimitingReaderBase` is accessed directly\n// while its limits have been changed.\ntemplate <typename Src>\nclass ScopedLimiterOrLimitingReader<\n    Src, std::enable_if_t<std::is_base_of_v<LimitingReaderBase, Src>>> {\n public:\n  using Options = LimitingReaderBase::Options;\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  explicit ScopedLimiterOrLimitingReader(Src* src ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                                         Options options)\n      : limiter_(src, options) {}\n\n  explicit ScopedLimiterOrLimitingReader(const ReaderSpan<Src>& src)\n      : limiter_(&src.reader(),\n                 LimitingReaderBase::Options().set_exact_length(src.length())) {\n  }\n\n  ScopedLimiterOrLimitingReader(const ScopedLimiterOrLimitingReader&) = delete;\n  ScopedLimiterOrLimitingReader& operator=(\n      const ScopedLimiterOrLimitingReader&) = delete;\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  ~ScopedLimiterOrLimitingReader() = default;\n\n  // Returns the `LimitingReaderBase` from which data should be read.\n  LimitingReaderBase& reader() { return limiter_.reader(); }\n\n  // Does nothing. The state of the original `LimitingReaderBase` is restored by\n  // the destructor.instead.\n  //\n  // For consistency with the primary template, `Close()` should still be\n  // called. The original `Reader` must not be changed between `Close()` and\n  // destroying the `ScopedLimiterOrLimitingReader`.\n  bool Close() { return reader().ok(); }\n\n private:\n  ScopedLimiter limiter_;\n};\n\ntemplate <typename Src>\nexplicit ScopedLimiterOrLimitingReader(Src* src,\n                                       LimitingReaderBase::Options options)\n    -> ScopedLimiterOrLimitingReader<Src>;\n\ntemplate <typename Src>\nexplicit ScopedLimiterOrLimitingReader(const ReaderSpan<Src>& src)\n    -> ScopedLimiterOrLimitingReader<Src>;\n\n// Specialization of `DependencyImpl<Reader*, ReaderSpan<ReaderType>>`.\ntemplate <typename ReaderType>\nclass DependencyImpl<Reader*, ReaderSpan<ReaderType>> {\n public:\n  explicit DependencyImpl(ReaderSpan<ReaderType> span)\n      : span_(std::move(span)),\n        scoped_limiter_(\n            &span_.reader(),\n            LimitingReaderBase::Options().set_exact_length(span_.length())) {}\n\n  ReaderSpan<ReaderType>& manager() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return span_;\n  }\n  const ReaderSpan<ReaderType>& manager() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return span_;\n  }\n\n  LimitingReaderBase* get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return &scoped_limiter_.reader();\n  }\n\n  bool IsOwning() const { return false; }\n\n  static constexpr bool kIsStable = false;\n\n protected:\n  DependencyImpl(const DependencyImpl&) = delete;\n  DependencyImpl& operator=(const DependencyImpl&) = delete;\n\n  ~DependencyImpl() = default;\n\n private:\n  ReaderSpan<ReaderType> span_;\n  mutable ScopedLimiterOrLimitingReader<ReaderType> scoped_limiter_;\n};\n\n// Implementation details follow.\n\ninline LimitingReaderBase::LimitingReaderBase(bool exact, bool fail_if_longer)\n    : exact_(exact), fail_if_longer_(fail_if_longer) {}\n\ninline LimitingReaderBase::LimitingReaderBase(\n    LimitingReaderBase&& that) noexcept\n    : Reader(static_cast<Reader&&>(that)),\n      max_pos_(that.max_pos_),\n      exact_(that.exact_),\n      fail_if_longer_(that.fail_if_longer_) {}\n\ninline LimitingReaderBase& LimitingReaderBase::operator=(\n    LimitingReaderBase&& that) noexcept {\n  Reader::operator=(static_cast<Reader&&>(that));\n  max_pos_ = that.max_pos_;\n  exact_ = that.exact_;\n  fail_if_longer_ = that.fail_if_longer_;\n  return *this;\n}\n\ninline void LimitingReaderBase::Reset(Closed) {\n  Reader::Reset(kClosed);\n  max_pos_ = std::numeric_limits<Position>::max();\n  exact_ = false;\n  fail_if_longer_ = false;\n}\n\ninline void LimitingReaderBase::Reset(bool exact, bool fail_if_longer) {\n  Reader::Reset();\n  max_pos_ = std::numeric_limits<Position>::max();\n  exact_ = exact;\n  fail_if_longer_ = fail_if_longer;\n}\n\ninline void LimitingReaderBase::restore_max_pos(Position max_pos) {\n  RIEGELI_ASSERT_GE(max_pos, max_pos_)\n      << \"Failed precondition of LimitingReaderBase::restore_max_pos(): \"\n         \"the limit is being reduced\";\n  max_pos_ = max_pos;\n}\n\ninline Position LimitingReaderBase::max_length() const {\n  RIEGELI_ASSERT_GE(max_pos_, pos())\n      << \"Failed invariant of LimitingReaderBase: \"\n         \"position already exceeds its limit\";\n  return max_pos_ - pos();\n}\n\ninline void LimitingReaderBase::set_max_pos(Position max_pos) {\n  max_pos_ = max_pos;\n  if (ABSL_PREDICT_FALSE(limit_pos() > max_pos_)) MakeBufferSlow();\n}\n\ninline void LimitingReaderBase::set_max_length(Position max_length) {\n  max_pos_ = pos() + max_length;  // Wrap-around is not an error.\n  if (ABSL_PREDICT_FALSE(max_pos_ < max_length)) {\n    max_pos_ = std::numeric_limits<Position>::max();\n    if (exact_) FailLengthOverflow(max_length);\n    return;\n  }\n  if (limit_pos() > max_pos_) {\n    set_buffer(start(),\n               start_to_limit() - IntCast<size_t>(limit_pos() - max_pos_),\n               start_to_cursor());\n    set_limit_pos(max_pos_);\n  }\n}\n\ninline void LimitingReaderBase::SyncBuffer(Reader& src) {\n  src.set_cursor(cursor());\n}\n\ninline void LimitingReaderBase::MakeBuffer(Reader& src) {\n  set_buffer(src.start(), src.start_to_limit(), src.start_to_cursor());\n  set_limit_pos(src.limit_pos());\n  if (ABSL_PREDICT_FALSE(limit_pos() > max_pos_)) MakeBufferSlow();\n  if (ABSL_PREDICT_FALSE(!src.ok())) FailWithoutAnnotation(src.status());\n}\n\ntemplate <typename Src>\nclass LimitingReader<Src>::Mover {\n public:\n  static auto member() { return &LimitingReader::src_; }\n\n  explicit Mover(LimitingReader& self, LimitingReader& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `src_` is not moved yet so `src_` is taken from `that`.\n    if (uses_buffer_) self.SyncBuffer(*that.src_);\n  }\n\n  void Done(LimitingReader& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.src_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Src>\ninline LimitingReader<Src>::LimitingReader(Initializer<Src> src,\n                                           Options options)\n    : LimitingReaderBase(options.exact(), options.fail_if_longer()),\n      src_(std::move(src)) {\n  Initialize(src_.get(), options);\n}\n\ntemplate <typename Src>\ninline void LimitingReader<Src>::Reset(Closed) {\n  LimitingReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void LimitingReader<Src>::Reset(Initializer<Src> src, Options options) {\n  LimitingReaderBase::Reset(options.exact(), options.fail_if_longer());\n  src_.Reset(std::move(src));\n  Initialize(src_.get(), options);\n}\n\ntemplate <typename Src>\nvoid LimitingReader<Src>::Done() {\n  LimitingReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(src_->status());\n    }\n  }\n}\n\ntemplate <typename Src>\nbool LimitingReader<Src>::SyncImpl(SyncType sync_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer(*src_);\n  bool sync_ok = true;\n  if (sync_type != SyncType::kFromObject || src_.IsOwning()) {\n    sync_ok = src_->Sync(sync_type);\n  }\n  MakeBuffer(*src_);\n  return sync_ok;\n}\n\nABSL_ATTRIBUTE_ALWAYS_INLINE\ninline ScopedLimiter::ScopedLimiter(\n    LimitingReaderBase* reader ABSL_ATTRIBUTE_LIFETIME_BOUND, Options options)\n    : reader_(RIEGELI_EVAL_ASSERT_NOTNULL(reader)),\n      old_max_pos_(reader_->max_pos()),\n      old_exact_(reader_->exact()),\n      fail_if_longer_(options.fail_if_longer()) {\n  if (options.max_pos() != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(*options.max_pos() > reader_->max_pos())) {\n      if (options.exact()) reader_->FailNotEnoughAtPos(*options.max_pos());\n    } else {\n      reader_->set_max_pos(*options.max_pos());\n    }\n  } else if (options.max_length() != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(*options.max_length() > reader_->max_length())) {\n      if (options.exact())\n        reader_->FailNotEnoughAtLength(*options.max_length());\n    } else {\n      reader_->set_max_length(*options.max_length());\n    }\n  } else if (ABSL_PREDICT_FALSE(reader_->max_pos() <\n                                    std::numeric_limits<Position>::max() &&\n                                options.exact())) {\n    reader_->FailNotEnoughAtEnd();\n  }\n  reader_->set_exact(true);\n}\n\nABSL_ATTRIBUTE_ALWAYS_INLINE\ninline ScopedLimiter::~ScopedLimiter() {\n  RIEGELI_ASSERT_GE(old_max_pos_, reader_->max_pos())\n      << \"Failed precondtion of ~ScopedLimiter: \"\n         \"The underlying LimitingReader increased its limit \"\n         \"while the ScopedLimiter was active\";\n  const Position inner_max_pos = reader_->max_pos();\n  reader_->restore_max_pos(old_max_pos_);\n  reader_->set_exact(old_exact_);\n  if (fail_if_longer_ && reader_->pos() == inner_max_pos &&\n      ABSL_PREDICT_FALSE(reader_->Pull())) {\n    reader_->FailPositionLimitExceeded();\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_LIMITING_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/limiting_writer.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/limiting_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid LimitingWriterBase::Initialize(Writer* dest, const Options& options,\n                                    bool is_owning) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of LimitingWriter: null Writer pointer\";\n  if (is_owning && exact()) {\n    if (options.max_pos() != std::nullopt) {\n      dest->SetWriteSizeHint(SaturatingSub(*options.max_pos(), dest->pos()));\n    } else if (options.max_length() != std::nullopt) {\n      dest->SetWriteSizeHint(*options.max_length());\n    }\n  }\n  set_buffer(dest->start(), dest->start_to_limit(), dest->start_to_cursor());\n  set_start_pos(dest->start_pos());\n  if (ABSL_PREDICT_FALSE(!dest->ok())) FailWithoutAnnotation(dest->status());\n  if (options.max_pos() != std::nullopt) {\n    set_max_pos(*options.max_pos());\n  } else if (options.max_length() != std::nullopt) {\n    set_max_length(*options.max_length());\n  }\n}\n\nvoid LimitingWriterBase::set_max_pos(Position max_pos) {\n  max_pos_ = max_pos;\n  if (ABSL_PREDICT_FALSE(start_pos() > max_pos_)) {\n    set_buffer(cursor());\n    set_start_pos(max_pos_);\n    FailLimitExceeded();\n  }\n}\n\nvoid LimitingWriterBase::set_max_length(Position max_length) {\n  if (ABSL_PREDICT_FALSE(max_length >\n                         std::numeric_limits<Position>::max() - pos())) {\n    if (exact_) FailLengthOverflow(max_length);\n    max_pos_ = std::numeric_limits<Position>::max();\n    return;\n  }\n  set_max_pos(pos() + max_length);\n}\n\nvoid LimitingWriterBase::Done() {\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_TRUE(ok())) SyncBuffer(dest);\n  if (exact_ && ABSL_PREDICT_FALSE(pos() < max_pos_)) {\n    // Do not call `Fail()` because `AnnotateStatusImpl()` synchronizes the\n    // buffer again.\n    FailWithoutAnnotation(dest.AnnotateStatus(absl::InvalidArgumentError(\n        absl::StrCat(\"Not enough data: expected \", max_pos(), \" or \",\n                     max_length(), \" more\"))));\n  }\n  Writer::Done();\n}\n\nbool LimitingWriterBase::FailLimitExceeded() {\n  Writer& dest = *DestWriter();\n  return FailLimitExceeded(dest);\n}\n\nbool LimitingWriterBase::FailLimitExceeded(Writer& dest) {\n  set_start_pos(max_pos_);\n  set_buffer();\n  // Do not call `Fail()` because `AnnotateStatusImpl()` synchronizes the buffer\n  // again.\n  return FailWithoutAnnotation(dest.AnnotateStatus(\n      absl::ResourceExhaustedError(\"Position limit exceeded\")));\n}\n\ninline void LimitingWriterBase::FailLengthOverflow(Position max_length) {\n  Fail(absl::InvalidArgumentError(\n      absl::StrCat(\"Not enough data: expected \", max_length,\n                   \"more, which overflows the Writer position range\")));\n}\n\nabsl::Status LimitingWriterBase::AnnotateStatusImpl(absl::Status status) {\n  // Fully delegate annotations to `*DestWriter()`.\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    const bool sync_buffer_ok = SyncBuffer(dest);\n    status = dest.AnnotateStatus(std::move(status));\n    if (ABSL_PREDICT_TRUE(sync_buffer_ok)) MakeBuffer(dest);\n  }\n  return status;\n}\n\nbool LimitingWriterBase::PushSlow(size_t min_length,\n                                  size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  RIEGELI_ASSERT_LE(start_pos(), max_pos_)\n      << \"Failed invariant of LimitingWriterBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return false;\n  const bool push_ok = dest.Push(min_length, recommended_length);\n  MakeBuffer(dest);\n  return push_ok;\n}\n\nbool LimitingWriterBase::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of Writer::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  return WriteInternal(src, [](absl::string_view src, size_t length) {\n    src.remove_suffix(length);\n    return src;\n  });\n}\n\nbool LimitingWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  return WriteInternal(std::move(src), [](ExternalRef src, size_t length) {\n    Chain result(std::move(src));\n    result.RemoveSuffix(length);\n    return result;\n  });\n}\n\nbool LimitingWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  return WriteInternal(src, [](const Chain& src, size_t length) {\n    Chain result = src;\n    result.RemoveSuffix(length);\n    return result;\n  });\n}\n\nbool LimitingWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  return WriteInternal(std::move(src),\n                       [](Chain&& src, size_t length) -> Chain&& {\n                         src.RemoveSuffix(length);\n                         return std::move(src);\n                       });\n}\n\nbool LimitingWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  return WriteInternal(src, [](const absl::Cord& src, size_t length) {\n    absl::Cord result = src;\n    result.RemoveSuffix(length);\n    return result;\n  });\n}\n\nbool LimitingWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  return WriteInternal(std::move(src),\n                       [](absl::Cord&& src, size_t length) -> absl::Cord&& {\n                         src.RemoveSuffix(length);\n                         return std::move(src);\n                       });\n}\n\ntemplate <typename Src, typename RemoveSuffix>\ninline bool LimitingWriterBase::WriteInternal(Src&& src,\n                                              RemoveSuffix&& remove_suffix) {\n  RIEGELI_ASSERT_LE(start_pos(), max_pos_)\n      << \"Failed invariant of LimitingWriterBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return false;\n  const Position max_length = max_pos_ - pos();\n  if (ABSL_PREDICT_TRUE(src.size() <= max_length)) {\n    const bool write_ok = dest.Write(std::forward<Src>(src));\n    MakeBuffer(dest);\n    return write_ok;\n  }\n  if (ABSL_PREDICT_FALSE(!dest.Write(std::forward<RemoveSuffix>(remove_suffix)(\n          std::forward<Src>(src), src.size() - max_length)))) {\n    MakeBuffer(dest);\n    return false;\n  }\n  return FailLimitExceeded(dest);\n}\n\nbool LimitingWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  RIEGELI_ASSERT_LE(start_pos(), max_pos_)\n      << \"Failed invariant of LimitingWriterBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return false;\n  const Position max_length = max_pos_ - pos();\n  if (ABSL_PREDICT_TRUE(src.size() <= max_length)) {\n    const bool write_ok = dest.Write(src);\n    MakeBuffer(dest);\n    return write_ok;\n  }\n  if (ABSL_PREDICT_FALSE(!dest.Write(src.Extract(max_length)))) {\n    MakeBuffer(dest);\n    return false;\n  }\n  return FailLimitExceeded(dest);\n}\n\nbool LimitingWriterBase::SupportsRandomAccess() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsRandomAccess();\n}\n\nbool LimitingWriterBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of Writer::SeekSlow(): \"\n         \"position unchanged, use Seek() instead\";\n  RIEGELI_ASSERT_LE(start_pos(), max_pos_)\n      << \"Failed invariant of LimitingWriterBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return false;\n  const Position pos_to_seek = UnsignedMin(new_pos, max_pos_);\n  const bool seek_ok = dest.Seek(pos_to_seek);\n  MakeBuffer(dest);\n  return seek_ok && pos_to_seek == new_pos;\n}\n\nstd::optional<Position> LimitingWriterBase::SizeImpl() {\n  RIEGELI_ASSERT_LE(start_pos(), max_pos_)\n      << \"Failed invariant of LimitingWriterBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return std::nullopt;\n  const std::optional<Position> size = dest.Size();\n  MakeBuffer(dest);\n  return size;\n}\n\nbool LimitingWriterBase::SupportsTruncate() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsTruncate();\n}\n\nbool LimitingWriterBase::TruncateImpl(Position new_size) {\n  RIEGELI_ASSERT_LE(start_pos(), max_pos_)\n      << \"Failed invariant of LimitingWriterBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(pos() > max_pos_) && new_size <= max_pos_) {\n    set_cursor(cursor() - IntCast<size_t>(pos() - max_pos_));\n  }\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return false;\n  const bool truncate_ok = dest.Truncate(new_size);\n  MakeBuffer(dest);\n  return truncate_ok;\n}\n\nbool LimitingWriterBase::SupportsReadMode() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsReadMode();\n}\n\nReader* LimitingWriterBase::ReadModeImpl(Position initial_pos) {\n  RIEGELI_ASSERT_LE(start_pos(), max_pos_)\n      << \"Failed invariant of LimitingWriterBase: \"\n         \"position already exceeds its limit\";\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return nullptr;\n  Reader* const reader = dest.ReadMode(initial_pos);\n  MakeBuffer(dest);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/limiting_writer.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_LIMITING_WRITER_H_\n#define RIEGELI_BYTES_LIMITING_WRITER_H_\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nclass Reader;\n\n// Template parameter independent part of `LimitingWriter`.\nclass LimitingWriterBase : public Writer {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // The limit expressed as an absolute position.\n    //\n    // `std::nullopt` means no limit, unless `max_length()` is set.\n    //\n    // Default: `std::nullopt`.\n    Options& set_max_pos(std::optional<Position> max_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      max_pos_ = max_pos;\n      max_length_ = std::nullopt;\n      return *this;\n    }\n    Options&& set_max_pos(std::optional<Position> max_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_pos(max_pos));\n    }\n    std::optional<Position> max_pos() const { return max_pos_; }\n\n    // A shortcut for `set_max_pos(pos)` with `set_exact(true)`.\n    Options& set_exact_pos(Position exact_pos) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_max_pos(exact_pos).set_exact(true);\n    }\n    Options&& set_exact_pos(Position exact_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_exact_pos(exact_pos));\n    }\n\n    // The limit expressed as a length relative to the current position.\n    //\n    // `std::nullopt` means no limit, unless `max_pos()` is set.\n    //\n    // Default: `std::nullopt`.\n    Options& set_max_length(std::optional<Position> max_length) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      max_length_ = max_length;\n      max_pos_ = std::nullopt;\n      return *this;\n    }\n    Options&& set_max_length(std::optional<Position> max_length) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_length(max_length));\n    }\n    std::optional<Position> max_length() const { return max_length_; }\n\n    // A shortcut for `set_max_length(length)` with `set_exact(true)`.\n    Options& set_exact_length(Position exact_length) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_max_length(exact_length).set_exact(true);\n    }\n    Options&& set_exact_length(Position exact_length) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_exact_length(exact_length));\n    }\n\n    // If `false`, `LimitingWriter` will write data at most up to the limit.\n    // Writing will fail if the limit is exceeded.\n    //\n    // If `true`, `LimitingWriter` will write data exactly up to the limit.\n    // Writing will fail if the limit is exceeded, and `Close()` will fail if\n    // the current position at that time is before the limit.\n    //\n    // Default: `false`.\n    Options& set_exact(bool exact) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      exact_ = exact;\n      return *this;\n    }\n    Options&& set_exact(bool exact) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_exact(exact));\n    }\n    bool exact() const { return exact_; }\n\n   private:\n    std::optional<Position> max_pos_;\n    std::optional<Position> max_length_;\n    bool exact_ = false;\n  };\n\n  // Returns the original `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Accesses the limit expressed as an absolute position.\n  //\n  // If `set_max_length()` was used, `max_pos()` returns the same limit\n  // translated to an absolute position.\n  //\n  // If no limit is set, returns `std::numeric_limits<Position>::max()`.\n  void set_max_pos(Position max_pos);\n  Position max_pos() const { return max_pos_; }\n\n  // Accesses the limit expressed as a length relative to the current position,\n  // i.e. the length remaining to the limit.\n  //\n  // If `set_max_pos()` was used, `max_length()` returns the same limit\n  // translated to a length relative to the current position.\n  //\n  // If no limit is set, returns `std::numeric_limits<Position>::max() - pos()`.\n  void set_max_length(Position max_length);\n  Position max_length() const { return SaturatingSub(max_pos_, pos()); }\n\n  // Clears the limit.\n  void clear_limit() { max_pos_ = std::numeric_limits<Position>::max(); }\n\n  bool SupportsRandomAccess() override;\n  bool SupportsTruncate() override;\n  bool SupportsReadMode() override;\n\n protected:\n  explicit LimitingWriterBase(Closed) noexcept : Writer(kClosed) {}\n\n  explicit LimitingWriterBase(bool exact);\n\n  LimitingWriterBase(LimitingWriterBase&& that) noexcept;\n  LimitingWriterBase& operator=(LimitingWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(bool exact);\n  void Initialize(Writer* dest, const Options& options, bool is_owning);\n  bool exact() const { return exact_; }\n\n  // Sets cursor of `dest` to cursor of `*this`. Fails `*this` if the limit is\n  // exceeded.\n  bool SyncBuffer(Writer& dest);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `dest`. Fails `*this`\n  // if `dest` failed.\n  void MakeBuffer(Writer& dest);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  bool TruncateImpl(Position new_size) override;\n  Reader* ReadModeImpl(Position initial_pos) override;\n\n private:\n  ABSL_ATTRIBUTE_COLD bool FailLimitExceeded();\n  ABSL_ATTRIBUTE_COLD bool FailLimitExceeded(Writer& dest);\n  ABSL_ATTRIBUTE_COLD void FailLengthOverflow(Position max_length);\n\n  // This template is defined and used only in limiting_writer.cc.\n  template <typename Src, typename RemoveSuffix>\n  bool WriteInternal(Src&& src, RemoveSuffix&& remove_suffix);\n\n  // Invariant: `start_pos() <= max_pos_`\n  Position max_pos_ = std::numeric_limits<Position>::max();\n\n  bool exact_ = false;\n\n  // Invariants if `ok()`:\n  //   `start() == DestWriter()->start()`\n  //   `limit() == DestWriter()->limit()`\n  //   `start_pos() == DestWriter()->start_pos()`\n};\n\n// A `Writer` which writes to another `Writer` up to the specified size limit.\n// An attempt to write more fails, after writing to the destination everything\n// up to the limit.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the original `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The original `Writer` must not be accessed until the `LimitingWriter` is\n// closed or no longer used, except that it is allowed to read the destination\n// of the original `Writer` immediately after `Flush()`.\ntemplate <typename Dest = Writer*>\nclass LimitingWriter : public LimitingWriterBase {\n public:\n  // Creates a closed `LimitingWriter`.\n  explicit LimitingWriter(Closed) noexcept : LimitingWriterBase(kClosed) {}\n\n  // Will write to the original `Writer` provided by `dest`.\n  explicit LimitingWriter(Initializer<Dest> dest, Options options = Options());\n\n  LimitingWriter(LimitingWriter&& that) = default;\n  LimitingWriter& operator=(LimitingWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `LimitingWriter`. This\n  // avoids constructing a temporary `LimitingWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the original `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the original `Writer`.\n  MovingDependency<Writer*, Dest, Mover> dest_;\n};\n\nexplicit LimitingWriter(Closed) -> LimitingWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit LimitingWriter(Dest&& dest, LimitingWriterBase::Options options =\n                                         LimitingWriterBase::Options())\n    -> LimitingWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline LimitingWriterBase::LimitingWriterBase(bool exact) : exact_(exact) {}\n\ninline LimitingWriterBase::LimitingWriterBase(\n    LimitingWriterBase&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)),\n      max_pos_(that.max_pos_),\n      exact_(that.exact_) {}\n\ninline LimitingWriterBase& LimitingWriterBase::operator=(\n    LimitingWriterBase&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  max_pos_ = that.max_pos_;\n  exact_ = that.exact_;\n  return *this;\n}\n\ninline void LimitingWriterBase::Reset(Closed) {\n  Writer::Reset(kClosed);\n  max_pos_ = std::numeric_limits<Position>::max();\n  exact_ = false;\n}\n\ninline void LimitingWriterBase::Reset(bool exact) {\n  Writer::Reset();\n  max_pos_ = std::numeric_limits<Position>::max();\n  exact_ = exact;\n}\n\ninline bool LimitingWriterBase::SyncBuffer(Writer& dest) {\n  if (ABSL_PREDICT_FALSE(pos() > max_pos_)) {\n    dest.set_cursor(cursor() - IntCast<size_t>(pos() - max_pos_));\n    return FailLimitExceeded(dest);\n  }\n  dest.set_cursor(cursor());\n  return true;\n}\n\ninline void LimitingWriterBase::MakeBuffer(Writer& dest) {\n  set_buffer(dest.start(), dest.start_to_limit(), dest.start_to_cursor());\n  set_start_pos(dest.start_pos());\n  if (ABSL_PREDICT_FALSE(start_pos() > max_pos_)) {\n    set_buffer(cursor());\n    set_start_pos(max_pos_);\n    FailLimitExceeded();\n  }\n  if (ABSL_PREDICT_FALSE(!dest.ok())) FailWithoutAnnotation(dest.status());\n}\n\ntemplate <typename Dest>\nclass LimitingWriter<Dest>::Mover {\n public:\n  static auto member() { return &LimitingWriter::dest_; }\n\n  explicit Mover(LimitingWriter& self, LimitingWriter& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `dest_` is not moved yet so `dest_` is taken from `that`.\n    if (uses_buffer_) {\n      if (ABSL_PREDICT_FALSE(!self.SyncBuffer(*that.dest_))) {\n        uses_buffer_ = false;\n      }\n    }\n  }\n\n  void Done(LimitingWriter& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.dest_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Dest>\ninline LimitingWriter<Dest>::LimitingWriter(Initializer<Dest> dest,\n                                            Options options)\n    : LimitingWriterBase(options.exact()), dest_(std::move(dest)) {\n  Initialize(dest_.get(), options, dest_.IsOwning());\n}\n\ntemplate <typename Dest>\ninline void LimitingWriter<Dest>::Reset(Closed) {\n  LimitingWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void LimitingWriter<Dest>::Reset(Initializer<Dest> dest,\n                                        Options options) {\n  LimitingWriterBase::Reset(options.exact());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options, dest_.IsOwning());\n}\n\ntemplate <typename Dest>\nvoid LimitingWriter<Dest>::Done() {\n  LimitingWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(dest_->status());\n    }\n  }\n}\n\ntemplate <typename Dest>\nvoid LimitingWriter<Dest>::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (dest_.IsOwning() && !exact()) {\n    if (ABSL_PREDICT_FALSE(!SyncBuffer(*dest_))) return;\n    dest_->SetWriteSizeHint(\n        write_size_hint == std::nullopt\n            ? std::nullopt\n            : std::make_optional(UnsignedMin(*write_size_hint, max_length())));\n    MakeBuffer(*dest_);\n  }\n}\n\ntemplate <typename Dest>\nbool LimitingWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(*dest_))) return false;\n  bool flush_ok = true;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    flush_ok = dest_->Flush(flush_type);\n  }\n  MakeBuffer(*dest_);\n  return flush_ok;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_LIMITING_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/null_backward_writer.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/null_backward_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n\nnamespace riegeli {\n\nvoid NullBackwardWriter::Done() {\n  BackwardWriter::Done();\n  buffer_ = Buffer();\n}\n\ninline void NullBackwardWriter::SyncBuffer() {\n  set_start_pos(pos());\n  set_cursor(start());\n}\n\ninline bool NullBackwardWriter::MakeBuffer(size_t min_length,\n                                           size_t recommended_length) {\n  if (ABSL_PREDICT_FALSE(min_length >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  const size_t buffer_length = UnsignedMin(\n      buffer_sizer_.BufferLength(start_pos(), min_length, recommended_length),\n      std::numeric_limits<Position>::max() - start_pos());\n  buffer_.Reset(buffer_length);\n  set_buffer(buffer_.data(), buffer_length);\n  return true;\n}\n\nvoid NullBackwardWriter::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  buffer_sizer_.set_write_size_hint(pos(), write_size_hint);\n}\n\nbool NullBackwardWriter::PushSlow(size_t min_length,\n                                  size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of BackwardWriter::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  return MakeBuffer(min_length, recommended_length);\n}\n\nbool NullBackwardWriter::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  return MakeBuffer();\n}\n\nbool NullBackwardWriter::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  return MakeBuffer();\n}\n\nbool NullBackwardWriter::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  return MakeBuffer();\n}\n\nbool NullBackwardWriter::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  return MakeBuffer();\n}\n\nbool NullBackwardWriter::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  return MakeBuffer();\n}\n\nbool NullBackwardWriter::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (new_size >= start_pos()) {\n    if (ABSL_PREDICT_FALSE(new_size > pos())) return false;\n    set_cursor(start() - IntCast<size_t>(new_size - start_pos()));\n    return true;\n  }\n  buffer_sizer_.EndRun(pos());\n  set_start_pos(new_size);\n  buffer_sizer_.BeginRun(start_pos());\n  return MakeBuffer();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/null_backward_writer.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_NULL_BACKWARD_WRITER_H_\n#define RIEGELI_BYTES_NULL_BACKWARD_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n\nnamespace riegeli {\n\n// A `BackwardWriter` which discards all output.\n//\n// It tracks `pos()` normally.\nclass NullBackwardWriter : public BackwardWriter {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // `NullBackwardWriter` has a smaller default buffer size (512) so that\n    // writing larger values is skipped altogether.\n    static constexpr size_t kDefaultMinBufferSize = kMaxBytesToCopy + 1;\n    static constexpr size_t kDefaultMaxBufferSize = kMaxBytesToCopy + 1;\n  };\n\n  // Creates a closed `NullBackwardWriter`.\n  explicit NullBackwardWriter(Closed) noexcept : BackwardWriter(kClosed) {}\n\n  // Will discard all output.\n  NullBackwardWriter(Options options = Options()) noexcept;\n\n  NullBackwardWriter(NullBackwardWriter&& that) noexcept;\n  NullBackwardWriter& operator=(NullBackwardWriter&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `NullBackwardWriter`. This\n  // avoids constructing a temporary `NullBackwardWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Options options = Options());\n\n  bool SupportsTruncate() override { return true; }\n\n protected:\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using BackwardWriter::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool TruncateImpl(Position new_size) override;\n\n private:\n  // Resets buffer pointers to the beginning of the buffer.\n  void SyncBuffer();\n\n  // Ensures that the buffer has a sufficient size.\n  bool MakeBuffer(size_t min_length = 0, size_t recommended_length = 0);\n\n  WriteBufferSizer buffer_sizer_;\n  Buffer buffer_;\n};\n\n// Implementation details follow.\n\ninline NullBackwardWriter::NullBackwardWriter(Options options) noexcept\n    : buffer_sizer_(options.buffer_options()) {}\n\ninline NullBackwardWriter::NullBackwardWriter(\n    NullBackwardWriter&& that) noexcept\n    : BackwardWriter(static_cast<BackwardWriter&&>(that)),\n      buffer_sizer_(that.buffer_sizer_),\n      buffer_(std::move(that.buffer_)) {}\n\ninline NullBackwardWriter& NullBackwardWriter::operator=(\n    NullBackwardWriter&& that) noexcept {\n  BackwardWriter::operator=(static_cast<BackwardWriter&&>(that));\n  buffer_sizer_ = that.buffer_sizer_;\n  buffer_ = std::move(that.buffer_);\n  return *this;\n}\n\ninline void NullBackwardWriter::Reset(Closed) {\n  BackwardWriter::Reset(kClosed);\n  buffer_sizer_.Reset();\n  buffer_ = Buffer();\n}\n\ninline void NullBackwardWriter::Reset(Options options) {\n  BackwardWriter::Reset();\n  buffer_sizer_.Reset(options.buffer_options());\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_NULL_BACKWARD_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/null_writer.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/null_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid NullWriter::Done() {\n  Writer::Done();\n  buffer_ = Buffer();\n}\n\ninline void NullWriter::SyncBuffer() {\n  set_start_pos(pos());\n  set_cursor(start());\n}\n\ninline bool NullWriter::MakeBuffer(size_t min_length,\n                                   size_t recommended_length) {\n  if (ABSL_PREDICT_FALSE(min_length >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  const size_t buffer_length = UnsignedMin(\n      buffer_sizer_.BufferLength(start_pos(), min_length, recommended_length),\n      std::numeric_limits<Position>::max() - start_pos());\n  buffer_.Reset(buffer_length);\n  set_buffer(buffer_.data(), buffer_length);\n  return true;\n}\n\nvoid NullWriter::SetWriteSizeHintImpl(std::optional<Position> write_size_hint) {\n  buffer_sizer_.set_write_size_hint(pos(), write_size_hint);\n}\n\nbool NullWriter::PushSlow(size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  return MakeBuffer(min_length, recommended_length);\n}\n\nbool NullWriter::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of Writer::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  return MakeBuffer();\n}\n\nbool NullWriter::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  return MakeBuffer();\n}\n\nbool NullWriter::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  return MakeBuffer();\n}\n\nbool NullWriter::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  return MakeBuffer();\n}\n\nbool NullWriter::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  return MakeBuffer();\n}\n\nbool NullWriter::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of Writer::SeekSlow(): \"\n         \"position unchanged, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const Position size = UnsignedMax(pos(), written_size_);\n  if (new_pos >= start_pos() && new_pos <= pos()) {\n    written_size_ = size;\n    set_cursor(start() + IntCast<size_t>(new_pos - start_pos()));\n    return true;\n  }\n  buffer_sizer_.EndRun(pos());\n  if (ABSL_PREDICT_FALSE(new_pos > size)) {\n    set_start_pos(size);\n    buffer_sizer_.BeginRun(start_pos());\n    MakeBuffer();\n    return false;\n  }\n  written_size_ = size;\n  set_start_pos(new_pos);\n  buffer_sizer_.BeginRun(start_pos());\n  return MakeBuffer();\n}\n\nstd::optional<Position> NullWriter::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  return UnsignedMax(pos(), written_size_);\n}\n\nbool NullWriter::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const Position size = UnsignedMax(pos(), written_size_);\n  if (new_size >= start_pos() && new_size <= pos()) {\n    written_size_ = new_size;\n    set_cursor(start() + IntCast<size_t>(new_size - start_pos()));\n    return true;\n  }\n  buffer_sizer_.EndRun(pos());\n  if (ABSL_PREDICT_FALSE(new_size > size)) {\n    set_start_pos(size);\n    buffer_sizer_.BeginRun(start_pos());\n    MakeBuffer();\n    return false;\n  }\n  written_size_ = new_size;\n  set_start_pos(new_size);\n  buffer_sizer_.BeginRun(start_pos());\n  return MakeBuffer();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/null_writer.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_NULL_WRITER_H_\n#define RIEGELI_BYTES_NULL_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// A `Writer` which discards all output.\n//\n// It tracks `pos()` normally.\nclass NullWriter : public Writer {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // `NullWriter` has a smaller default buffer size (1024) so that writing\n    // larger values is skipped altogether.\n    static constexpr size_t kDefaultMinBufferSize = 1 << 10;\n    static constexpr size_t kDefaultMaxBufferSize = 1 << 10;\n  };\n\n  // Creates a closed `NullWriter`.\n  explicit NullWriter(Closed) noexcept : Writer(kClosed) {}\n\n  // Will discard all output.\n  explicit NullWriter(Options options = Options()) noexcept;\n\n  NullWriter(NullWriter&& that) noexcept;\n  NullWriter& operator=(NullWriter&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `NullWriter`. This avoids\n  // constructing a temporary `NullWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Options options = Options());\n\n  bool SupportsRandomAccess() override { return true; }\n\n protected:\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  bool TruncateImpl(Position new_size) override;\n\n private:\n  // Resets buffer pointers to the beginning of the buffer.\n  void SyncBuffer();\n\n  // Ensures that the buffer has a sufficient size.\n  bool MakeBuffer(size_t min_length = 0, size_t recommended_length = 0);\n\n  WriteBufferSizer buffer_sizer_;\n  Buffer buffer_;\n\n  // Size of written data is always `UnsignedMax(pos(), written_size_)`.\n  // This is used to determine the size after seeking backwards.\n  Position written_size_ = 0;\n};\n\n// Implementation details follow.\n\ninline NullWriter::NullWriter(Options options) noexcept\n    : buffer_sizer_(options.buffer_options()) {}\n\ninline NullWriter::NullWriter(NullWriter&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)),\n      buffer_sizer_(that.buffer_sizer_),\n      buffer_(std::move(that.buffer_)),\n      written_size_(that.written_size_) {}\n\ninline NullWriter& NullWriter::operator=(NullWriter&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  buffer_sizer_ = that.buffer_sizer_;\n  buffer_ = std::move(that.buffer_);\n  written_size_ = that.written_size_;\n  return *this;\n}\n\ninline void NullWriter::Reset(Closed) {\n  Writer::Reset(kClosed);\n  buffer_sizer_.Reset();\n  buffer_ = Buffer();\n  written_size_ = 0;\n}\n\ninline void NullWriter::Reset(Options options) {\n  Writer::Reset();\n  buffer_sizer_.Reset(options.buffer_options());\n  written_size_ = 0;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_NULL_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/ostream_writer.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/ostream_writer.h\"\n\n#include <stddef.h>\n\n#include <cerrno>\n#include <ios>\n#include <istream>\n#include <limits>\n#include <optional>\n#include <ostream>\n#include <string>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/global.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/istream_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid OStreamWriterBase::Initialize(std::ostream* dest,\n                                   std::optional<Position> assumed_pos,\n                                   bool assumed_append) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of OStreamWriter: null stream pointer\";\n  RIEGELI_ASSERT_EQ(supports_random_access_, LazyBoolState::kUnknown)\n      << \"Failed precondition of OStreamWriterBase::Initialize(): \"\n         \"supports_random_access_ not reset\";\n  RIEGELI_ASSERT_EQ(supports_read_mode_, LazyBoolState::kUnknown)\n      << \"Failed precondition of OStreamWriterBase::Initialize(): \"\n         \"supports_read_mode_ not reset\";\n  RIEGELI_ASSERT_OK(random_access_status_)\n      << \"Failed precondition of OStreamWriterBase::Initialize(): \"\n         \"random_access_status_ not reset\";\n  RIEGELI_ASSERT_OK(read_mode_status_)\n      << \"Failed precondition of OStreamWriterBase::Initialize(): \"\n         \"read_mode_status_ not reset\";\n  if (ABSL_PREDICT_FALSE(dest->fail())) {\n    // Either constructing the stream failed or the stream was already in a\n    // failed state. In any case `OStreamWriterBase` should fail.\n    FailOperation(\"ostream::ostream()\");\n    return;\n  }\n  if (assumed_pos != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(\n            *assumed_pos >\n            Position{std::numeric_limits<std::streamoff>::max()})) {\n      FailOverflow();\n      return;\n    }\n    set_start_pos(*assumed_pos);\n    supports_random_access_ = LazyBoolState::kFalse;\n    supports_read_mode_ = LazyBoolState::kFalse;\n    random_access_status_ = Global([] {\n      return absl::UnimplementedError(\n          \"OStreamWriterBase::Options::assumed_pos() excludes random access\");\n    });\n    read_mode_status_ = random_access_status_;\n  } else {\n    errno = 0;\n    const std::streamoff stream_pos = dest->tellp();\n    if (stream_pos < 0) {\n      // Random access is not supported. Assume 0 as the initial position.\n      supports_random_access_ = LazyBoolState::kFalse;\n      supports_read_mode_ = LazyBoolState::kFalse;\n      random_access_status_ = FailedOperationStatus(\"ostream::tellp()\");\n      read_mode_status_ = random_access_status_;\n      return;\n    }\n    set_start_pos(IntCast<Position>(stream_pos));\n    if (assumed_append) {\n      supports_random_access_ = LazyBoolState::kFalse;\n      // `supports_read_mode_` is left as `LazyBoolState::kUnknown`.\n      random_access_status_ = Global([] {\n        return absl::UnimplementedError(\"Append mode excludes random access\");\n      });\n    } else {\n      // `std::ostream::tellp()` succeeded, and `std::ostream::seekp()` will be\n      // checked later. `supports_random_access_` and `supports_read_mode_` are\n      // left as `LazyBoolState::kUnknown`.\n    }\n  }\n  BeginRun();\n}\n\nvoid OStreamWriterBase::Done() {\n  BufferedWriter::Done();\n  random_access_status_ = absl::OkStatus();\n  read_mode_status_ = absl::OkStatus();\n  associated_reader_.Reset();\n}\n\ninline absl::Status OStreamWriterBase::FailedOperationStatus(\n    absl::string_view operation) {\n  // There is no way to get details why a stream operation failed without\n  // letting the stream throw exceptions. Hopefully low level failures have set\n  // `errno` as a side effect.\n  //\n  // This requires resetting `errno` to 0 before the stream operation because\n  // the operation may fail without setting `errno`.\n  const int error_number = errno;\n  const std::string message = absl::StrCat(operation, \" failed\");\n  return error_number == 0 ? absl::UnknownError(message)\n                           : absl::ErrnoToStatus(error_number, message);\n}\n\nbool OStreamWriterBase::FailOperation(absl::string_view operation) {\n  return Fail(FailedOperationStatus(operation));\n}\n\nbool OStreamWriterBase::SupportsRandomAccess() {\n  if (ABSL_PREDICT_TRUE(supports_random_access_ != LazyBoolState::kUnknown)) {\n    return supports_random_access_ == LazyBoolState::kTrue;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::ostream& dest = *DestStream();\n  errno = 0;\n  dest.seekp(0, std::ios_base::end);\n  if (dest.fail()) {\n    // Not supported.\n    supports_random_access_ = LazyBoolState::kFalse;\n    random_access_status_ = FailedOperationStatus(\"ostream::seekp()\");\n    dest.clear(dest.rdstate() & ~std::ios_base::failbit);\n    return false;\n  }\n  // Supported.\n  dest.seekp(IntCast<std::streamoff>(start_pos()), std::ios_base::beg);\n  if (ABSL_PREDICT_FALSE(dest.fail())) return FailOperation(\"ostream::seekp()\");\n  supports_random_access_ = LazyBoolState::kTrue;\n  return true;\n}\n\nbool OStreamWriterBase::SupportsReadMode() {\n  if (ABSL_PREDICT_TRUE(supports_read_mode_ != LazyBoolState::kUnknown)) {\n    return supports_read_mode_ == LazyBoolState::kTrue;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::istream* const src = SrcStream();\n  if (src == nullptr) {\n    supports_read_mode_ = LazyBoolState::kFalse;\n    read_mode_status_ = Global([] {\n      return absl::UnimplementedError(\n          \"Read mode requires the static type of the destination \"\n          \"deriving from std::istream\");\n    });\n    return false;\n  }\n  errno = 0;\n  const std::streamoff stream_pos = src->tellg();\n  if (stream_pos < 0) {\n    // Not supported.\n    supports_read_mode_ = LazyBoolState::kFalse;\n    read_mode_status_ = FailedOperationStatus(\"istream::tellg()\");\n    return false;\n  }\n  src->seekg(0, std::ios_base::end);\n  if (src->fail()) {\n    // Not supported.\n    supports_read_mode_ = LazyBoolState::kFalse;\n    read_mode_status_ = FailedOperationStatus(\"istream::seekg()\");\n    src->clear(src->rdstate() & ~std::ios_base::failbit);\n    return false;\n  }\n  // Supported.\n  supports_read_mode_ = LazyBoolState::kTrue;\n  std::ostream& dest = *DestStream();\n  dest.seekp(IntCast<std::streamoff>(start_pos()), std::ios_base::beg);\n  if (ABSL_PREDICT_FALSE(dest.fail())) return FailOperation(\"ostream::seekp()\");\n  return true;\n}\n\ninline bool OStreamWriterBase::WriteMode() {\n  if (ABSL_PREDICT_TRUE(!read_mode_)) return true;\n  read_mode_ = false;\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::ostream& dest = *DestStream();\n  errno = 0;\n  dest.seekp(IntCast<std::streamoff>(start_pos()), std::ios_base::beg);\n  if (ABSL_PREDICT_FALSE(dest.fail())) return FailOperation(\"ostream::seekp()\");\n  return true;\n}\n\nbool OStreamWriterBase::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of BufferedWriter::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedWriter::WriteInternal()\";\n  if (ABSL_PREDICT_FALSE(!WriteMode())) return false;\n  std::ostream& dest = *DestStream();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         Position{std::numeric_limits<std::streamoff>::max()} -\n                             start_pos())) {\n    return FailOverflow();\n  }\n  errno = 0;\n  do {\n    dest.write(src.data(), IntCast<std::streamsize>(src.size()));\n    if (ABSL_PREDICT_FALSE(dest.fail())) {\n      return FailOperation(\"ostream::write()\");\n    }\n    move_start_pos(src.size());\n    src.remove_prefix(src.size());\n  } while (!src.empty());\n  return true;\n}\n\nbool OStreamWriterBase::FlushBehindBuffer(absl::string_view src,\n                                          FlushType flush_type) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::FlushBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!WriteMode())) return false;\n  return BufferedWriter::FlushBehindBuffer(src, flush_type);\n}\n\nbool OStreamWriterBase::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of BufferedWriter::SeekBehindBuffer(): \"\n         \"position unchanged, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!OStreamWriterBase::SupportsRandomAccess())) {\n    if (ok()) Fail(random_access_status_);\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  read_mode_ = false;\n  std::ostream& dest = *DestStream();\n  errno = 0;\n  if (new_pos > start_pos()) {\n    // Seeking forwards.\n    dest.seekp(0, std::ios_base::end);\n    if (ABSL_PREDICT_FALSE(dest.fail())) {\n      return FailOperation(\"ostream::seekp()\");\n    }\n    const std::streamoff stream_size = dest.tellp();\n    if (ABSL_PREDICT_FALSE(stream_size < 0)) {\n      return FailOperation(\"ostream::tellp()\");\n    }\n    if (ABSL_PREDICT_FALSE(new_pos > IntCast<Position>(stream_size))) {\n      // Stream ends.\n      set_start_pos(IntCast<Position>(stream_size));\n      return false;\n    }\n  }\n  dest.seekp(IntCast<std::streamoff>(new_pos), std::ios_base::beg);\n  if (ABSL_PREDICT_FALSE(dest.fail())) return FailOperation(\"ostream::seekp()\");\n  set_start_pos(new_pos);\n  return true;\n}\n\nstd::optional<Position> OStreamWriterBase::SizeBehindBuffer() {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::SizeBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!OStreamWriterBase::SupportsRandomAccess())) {\n    if (ok()) Fail(random_access_status_);\n    return std::nullopt;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  read_mode_ = false;\n  std::ostream& dest = *DestStream();\n  errno = 0;\n  dest.seekp(0, std::ios_base::end);\n  if (ABSL_PREDICT_FALSE(dest.fail())) {\n    FailOperation(\"ostream::seekp()\");\n    return std::nullopt;\n  }\n  const std::streamoff stream_size = dest.tellp();\n  if (ABSL_PREDICT_FALSE(stream_size < 0)) {\n    FailOperation(\"ostream::tellp()\");\n    return std::nullopt;\n  }\n  dest.seekp(IntCast<std::streamoff>(start_pos()), std::ios_base::beg);\n  if (ABSL_PREDICT_FALSE(dest.fail())) {\n    FailOperation(\"ostream::seekp()\");\n    return std::nullopt;\n  }\n  return IntCast<Position>(stream_size);\n}\n\nReader* OStreamWriterBase::ReadModeBehindBuffer(Position initial_pos) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::ReadModeBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!OStreamWriterBase::SupportsReadMode())) {\n    if (ok()) Fail(read_mode_status_);\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  std::istream& src = *SrcStream();\n  IStreamReader<>* const reader = associated_reader_.ResetReader(\n      &src, IStreamReaderBase::Options().set_buffer_options(buffer_options()));\n  reader->Seek(initial_pos);\n  read_mode_ = true;\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/ostream_writer.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_OSTREAM_WRITER_H_\n#define RIEGELI_BYTES_OSTREAM_WRITER_H_\n\n#include <stdint.h>\n\n#include <cerrno>\n#include <istream>\n#include <optional>\n#include <ostream>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/iostream_internal.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\ntemplate <typename Src>\nclass IStreamReader;\nclass Reader;\n\n// Template parameter independent part of `OStreamWriter`.\nclass OStreamWriterBase : public BufferedWriter {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // If `std::nullopt`, the current position reported by `pos()` corresponds\n    // to the current stream position if possible, otherwise 0 is assumed as the\n    // initial position. Random access is supported if the stream supports\n    // random access.\n    //\n    // If not `std::nullopt`, this position is assumed initially, to be reported\n    // by `pos()`. It does not need to correspond to the current stream\n    // position. Random access is not supported.\n    //\n    // Warning: On Windows this must not be `std::nullopt` if the stream is a\n    // `std::ofstream` or `std::fstream` opened in text mode.\n    //\n    // Default: `std::nullopt`.\n    Options& set_assumed_pos(std::optional<Position> assumed_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      assumed_pos_ = assumed_pos;\n      return *this;\n    }\n    Options&& set_assumed_pos(std::optional<Position> assumed_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_pos(assumed_pos));\n    }\n    std::optional<Position> assumed_pos() const { return assumed_pos_; }\n\n    // If `assumed_pos()` is not set, `assumed_append()` should be set to `true`\n    // if the `std::ostream` refers a file open in append mode, i.e. if all\n    // writes happen at the end. This lets `SupportsRandomAccess()` correctly\n    // return `false`.\n    //\n    // Default: `false`.\n    Options& set_assumed_append(bool assumed_append) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      assumed_append_ = assumed_append;\n      return *this;\n    }\n    Options&& set_assumed_append(bool assumed_append) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_append(assumed_append));\n    }\n    bool assumed_append() const { return assumed_append_; }\n\n   private:\n    std::optional<Position> assumed_pos_;\n    bool assumed_append_ = false;\n  };\n\n  // Returns the stream being written to. Unchanged by `Close()`.\n  virtual std::ostream* DestStream() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsRandomAccess() override;\n  bool SupportsTruncate() override { return false; }\n  bool SupportsReadMode() override;\n\n protected:\n  explicit OStreamWriterBase(Closed) noexcept : BufferedWriter(kClosed) {}\n\n  explicit OStreamWriterBase(BufferOptions buffer_options);\n\n  OStreamWriterBase(OStreamWriterBase&& that) noexcept;\n  OStreamWriterBase& operator=(OStreamWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options);\n  void Initialize(std::ostream* dest, std::optional<Position> assumed_pos,\n                  bool assumed_append);\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation);\n\n  // Returns the stream pointer as `std::istream*` if the static type of the\n  // destination derives from `std::istream`, otherwise returns `nullptr`.\n  virtual std::istream* SrcStream() const = 0;\n\n  void Done() override;\n  bool WriteInternal(absl::string_view src) override;\n  bool FlushBehindBuffer(absl::string_view src, FlushType flush_type) override;\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::optional<Position> SizeBehindBuffer() override;\n  Reader* ReadModeBehindBuffer(Position initial_pos) override;\n\n private:\n  // Encodes a `bool` or a marker that the value is not resolved yet.\n  enum class LazyBoolState : uint8_t { kUnknown, kTrue, kFalse };\n\n  absl::Status FailedOperationStatus(absl::string_view operation);\n\n  bool WriteMode();\n\n  LazyBoolState supports_random_access_ = LazyBoolState::kUnknown;\n  LazyBoolState supports_read_mode_ = LazyBoolState::kUnknown;\n  absl::Status random_access_status_;\n  absl::Status read_mode_status_;\n\n  AssociatedReader<IStreamReader<std::istream*>> associated_reader_;\n  bool read_mode_ = false;\n\n  // Invariant: `start_pos() <= std::numeric_limits<std::streamoff>::max()`\n};\n\n// A `Writer` which writes to a `std::ostream`.\n//\n// `OStreamWriter` supports random access if\n// `Options::assumed_pos() == std::nullopt` and the stream supports random\n// access (this is checked by calling `std::ostream::tellp()` and\n// `std::ostream::seekp()` to the end and back).\n//\n// `OStreamWriter` supports `ReadMode()` if the static type of the stream\n// derives also from `std::istream`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the stream being written to. `Dest` must support\n// `Dependency<std::ostream*, Dest>`, e.g. `std::ostream*` (not owned, default),\n// `std::ofstream` (owned), `std::unique_ptr<std::ostream>` (owned),\n// `Any<std::ostream*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// Until the `OStreamWriter` is closed or no longer used, the `std::ostream`\n// must not be closed nor have its position changed, except that if random\n// access is not used, careful interleaving of multiple writers is possible:\n// `Flush()` is needed before switching to another writer, and `pos()` does not\n// take other writers into account.\ntemplate <typename Dest = std::ostream*>\nclass OStreamWriter : public OStreamWriterBase {\n public:\n  // Creates a closed `OStreamWriter`.\n  explicit OStreamWriter(Closed) noexcept : OStreamWriterBase(kClosed) {}\n\n  // Will write to the stream provided by `dest`.\n  explicit OStreamWriter(Initializer<Dest> dest, Options options = Options());\n\n  OStreamWriter(OStreamWriter&& that) = default;\n  OStreamWriter& operator=(OStreamWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `OStreamWriter`. This\n  // avoids constructing a temporary `OStreamWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the stream being written\n  // to. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  std::ostream* DestStream() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  std::istream* SrcStream() const override;\n\n  void Done() override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  // The object providing and possibly owning the stream being written to.\n  Dependency<std::ostream*, Dest> dest_;\n};\n\nexplicit OStreamWriter(Closed) -> OStreamWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit OStreamWriter(Dest&& dest, OStreamWriterBase::Options options =\n                                        OStreamWriterBase::Options())\n    -> OStreamWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline OStreamWriterBase::OStreamWriterBase(BufferOptions buffer_options)\n    : BufferedWriter(buffer_options) {\n  // Clear `errno` so that `Initialize()` can attribute failures to opening the\n  // stream.\n  errno = 0;\n}\n\ninline OStreamWriterBase::OStreamWriterBase(OStreamWriterBase&& that) noexcept\n    : BufferedWriter(static_cast<BufferedWriter&&>(that)),\n      supports_random_access_(\n          std::exchange(that.supports_random_access_, LazyBoolState::kUnknown)),\n      supports_read_mode_(\n          std::exchange(that.supports_read_mode_, LazyBoolState::kUnknown)),\n      random_access_status_(std::move(that.random_access_status_)),\n      read_mode_status_(std::move(that.read_mode_status_)),\n      associated_reader_(std::move(that.associated_reader_)),\n      read_mode_(that.read_mode_) {}\n\ninline OStreamWriterBase& OStreamWriterBase::operator=(\n    OStreamWriterBase&& that) noexcept {\n  BufferedWriter::operator=(static_cast<BufferedWriter&&>(that));\n  supports_random_access_ =\n      std::exchange(that.supports_random_access_, LazyBoolState::kUnknown);\n  supports_read_mode_ =\n      std::exchange(that.supports_read_mode_, LazyBoolState::kUnknown);\n  random_access_status_ = std::move(that.random_access_status_);\n  read_mode_status_ = std::move(that.read_mode_status_);\n  associated_reader_ = std::move(that.associated_reader_);\n  read_mode_ = that.read_mode_;\n  return *this;\n}\n\ninline void OStreamWriterBase::Reset(Closed) {\n  BufferedWriter::Reset(kClosed);\n  supports_random_access_ = LazyBoolState::kUnknown;\n  supports_read_mode_ = LazyBoolState::kUnknown;\n  random_access_status_ = absl::OkStatus();\n  read_mode_status_ = absl::OkStatus();\n  associated_reader_.Reset();\n  read_mode_ = false;\n}\n\ninline void OStreamWriterBase::Reset(BufferOptions buffer_options) {\n  BufferedWriter::Reset(buffer_options);\n  supports_random_access_ = LazyBoolState::kUnknown;\n  supports_read_mode_ = LazyBoolState::kUnknown;\n  random_access_status_ = absl::OkStatus();\n  read_mode_status_ = absl::OkStatus();\n  associated_reader_.Reset();\n  read_mode_ = false;\n  // Clear `errno` so that `Initialize()` can attribute failures to opening the\n  // stream.\n  errno = 0;\n}\n\ntemplate <typename Dest>\ninline OStreamWriter<Dest>::OStreamWriter(Initializer<Dest> dest,\n                                          Options options)\n    : OStreamWriterBase(options.buffer_options()), dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.assumed_pos(), options.assumed_append());\n}\n\ntemplate <typename Dest>\ninline void OStreamWriter<Dest>::Reset(Closed) {\n  OStreamWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void OStreamWriter<Dest>::Reset(Initializer<Dest> dest,\n                                       Options options) {\n  OStreamWriterBase::Reset(options.buffer_options());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.assumed_pos(), options.assumed_append());\n}\n\ntemplate <typename Dest>\ninline std::istream* OStreamWriter<Dest>::SrcStream() const {\n  return iostream_internal::DetectIStream(dest_.get());\n}\n\ntemplate <typename Dest>\nvoid OStreamWriter<Dest>::Done() {\n  OStreamWriterBase::Done();\n  if (dest_.IsOwning()) {\n    errno = 0;\n    iostream_internal::Close(*dest_);\n    if (ABSL_PREDICT_FALSE(dest_->fail()) && ABSL_PREDICT_TRUE(ok())) {\n      FailOperation(\"ostream::close()\");\n    }\n  }\n}\n\ntemplate <typename Dest>\nbool OStreamWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!OStreamWriterBase::FlushImpl(flush_type))) {\n    return false;\n  }\n  switch (flush_type) {\n    case FlushType::kFromObject:\n      if (!dest_.IsOwning()) return true;\n      ABSL_FALLTHROUGH_INTENDED;\n    case FlushType::kFromProcess:\n    case FlushType::kFromMachine:\n      errno = 0;\n      dest_->flush();\n      if (ABSL_PREDICT_FALSE(dest_->fail())) {\n        return FailOperation(\"ostream::flush()\");\n      }\n      return true;\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown flush type: \" << static_cast<int>(flush_type);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_OSTREAM_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/path_ref.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_PATH_REF_H_\n#define RIEGELI_BYTES_PATH_REF_H_\n\n#include <stddef.h>\n\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/invoker.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/string_ref.h\"\n#include \"riegeli/base/temporary_storage.h\"\n#include \"riegeli/base/type_traits.h\"\n\n#if defined(__cpp_lib_filesystem) && __cpp_lib_filesystem >= 201703\n#include <filesystem>\n#endif\n\nnamespace riegeli {\n\n// Filename used for default-constructed or moved-from objects.\nconstexpr char kDefaultFilenameCStr[] = \"<none>\";\nconstexpr absl::string_view kDefaultFilename = kDefaultFilenameCStr;\n\n// `PathRef` stores an `absl::string_view` representing a file path.\n//\n// It is intended for function parameters when the implementation needs\n// an `absl::string_view`, and the caller might have another representation\n// of the string.\n//\n// It is convertible from:\n//  * types convertible to `absl::string_view`\n//  * types convertible to `std::string`, e.g. `PathInitializer`\n//  * `std::filesystem::path`\n//\n// For `std::filesystem::path` with `value_type = char`, it refers to\n// `path.native()`.\n//\n// For `std::filesystem::path` with `value_type = wchar_t`, it refers to\n// `path.string()` stored in a storage object passed as a default argument to\n// the constructor.\n//\n// `PathRef` does not own path contents and is efficiently copyable.\nclass PathRef : public StringRef, public WithCompare<PathRef> {\n public:\n  // Stores an empty `absl::string_view`.\n  PathRef() = default;\n\n  // Stores `str` converted to `absl::string_view`.\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  /*implicit*/ PathRef(const char* str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : StringRef(absl::string_view(str)) {}\n\n  // Stores `str` converted to `StringRef` and then to `absl::string_view`.\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<PathRef, T>,\n                                   std::is_convertible<T&&, absl::string_view>>,\n                int> = 0>\n  /*implicit*/ PathRef(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : StringRef(std::forward<T>(str)) {}\n\n  // Stores `str` materialized, then converted to `StringRef` and then to\n  // `absl::string_view`.\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<\n                    NotSameRef<PathRef, T>,\n                    std::negation<std::is_convertible<T&&, absl::string_view>>,\n                    std::is_convertible<T&&, std::string>>,\n                int> = 0>\n  /*implicit*/ PathRef(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                       TemporaryStorage<std::string>&& storage\n                           ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : StringRef(std::forward<T>(str), std::move(storage)) {}\n\n#if __cpp_lib_filesystem >= 201703\n\n  // For `std::filesystem::path` with `value_type = char`, stores a reference to\n  // `path.native()`.\n  template <\n      typename DependentPath = std::filesystem::path,\n      std::enable_if_t<std::is_same_v<typename DependentPath::value_type, char>,\n                       int> = 0>\n  /*implicit*/ PathRef(\n      const std::filesystem::path& path ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : StringRef(static_cast<const DependentPath&>(path).native()) {}\n  template <\n      typename DependentPath = std::filesystem::path,\n      std::enable_if_t<std::is_same_v<typename DependentPath::value_type, char>,\n                       int> = 0>\n  /*implicit*/ PathRef(\n      const std::filesystem::path& path ABSL_ATTRIBUTE_LIFETIME_BOUND,\n      ABSL_ATTRIBUTE_UNUSED TemporaryStorage<std::string>&& storage)\n      : PathRef(path) {}\n\n  // For `std::filesystem::path` with `value_type = wchar_t`, stores a reference\n  // to `path.string()` stored in a storage object passed as a default argument\n  // to this constructor.\n  template <\n      typename DependentPath = std::filesystem::path,\n      std::enable_if_t<\n          !std::is_same_v<typename DependentPath::value_type, char>, int> = 0>\n  /*implicit*/ PathRef(const std::filesystem::path& path,\n                       TemporaryStorage<std::string>&& storage\n                           ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : StringRef(std::move(storage).emplace(\n            riegeli::Invoker([&path] { return path.string(); }))) {}\n\n#endif\n\n  PathRef(const PathRef& that) = default;\n  PathRef& operator=(const PathRef&) = delete;\n\n  friend bool operator==(PathRef a, PathRef b) {\n    return absl::string_view(a) == absl::string_view(b);\n  }\n  friend riegeli::StrongOrdering RIEGELI_COMPARE(PathRef a, PathRef b) {\n    return riegeli::Compare(absl::string_view(a), absl::string_view(b));\n  }\n\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<NotSameRef<PathRef, T>,\n                                          std::is_convertible<T&&, StringRef>>,\n                       int> = 0>\n  friend bool operator==(PathRef a, T&& b) {\n    return a == PathRef(std::forward<T>(b));\n  }\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<NotSameRef<PathRef, T>,\n                                          std::is_convertible<T&&, StringRef>>,\n                       int> = 0>\n  friend riegeli::StrongOrdering RIEGELI_COMPARE(PathRef a, T&& b) {\n    return riegeli::Compare(a, PathRef(std::forward<T>(b)));\n  }\n\n#if __cpp_lib_filesystem >= 201703\n\n  friend bool operator==(PathRef a, const std::filesystem::path& b) {\n    return a == PathRef(b);\n  }\n  friend riegeli::StrongOrdering RIEGELI_COMPARE(\n      PathRef a, const std::filesystem::path& b) {\n    return riegeli::Compare(a, PathRef(b));\n  }\n\n#endif\n};\n\n// `PathInitializer` is convertible from the same types as `PathRef`,\n// but efficiently takes ownership of `std::string`.\n//\n// `PathInitializer` behaves like `Initializer<std::string>`.\nclass PathInitializer : public Initializer<std::string> {\n public:\n#if __cpp_lib_filesystem >= 201703\n  class StringFromPath {\n   public:\n    explicit StringFromPath(\n        const std::filesystem::path& path ABSL_ATTRIBUTE_LIFETIME_BOUND)\n        : path_(path) {}\n\n    /*implicit*/ operator std::string() const { return path_.string(); }\n\n   private:\n    const std::filesystem::path& path_;\n  };\n#endif\n\n  PathInitializer() = default;\n\n  // Stores `str` converted to `absl::string_view` and then to `std::string`.\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  /*implicit*/ PathInitializer(const char* str ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                               TemporaryStorage<MakerType<absl::string_view>>&&\n                                   storage ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : Initializer(std::move(storage).emplace(absl::string_view(str))) {}\n\n  // Stores `str` converted to `std::string`.\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<PathInitializer, T>,\n                                   std::is_convertible<T&&, std::string>>,\n                int> = 0>\n  /*implicit*/ PathInitializer(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : Initializer(std::forward<T>(str)) {}\n\n#if __cpp_lib_filesystem >= 201703\n  // Stores `path.string()`.\n  /*implicit*/ PathInitializer(const std::filesystem::path& path\n                                   ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                               TemporaryStorage<StringFromPath>&& storage\n                                   ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : Initializer(std::move(storage).emplace(path)) {}\n#endif\n\n  // Stores `str` converted to `PathRef`, then to `absl::string_view`, and then\n  // to `std::string`.\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<\n                           NotSameRef<PathInitializer, T>,\n                           std::negation<std::is_convertible<T&&, std::string>>,\n#if __cpp_lib_filesystem >= 201703\n                           NotSameRef<std::filesystem::path, T>,\n#endif\n                           std::is_convertible<T&&, StringRef>>,\n                       int> = 0>\n  /*implicit*/ PathInitializer(T&& str ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                               TemporaryStorage<MakerType<absl::string_view>>&&\n                                   storage ABSL_ATTRIBUTE_LIFETIME_BOUND = {})\n      : Initializer(\n            std::move(storage).emplace(StringRef(std::forward<T>(str)))) {\n  }\n\n  PathInitializer(PathInitializer&& that) = default;\n  PathInitializer& operator=(PathInitializer&&) = delete;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_PATH_REF_H_\n"
  },
  {
    "path": "riegeli/bytes/position_shifting_backward_writer.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/position_shifting_backward_writer.h\"\n\n#include <stddef.h>\n\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\nvoid PositionShiftingBackwardWriterBase::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    BackwardWriter& dest = *DestWriter();\n    SyncBuffer(dest);\n  }\n  BackwardWriter::Done();\n}\n\nbool PositionShiftingBackwardWriterBase::FailUnderflow(Position new_pos,\n                                                       Object& object) {\n  return object.Fail(absl::InvalidArgumentError(\n      absl::StrCat(\"PositionShiftingBackwardWriter does not support \"\n                   \"truncating before the base position: \",\n                   new_pos, \" < \", base_pos_)));\n}\n\nabsl::Status PositionShiftingBackwardWriterBase::AnnotateStatusImpl(\n    absl::Status status) {\n  if (is_open()) {\n    BackwardWriter& dest = *DestWriter();\n    SyncBuffer(dest);\n    status = dest.AnnotateStatus(std::move(status));\n    MakeBuffer(dest);\n  }\n  // The status might have been annotated by `dest` with the original position.\n  // Clarify that the current position is the relative position instead of\n  // delegating to `BackwardWriter::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status PositionShiftingBackwardWriterBase::AnnotateOverDest(\n    absl::Status status) {\n  if (is_open() && base_pos_ > 0) {\n    return Annotate(status,\n                    absl::StrCat(\"with relative position at byte \", pos()));\n  }\n  return status;\n}\n\nbool PositionShiftingBackwardWriterBase::PushSlow(size_t min_length,\n                                                  size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of BackwardWriter::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  BackwardWriter& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool push_ok = dest.Push(min_length, recommended_length);\n  return MakeBuffer(dest, min_length) && push_ok;\n}\n\nbool PositionShiftingBackwardWriterBase::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  return WriteInternal(src);\n}\n\nbool PositionShiftingBackwardWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool PositionShiftingBackwardWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  return WriteInternal(src);\n}\n\nbool PositionShiftingBackwardWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool PositionShiftingBackwardWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  return WriteInternal(src);\n}\n\nbool PositionShiftingBackwardWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool PositionShiftingBackwardWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  return WriteInternal(src);\n}\n\ntemplate <typename Src>\ninline bool PositionShiftingBackwardWriterBase::WriteInternal(Src&& src) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  BackwardWriter& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool write_ok = dest.Write(std::forward<Src>(src));\n  return MakeBuffer(dest) && write_ok;\n}\n\nbool PositionShiftingBackwardWriterBase::SupportsTruncate() {\n  BackwardWriter* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsTruncate();\n}\n\nbool PositionShiftingBackwardWriterBase::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(new_size < base_pos_)) {\n    return FailUnderflow(new_size, *this);\n  }\n  BackwardWriter& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool truncate_ok = dest.Truncate(new_size - base_pos_);\n  return MakeBuffer(dest) && truncate_ok;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/position_shifting_backward_writer.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_POSITION_SHIFTING_BACKWARD_WRITER_H_\n#define RIEGELI_BYTES_POSITION_SHIFTING_BACKWARD_WRITER_H_\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `PositionShiftingBackwardWriter`.\nclass PositionShiftingBackwardWriterBase : public BackwardWriter {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // The base position of the new `BackwardWriter`.\n    //\n    // Default: 0.\n    Options& set_base_pos(Position base_pos) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      base_pos_ = base_pos;\n      return *this;\n    }\n    Options&& set_base_pos(Position base_pos) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_base_pos(base_pos));\n    }\n    Position base_pos() const { return base_pos_; }\n\n   private:\n    Position base_pos_ = 0;\n  };\n\n  // Returns the new `BackwardWriter`. Unchanged by `Close()`.\n  virtual BackwardWriter* DestWriter() const = 0;\n\n  // Returns the base position of the original `BackwardWriter`.\n  Position base_pos() const { return base_pos_; }\n\n  bool SupportsTruncate() override;\n\n protected:\n  explicit PositionShiftingBackwardWriterBase(Closed) noexcept\n      : BackwardWriter(kClosed) {}\n\n  explicit PositionShiftingBackwardWriterBase(Position base_pos);\n\n  PositionShiftingBackwardWriterBase(\n      PositionShiftingBackwardWriterBase&& that) noexcept;\n  PositionShiftingBackwardWriterBase& operator=(\n      PositionShiftingBackwardWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(Position base_pos);\n  void Initialize(BackwardWriter* dest);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  // Sets cursor of `dest` to cursor of `*this`.\n  void SyncBuffer(BackwardWriter& dest);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `dest`, adjusting\n  // `start()` to hide data already written. Fails `*this` if `dest` failed\n  // or there is not enough `Position` space for `min_length`.\n  bool MakeBuffer(BackwardWriter& dest, size_t min_length = 0);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using BackwardWriter::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool TruncateImpl(Position new_size) override;\n\n private:\n  ABSL_ATTRIBUTE_COLD bool FailUnderflow(Position new_pos, Object& object);\n\n  // This template is defined and used only in position_shifting_writer.cc.\n  template <typename Src>\n  bool WriteInternal(Src&& src);\n\n  Position base_pos_ = 0;\n\n  // Invariants if `ok()`:\n  //   `start() == DestWriter()->cursor()`\n  //   `limit() == DestWriter()->limit()`\n  //   `start_pos() == DestWriter()->pos() + base_pos_`\n};\n\n// A `BackwardWriter` which writes to another `BackwardWriter`, reporting\n// positions shifted so that the beginning appears as the given base position.\n//\n// `PrefixLimitingBackwardWriter` can be used for shifting positions in the\n// other direction.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the original `BackwardWriter`. `Dest` must support\n// `Dependency<BackwardWriter*, Dest>`, e.g.\n// `BackwardWriter*` (not owned, default),\n// `ChainNackwardWriter<>` (owned), `std::unique_ptr<BackwardWriter>` (owned),\n// `Any<BackwardWriter*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The original `BackwardWriter` must not be accessed until the\n// `PositionShiftingBackwardWriter` is closed or no longer used, except that\n// it is allowed to read the destination of the original `BackwardWriter`\n// immediately after `Flush()`.\ntemplate <typename Dest = BackwardWriter*>\nclass PositionShiftingBackwardWriter\n    : public PositionShiftingBackwardWriterBase {\n public:\n  // Creates a closed `PositionShiftingBackwardWriter`.\n  explicit PositionShiftingBackwardWriter(Closed) noexcept\n      : PositionShiftingBackwardWriterBase(kClosed) {}\n\n  // Will write to the original `BackwardWriter` provided by `dest`.\n  explicit PositionShiftingBackwardWriter(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  PositionShiftingBackwardWriter(PositionShiftingBackwardWriter&& that) =\n      default;\n  PositionShiftingBackwardWriter& operator=(\n      PositionShiftingBackwardWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed\n  // `PositionShiftingBackwardWriter`. This avoids constructing a temporary\n  // `PositionShiftingBackwardWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the original\n  // `BackwardWriter`. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  BackwardWriter* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the original `BackwardWriter`.\n  MovingDependency<BackwardWriter*, Dest, Mover> dest_;\n};\n\nexplicit PositionShiftingBackwardWriter(Closed)\n    -> PositionShiftingBackwardWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit PositionShiftingBackwardWriter(\n    Dest&& dest, PositionShiftingBackwardWriterBase::Options options =\n                     PositionShiftingBackwardWriterBase::Options())\n    -> PositionShiftingBackwardWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline PositionShiftingBackwardWriterBase::PositionShiftingBackwardWriterBase(\n    Position base_pos)\n    : base_pos_(base_pos) {}\n\ninline PositionShiftingBackwardWriterBase::PositionShiftingBackwardWriterBase(\n    PositionShiftingBackwardWriterBase&& that) noexcept\n    : BackwardWriter(static_cast<BackwardWriter&&>(that)),\n      base_pos_(that.base_pos_) {}\n\ninline PositionShiftingBackwardWriterBase&\nPositionShiftingBackwardWriterBase::operator=(\n    PositionShiftingBackwardWriterBase&& that) noexcept {\n  BackwardWriter::operator=(static_cast<BackwardWriter&&>(that));\n  base_pos_ = that.base_pos_;\n  return *this;\n}\n\ninline void PositionShiftingBackwardWriterBase::Reset(Closed) {\n  BackwardWriter::Reset(kClosed);\n  base_pos_ = 0;\n}\n\ninline void PositionShiftingBackwardWriterBase::Reset(Position base_pos) {\n  BackwardWriter::Reset();\n  base_pos_ = base_pos;\n}\n\ninline void PositionShiftingBackwardWriterBase::Initialize(\n    BackwardWriter* dest) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of PositionShiftingBackwardWriter: \"\n         \"null BackwardWriter pointer\";\n  MakeBuffer(*dest);\n}\n\ninline void PositionShiftingBackwardWriterBase::SyncBuffer(\n    BackwardWriter& dest) {\n  dest.set_cursor(cursor());\n}\n\ninline bool PositionShiftingBackwardWriterBase::MakeBuffer(BackwardWriter& dest,\n                                                           size_t min_length) {\n  const Position max_pos = std::numeric_limits<Position>::max() - base_pos_;\n  if (ABSL_PREDICT_FALSE(dest.limit_pos() > max_pos)) {\n    if (ABSL_PREDICT_FALSE(dest.pos() > max_pos)) {\n      set_buffer(dest.cursor());\n      set_start_pos(std::numeric_limits<Position>::max());\n      return FailOverflow();\n    }\n    set_buffer(dest.cursor() - IntCast<size_t>(max_pos - dest.pos()),\n               IntCast<size_t>(max_pos - dest.pos()));\n    set_start_pos(dest.pos() + base_pos_);\n    if (ABSL_PREDICT_FALSE(available() < min_length)) return FailOverflow();\n  } else {\n    set_buffer(dest.limit(), dest.available());\n    set_start_pos(dest.pos() + base_pos_);\n  }\n  if (ABSL_PREDICT_FALSE(!dest.ok())) {\n    return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n  return true;\n}\n\ntemplate <typename Dest>\nclass PositionShiftingBackwardWriter<Dest>::Mover {\n public:\n  static auto member() { return &PositionShiftingBackwardWriter::dest_; }\n\n  explicit Mover(PositionShiftingBackwardWriter& self,\n                 PositionShiftingBackwardWriter& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `dest_` is not moved yet so `dest_` is taken from `that`.\n    if (uses_buffer_) self.SyncBuffer(*that.dest_);\n  }\n\n  void Done(PositionShiftingBackwardWriter& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.dest_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Dest>\ninline PositionShiftingBackwardWriter<Dest>::PositionShiftingBackwardWriter(\n    Initializer<Dest> dest, Options options)\n    : PositionShiftingBackwardWriterBase(options.base_pos()),\n      dest_(std::move(dest)) {\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\ninline void PositionShiftingBackwardWriter<Dest>::Reset(Closed) {\n  PositionShiftingBackwardWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void PositionShiftingBackwardWriter<Dest>::Reset(Initializer<Dest> dest,\n                                                        Options options) {\n  PositionShiftingBackwardWriterBase::Reset(options.base_pos());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\nvoid PositionShiftingBackwardWriter<Dest>::Done() {\n  PositionShiftingBackwardWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <typename Dest>\nvoid PositionShiftingBackwardWriter<Dest>::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (dest_.IsOwning()) {\n    SyncBuffer(*dest_);\n    dest_->SetWriteSizeHint(\n        write_size_hint == std::nullopt\n            ? std::nullopt\n            : std::make_optional(SaturatingAdd(base_pos(), *write_size_hint)));\n    MakeBuffer(*dest_);\n  }\n}\n\ntemplate <typename Dest>\nbool PositionShiftingBackwardWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer(*dest_);\n  bool flush_ok = true;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    flush_ok = dest_->Flush(flush_type);\n  }\n  return MakeBuffer(*dest_) && flush_ok;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_POSITION_SHIFTING_BACKWARD_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/position_shifting_reader.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/position_shifting_reader.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid PositionShiftingReaderBase::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    Reader& src = *SrcReader();\n    SyncBuffer(src);\n  }\n  Reader::Done();\n}\n\nbool PositionShiftingReaderBase::FailUnderflow(Position new_pos,\n                                               Object& object) {\n  return object.Fail(absl::InvalidArgumentError(\n      absl::StrCat(\"PositionShiftingReader does not support \"\n                   \"seeking before the base position: \",\n                   new_pos, \" < \", base_pos_)));\n}\n\nabsl::Status PositionShiftingReaderBase::AnnotateStatusImpl(\n    absl::Status status) {\n  if (is_open()) {\n    Reader& src = *SrcReader();\n    SyncBuffer(src);\n    status = src.AnnotateStatus(std::move(status));\n    MakeBuffer(src);\n  }\n  // The status might have been annotated by `src` with the original position.\n  // Clarify that the current position is the relative position instead of\n  // delegating to `Reader::AnnotateStatusImpl()`.\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status PositionShiftingReaderBase::AnnotateOverSrc(absl::Status status) {\n  if (is_open() && base_pos_ > 0) {\n    return Annotate(status,\n                    absl::StrCat(\"with relative position at byte \", pos()));\n  }\n  return status;\n}\n\nbool PositionShiftingReaderBase::PullSlow(size_t min_length,\n                                          size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::PullSlow(): \"\n         \"enough data available, use Pull() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool pull_ok = src.Pull(min_length, recommended_length);\n  return MakeBuffer(src, min_length) && pull_ok;\n}\n\nbool PositionShiftingReaderBase::ReadSlow(size_t length, char* dest) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of Reader::ReadSlow(char*): \"\n         \"enough data available, use Read(char*) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool read_ok = src.Read(length, dest);\n  return MakeBuffer(src) && read_ok;\n}\n\nbool PositionShiftingReaderBase::ReadSlow(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"Chain size overflow\";\n  return ReadInternal(length, dest);\n}\n\nbool PositionShiftingReaderBase::ReadSlow(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"Cord size overflow\";\n  return ReadInternal(length, dest);\n}\n\ntemplate <typename Dest>\ninline bool PositionShiftingReaderBase::ReadInternal(size_t length,\n                                                     Dest& dest) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool read_ok = src.ReadAndAppend(length, dest);\n  return MakeBuffer(src) && read_ok;\n}\n\nbool PositionShiftingReaderBase::CopySlow(Position length, Writer& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool copy_ok = src.Copy(length, dest);\n  return MakeBuffer(src) && copy_ok;\n}\n\nbool PositionShiftingReaderBase::CopySlow(size_t length, BackwardWriter& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(BackwardWriter&): \"\n         \"enough data available, use Copy(BackwardWriter&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool copy_ok = src.Copy(length, dest);\n  return MakeBuffer(src) && copy_ok;\n}\n\nbool PositionShiftingReaderBase::ReadSomeSlow(size_t max_length, char* dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"nothing to read, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"some data available, use ReadSome(char*) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool read_ok = src.ReadSome(max_length, dest);\n  return MakeBuffer(src) && read_ok;\n}\n\nbool PositionShiftingReaderBase::CopySomeSlow(size_t max_length, Writer& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"nothing to read, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"some data available, use CopySome(Writer&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool copy_ok = src.CopySome(max_length, dest);\n  return MakeBuffer(src) && copy_ok;\n}\n\nvoid PositionShiftingReaderBase::ReadHintSlow(size_t min_length,\n                                              size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::ReadHintSlow(): \"\n         \"enough data available, use ReadHint() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  src.ReadHint(min_length, recommended_length);\n  MakeBuffer(src);\n}\n\nbool PositionShiftingReaderBase::ToleratesReadingAhead() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->ToleratesReadingAhead();\n}\n\nbool PositionShiftingReaderBase::SupportsRandomAccess() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRandomAccess();\n}\n\nbool PositionShiftingReaderBase::SupportsRewind() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRewind();\n}\n\nbool PositionShiftingReaderBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of Reader::SeekSlow(): \"\n         \"position in the buffer, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(new_pos < base_pos_)) {\n    return FailUnderflow(new_pos, *this);\n  }\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool seek_ok = src.Seek(new_pos - base_pos_);\n  return MakeBuffer(src) && seek_ok;\n}\n\nbool PositionShiftingReaderBase::SupportsSize() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsSize();\n}\n\nstd::optional<Position> PositionShiftingReaderBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const std::optional<Position> size = src.Size();\n  if (ABSL_PREDICT_FALSE(!MakeBuffer(src) || size == std::nullopt)) {\n    return std::nullopt;\n  }\n  if (ABSL_PREDICT_FALSE(*size >\n                         std::numeric_limits<Position>::max() - base_pos_)) {\n    FailOverflow();\n    return std::nullopt;\n  }\n  return *size + base_pos_;\n}\n\nbool PositionShiftingReaderBase::SupportsNewReader() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsNewReader();\n}\n\nstd::unique_ptr<Reader> PositionShiftingReaderBase::NewReaderImpl(\n    Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point\n  // if `SrcReader()->SupportsNewReader()`.\n  Reader& src = *SrcReader();\n  std::unique_ptr<Reader> base_reader =\n      src.NewReader(SaturatingSub(initial_pos, base_pos_));\n  if (ABSL_PREDICT_FALSE(base_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    return nullptr;\n  }\n  std::unique_ptr<Reader> reader =\n      std::make_unique<PositionShiftingReader<std::unique_ptr<Reader>>>(\n          std::move(base_reader),\n          PositionShiftingReaderBase::Options().set_base_pos(base_pos_));\n  if (ABSL_PREDICT_FALSE(initial_pos < base_pos_)) {\n    FailUnderflow(initial_pos, *reader);\n  }\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/position_shifting_reader.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_POSITION_SHIFTING_READER_H_\n#define RIEGELI_BYTES_POSITION_SHIFTING_READER_H_\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nclass BackwardWriter;\nclass Writer;\n\n// Template parameter independent part of `PositionShiftingReader`.\nclass PositionShiftingReaderBase : public Reader {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // The base position of the new `Reader`.\n    //\n    // Default: 0.\n    Options& set_base_pos(Position base_pos) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      base_pos_ = base_pos;\n      return *this;\n    }\n    Options&& set_base_pos(Position base_pos) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_base_pos(base_pos));\n    }\n    Position base_pos() const { return base_pos_; }\n\n   private:\n    Position base_pos_ = 0;\n  };\n\n  // Returns the original `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the base position of the new `Reader`.\n  Position base_pos() const { return base_pos_; }\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRandomAccess() override;\n  bool SupportsRewind() override;\n  bool SupportsSize() override;\n  bool SupportsNewReader() override;\n\n protected:\n  explicit PositionShiftingReaderBase(Closed) noexcept : Reader(kClosed) {}\n\n  explicit PositionShiftingReaderBase(Position base_pos);\n\n  PositionShiftingReaderBase(PositionShiftingReaderBase&& that) noexcept;\n  PositionShiftingReaderBase& operator=(\n      PositionShiftingReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(Position base_pos);\n  void Initialize(Reader* src);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  // Sets cursor of `src` to cursor of `*this`.\n  void SyncBuffer(Reader& src);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `src`. Fails `*this`\n  // if `src` failed or there is not enough `Position` space for `min_length`.\n  bool MakeBuffer(Reader& src, size_t min_length = 0);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PullSlow(size_t min_length, size_t recommended_length) override;\n  using Reader::ReadSlow;\n  bool ReadSlow(size_t length, char* dest) override;\n  bool ReadSlow(size_t length, Chain& dest) override;\n  bool ReadSlow(size_t length, absl::Cord& dest) override;\n  using Reader::CopySlow;\n  bool CopySlow(Position length, Writer& dest) override;\n  bool CopySlow(size_t length, BackwardWriter& dest) override;\n  using Reader::ReadSomeSlow;\n  bool ReadSomeSlow(size_t max_length, char* dest) override;\n  using Reader::CopySomeSlow;\n  bool CopySomeSlow(size_t max_length, Writer& dest) override;\n  void ReadHintSlow(size_t min_length, size_t recommended_length) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  ABSL_ATTRIBUTE_COLD bool FailUnderflow(Position new_pos, Object& object);\n\n  // This template is defined and used only in position_shifting_reader.cc.\n  template <typename Dest>\n  bool ReadInternal(size_t length, Dest& dest);\n\n  Position base_pos_ = 0;\n\n  // Invariants if `ok()`:\n  //   `start() == SrcReader()->start()`\n  //   `limit() == SrcReader()->limit()`\n  //   `start_pos() == SrcReader()->start_pos() + base_pos_`\n};\n\n// A `Reader` which reads from another `Reader`, reporting positions shifted so\n// that the beginning appears as the given base position. Seeking back before\n// the base position fails.\n//\n// `PrefixLimitingReader` can be used for shifting positions in the other\n// direction.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the original `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The original `Reader` must not be accessed until the `PositionShiftingReader`\n// is closed or no longer used.\ntemplate <typename Src = Reader*>\nclass PositionShiftingReader : public PositionShiftingReaderBase {\n public:\n  // Creates a closed `PositionShiftingReader`.\n  explicit PositionShiftingReader(Closed) noexcept\n      : PositionShiftingReaderBase(kClosed) {}\n\n  // Will read from the original `Reader` provided by `src`.\n  explicit PositionShiftingReader(Initializer<Src> src,\n                                  Options options = Options());\n\n  PositionShiftingReader(PositionShiftingReader&& that) = default;\n  PositionShiftingReader& operator=(PositionShiftingReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `PositionShiftingReader`.\n  // This avoids constructing a temporary `PositionShiftingReader` and moving\n  // from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the original `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  void VerifyEndImpl() override;\n  bool SyncImpl(SyncType sync_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the original `Reader`.\n  MovingDependency<Reader*, Src, Mover> src_;\n};\n\nexplicit PositionShiftingReader(Closed)\n    -> PositionShiftingReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit PositionShiftingReader(Src&& src,\n                                PositionShiftingReaderBase::Options options =\n                                    PositionShiftingReaderBase::Options())\n    -> PositionShiftingReader<TargetT<Src>>;\n\n// Implementation details follow.\n\ninline PositionShiftingReaderBase::PositionShiftingReaderBase(Position base_pos)\n    : base_pos_(base_pos) {}\n\ninline PositionShiftingReaderBase::PositionShiftingReaderBase(\n    PositionShiftingReaderBase&& that) noexcept\n    : Reader(static_cast<Reader&&>(that)), base_pos_(that.base_pos_) {}\n\ninline PositionShiftingReaderBase& PositionShiftingReaderBase::operator=(\n    PositionShiftingReaderBase&& that) noexcept {\n  Reader::operator=(static_cast<Reader&&>(that));\n  base_pos_ = that.base_pos_;\n  return *this;\n}\n\ninline void PositionShiftingReaderBase::Reset(Closed) {\n  Reader::Reset(kClosed);\n  base_pos_ = 0;\n}\n\ninline void PositionShiftingReaderBase::Reset(Position base_pos) {\n  Reader::Reset();\n  base_pos_ = base_pos;\n}\n\ninline void PositionShiftingReaderBase::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of PositionShiftingReader: null Reader pointer\";\n  MakeBuffer(*src);\n}\n\ninline void PositionShiftingReaderBase::SyncBuffer(Reader& src) {\n  src.set_cursor(cursor());\n}\n\ninline bool PositionShiftingReaderBase::MakeBuffer(Reader& src,\n                                                   size_t min_length) {\n  const Position max_pos = std::numeric_limits<Position>::max() - base_pos_;\n  if (ABSL_PREDICT_FALSE(src.limit_pos() > max_pos)) {\n    if (ABSL_PREDICT_FALSE(src.pos() > max_pos)) {\n      set_buffer(src.cursor());\n      set_limit_pos(std::numeric_limits<Position>::max());\n      return FailOverflow();\n    }\n    set_buffer(src.start(), IntCast<size_t>(max_pos - src.start_pos()),\n               src.start_to_cursor());\n    set_limit_pos(std::numeric_limits<Position>::max());\n    if (ABSL_PREDICT_FALSE(available() < min_length)) return FailOverflow();\n  } else {\n    set_buffer(src.start(), src.start_to_limit(), src.start_to_cursor());\n    set_limit_pos(src.limit_pos() + base_pos_);\n  }\n  if (ABSL_PREDICT_FALSE(!src.ok())) {\n    return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n  }\n  return true;\n}\n\ntemplate <typename Src>\ninline PositionShiftingReader<Src>::PositionShiftingReader(Initializer<Src> src,\n                                                           Options options)\n    : PositionShiftingReaderBase(options.base_pos()), src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void PositionShiftingReader<Src>::Reset(Closed) {\n  PositionShiftingReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void PositionShiftingReader<Src>::Reset(Initializer<Src> src,\n                                               Options options) {\n  PositionShiftingReaderBase::Reset(options.base_pos());\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\nclass PositionShiftingReader<Src>::Mover {\n public:\n  static auto member() { return &PositionShiftingReader::src_; }\n\n  explicit Mover(PositionShiftingReader& self, PositionShiftingReader& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `src_` is not moved yet so `src_` is taken from `that`.\n    if (uses_buffer_) self.SyncBuffer(*that.src_);\n  }\n\n  void Done(PositionShiftingReader& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.src_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Src>\nvoid PositionShiftingReader<Src>::Done() {\n  PositionShiftingReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(AnnotateOverSrc(src_->status()));\n    }\n  }\n}\n\ntemplate <typename Src>\nvoid PositionShiftingReader<Src>::SetReadAllHintImpl(bool read_all_hint) {\n  if (src_.IsOwning()) {\n    SyncBuffer(*src_);\n    src_->SetReadAllHint(read_all_hint);\n    MakeBuffer(*src_);\n  }\n}\n\ntemplate <typename Src>\nvoid PositionShiftingReader<Src>::VerifyEndImpl() {\n  if (!src_.IsOwning()) {\n    PositionShiftingReaderBase::VerifyEndImpl();\n  } else if (ABSL_PREDICT_TRUE(ok())) {\n    SyncBuffer(*src_);\n    src_->VerifyEnd();\n    MakeBuffer(*src_);\n  }\n}\n\ntemplate <typename Src>\nbool PositionShiftingReader<Src>::SyncImpl(SyncType sync_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer(*src_);\n  bool sync_ok = true;\n  if (sync_type != SyncType::kFromObject || src_.IsOwning()) {\n    sync_ok = src_->Sync(sync_type);\n  }\n  return MakeBuffer(*src_) && sync_ok;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_POSITION_SHIFTING_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/position_shifting_writer.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/position_shifting_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/position_shifting_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid PositionShiftingWriterBase::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    Writer& dest = *DestWriter();\n    SyncBuffer(dest);\n  }\n  Writer::Done();\n  associated_reader_.Reset();\n}\n\nbool PositionShiftingWriterBase::FailUnderflow(absl::string_view operation,\n                                               Position new_pos,\n                                               Object& object) {\n  return object.Fail(absl::InvalidArgumentError(\n      absl::StrCat(\"PositionShiftingWriter does not support \", operation,\n                   \" before the base position: \", new_pos, \" < \", base_pos_)));\n}\n\nabsl::Status PositionShiftingWriterBase::AnnotateStatusImpl(\n    absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    SyncBuffer(dest);\n    status = dest.AnnotateStatus(std::move(status));\n    MakeBuffer(dest);\n  }\n  // The status might have been annotated by `dest` with the original position.\n  // Clarify that the current position is the relative position instead of\n  // delegating to `Writer::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status PositionShiftingWriterBase::AnnotateOverDest(absl::Status status) {\n  if (is_open() && base_pos_ > 0) {\n    return Annotate(status,\n                    absl::StrCat(\"with relative position at byte \", pos()));\n  }\n  return status;\n}\n\nbool PositionShiftingWriterBase::PushSlow(size_t min_length,\n                                          size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool push_ok = dest.Push(min_length, recommended_length);\n  return MakeBuffer(dest, min_length) && push_ok;\n}\n\nbool PositionShiftingWriterBase::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of Writer::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  return WriteInternal(src);\n}\n\nbool PositionShiftingWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool PositionShiftingWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  return WriteInternal(src);\n}\n\nbool PositionShiftingWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool PositionShiftingWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  return WriteInternal(src);\n}\n\nbool PositionShiftingWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool PositionShiftingWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  return WriteInternal(src);\n}\n\ntemplate <typename Src>\ninline bool PositionShiftingWriterBase::WriteInternal(Src&& src) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool write_ok = dest.Write(std::forward<Src>(src));\n  return MakeBuffer(dest) && write_ok;\n}\n\nbool PositionShiftingWriterBase::SupportsRandomAccess() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsRandomAccess();\n}\n\nbool PositionShiftingWriterBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of Writer::SeekSlow(): \"\n         \"position unchanged, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(new_pos < base_pos_)) {\n    return FailUnderflow(\"seeking\", new_pos, *this);\n  }\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool seek_ok = dest.Seek(new_pos - base_pos_);\n  return MakeBuffer(dest) && seek_ok;\n}\n\nstd::optional<Position> PositionShiftingWriterBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  const std::optional<Position> size = dest.Size();\n  if (ABSL_PREDICT_FALSE(!MakeBuffer(dest) || size == std::nullopt)) {\n    return std::nullopt;\n  }\n  if (ABSL_PREDICT_FALSE(*size >\n                         std::numeric_limits<Position>::max() - base_pos_)) {\n    FailOverflow();\n    return std::nullopt;\n  }\n  return *size + base_pos_;\n}\n\nbool PositionShiftingWriterBase::SupportsTruncate() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsTruncate();\n}\n\nbool PositionShiftingWriterBase::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(new_size < base_pos_)) {\n    return FailUnderflow(\"truncating\", new_size, *this);\n  }\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool truncate_ok = dest.Truncate(new_size - base_pos_);\n  return MakeBuffer(dest) && truncate_ok;\n}\n\nbool PositionShiftingWriterBase::SupportsReadMode() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsReadMode();\n}\n\nReader* PositionShiftingWriterBase::ReadModeImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  Reader* const base_reader =\n      dest.ReadMode(SaturatingSub(initial_pos, base_pos_));\n  if (ABSL_PREDICT_FALSE(!MakeBuffer(dest) || base_reader == nullptr)) {\n    return nullptr;\n  }\n  PositionShiftingReader<>* const reader = associated_reader_.ResetReader(\n      base_reader,\n      PositionShiftingReaderBase::Options().set_base_pos(base_pos_));\n  if (ABSL_PREDICT_FALSE(initial_pos < base_pos_)) {\n    FailUnderflow(\"seeking\", initial_pos, *reader);\n  }\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/position_shifting_writer.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_POSITION_SHIFTING_WRITER_H_\n#define RIEGELI_BYTES_POSITION_SHIFTING_WRITER_H_\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\ntemplate <typename Src>\nclass PositionShiftingReader;\nclass Reader;\n\n// Template parameter independent part of `PositionShiftingWriter`.\nclass PositionShiftingWriterBase : public Writer {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // The base position of the new `Writer`.\n    //\n    // Default: 0.\n    Options& set_base_pos(Position base_pos) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      base_pos_ = base_pos;\n      return *this;\n    }\n    Options&& set_base_pos(Position base_pos) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_base_pos(base_pos));\n    }\n    Position base_pos() const { return base_pos_; }\n\n   private:\n    Position base_pos_ = 0;\n  };\n\n  // Returns the new `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const = 0;\n\n  // Returns the base position of the original `Writer`.\n  Position base_pos() const { return base_pos_; }\n\n  bool SupportsRandomAccess() override;\n  bool SupportsTruncate() override;\n  bool SupportsReadMode() override;\n\n protected:\n  explicit PositionShiftingWriterBase(Closed) noexcept : Writer(kClosed) {}\n\n  explicit PositionShiftingWriterBase(Position base_pos);\n\n  PositionShiftingWriterBase(PositionShiftingWriterBase&& that) noexcept;\n  PositionShiftingWriterBase& operator=(\n      PositionShiftingWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(Position base_pos);\n  void Initialize(Writer* dest);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  // Sets cursor of `dest` to cursor of `*this`.\n  void SyncBuffer(Writer& dest);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `dest`, adjusting\n  // `start()` to hide data already written. Fails `*this` if `dest` failed\n  // or there is not enough `Position` space for `min_length`.\n  bool MakeBuffer(Writer& dest, size_t min_length = 0);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  bool TruncateImpl(Position new_size) override;\n  Reader* ReadModeImpl(Position initial_pos) override;\n\n private:\n  ABSL_ATTRIBUTE_COLD bool FailUnderflow(absl::string_view operation,\n                                         Position new_pos, Object& object);\n\n  // This template is defined and used only in position_shifting_writer.cc.\n  template <typename Src>\n  bool WriteInternal(Src&& src);\n\n  Position base_pos_ = 0;\n\n  AssociatedReader<PositionShiftingReader<Reader*>> associated_reader_;\n\n  // Invariants if `ok()`:\n  //   `start() == DestWriter()->cursor()`\n  //   `limit() == DestWriter()->limit()`\n  //   `start_pos() == DestWriter()->pos() + base_pos_`\n};\n\n// A `Writer` which writes to another `Writer`, reporting positions shifted so\n// that the beginning appears as the given base position. Seeking back before\n// the base position fails.\n//\n// `PrefixLimitingWriter` can be used for shifting positions in the other\n// direction.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the original `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The original `Writer` must not be accessed until the `PositionShiftingWriter`\n// is closed or no longer used, except that it is allowed to read the\n// destination of the original `Writer` immediately after `Flush()`.\ntemplate <typename Dest = Writer*>\nclass PositionShiftingWriter : public PositionShiftingWriterBase {\n public:\n  // Creates a closed `PositionShiftingWriter`.\n  explicit PositionShiftingWriter(Closed) noexcept\n      : PositionShiftingWriterBase(kClosed) {}\n\n  // Will write to the original `Writer` provided by `dest`.\n  explicit PositionShiftingWriter(Initializer<Dest> dest,\n                                  Options options = Options());\n\n  PositionShiftingWriter(PositionShiftingWriter&& that) = default;\n  PositionShiftingWriter& operator=(PositionShiftingWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `PositionShiftingWriter`.\n  // This avoids constructing a temporary `PositionShiftingWriter` and moving\n  // from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the original `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the original `Writer`.\n  MovingDependency<Writer*, Dest, Mover> dest_;\n};\n\nexplicit PositionShiftingWriter(Closed)\n    -> PositionShiftingWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit PositionShiftingWriter(Dest&& dest,\n                                PositionShiftingWriterBase::Options options =\n                                    PositionShiftingWriterBase::Options())\n    -> PositionShiftingWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline PositionShiftingWriterBase::PositionShiftingWriterBase(Position base_pos)\n    : base_pos_(base_pos) {}\n\ninline PositionShiftingWriterBase::PositionShiftingWriterBase(\n    PositionShiftingWriterBase&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)),\n      base_pos_(that.base_pos_),\n      associated_reader_(std::move(that.associated_reader_)) {}\n\ninline PositionShiftingWriterBase& PositionShiftingWriterBase::operator=(\n    PositionShiftingWriterBase&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  base_pos_ = that.base_pos_;\n  associated_reader_ = std::move(that.associated_reader_);\n  return *this;\n}\n\ninline void PositionShiftingWriterBase::Reset(Closed) {\n  Writer::Reset(kClosed);\n  base_pos_ = 0;\n  associated_reader_.Reset();\n}\n\ninline void PositionShiftingWriterBase::Reset(Position base_pos) {\n  Writer::Reset();\n  base_pos_ = base_pos;\n  associated_reader_.Reset();\n}\n\ninline void PositionShiftingWriterBase::Initialize(Writer* dest) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of PositionShiftingWriter: null Writer pointer\";\n  MakeBuffer(*dest);\n}\n\ninline void PositionShiftingWriterBase::SyncBuffer(Writer& dest) {\n  dest.set_cursor(cursor());\n}\n\ninline bool PositionShiftingWriterBase::MakeBuffer(Writer& dest,\n                                                   size_t min_length) {\n  const Position max_pos = std::numeric_limits<Position>::max() - base_pos_;\n  if (ABSL_PREDICT_FALSE(dest.limit_pos() > max_pos)) {\n    if (ABSL_PREDICT_FALSE(dest.pos() > max_pos)) {\n      set_buffer(dest.cursor());\n      set_start_pos(std::numeric_limits<Position>::max());\n      return FailOverflow();\n    }\n    set_buffer(dest.cursor(), IntCast<size_t>(max_pos - dest.pos()));\n    set_start_pos(dest.pos() + base_pos_);\n    if (ABSL_PREDICT_FALSE(available() < min_length)) return FailOverflow();\n  } else {\n    set_buffer(dest.cursor(), dest.available());\n    set_start_pos(dest.pos() + base_pos_);\n  }\n  if (ABSL_PREDICT_FALSE(!dest.ok())) {\n    return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n  return true;\n}\n\ntemplate <typename Dest>\nclass PositionShiftingWriter<Dest>::Mover {\n public:\n  static auto member() { return &PositionShiftingWriter::dest_; }\n\n  explicit Mover(PositionShiftingWriter& self, PositionShiftingWriter& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `dest_` is not moved yet so `dest_` is taken from `that`.\n    if (uses_buffer_) self.SyncBuffer(*that.dest_);\n  }\n\n  void Done(PositionShiftingWriter& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.dest_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Dest>\ninline PositionShiftingWriter<Dest>::PositionShiftingWriter(\n    Initializer<Dest> dest, Options options)\n    : PositionShiftingWriterBase(options.base_pos()), dest_(std::move(dest)) {\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\ninline void PositionShiftingWriter<Dest>::Reset(Closed) {\n  PositionShiftingWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void PositionShiftingWriter<Dest>::Reset(Initializer<Dest> dest,\n                                                Options options) {\n  PositionShiftingWriterBase::Reset(options.base_pos());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\nvoid PositionShiftingWriter<Dest>::Done() {\n  PositionShiftingWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <typename Dest>\nvoid PositionShiftingWriter<Dest>::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (dest_.IsOwning()) {\n    SyncBuffer(*dest_);\n    dest_->SetWriteSizeHint(\n        write_size_hint == std::nullopt\n            ? std::nullopt\n            : std::make_optional(SaturatingAdd(base_pos(), *write_size_hint)));\n    MakeBuffer(*dest_);\n  }\n}\n\ntemplate <typename Dest>\nbool PositionShiftingWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer(*dest_);\n  bool flush_ok = true;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    flush_ok = dest_->Flush(flush_type);\n  }\n  return MakeBuffer(*dest_) && flush_ok;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_POSITION_SHIFTING_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/prefix_limiting_backward_writer.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/prefix_limiting_backward_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\nvoid PrefixLimitingBackwardWriterBase::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    BackwardWriter& dest = *DestWriter();\n    SyncBuffer(dest);\n  }\n  BackwardWriter::Done();\n}\n\nabsl::Status PrefixLimitingBackwardWriterBase::AnnotateStatusImpl(\n    absl::Status status) {\n  if (is_open()) {\n    BackwardWriter& dest = *DestWriter();\n    SyncBuffer(dest);\n    status = dest.AnnotateStatus(std::move(status));\n    MakeBuffer(dest);\n  }\n  // The status might have been annotated by `dest` with the original position.\n  // Clarify that the current position is the relative position instead of\n  // delegating to `BackwardWriter::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status PrefixLimitingBackwardWriterBase::AnnotateOverDest(\n    absl::Status status) {\n  if (is_open() && base_pos_ > 0) {\n    return Annotate(status,\n                    absl::StrCat(\"with relative position at byte \", pos()));\n  }\n  return status;\n}\n\nbool PrefixLimitingBackwardWriterBase::PushSlow(size_t min_length,\n                                                size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of BackwardWriter::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  BackwardWriter& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool push_ok = dest.Push(min_length, recommended_length);\n  MakeBuffer(dest);\n  return push_ok;\n}\n\nbool PrefixLimitingBackwardWriterBase::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  return WriteInternal(src);\n}\n\nbool PrefixLimitingBackwardWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool PrefixLimitingBackwardWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  return WriteInternal(src);\n}\n\nbool PrefixLimitingBackwardWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool PrefixLimitingBackwardWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  return WriteInternal(src);\n}\n\nbool PrefixLimitingBackwardWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool PrefixLimitingBackwardWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  return WriteInternal(src);\n}\n\ntemplate <typename Src>\ninline bool PrefixLimitingBackwardWriterBase::WriteInternal(Src&& src) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  BackwardWriter& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool write_ok = dest.Write(std::forward<Src>(src));\n  MakeBuffer(dest);\n  return write_ok;\n}\n\nbool PrefixLimitingBackwardWriterBase::SupportsTruncate() {\n  BackwardWriter* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsTruncate();\n}\n\nbool PrefixLimitingBackwardWriterBase::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  BackwardWriter& dest = *DestWriter();\n  SyncBuffer(dest);\n  bool truncate_ok;\n  if (ABSL_PREDICT_FALSE(new_size >\n                         std::numeric_limits<Position>::max() - base_pos_)) {\n    truncate_ok = false;\n  } else {\n    truncate_ok = dest.Truncate(new_size + base_pos_);\n  }\n  MakeBuffer(dest);\n  return truncate_ok;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/prefix_limiting_backward_writer.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_PREFIX_LIMITING_BACKWARD_WRITER_H_\n#define RIEGELI_BYTES_PREFIX_LIMITING_BACKWARD_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\ntemplate <typename Src>\nclass PrefixLimitingReader;\nclass Reader;\n\n// Template parameter independent part of `PrefixLimitingBackwardWriter`.\nclass PrefixLimitingBackwardWriterBase : public BackwardWriter {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // The base position of the original `BackwardWriter`. It must be at least\n    // as large as the initial position.\n    //\n    // `std::nullopt` means the current position.\n    //\n    // Default: `std::nullopt`.\n    Options& set_base_pos(std::optional<Position> base_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      base_pos_ = base_pos;\n      return *this;\n    }\n    Options&& set_base_pos(std::optional<Position> base_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_base_pos(base_pos));\n    }\n    std::optional<Position> base_pos() const { return base_pos_; }\n\n   private:\n    std::optional<Position> base_pos_;\n  };\n\n  // Returns the original `BackwardWriter`. Unchanged by `Close()`.\n  virtual BackwardWriter* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the base position of the original `BackwardWriter`.\n  Position base_pos() const { return base_pos_; }\n  bool SupportsTruncate() override;\n\n protected:\n  using BackwardWriter::BackwardWriter;\n\n  PrefixLimitingBackwardWriterBase(\n      PrefixLimitingBackwardWriterBase&& that) noexcept;\n  PrefixLimitingBackwardWriterBase& operator=(\n      PrefixLimitingBackwardWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(BackwardWriter* dest, std::optional<Position> base_pos);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  // Sets cursor of `dest` to cursor of `*this`.\n  void SyncBuffer(BackwardWriter& dest);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `dest`, adjusting\n  // `start()` to hide data already written. Fails `*this` if `dest` failed.\n  void MakeBuffer(BackwardWriter& dest);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using BackwardWriter::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool TruncateImpl(Position new_size) override;\n\n private:\n  // This template is defined and used only in prefix_limiting_writer.cc.\n  template <typename Src>\n  bool WriteInternal(Src&& src);\n\n  Position base_pos_ = 0;\n\n  // Invariants if `ok()`:\n  //   `start() == DestWriter()->cursor()`\n  //   `limit() == DestWriter()->limit()`\n  //   `start_pos() == DestWriter()->pos() - base_pos_`\n};\n\n// A `BackwardWriter` which writes to another `BackwardWriter`, hiding data\n// before a base position, and reporting positions shifted so that the base\n// position appears as 0.\n//\n// `PositionShiftingBackwardWriter` can be used for shifting positions in the\n// other direction.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the original `BackwardWriter`. `Dest` must support\n// `Dependency<BackwardWriter*, Dest>`, e.g.\n// `BackwardWriter*` (not owned, default), `ChainBackwardWriter<>` (owned),\n// `std::unique_ptr<BackwardWriter>` (owned),\n// `Any<BackwardWriter*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The original `BackwardWriter` must not be accessed until the\n// `PrefixLimitingBackwardWriter` is closed or no longer used, except that it is\n// allowed to read the destination of the original `BackwardWriter` immediately\n// after `Flush()`.\ntemplate <typename Dest = BackwardWriter*>\nclass PrefixLimitingBackwardWriter : public PrefixLimitingBackwardWriterBase {\n public:\n  // Creates a closed `PrefixLimitingBackwardWriter`.\n  explicit PrefixLimitingBackwardWriter(Closed) noexcept\n      : PrefixLimitingBackwardWriterBase(kClosed) {}\n\n  // Will write to the original `BackwardWriter` provided by `dest`.\n  explicit PrefixLimitingBackwardWriter(Initializer<Dest> dest,\n                                        Options options = Options());\n\n  PrefixLimitingBackwardWriter(PrefixLimitingBackwardWriter&& that) = default;\n  PrefixLimitingBackwardWriter& operator=(PrefixLimitingBackwardWriter&& that) =\n      default;\n\n  // Makes `*this` equivalent to a newly constructed\n  // `PrefixLimitingBackwardWriter`. This avoids constructing a temporary\n  // `PrefixLimitingBackwardWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the original\n  // `BackwardWriter`. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  BackwardWriter* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the original `BackwardWriter`.\n  MovingDependency<BackwardWriter*, Dest, Mover> dest_;\n};\n\nexplicit PrefixLimitingBackwardWriter(Closed)\n    -> PrefixLimitingBackwardWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit PrefixLimitingBackwardWriter(\n    Dest&& dest, PrefixLimitingBackwardWriterBase::Options options =\n                     PrefixLimitingBackwardWriterBase::Options())\n    -> PrefixLimitingBackwardWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline PrefixLimitingBackwardWriterBase::PrefixLimitingBackwardWriterBase(\n    PrefixLimitingBackwardWriterBase&& that) noexcept\n    : BackwardWriter(static_cast<BackwardWriter&&>(that)),\n      base_pos_(that.base_pos_) {}\n\ninline PrefixLimitingBackwardWriterBase&\nPrefixLimitingBackwardWriterBase::operator=(\n    PrefixLimitingBackwardWriterBase&& that) noexcept {\n  BackwardWriter::operator=(static_cast<BackwardWriter&&>(that));\n  base_pos_ = that.base_pos_;\n  return *this;\n}\n\ninline void PrefixLimitingBackwardWriterBase::Reset(Closed) {\n  BackwardWriter::Reset(kClosed);\n  base_pos_ = 0;\n}\n\ninline void PrefixLimitingBackwardWriterBase::Reset() {\n  BackwardWriter::Reset();\n  // `base_pos_` will be set by `Initialize()`.\n}\n\ninline void PrefixLimitingBackwardWriterBase::Initialize(\n    BackwardWriter* dest, std::optional<Position> base_pos) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of PrefixLimitingBackwardWriter: \"\n         \"null BackwardWriter pointer\";\n  if (base_pos == std::nullopt) {\n    base_pos_ = dest->pos();\n  } else {\n    RIEGELI_ASSERT_LE(*base_pos, dest->pos())\n        << \"Failed precondition of PrefixLimitingBackwardWriter: \"\n           \"current position below the base position\";\n    base_pos_ = *base_pos;\n  }\n  MakeBuffer(*dest);\n}\n\ninline void PrefixLimitingBackwardWriterBase::SyncBuffer(BackwardWriter& dest) {\n  dest.set_cursor(cursor());\n}\n\ninline void PrefixLimitingBackwardWriterBase::MakeBuffer(BackwardWriter& dest) {\n  RIEGELI_ASSERT_GE(dest.pos(), base_pos_)\n      << \"PrefixLimitingBackwardWriter destination \"\n         \"changed position unexpectedly\";\n  set_buffer(dest.limit(), dest.available());\n  set_start_pos(dest.pos() - base_pos_);\n  if (ABSL_PREDICT_FALSE(!dest.ok())) {\n    FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n}\n\ntemplate <typename Dest>\nclass PrefixLimitingBackwardWriter<Dest>::Mover {\n public:\n  static auto member() { return &PrefixLimitingBackwardWriter::dest_; }\n\n  explicit Mover(PrefixLimitingBackwardWriter& self,\n                 PrefixLimitingBackwardWriter& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `dest_` is not moved yet so `dest_` is taken from `that`.\n    if (uses_buffer_) self.SyncBuffer(*that.dest_);\n  }\n\n  void Done(PrefixLimitingBackwardWriter& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.dest_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Dest>\ninline PrefixLimitingBackwardWriter<Dest>::PrefixLimitingBackwardWriter(\n    Initializer<Dest> dest, Options options)\n    : dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.base_pos());\n}\n\ntemplate <typename Dest>\ninline void PrefixLimitingBackwardWriter<Dest>::Reset(Closed) {\n  PrefixLimitingBackwardWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void PrefixLimitingBackwardWriter<Dest>::Reset(Initializer<Dest> dest,\n                                                      Options options) {\n  PrefixLimitingBackwardWriterBase::Reset();\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.base_pos());\n}\n\ntemplate <typename Dest>\nvoid PrefixLimitingBackwardWriter<Dest>::Done() {\n  PrefixLimitingBackwardWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <typename Dest>\nvoid PrefixLimitingBackwardWriter<Dest>::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (dest_.IsOwning()) {\n    SyncBuffer(*dest_);\n    dest_->SetWriteSizeHint(\n        write_size_hint == std::nullopt\n            ? std::nullopt\n            : std::make_optional(SaturatingAdd(base_pos(), *write_size_hint)));\n    MakeBuffer(*dest_);\n  }\n}\n\ntemplate <typename Dest>\nbool PrefixLimitingBackwardWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer(*dest_);\n  bool flush_ok = true;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    flush_ok = dest_->Flush(flush_type);\n  }\n  MakeBuffer(*dest_);\n  return flush_ok;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_PREFIX_LIMITING_BACKWARD_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/prefix_limiting_reader.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/prefix_limiting_reader.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid PrefixLimitingReaderBase::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    Reader& src = *SrcReader();\n    SyncBuffer(src);\n  }\n  Reader::Done();\n}\n\nabsl::Status PrefixLimitingReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Reader& src = *SrcReader();\n    SyncBuffer(src);\n    status = src.AnnotateStatus(std::move(status));\n    MakeBuffer(src);\n  }\n  // The status might have been annotated by `src` with the original position.\n  // Clarify that the current position is the relative position instead of\n  // delegating to `Reader::AnnotateStatusImpl()`.\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status PrefixLimitingReaderBase::AnnotateOverSrc(absl::Status status) {\n  if (is_open() && base_pos_ > 0) {\n    return Annotate(status,\n                    absl::StrCat(\"with relative position at byte \", pos()));\n  }\n  return status;\n}\n\nbool PrefixLimitingReaderBase::PullSlow(size_t min_length,\n                                        size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::PullSlow(): \"\n         \"enough data available, use Pull() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool pull_ok = src.Pull(min_length, recommended_length);\n  MakeBuffer(src);\n  return pull_ok;\n}\n\nbool PrefixLimitingReaderBase::ReadSlow(size_t length, char* dest) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of Reader::ReadSlow(char*): \"\n         \"enough data available, use Read(char*) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool read_ok = src.Read(length, dest);\n  MakeBuffer(src);\n  return read_ok;\n}\n\nbool PrefixLimitingReaderBase::ReadSlow(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"Chain size overflow\";\n  return ReadInternal(length, dest);\n}\n\nbool PrefixLimitingReaderBase::ReadSlow(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"Cord size overflow\";\n  return ReadInternal(length, dest);\n}\n\ntemplate <typename Dest>\ninline bool PrefixLimitingReaderBase::ReadInternal(size_t length, Dest& dest) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool read_ok = src.ReadAndAppend(length, dest);\n  MakeBuffer(src);\n  return read_ok;\n}\n\nbool PrefixLimitingReaderBase::CopySlow(Position length, Writer& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool copy_ok = src.Copy(length, dest);\n  MakeBuffer(src);\n  return copy_ok;\n}\n\nbool PrefixLimitingReaderBase::CopySlow(size_t length, BackwardWriter& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(BackwardWriter&): \"\n         \"enough data available, use Copy(BackwardWriter&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool copy_ok = src.Copy(length, dest);\n  MakeBuffer(src);\n  return copy_ok;\n}\n\nbool PrefixLimitingReaderBase::ReadSomeSlow(size_t max_length, char* dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"nothing to read, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"some data available, use ReadSome(char*) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool read_ok = src.ReadSome(max_length, dest);\n  MakeBuffer(src);\n  return read_ok;\n}\n\nbool PrefixLimitingReaderBase::CopySomeSlow(size_t max_length, Writer& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"nothing to copy, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"some data available, use CopySome(Writer&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool copy_ok = src.CopySome(max_length, dest);\n  MakeBuffer(src);\n  return copy_ok;\n}\n\nvoid PrefixLimitingReaderBase::ReadHintSlow(size_t min_length,\n                                            size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::ReadHintSlow(): \"\n         \"enough data available, use ReadHint() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  src.ReadHint(min_length, recommended_length);\n  MakeBuffer(src);\n}\n\nbool PrefixLimitingReaderBase::ToleratesReadingAhead() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->ToleratesReadingAhead();\n}\n\nbool PrefixLimitingReaderBase::SupportsRandomAccess() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRandomAccess();\n}\n\nbool PrefixLimitingReaderBase::SupportsRewind() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRewind();\n}\n\nbool PrefixLimitingReaderBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of Reader::SeekSlow(): \"\n         \"position in the buffer, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  bool seek_ok;\n  if (ABSL_PREDICT_FALSE(new_pos >\n                         std::numeric_limits<Position>::max() - base_pos_)) {\n    src.Seek(std::numeric_limits<Position>::max());\n    seek_ok = false;\n  } else {\n    seek_ok = src.Seek(new_pos + base_pos_);\n  }\n  MakeBuffer(src);\n  return seek_ok;\n}\n\nbool PrefixLimitingReaderBase::SupportsSize() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsSize();\n}\n\nstd::optional<Position> PrefixLimitingReaderBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const std::optional<Position> size = src.Size();\n  MakeBuffer(src);\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) return std::nullopt;\n  return SaturatingSub(*size, base_pos_);\n}\n\nbool PrefixLimitingReaderBase::SupportsNewReader() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsNewReader();\n}\n\nstd::unique_ptr<Reader> PrefixLimitingReaderBase::NewReaderImpl(\n    Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point\n  // if `SrcReader()->SupportsNewReader()`.\n  Reader& src = *SrcReader();\n  std::unique_ptr<Reader> reader =\n      src.NewReader(SaturatingAdd(initial_pos, base_pos_));\n  if (ABSL_PREDICT_FALSE(reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    return nullptr;\n  }\n  return std::make_unique<PrefixLimitingReader<std::unique_ptr<Reader>>>(\n      std::move(reader),\n      PrefixLimitingReaderBase::Options().set_base_pos(base_pos_));\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/prefix_limiting_reader.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_PREFIX_LIMITING_READER_H_\n#define RIEGELI_BYTES_PREFIX_LIMITING_READER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nclass BackwardWriter;\nclass Writer;\n\n// Template parameter independent part of `PrefixLimitingReader`.\nclass PrefixLimitingReaderBase : public Reader {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // The base position of the original `Reader`. It must be at least as large\n    // as the initial position.\n    //\n    // `std::nullopt` means the current position.\n    //\n    // Default: `std::nullopt`.\n    Options& set_base_pos(std::optional<Position> base_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      base_pos_ = base_pos;\n      return *this;\n    }\n    Options&& set_base_pos(std::optional<Position> base_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_base_pos(base_pos));\n    }\n    std::optional<Position> base_pos() const { return base_pos_; }\n\n   private:\n    std::optional<Position> base_pos_;\n  };\n\n  // Returns the original `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the base position of the original `Reader`.\n  Position base_pos() const { return base_pos_; }\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRandomAccess() override;\n  bool SupportsRewind() override;\n  bool SupportsSize() override;\n  bool SupportsNewReader() override;\n\n protected:\n  using Reader::Reader;\n\n  PrefixLimitingReaderBase(PrefixLimitingReaderBase&& that) noexcept;\n  PrefixLimitingReaderBase& operator=(PrefixLimitingReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(Reader* src, std::optional<Position> base_pos);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  // Sets cursor of `src` to cursor of `*this`.\n  void SyncBuffer(Reader& src);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `src`, adjusting\n  // `start()` to hide data already read. Fails `*this` if `src` failed.\n  void MakeBuffer(Reader& src);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PullSlow(size_t min_length, size_t recommended_length) override;\n  using Reader::ReadSlow;\n  bool ReadSlow(size_t length, char* dest) override;\n  bool ReadSlow(size_t length, Chain& dest) override;\n  bool ReadSlow(size_t length, absl::Cord& dest) override;\n  using Reader::CopySlow;\n  bool CopySlow(Position length, Writer& dest) override;\n  bool CopySlow(size_t length, BackwardWriter& dest) override;\n  using Reader::ReadSomeSlow;\n  bool ReadSomeSlow(size_t max_length, char* dest) override;\n  using Reader::CopySomeSlow;\n  bool CopySomeSlow(size_t max_length, Writer& dest) override;\n  void ReadHintSlow(size_t min_length, size_t recommended_length) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  // This template is defined and used only in prefix_limiting_reader.cc.\n  template <typename Dest>\n  bool ReadInternal(size_t length, Dest& dest);\n\n  Position base_pos_ = 0;\n\n  // Invariants if `is_open()`:\n  //   `start() >= SrcReader()->cursor()`\n  //   `limit() == SrcReader()->limit()`\n  //   `limit_pos() == SrcReader()->limit_pos() - base_pos_`\n};\n\n// A `Reader` which reads from another `Reader`, hiding data before a base\n// position, and reporting positions shifted so that the base position appears\n// as 0.\n//\n// `PositionShiftingReader` can be used for shifting positions in the other\n// direction.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the original `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The original `Reader` must not be accessed until the `PrefixLimitingReader`\n// is closed or no longer used.\ntemplate <typename Src = Reader*>\nclass PrefixLimitingReader : public PrefixLimitingReaderBase {\n public:\n  // Creates a closed `PrefixLimitingReader`.\n  explicit PrefixLimitingReader(Closed) noexcept\n      : PrefixLimitingReaderBase(kClosed) {}\n\n  // Will read from the original `Reader` provided by `src`.\n  explicit PrefixLimitingReader(Initializer<Src> src,\n                                Options options = Options());\n\n  PrefixLimitingReader(PrefixLimitingReader&& that) = default;\n  PrefixLimitingReader& operator=(PrefixLimitingReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `PrefixLimitingReader`.\n  // This avoids constructing a temporary `PrefixLimitingReader` and moving\n  // from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the original `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  void VerifyEndImpl() override;\n  bool SyncImpl(SyncType sync_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the original `Reader`.\n  MovingDependency<Reader*, Src, Mover> src_;\n};\n\nexplicit PrefixLimitingReader(Closed)\n    -> PrefixLimitingReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit PrefixLimitingReader(Src&& src,\n                              PrefixLimitingReaderBase::Options options =\n                                  PrefixLimitingReaderBase::Options())\n    -> PrefixLimitingReader<TargetT<Src>>;\n\n// Implementation details follow.\n\ninline PrefixLimitingReaderBase::PrefixLimitingReaderBase(\n    PrefixLimitingReaderBase&& that) noexcept\n    : Reader(static_cast<Reader&&>(that)), base_pos_(that.base_pos_) {}\n\ninline PrefixLimitingReaderBase& PrefixLimitingReaderBase::operator=(\n    PrefixLimitingReaderBase&& that) noexcept {\n  Reader::operator=(static_cast<Reader&&>(that));\n  base_pos_ = that.base_pos_;\n  return *this;\n}\n\ninline void PrefixLimitingReaderBase::Reset(Closed) {\n  Reader::Reset(kClosed);\n  base_pos_ = 0;\n}\n\ninline void PrefixLimitingReaderBase::Reset() {\n  Reader::Reset();\n  // `base_pos_` will be set by `Initialize()`.\n}\n\ninline void PrefixLimitingReaderBase::Initialize(\n    Reader* src, std::optional<Position> base_pos) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of PrefixLimitingReader: null Reader pointer\";\n  if (base_pos == std::nullopt) {\n    base_pos_ = src->pos();\n  } else {\n    RIEGELI_ASSERT_LE(*base_pos, src->pos())\n        << \"Failed precondition of PrefixLimitingReader: \"\n           \"current position below the base position\";\n    base_pos_ = *base_pos;\n  }\n  MakeBuffer(*src);\n}\n\ninline void PrefixLimitingReaderBase::SyncBuffer(Reader& src) {\n  src.set_cursor(cursor());\n}\n\ninline void PrefixLimitingReaderBase::MakeBuffer(Reader& src) {\n  RIEGELI_ASSERT_GE(src.pos(), base_pos_)\n      << \"PrefixLimitingReader source changed position unexpectedly\";\n  set_buffer(src.cursor(), src.available());\n  set_limit_pos(src.limit_pos() - base_pos_);\n  if (ABSL_PREDICT_FALSE(!src.ok())) {\n    FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n  }\n}\n\ntemplate <typename Src>\ninline PrefixLimitingReader<Src>::PrefixLimitingReader(Initializer<Src> src,\n                                                       Options options)\n    : src_(std::move(src)) {\n  Initialize(src_.get(), options.base_pos());\n}\n\ntemplate <typename Src>\nclass PrefixLimitingReader<Src>::Mover {\n public:\n  static auto member() { return &PrefixLimitingReader::src_; }\n\n  explicit Mover(PrefixLimitingReader& self, PrefixLimitingReader& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `src_` is not moved yet so `src_` is taken from `that`.\n    if (uses_buffer_) self.SyncBuffer(*that.src_);\n  }\n\n  void Done(PrefixLimitingReader& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.src_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Src>\ninline void PrefixLimitingReader<Src>::Reset(Closed) {\n  PrefixLimitingReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void PrefixLimitingReader<Src>::Reset(Initializer<Src> src,\n                                             Options options) {\n  PrefixLimitingReaderBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(src_.get(), options.base_pos());\n}\n\ntemplate <typename Src>\nvoid PrefixLimitingReader<Src>::Done() {\n  PrefixLimitingReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(AnnotateOverSrc(src_->status()));\n    }\n  }\n}\n\ntemplate <typename Src>\nvoid PrefixLimitingReader<Src>::SetReadAllHintImpl(bool read_all_hint) {\n  if (src_.IsOwning()) {\n    SyncBuffer(*src_);\n    src_->SetReadAllHint(read_all_hint);\n    MakeBuffer(*src_);\n  }\n}\n\ntemplate <typename Src>\nvoid PrefixLimitingReader<Src>::VerifyEndImpl() {\n  if (!src_.IsOwning()) {\n    PrefixLimitingReaderBase::VerifyEndImpl();\n  } else if (ABSL_PREDICT_TRUE(ok())) {\n    SyncBuffer(*src_);\n    src_->VerifyEnd();\n    MakeBuffer(*src_);\n  }\n}\n\ntemplate <typename Src>\nbool PrefixLimitingReader<Src>::SyncImpl(SyncType sync_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer(*src_);\n  bool sync_ok = true;\n  if (sync_type != SyncType::kFromObject || src_.IsOwning()) {\n    sync_ok = src_->Sync(sync_type);\n  }\n  MakeBuffer(*src_);\n  return sync_ok;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_PREFIX_LIMITING_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/prefix_limiting_writer.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/prefix_limiting_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/prefix_limiting_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid PrefixLimitingWriterBase::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    Writer& dest = *DestWriter();\n    SyncBuffer(dest);\n  }\n  Writer::Done();\n  associated_reader_.Reset();\n}\n\nabsl::Status PrefixLimitingWriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    SyncBuffer(dest);\n    status = dest.AnnotateStatus(std::move(status));\n    MakeBuffer(dest);\n  }\n  // The status might have been annotated by `dest` with the original position.\n  // Clarify that the current position is the relative position instead of\n  // delegating to `Writer::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status PrefixLimitingWriterBase::AnnotateOverDest(absl::Status status) {\n  if (is_open() && base_pos_ > 0) {\n    return Annotate(status,\n                    absl::StrCat(\"with relative position at byte \", pos()));\n  }\n  return status;\n}\n\nbool PrefixLimitingWriterBase::PushSlow(size_t min_length,\n                                        size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool push_ok = dest.Push(min_length, recommended_length);\n  MakeBuffer(dest);\n  return push_ok;\n}\n\nbool PrefixLimitingWriterBase::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of Writer::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  return WriteInternal(src);\n}\n\nbool PrefixLimitingWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool PrefixLimitingWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  return WriteInternal(src);\n}\n\nbool PrefixLimitingWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool PrefixLimitingWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  return WriteInternal(src);\n}\n\nbool PrefixLimitingWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool PrefixLimitingWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  return WriteInternal(src);\n}\n\ntemplate <typename Src>\ninline bool PrefixLimitingWriterBase::WriteInternal(Src&& src) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool write_ok = dest.Write(std::forward<Src>(src));\n  MakeBuffer(dest);\n  return write_ok;\n}\n\nbool PrefixLimitingWriterBase::SupportsRandomAccess() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsRandomAccess();\n}\n\nbool PrefixLimitingWriterBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of Writer::SeekSlow(): \"\n         \"position unchanged, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  bool seek_ok;\n  if (ABSL_PREDICT_FALSE(new_pos >\n                         std::numeric_limits<Position>::max() - base_pos_)) {\n    dest.Seek(std::numeric_limits<Position>::max());\n    seek_ok = false;\n  } else {\n    seek_ok = dest.Seek(new_pos + base_pos_);\n  }\n  MakeBuffer(dest);\n  return seek_ok;\n}\n\nstd::optional<Position> PrefixLimitingWriterBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  const std::optional<Position> size = dest.Size();\n  MakeBuffer(dest);\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) return std::nullopt;\n  return SaturatingSub(*size, base_pos_);\n}\n\nbool PrefixLimitingWriterBase::SupportsTruncate() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsTruncate();\n}\n\nbool PrefixLimitingWriterBase::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  bool truncate_ok;\n  if (ABSL_PREDICT_FALSE(new_size >\n                         std::numeric_limits<Position>::max() - base_pos_)) {\n    dest.Seek(std::numeric_limits<Position>::max());\n    truncate_ok = false;\n  } else {\n    truncate_ok = dest.Truncate(new_size + base_pos_);\n  }\n  MakeBuffer(dest);\n  return truncate_ok;\n}\n\nbool PrefixLimitingWriterBase::SupportsReadMode() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsReadMode();\n}\n\nReader* PrefixLimitingWriterBase::ReadModeImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  Reader* const reader = dest.ReadMode(SaturatingAdd(initial_pos, base_pos_));\n  MakeBuffer(dest);\n  if (ABSL_PREDICT_FALSE(reader == nullptr)) return nullptr;\n  return associated_reader_.ResetReader(\n      reader, PrefixLimitingReaderBase::Options().set_base_pos(base_pos_));\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/prefix_limiting_writer.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_PREFIX_LIMITING_WRITER_H_\n#define RIEGELI_BYTES_PREFIX_LIMITING_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\ntemplate <typename Src>\nclass PrefixLimitingReader;\nclass Reader;\n\n// Template parameter independent part of `PrefixLimitingWriter`.\nclass PrefixLimitingWriterBase : public Writer {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // The base position of the original `Writer`. It must be at least as large\n    // as the initial position.\n    //\n    // `std::nullopt` means the current position.\n    //\n    // Default: `std::nullopt`.\n    Options& set_base_pos(std::optional<Position> base_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      base_pos_ = base_pos;\n      return *this;\n    }\n    Options&& set_base_pos(std::optional<Position> base_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_base_pos(base_pos));\n    }\n    std::optional<Position> base_pos() const { return base_pos_; }\n\n   private:\n    std::optional<Position> base_pos_;\n  };\n\n  // Returns the original `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the base position of the original `Writer`.\n  Position base_pos() const { return base_pos_; }\n\n  bool SupportsRandomAccess() override;\n  bool SupportsTruncate() override;\n  bool SupportsReadMode() override;\n\n protected:\n  using Writer::Writer;\n\n  PrefixLimitingWriterBase(PrefixLimitingWriterBase&& that) noexcept;\n  PrefixLimitingWriterBase& operator=(PrefixLimitingWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(Writer* dest, std::optional<Position> base_pos);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  // Sets cursor of `dest` to cursor of `*this`.\n  void SyncBuffer(Writer& dest);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `dest`, adjusting\n  // `start()` to hide data already written. Fails `*this` if `dest` failed.\n  void MakeBuffer(Writer& dest);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  bool TruncateImpl(Position new_size) override;\n  Reader* ReadModeImpl(Position initial_pos) override;\n\n private:\n  // This template is defined and used only in prefix_limiting_writer.cc.\n  template <typename Src>\n  bool WriteInternal(Src&& src);\n\n  Position base_pos_ = 0;\n\n  AssociatedReader<PrefixLimitingReader<Reader*>> associated_reader_;\n\n  // Invariants if `ok()`:\n  //   `start() == DestWriter()->cursor()`\n  //   `limit() == DestWriter()->limit()`\n  //   `start_pos() == DestWriter()->pos() - base_pos_`\n};\n\n// A `Writer` which writes to another `Writer`, hiding data before a base\n// position, and reporting positions shifted so that the base position appears\n// as 0.\n//\n// `PositionShiftingWriter` can be used for shifting positions in the other\n// direction.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the original `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The original `Writer` must not be accessed until the `PrefixLimitingWriter`\n// is closed or no longer used, except that it is allowed to read the\n// destination of the original `Writer` immediately after `Flush()`.\ntemplate <typename Dest = Writer*>\nclass PrefixLimitingWriter : public PrefixLimitingWriterBase {\n public:\n  // Creates a closed `PrefixLimitingWriter`.\n  explicit PrefixLimitingWriter(Closed) noexcept\n      : PrefixLimitingWriterBase(kClosed) {}\n\n  // Will write to the original `Writer` provided by `dest`.\n  explicit PrefixLimitingWriter(Initializer<Dest> dest,\n                                Options options = Options());\n\n  PrefixLimitingWriter(PrefixLimitingWriter&& that) = default;\n  PrefixLimitingWriter& operator=(PrefixLimitingWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `PrefixLimitingWriter`.\n  // This avoids constructing a temporary `PrefixLimitingWriter` and moving\n  // from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the original `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the original `Writer`.\n  MovingDependency<Writer*, Dest, Mover> dest_;\n};\n\nexplicit PrefixLimitingWriter(Closed)\n    -> PrefixLimitingWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit PrefixLimitingWriter(Dest&& dest,\n                              PrefixLimitingWriterBase::Options options =\n                                  PrefixLimitingWriterBase::Options())\n    -> PrefixLimitingWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline PrefixLimitingWriterBase::PrefixLimitingWriterBase(\n    PrefixLimitingWriterBase&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)),\n      base_pos_(that.base_pos_),\n      associated_reader_(std::move(that.associated_reader_)) {}\n\ninline PrefixLimitingWriterBase& PrefixLimitingWriterBase::operator=(\n    PrefixLimitingWriterBase&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  base_pos_ = that.base_pos_;\n  associated_reader_ = std::move(that.associated_reader_);\n  return *this;\n}\n\ninline void PrefixLimitingWriterBase::Reset(Closed) {\n  Writer::Reset(kClosed);\n  base_pos_ = 0;\n  associated_reader_.Reset();\n}\n\ninline void PrefixLimitingWriterBase::Reset() {\n  Writer::Reset();\n  // `base_pos_` will be set by `Initialize()`.\n  associated_reader_.Reset();\n}\n\ninline void PrefixLimitingWriterBase::Initialize(\n    Writer* dest, std::optional<Position> base_pos) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of PrefixLimitingWriter: null Writer pointer\";\n  if (base_pos == std::nullopt) {\n    base_pos_ = dest->pos();\n  } else {\n    RIEGELI_ASSERT_LE(*base_pos, dest->pos())\n        << \"Failed precondition of PrefixLimitingWriter: \"\n           \"current position below the base position\";\n    base_pos_ = *base_pos;\n  }\n  MakeBuffer(*dest);\n}\n\ninline void PrefixLimitingWriterBase::SyncBuffer(Writer& dest) {\n  dest.set_cursor(cursor());\n}\n\ninline void PrefixLimitingWriterBase::MakeBuffer(Writer& dest) {\n  RIEGELI_ASSERT_GE(dest.pos(), base_pos_)\n      << \"PrefixLimitingWriter destination changed position unexpectedly\";\n  set_buffer(dest.cursor(), dest.available());\n  set_start_pos(dest.pos() - base_pos_);\n  if (ABSL_PREDICT_FALSE(!dest.ok())) {\n    FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n}\n\ntemplate <typename Dest>\nclass PrefixLimitingWriter<Dest>::Mover {\n public:\n  static auto member() { return &PrefixLimitingWriter::dest_; }\n\n  explicit Mover(PrefixLimitingWriter& self, PrefixLimitingWriter& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `dest_` is not moved yet so `dest_` is taken from `that`.\n    if (uses_buffer_) self.SyncBuffer(*that.dest_);\n  }\n\n  void Done(PrefixLimitingWriter& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.dest_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Dest>\ninline PrefixLimitingWriter<Dest>::PrefixLimitingWriter(Initializer<Dest> dest,\n                                                        Options options)\n    : dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.base_pos());\n}\n\ntemplate <typename Dest>\ninline void PrefixLimitingWriter<Dest>::Reset(Closed) {\n  PrefixLimitingWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void PrefixLimitingWriter<Dest>::Reset(Initializer<Dest> dest,\n                                              Options options) {\n  PrefixLimitingWriterBase::Reset();\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.base_pos());\n}\n\ntemplate <typename Dest>\nvoid PrefixLimitingWriter<Dest>::Done() {\n  PrefixLimitingWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <typename Dest>\nvoid PrefixLimitingWriter<Dest>::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (dest_.IsOwning()) {\n    SyncBuffer(*dest_);\n    dest_->SetWriteSizeHint(\n        write_size_hint == std::nullopt\n            ? std::nullopt\n            : std::make_optional(SaturatingAdd(base_pos(), *write_size_hint)));\n    MakeBuffer(*dest_);\n  }\n}\n\ntemplate <typename Dest>\nbool PrefixLimitingWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer(*dest_);\n  bool flush_ok = true;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    flush_ok = dest_->Flush(flush_type);\n  }\n  MakeBuffer(*dest_);\n  return flush_ok;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_PREFIX_LIMITING_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/pullable_reader.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/pullable_reader.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <limits>\n#include <memory>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/cord_buffer.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_utils.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/sized_shared_buffer.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid PullableReader::Done() {\n  if (ABSL_PREDICT_FALSE(scratch_used()) && !ScratchEnds()) {\n    if (!SupportsRandomAccess()) {\n      // Seeking back is not feasible.\n      Reader::Done();\n      scratch_.reset();\n      return;\n    }\n    const Position new_pos = pos();\n    SyncScratch();\n    Seek(new_pos);\n  }\n  DoneBehindScratch();\n  Reader::Done();\n  scratch_.reset();\n}\n\nvoid PullableReader::DoneBehindScratch() {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::DoneBehindScratch(): \"\n         \"scratch used\";\n  SyncBehindScratch(SyncType::kFromObject);\n}\n\ninline void PullableReader::SyncScratch() {\n  RIEGELI_ASSERT(scratch_used())\n      << \"Failed precondition of PullableReader::SyncScratch(): \"\n         \"scratch not used\";\n  RIEGELI_ASSERT_EQ(start(), scratch_->buffer.data())\n      << \"Failed invariant of PullableReader: \"\n         \"scratch used but buffer pointers do not point to scratch\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), scratch_->buffer.size())\n      << \"Failed invariant of PullableReader: \"\n         \"scratch used but buffer pointers do not point to scratch\";\n  ClearScratch();\n}\n\ninline void PullableReader::ClearScratch() {\n  scratch_->buffer.ClearAndShrink();\n  set_buffer(scratch_->original_start, scratch_->original_start_to_limit,\n             scratch_->original_start_to_cursor);\n  move_limit_pos(available());\n}\n\ninline bool PullableReader::ScratchEnds() {\n  RIEGELI_ASSERT(scratch_used())\n      << \"Failed precondition of PullableReader::ScratchEnds(): \"\n         \"scratch not used\";\n  const size_t available_length = available();\n  if (scratch_->original_start_to_cursor >= available_length) {\n    SyncScratch();\n    set_cursor(cursor() - available_length);\n    return true;\n  }\n  return false;\n}\n\nbool PullableReader::PullSlow(size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::PullSlow(): \"\n         \"enough data available, use Pull() instead\";\n  if (ABSL_PREDICT_TRUE(min_length == 1)) {\n    if (ABSL_PREDICT_FALSE(scratch_used())) {\n      SyncScratch();\n      if (available() > 0) return true;\n    }\n    return PullBehindScratch(recommended_length);\n  }\n  if (scratch_used() && ScratchEnds() && available() >= min_length) return true;\n  if (available() == 0) {\n    RIEGELI_ASSERT(!scratch_used())\n        << \"Scratch should have ended but is still used\";\n    if (ABSL_PREDICT_FALSE(!PullBehindScratch(recommended_length))) {\n      return false;\n    }\n    if (available() >= min_length) return true;\n  }\n  size_t remaining_min_length = min_length;\n  recommended_length = UnsignedMax(min_length, recommended_length);\n  std::unique_ptr<Scratch> new_scratch;\n  if (ABSL_PREDICT_FALSE(scratch_ == nullptr)) {\n    new_scratch = std::make_unique<Scratch>();\n  } else {\n    new_scratch = std::move(scratch_);\n    if (!new_scratch->buffer.empty()) {\n      // Scratch is used but it does not have enough data after the cursor.\n      new_scratch->buffer.RemovePrefix(start_to_cursor());\n      new_scratch->buffer.Shrink(recommended_length);\n      remaining_min_length -= new_scratch->buffer.size();\n      recommended_length -= new_scratch->buffer.size();\n      set_buffer(new_scratch->original_start,\n                 new_scratch->original_start_to_limit,\n                 new_scratch->original_start_to_cursor);\n      move_limit_pos(available());\n    }\n  }\n  const absl::Span<char> flat_buffer = new_scratch->buffer.AppendBuffer(\n      remaining_min_length, recommended_length);\n  char* dest = flat_buffer.data();\n  char* const min_limit = flat_buffer.data() + remaining_min_length;\n  char* const recommended_limit = flat_buffer.data() + recommended_length;\n  char* const max_limit = flat_buffer.data() + flat_buffer.size();\n  do {\n    const size_t length =\n        UnsignedMin(available(), PtrDistance(dest, max_limit));\n    riegeli::null_safe_memcpy(dest, cursor(), length);\n    move_cursor(length);\n    dest += length;\n    if (dest >= min_limit) break;\n    if (ABSL_PREDICT_FALSE(scratch_used())) {\n      SyncScratch();\n      if (available() > 0) continue;\n    }\n  } while (PullBehindScratch(PtrDistance(dest, recommended_limit)));\n  new_scratch->buffer.RemoveSuffix(PtrDistance(dest, max_limit));\n  set_limit_pos(pos());\n  new_scratch->original_start = start();\n  new_scratch->original_start_to_limit = start_to_limit();\n  new_scratch->original_start_to_cursor = start_to_cursor();\n  scratch_ = std::move(new_scratch);\n  set_buffer(scratch_->buffer.data(), scratch_->buffer.size());\n  return available() >= min_length;\n}\n\nbool PullableReader::ReadBehindScratch(size_t length, char* dest) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of PullableReader::ReadBehindScratch(char*): \"\n         \"enough data available, use Read(char*) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(char*): \"\n         \"scratch used\";\n  do {\n    const size_t available_length = available();\n    riegeli::null_safe_memcpy(dest, cursor(), available_length);\n    move_cursor(available_length);\n    dest += available_length;\n    length -= available_length;\n    if (ABSL_PREDICT_FALSE(!PullBehindScratch(length))) return false;\n  } while (length > available());\n  std::memcpy(dest, cursor(), length);\n  move_cursor(length);\n  return true;\n}\n\nbool PullableReader::ReadBehindScratch(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"Chain size overflow\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"scratch used\";\n  do {\n    const absl::Span<char> buffer = dest.AppendBuffer(1, length, length);\n    size_t length_read;\n    if (ABSL_PREDICT_FALSE(!Read(buffer.size(), buffer.data(), &length_read))) {\n      dest.RemoveSuffix(buffer.size() - length_read);\n      return false;\n    }\n    length -= length_read;\n  } while (length > 0);\n  return true;\n}\n\nbool PullableReader::ReadBehindScratch(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"Cord size overflow\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"scratch used\";\n  absl::CordBuffer buffer = dest.GetCustomAppendBuffer(\n      cord_internal::kCordBufferBlockSize, length, 1);\n  absl::Span<char> span = buffer.available_up_to(length);\n  if (buffer.capacity() < kDefaultMinBlockSize && length > span.size()) {\n    absl::CordBuffer new_buffer = absl::CordBuffer::CreateWithCustomLimit(\n        cord_internal::kCordBufferBlockSize, buffer.length() + length);\n    std::memcpy(new_buffer.data(), buffer.data(), buffer.length());\n    new_buffer.SetLength(buffer.length());\n    buffer = std::move(new_buffer);\n    span = buffer.available_up_to(length);\n  }\n  for (;;) {\n    size_t length_read;\n    const bool read_ok = Read(span.size(), span.data(), &length_read);\n    buffer.IncreaseLengthBy(length_read);\n    dest.Append(std::move(buffer));\n    if (ABSL_PREDICT_FALSE(!read_ok)) return false;\n    length -= length_read;\n    if (length == 0) return true;\n    buffer = absl::CordBuffer::CreateWithCustomLimit(\n        cord_internal::kCordBufferBlockSize, length);\n    span = buffer.available_up_to(length);\n  }\n}\n\nbool PullableReader::CopyBehindScratch(Position length, Writer& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::CopyBehindScratch(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::CopyBehindScratch(Writer&): \"\n         \"scratch used\";\n  while (length > available()) {\n    const absl::string_view data(cursor(), available());\n    move_cursor(data.size());\n    if (ABSL_PREDICT_FALSE(!dest.Write(data))) return false;\n    length -= data.size();\n    if (ABSL_PREDICT_FALSE(!PullBehindScratch(length))) return false;\n  }\n  const absl::string_view data(cursor(), IntCast<size_t>(length));\n  move_cursor(IntCast<size_t>(length));\n  return dest.Write(data);\n}\n\nbool PullableReader::CopyBehindScratch(size_t length, BackwardWriter& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of \"\n         \"PullableReader::CopyBehindScratch(BackwardWriter&): \"\n         \"enough data available, use Copy(BackwardWriter&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PullableReader::CopyBehindScratch(BackwardWriter&): \"\n         \"scratch used\";\n  if (length <= available()) {\n    const absl::string_view data(cursor(), length);\n    move_cursor(length);\n    return dest.Write(data);\n  }\n  if (length <= kMaxBytesToCopy) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(length))) return false;\n    dest.move_cursor(length);\n    if (ABSL_PREDICT_FALSE(!ReadBehindScratch(length, dest.cursor()))) {\n      dest.set_cursor(dest.cursor() + length);\n      return false;\n    }\n    return true;\n  }\n  Chain data;\n  if (ABSL_PREDICT_FALSE(!ReadBehindScratch(length, data))) return false;\n  return dest.Write(std::move(data));\n}\n\nbool PullableReader::ReadSomeBehindScratch(size_t max_length, char* dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of PullableReader::ReadSomeBehindScratch(char*): \"\n         \"nothing to read, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PullableReader::ReadSomeBehindScratch(char*): \"\n         \"some data available, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadSomeBehindScratch(char*): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!PullBehindScratch(max_length))) return false;\n  max_length = UnsignedMin(max_length, available());\n  std::memcpy(dest, cursor(), max_length);\n  move_cursor(max_length);\n  return true;\n}\n\nbool PullableReader::CopySomeBehindScratch(size_t max_length, Writer& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of \"\n         \"PullableReader::CopySomeBehindScratch(Writer&): \"\n         \"nothing to read, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of \"\n         \"PullableReader::CopySomeBehindScratch(Writer&): \"\n         \"some data available, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PullableReader::CopySomeBehindScratch(Writer&): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!PullBehindScratch(max_length))) return false;\n  max_length = UnsignedMin(max_length, available());\n  if (available() >= max_length && max_length <= kMaxBytesToCopy) {\n    const absl::string_view data(cursor(), max_length);\n    move_cursor(max_length);\n    return dest.Write(data);\n  }\n  return CopyBehindScratch(max_length, dest);\n}\n\nvoid PullableReader::ReadHintBehindScratch(size_t min_length,\n                                           size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of PullableReader::ReadHintBehindScratch(): \"\n         \"enough data available, use ReadHint() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadHintBehindScratch(): \"\n         \"scratch used\";\n}\n\nbool PullableReader::SyncBehindScratch(SyncType sync_type) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::SyncBehindScratch(): \"\n         \"scratch used\";\n  return ok();\n}\n\nbool PullableReader::SeekBehindScratch(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(new_pos <= limit_pos())) {\n    return Fail(\n        absl::UnimplementedError(\"Reader::Seek() backwards not supported\"));\n  }\n  // Seeking forwards.\n  do {\n    move_cursor(available());\n    if (ABSL_PREDICT_FALSE(!PullBehindScratch(0))) return false;\n  } while (new_pos > limit_pos());\n  const Position available_length = limit_pos() - new_pos;\n  RIEGELI_ASSERT_LE(available_length, start_to_limit())\n      << \"PullableReader::PullBehindScratch() skipped some data\";\n  set_cursor(limit() - available_length);\n  return true;\n}\n\nbool PullableReader::ReadSlow(size_t length, char* dest) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of Reader::ReadSlow(char*): \"\n         \"enough data available, use Read(char*) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (!ScratchEnds()) {\n      const size_t length_to_read = available();\n      std::memcpy(dest, cursor(), length_to_read);\n      dest += length_to_read;\n      length -= length_to_read;\n      move_cursor(length_to_read);\n      SyncScratch();\n    }\n    if (available() >= length) {\n      riegeli::null_safe_memcpy(dest, cursor(), length);\n      move_cursor(length);\n      return true;\n    }\n  }\n  return ReadBehindScratch(length, dest);\n}\n\nbool PullableReader::ReadSlow(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"Chain size overflow\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (!ScratchEnds()) {\n      if (available() >= length) {\n        dest.Append(\n            ExternalRef(scratch_->buffer, absl::string_view(cursor(), length)));\n        move_cursor(length);\n        return true;\n      }\n      length -= available();\n      dest.Append(ExternalRef(std::move(scratch_->buffer),\n                              absl::string_view(cursor(), available())));\n      ClearScratch();\n    }\n    if (available() >= length && length <= kMaxBytesToCopy) {\n      dest.Append(absl::string_view(cursor(), length));\n      move_cursor(length);\n      return true;\n    }\n  }\n  return ReadBehindScratch(length, dest);\n}\n\nbool PullableReader::ReadSlow(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"Cord size overflow\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (!ScratchEnds()) {\n      if (available() >= length) {\n        ExternalRef(scratch_->buffer, absl::string_view(cursor(), length))\n            .AppendTo(dest);\n        move_cursor(length);\n        return true;\n      }\n      length -= available();\n      ExternalRef(std::move(scratch_->buffer),\n                  absl::string_view(cursor(), available()))\n          .AppendTo(dest);\n      ClearScratch();\n    }\n    if (available() >= length && length <= kMaxBytesToCopy) {\n      dest.Append(absl::string_view(cursor(), length));\n      move_cursor(length);\n      return true;\n    }\n  }\n  return ReadBehindScratch(length, dest);\n}\n\nbool PullableReader::CopySlow(Position length, Writer& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (!ScratchEnds()) {\n      if (available() >= length) {\n        const absl::string_view data(cursor(), length);\n        move_cursor(length);\n        return dest.Write(ExternalRef(scratch_->buffer, data));\n      }\n      length -= available();\n      const bool write_ok =\n          dest.Write(ExternalRef(std::move(scratch_->buffer),\n                                 absl::string_view(cursor(), available())));\n      ClearScratch();\n      if (ABSL_PREDICT_FALSE(!write_ok)) return false;\n    }\n    if (available() >= length && length <= kMaxBytesToCopy) {\n      const absl::string_view data(cursor(), IntCast<size_t>(length));\n      move_cursor(IntCast<size_t>(length));\n      return dest.Write(data);\n    }\n  }\n  return CopyBehindScratch(length, dest);\n}\n\nbool PullableReader::CopySlow(size_t length, BackwardWriter& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(BackwardWriter&): \"\n         \"enough data available, use Copy(BackwardWriter&) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    Chain from_scratch;\n    if (!ScratchEnds()) {\n      if (available() >= length) {\n        const absl::string_view data(cursor(), length);\n        move_cursor(length);\n        return dest.Write(ExternalRef(scratch_->buffer, data));\n      }\n      length -= available();\n      from_scratch =\n          Chain(ExternalRef(std::move(scratch_->buffer),\n                            absl::string_view(cursor(), available())));\n      ClearScratch();\n    }\n    if (available() >= length && length <= kMaxBytesToCopy) {\n      const absl::string_view data(cursor(), length);\n      move_cursor(length);\n      if (ABSL_PREDICT_FALSE(!dest.Write(data))) return false;\n    } else {\n      if (ABSL_PREDICT_FALSE(!CopyBehindScratch(length, dest))) return false;\n    }\n    return dest.Write(std::move(from_scratch));\n  }\n  return CopyBehindScratch(length, dest);\n}\n\nbool PullableReader::ReadSomeSlow(size_t max_length, char* dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"nothing to read, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"some data available, use ReadSome(char*) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    SyncScratch();\n    if (available() > 0) {\n      max_length = UnsignedMin(max_length, available());\n      riegeli::null_safe_memcpy(dest, cursor(), max_length);\n      move_cursor(max_length);\n      return true;\n    }\n  }\n  return ReadSomeBehindScratch(max_length, dest);\n}\n\nbool PullableReader::CopySomeSlow(size_t max_length, Writer& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"nothing to read, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"some data available, use CopySome(Writer&) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (!ScratchEnds()) {\n      max_length = UnsignedMin(max_length, available());\n      const absl::string_view data(cursor(), max_length);\n      move_cursor(max_length);\n      return dest.Write(ExternalRef(scratch_->buffer, data));\n    }\n    if (available() > 0) {\n      max_length = UnsignedMin(max_length, available());\n      if (ABSL_PREDICT_TRUE(max_length <= kMaxBytesToCopy)) {\n        const absl::string_view data(cursor(), max_length);\n        move_cursor(max_length);\n        return dest.Write(data);\n      }\n      return CopyBehindScratch(max_length, dest);\n    }\n  }\n  return CopySomeBehindScratch(max_length, dest);\n}\n\nvoid PullableReader::ReadHintSlow(size_t min_length,\n                                  size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::ReadHintSlow(): \"\n         \"enough data available, use ReadHint() instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (!ScratchEnds()) {\n      recommended_length = UnsignedMax(recommended_length, min_length);\n      min_length -= available();\n      recommended_length -= available();\n      BehindScratch behind_scratch(this);\n      if (available() < min_length) {\n        ReadHintBehindScratch(min_length, recommended_length);\n      }\n      return;\n    }\n    if (available() >= min_length) return;\n  }\n  ReadHintBehindScratch(min_length, recommended_length);\n}\n\nbool PullableReader::SyncImpl(SyncType sync_type) {\n  if (ABSL_PREDICT_FALSE(scratch_used()) && !ScratchEnds()) {\n    if (!SupportsRandomAccess()) {\n      // Seeking back is not feasible.\n      return ok();\n    }\n    const Position new_pos = pos();\n    SyncScratch();\n    Seek(new_pos);\n  }\n  return SyncBehindScratch(sync_type);\n}\n\nbool PullableReader::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of Reader::SeekSlow(): \"\n         \"position in the buffer, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    SyncScratch();\n    if (new_pos >= start_pos() && new_pos <= limit_pos()) {\n      set_cursor(limit() - (limit_pos() - new_pos));\n      return true;\n    }\n  }\n  return SeekBehindScratch(new_pos);\n}\n\nvoid PullableReader::BehindScratch::Enter() {\n  RIEGELI_ASSERT(context_->scratch_used())\n      << \"Failed precondition of PullableReader::BehindScratch::Enter(): \"\n         \"scratch not used\";\n  RIEGELI_ASSERT_EQ(context_->start(), context_->scratch_->buffer.data())\n      << \"Failed invariant of PullableReader: \"\n         \"scratch used but buffer pointers do not point to scratch\";\n  RIEGELI_ASSERT_EQ(context_->start_to_limit(),\n                    context_->scratch_->buffer.size())\n      << \"Failed invariant of PullableReader: \"\n         \"scratch used but buffer pointers do not point to scratch\";\n  scratch_ = std::move(context_->scratch_);\n  read_from_scratch_ = context_->start_to_cursor();\n  context_->set_buffer(scratch_->original_start,\n                       scratch_->original_start_to_limit,\n                       scratch_->original_start_to_cursor);\n  context_->move_limit_pos(context_->available());\n}\n\nvoid PullableReader::BehindScratch::Leave() {\n  RIEGELI_ASSERT_NE(scratch_, nullptr)\n      << \"Failed precondition of PullableReader::BehindScratch::Leave(): \"\n         \"scratch not used\";\n  context_->set_limit_pos(context_->pos());\n  scratch_->original_start = context_->start();\n  scratch_->original_start_to_limit = context_->start_to_limit();\n  scratch_->original_start_to_cursor = context_->start_to_cursor();\n  context_->set_buffer(scratch_->buffer.data(), scratch_->buffer.size(),\n                       read_from_scratch_);\n  context_->scratch_ = std::move(scratch_);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/pullable_reader.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_PULLABLE_READER_H_\n#define RIEGELI_BYTES_PULLABLE_READER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/sized_shared_buffer.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nclass BackwardWriter;\nclass Writer;\n\n// Abstract class `PullableReader` helps to implement\n// `Reader::PullSlow(min_length, recommended_length)` with `min_length > 1`.\n//\n// `PullableReader` accumulates pulled data in a scratch buffer if needed.\nclass PullableReader : public Reader {\n protected:\n  class BehindScratch;\n\n  using Reader::Reader;\n\n  PullableReader(PullableReader&& that) noexcept;\n  PullableReader& operator=(PullableReader&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `PullableReader`. This\n  // avoids constructing a temporary `PullableReader` and moving from it.\n  // Derived classes which redefine `Reset()` should include a call to\n  // `PullableReader::Reset()`.\n  void Reset(Closed);\n  void Reset();\n\n  void Done() override;\n\n  // Returns `true` if scratch is used, which means that buffer pointers are\n  // temporarily unrelated to the source. This is exposed for assertions.\n  bool scratch_used() const;\n\n  // `PullableReader::{Done,SyncImpl}()` seek the source back to the current\n  // position if scratch is used but not all data from scratch were read.\n  // This is feasible only if `SupportsRandomAccess()`.\n  //\n  // Warning: if `!SupportsRandomAccess()`, the source will have an\n  // unpredictable amount of extra data consumed because of buffering.\n  //\n  // For propagating `{Close,Sync}()` to dependencies, `{Done,SyncImpl}()`\n  // should be overridden to call `PullableReader::{Done,SyncImpl}()` and then\n  // close/sync the dependencies.\n\n  // Implementation of `Done()`, called while scratch is not used. This is\n  // called before buffer pointers are reset.\n  //\n  // If scratch was used but not all data from scratch were read and\n  // `!SupportsRandomAccess()`, seeking back is not feasible and\n  // `DoneBehindScratch()` is not called.\n  //\n  // By default calls `SyncBehindScratch(SyncType::kFromObject)`, which by\n  // default does nothing.\n  //\n  // Precondition: `!scratch_used()`\n  virtual void DoneBehindScratch();\n\n  // Implementation of `PullSlow(1, recommended_length)`, called while scratch\n  // is not used.\n  //\n  // Preconditions:\n  //   `available() == 0`\n  //   `!scratch_used()`\n  virtual bool PullBehindScratch(size_t recommended_length) = 0;\n\n  // Implementation of `ReadSlow()`, `CopySlow()`, `ReadSomeSlow()`,\n  // `CopySomeSlow()`, `ReadHintSlow()`, `SyncImpl()`, and `SeekSlow()`, called\n  // while scratch is not used.\n  //\n  // Regarding `SyncBehindScratch()`, if scratch was used but not all data from\n  // scratch were read and `!SupportsRandomAccess()`, seeking back is not\n  // feasible and `SyncBehindScratch()` is not called.\n  //\n  // By default they are implemented analogously to the corresponding `Reader`\n  // functions.\n  //\n  // Preconditions:\n  //   like the corresponding `Reader` functions\n  //   `!scratch_used()`\n  virtual bool ReadBehindScratch(size_t length, char* dest);\n  virtual bool ReadBehindScratch(size_t length, Chain& dest);\n  virtual bool ReadBehindScratch(size_t length, absl::Cord& dest);\n  virtual bool CopyBehindScratch(Position length, Writer& dest);\n  virtual bool CopyBehindScratch(size_t length, BackwardWriter& dest);\n  virtual bool ReadSomeBehindScratch(size_t max_length, char* dest);\n  virtual bool CopySomeBehindScratch(size_t max_length, Writer& dest);\n  virtual void ReadHintBehindScratch(size_t min_length,\n                                     size_t recommended_length);\n  virtual bool SyncBehindScratch(SyncType sync_type);\n  virtual bool SeekBehindScratch(Position new_pos);\n\n  bool PullSlow(size_t min_length, size_t recommended_length) override;\n  using Reader::ReadSlow;\n  bool ReadSlow(size_t length, char* dest) override;\n  bool ReadSlow(size_t length, Chain& dest) override;\n  bool ReadSlow(size_t length, absl::Cord& dest) override;\n  using Reader::CopySlow;\n  bool CopySlow(Position length, Writer& dest) override;\n  bool CopySlow(size_t length, BackwardWriter& dest) override;\n  using Reader::ReadSomeSlow;\n  bool ReadSomeSlow(size_t max_length, char* dest) override;\n  using Reader::CopySomeSlow;\n  bool CopySomeSlow(size_t max_length, Writer& dest) override;\n  void ReadHintSlow(size_t min_length, size_t recommended_length) override;\n  bool SyncImpl(SyncType sync_type) override;\n  bool SeekSlow(Position new_pos) override;\n\n private:\n  struct Scratch {\n    SizedSharedBuffer buffer;\n    const char* original_start = nullptr;\n    size_t original_start_to_limit = 0;\n    size_t original_start_to_cursor = 0;\n  };\n\n  void SyncScratch();\n  void ClearScratch();\n\n  // Stops using scratch and returns `true` if all remaining data in scratch\n  // come from a single fragment of the original source.\n  bool ScratchEnds();\n\n  std::unique_ptr<Scratch> scratch_;\n\n  // Invariants if `scratch_used()`:\n  //   `start() == scratch_->buffer.data()`\n  //   `start_to_limit() == scratch_->buffer.size()`\n};\n\n// Helps to implement move constructor or move assignment if scratch is used.\n//\n// Moving the source should be in scope of a `BehindScratch` object, unless\n// source buffer pointers are known to remain unchanged during a move or their\n// change does not need to be reflected elsewhere.\n//\n// This temporarily reveals the relationship between the source and the buffer\n// pointers, in case it was hidden behind scratch usage. In a `BehindScratch`\n// scope, scratch is not used, and buffer pointers may be changed. The current\n// position reflects what has been read from the source and must not be changed.\nclass PullableReader::BehindScratch {\n public:\n  explicit BehindScratch(PullableReader* context ABSL_ATTRIBUTE_LIFETIME_BOUND);\n\n  BehindScratch(BehindScratch&& that) = default;\n  BehindScratch& operator=(BehindScratch&&) = delete;\n\n  ~BehindScratch();\n\n private:\n  void Enter();\n  void Leave();\n\n  PullableReader* context_;\n  std::unique_ptr<Scratch> scratch_;\n  size_t read_from_scratch_;\n};\n\n// Implementation details follow.\n\ninline PullableReader::PullableReader(PullableReader&& that) noexcept\n    : Reader(static_cast<Reader&&>(that)), scratch_(std::move(that.scratch_)) {}\n\ninline PullableReader& PullableReader::operator=(\n    PullableReader&& that) noexcept {\n  Reader::operator=(static_cast<Reader&&>(that));\n  scratch_ = std::move(that.scratch_);\n  return *this;\n}\n\ninline void PullableReader::Reset(Closed) {\n  Reader::Reset(kClosed);\n  scratch_.reset();\n}\n\ninline void PullableReader::Reset() {\n  Reader::Reset();\n  if (ABSL_PREDICT_FALSE(scratch_used())) scratch_->buffer.Clear();\n}\n\ninline bool PullableReader::scratch_used() const {\n  return scratch_ != nullptr && !scratch_->buffer.empty();\n}\n\ninline PullableReader::BehindScratch::BehindScratch(\n    PullableReader* context ABSL_ATTRIBUTE_LIFETIME_BOUND)\n    : context_(context) {\n  if (ABSL_PREDICT_FALSE(context_->scratch_used())) Enter();\n}\n\ninline PullableReader::BehindScratch::~BehindScratch() {\n  if (ABSL_PREDICT_FALSE(scratch_ != nullptr)) Leave();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_PULLABLE_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/pushable_backward_writer.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/pushable_backward_writer.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <memory>\n#include <optional>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_utils.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/sized_shared_buffer.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\nvoid PushableBackwardWriter::Done() {\n  if (ABSL_PREDICT_TRUE(!scratch_used()) || ABSL_PREDICT_TRUE(SyncScratch())) {\n    DoneBehindScratch();\n  }\n  BackwardWriter::Done();\n  scratch_.reset();\n}\n\nvoid PushableBackwardWriter::OnFail() {\n  BackwardWriter::OnFail();\n  scratch_.reset();\n}\n\nvoid PushableBackwardWriter::DoneBehindScratch() {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableBackwardWriter::DoneBehindScratch(): \"\n         \"scratch used\";\n  FlushBehindScratch(FlushType::kFromObject);\n}\n\ninline bool PushableBackwardWriter::SyncScratch() {\n  RIEGELI_ASSERT(scratch_used())\n      << \"Failed precondition of PushableBackwardWriter::SyncScratch(): \"\n         \"scratch not used\";\n  RIEGELI_ASSERT_EQ(limit(), scratch_->buffer.data())\n      << \"Failed invariant of PushableBackwardWriter: \"\n         \"scratch used but buffer pointers do not point to scratch\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), scratch_->buffer.size())\n      << \"Failed invariant of PushableBackwardWriter: \"\n         \"scratch used but buffer pointers do not point to scratch\";\n  const size_t length_to_write = start_to_cursor();\n  set_buffer(scratch_->original_limit, scratch_->original_start_to_limit,\n             scratch_->original_start_to_cursor);\n  set_start_pos(start_pos() - start_to_cursor());\n  SizedSharedBuffer buffer = std::move(scratch_->buffer);\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Moving should have left the source SizedSharedBuffer cleared\";\n  const char* const data = buffer.data() + buffer.size() - length_to_write;\n  if (ABSL_PREDICT_FALSE(!Write(ExternalRef(\n          std::move(buffer), absl::string_view(data, length_to_write))))) {\n    return false;\n  }\n  RIEGELI_ASSERT(!scratch_used())\n      << \"WriteSlow(absl::string_view) must not start using scratch, \"\n         \"in particular if PushBehindScratch() calls ForcePushUsingScratch() \"\n         \"then WriteSlow(absl::string_view) must be overridden to avoid \"\n         \"indirectly calling ForcePushUsingScratch()\";\n  // Restore buffer allocation.\n  buffer.ClearAndShrink();\n  scratch_->buffer = std::move(buffer);\n  return true;\n}\n\nbool PushableBackwardWriter::PushSlow(size_t min_length,\n                                      size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of BackwardWriter::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    RIEGELI_ASSERT_EQ(limit(), scratch_->buffer.data())\n        << \"Failed invariant of PushableBackwardWriter: \"\n           \"scratch used but buffer pointers do not point to scratch\";\n    RIEGELI_ASSERT_EQ(start_to_limit(), scratch_->buffer.size())\n        << \"Failed invariant of PushableBackwardWriter: \"\n           \"scratch used but buffer pointers do not point to scratch\";\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= min_length) return true;\n  }\n  if (ABSL_PREDICT_TRUE(min_length == 1)) {\n    return PushBehindScratch(recommended_length);\n  }\n  if (available() == 0) {\n    if (ABSL_PREDICT_FALSE(!PushBehindScratch(recommended_length))) {\n      return false;\n    }\n    if (available() >= min_length) return true;\n    if (ABSL_PREDICT_FALSE(scratch_used())) {\n      // `PushBehindScratch()` must have called `ForcePushUsingScratch()` but\n      // scratch is too small.\n      if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n      if (available() >= min_length) return true;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(scratch_ == nullptr)) {\n    scratch_ = std::make_unique<Scratch>();\n  }\n  const absl::Span<char> flat_buffer =\n      scratch_->buffer.PrependBuffer(min_length, recommended_length);\n  set_start_pos(pos());\n  scratch_->original_limit = limit();\n  scratch_->original_start_to_limit = start_to_limit();\n  scratch_->original_start_to_cursor = start_to_cursor();\n  set_buffer(flat_buffer.data(), flat_buffer.size());\n  return true;\n}\n\nbool PushableBackwardWriter::ForcePushUsingScratch() {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::ForcePushUsingScratch(): \"\n         \"some space available, nothing to do\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::ForcePushUsingScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(scratch_ == nullptr)) {\n    scratch_ = std::make_unique<Scratch>();\n  }\n  const absl::Span<char> flat_buffer = scratch_->buffer.PrependBuffer(1);\n  set_start_pos(pos());\n  scratch_->original_limit = limit();\n  scratch_->original_start_to_limit = start_to_limit();\n  scratch_->original_start_to_cursor = start_to_cursor();\n  set_buffer(flat_buffer.data(), flat_buffer.size());\n  return true;\n}\n\nbool PushableBackwardWriter::WriteBehindScratch(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(string_view): \"\n         \"scratch used\";\n  do {\n    const size_t available_length = available();\n    move_cursor(available_length);\n    riegeli::null_safe_memcpy(\n        cursor(), src.data() + src.size() - available_length, available_length);\n    src.remove_suffix(available_length);\n    if (ABSL_PREDICT_FALSE(!PushBehindScratch(src.size()))) return false;\n  } while (src.size() > available());\n  move_cursor(src.size());\n  std::memcpy(cursor(), src.data(), src.size());\n  return true;\n}\n\nbool PushableBackwardWriter::WriteBehindScratch(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(ExternalRef): \"\n         \"scratch used\";\n  return Write(absl::string_view(src));\n}\n\nbool PushableBackwardWriter::WriteBehindScratch(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(Chain): \"\n         \"scratch used\";\n  for (Chain::Blocks::const_reverse_iterator iter = src.blocks().crbegin();\n       iter != src.blocks().crend(); ++iter) {\n    if (ABSL_PREDICT_FALSE(!Write(absl::string_view(*iter)))) return false;\n  }\n  return true;\n}\n\nbool PushableBackwardWriter::WriteBehindScratch(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(Chain&&): \"\n         \"scratch used\";\n  // Not `std::move(src)`: forward to `WriteBehindScratch(const Chain&)`.\n  return WriteBehindScratch(src);\n}\n\nbool PushableBackwardWriter::WriteBehindScratch(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(Cord): \"\n         \"scratch used\";\n  if (const std::optional<absl::string_view> flat = src.TryFlat();\n      flat != std::nullopt) {\n    return Write(*flat);\n  }\n  if (src.size() <= available()) {\n    move_cursor(src.size());\n    cord_internal::CopyCordToArray(src, cursor());\n    return true;\n  }\n  std::vector<absl::string_view> fragments(src.chunk_begin(), src.chunk_end());\n  for (std::vector<absl::string_view>::const_reverse_iterator iter =\n           fragments.crbegin();\n       iter != fragments.crend(); ++iter) {\n    if (ABSL_PREDICT_FALSE(!Write(*iter))) return false;\n  }\n  return true;\n}\n\nbool PushableBackwardWriter::WriteBehindScratch(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(Cord&&): \"\n         \"scratch used\";\n  // Not `std::move(src)`: forward to `WriteBehindScratch(const absl::Cord&)`.\n  return WriteBehindScratch(src);\n}\n\nbool PushableBackwardWriter::WriteBehindScratch(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::WriteBehindScratch(ByteFill): \"\n         \"scratch used\";\n  while (src.size() > available()) {\n    const size_t available_length = available();\n    move_cursor(available_length);\n    riegeli::null_safe_memset(cursor(), src.fill(), available_length);\n    src.Extract(available_length);\n    if (ABSL_PREDICT_FALSE(\n            !PushBehindScratch(SaturatingIntCast<size_t>(src.size())))) {\n      return false;\n    }\n  }\n  move_cursor(IntCast<size_t>(src.size()));\n  std::memset(cursor(), src.fill(), IntCast<size_t>(src.size()));\n  return true;\n}\n\nbool PushableBackwardWriter::FlushBehindScratch(FlushType flush_type) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableBackwardWriter::FlushBehindScratch(): \"\n         \"scratch used\";\n  return ok();\n}\n\nbool PushableBackwardWriter::TruncateBehindScratch(Position new_size) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::TruncateBehindScratch(): \"\n         \"scratch used\";\n  return Fail(\n      absl::UnimplementedError(\"BackwardWriter::Truncate() not supported\"));\n}\n\nbool PushableBackwardWriter::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size()) {\n      move_cursor(src.size());\n      riegeli::null_safe_memcpy(cursor(), src.data(), src.size());\n      return true;\n    }\n  }\n  return WriteBehindScratch(src);\n}\n\nbool PushableBackwardWriter::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size() && src.size() <= kMaxBytesToCopy) {\n      move_cursor(src.size());\n      riegeli::null_safe_memcpy(cursor(), src.data(), src.size());\n      return true;\n    }\n  }\n  return WriteBehindScratch(std::move(src));\n}\n\nbool PushableBackwardWriter::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size() && src.size() <= kMaxBytesToCopy) {\n      move_cursor(src.size());\n      src.CopyTo(cursor());\n      return true;\n    }\n  }\n  return WriteBehindScratch(src);\n}\n\nbool PushableBackwardWriter::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size() && src.size() <= kMaxBytesToCopy) {\n      move_cursor(src.size());\n      src.CopyTo(cursor());\n      return true;\n    }\n  }\n  return WriteBehindScratch(std::move(src));\n}\n\nbool PushableBackwardWriter::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size() && src.size() <= kMaxBytesToCopy) {\n      move_cursor(src.size());\n      cord_internal::CopyCordToArray(src, cursor());\n      return true;\n    }\n  }\n  return WriteBehindScratch(src);\n}\n\nbool PushableBackwardWriter::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size() && src.size() <= kMaxBytesToCopy) {\n      move_cursor(src.size());\n      cord_internal::CopyCordToArray(src, cursor());\n      return true;\n    }\n  }\n  return WriteBehindScratch(std::move(src));\n}\n\nbool PushableBackwardWriter::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size() && src.size() <= kMaxBytesToCopy) {\n      riegeli::null_safe_memset(cursor(), src.fill(),\n                                IntCast<size_t>(src.size()));\n      move_cursor(IntCast<size_t>(src.size()));\n      return true;\n    }\n  }\n  return WriteBehindScratch(src);\n}\n\nbool PushableBackwardWriter::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n  }\n  return FlushBehindScratch(flush_type);\n}\n\nbool PushableBackwardWriter::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n  }\n  return TruncateBehindScratch(new_size);\n}\n\nvoid PushableBackwardWriter::BehindScratch::Enter() {\n  RIEGELI_ASSERT(context_->scratch_used())\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::BehindScratch::Enter(): \"\n         \"scratch not used\";\n  RIEGELI_ASSERT_EQ(context_->limit(), context_->scratch_->buffer.data())\n      << \"Failed invariant of PushableBackwardWriter: \"\n         \"scratch used but buffer pointers do not point to scratch\";\n  RIEGELI_ASSERT_EQ(context_->start_to_limit(),\n                    context_->scratch_->buffer.size())\n      << \"Failed invariant of PushableBackwardWriter: \"\n         \"scratch used but buffer pointers do not point to scratch\";\n  scratch_ = std::move(context_->scratch_);\n  written_to_scratch_ = context_->start_to_cursor();\n  context_->set_buffer(scratch_->original_limit,\n                       scratch_->original_start_to_limit,\n                       scratch_->original_start_to_cursor);\n  context_->set_start_pos(context_->start_pos() - context_->start_to_cursor());\n}\n\nvoid PushableBackwardWriter::BehindScratch::Leave() {\n  RIEGELI_ASSERT_NE(scratch_, nullptr)\n      << \"Failed precondition of \"\n         \"PushableBackwardWriter::BehindScratch::Leave(): \"\n         \"scratch not used\";\n  context_->set_start_pos(context_->pos());\n  scratch_->original_limit = context_->limit();\n  scratch_->original_start_to_limit = context_->start_to_limit();\n  scratch_->original_limit = context_->limit();\n  context_->set_buffer(const_cast<char*>(scratch_->buffer.data()),\n                       scratch_->buffer.size(), written_to_scratch_);\n  context_->scratch_ = std::move(scratch_);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/pushable_backward_writer.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_PUSHABLE_BACKWARD_WRITER_H_\n#define RIEGELI_BYTES_PUSHABLE_BACKWARD_WRITER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/sized_shared_buffer.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\n// Abstract class `PushableBackwardWriter` helps to implement\n// `BackwardWriter::PushSlow(min_length, recommended_length)` with\n// `min_length > 1`.\n//\n// `PushableBackwardWriter` accumulates data to be pushed in a scratch buffer if\n// needed.\nclass PushableBackwardWriter : public BackwardWriter {\n protected:\n  class BehindScratch;\n\n  using BackwardWriter::BackwardWriter;\n\n  PushableBackwardWriter(PushableBackwardWriter&& that) noexcept;\n  PushableBackwardWriter& operator=(PushableBackwardWriter&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `PushableBackwardWriter`.\n  // This avoids constructing a temporary `PushableBackwardWriter` and moving\n  // from it. Derived classes which redefine `Reset()` should include a call to\n  // `PushableBackwardWriter::Reset()`.\n  void Reset(Closed);\n  void Reset();\n\n  void Done() override;\n  void OnFail() override;\n\n  // Returns `true` if scratch is currently in use, which means that buffer\n  // pointers are temporarily unrelated to the destination. This is exposed for\n  // assertions.\n  bool scratch_used() const;\n\n  // `PushableBackwardWriter::{Done,FlushImpl}()` write the scratch if needed,\n  // and call\n  // `{Done,Flush}BehindScratch()` to close/flush the destination before buffer\n  // pointers are reset.\n  //\n  // For propagating `{Close,Flush}()` to dependencies, `{Done,FlushImpl}()`\n  // should be overridden to call `PushableBackwardWriter::{Done,FlushImpl}()`\n  // and then close/flush the dependencies.\n\n  // Implementation of `Done()`, called while scratch is not used, and only if\n  // writing the scratch succeeded. This is called before buffer pointers are\n  // reset.\n  //\n  // By default calls `FlushBehindScratch(FlushType::kFromObject)`, which by\n  // default does nothing.\n  //\n  // Precondition: `!scratch_used()`\n  virtual void DoneBehindScratch();\n\n  // Implementation of `PushSlow(1, recommended_length)`, called while scratch\n  // is not used.\n  //\n  // Preconditions:\n  //   `available() == 0`\n  //   `!scratch_used()`\n  virtual bool PushBehindScratch(size_t recommended_length) = 0;\n\n  // Force using scratch as the buffer.\n  //\n  // This can be used in the implementation of `PushBehindScratch()` if in some\n  // circumstances scratch should be used even when `min_length == 1`.\n  //\n  // These circumstances should be rare, otherwise performance would be poor\n  // because `Push()` would call `PushSlow()` too often.\n  //\n  // Warning: `WriteSlow(absl::string_view)` or `WriteSlow(Chain)` is\n  // called to stop using scratch by writing scratch contents, and the\n  // default implementation of `WriteSlow(absl::string_view)` calls\n  // `PushBehindScratch()`. This means that if `PushBehindScratch()` calls\n  // `ForcePushUsingScratch()`, then `WriteSlow(absl::string_view)` must be\n  // overridden to avoid indirectly calling `ForcePushUsingScratch()` or\n  // `PushSlow(min_length > 1)`.\n  //\n  // Preconditions:\n  //   `available() == 0`\n  //   `!scratch_used()`\n  //\n  // Always returns `true`.\n  //\n  // Postconditions:\n  //   `available() > 0`\n  //   `scratch_used()`\n  bool ForcePushUsingScratch();\n\n  // Implementation of `WriteSlow()`, `FlushImpl()`, and `TruncateImpl()`,\n  // called while scratch is not used.\n  //\n  // By default they are implemented analogously to the corresponding\n  // `BackwardWriter` functions.\n  //\n  // Preconditions:\n  //   like the corresponding `BackwardWriter` functions\n  //   `!scratch_used()`\n  virtual bool WriteBehindScratch(absl::string_view src);\n  virtual bool WriteBehindScratch(ExternalRef src);\n  virtual bool WriteBehindScratch(const Chain& src);\n  virtual bool WriteBehindScratch(Chain&& src);\n  virtual bool WriteBehindScratch(const absl::Cord& src);\n  virtual bool WriteBehindScratch(absl::Cord&& src);\n  virtual bool WriteBehindScratch(ByteFill src);\n  virtual bool FlushBehindScratch(FlushType flush_type);\n  virtual bool TruncateBehindScratch(Position new_size);\n\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool FlushImpl(FlushType flush_type) override;\n  bool TruncateImpl(Position new_size) override;\n\n private:\n  struct Scratch {\n    SizedSharedBuffer buffer;\n    char* original_limit = nullptr;\n    size_t original_start_to_limit = 0;\n    size_t original_start_to_cursor = 0;\n  };\n\n  bool SyncScratch();\n\n  std::unique_ptr<Scratch> scratch_;\n\n  // Invariants if `scratch_used()`:\n  //   `limit() == scratch_->buffer.data()`\n  //   `start_to_limit() == scratch_->buffer.data()`\n};\n\n// Helps to implement move constructor or move assignment if scratch is used.\n//\n// Moving the destination should be in scope of a `BehindScratch` object, unless\n// destination buffer pointers are known to remain unchanged during a move or\n// their change does not need to be reflected elsewhere.\n//\n// This temporarily reveals the relationship between the destination and the\n// buffer pointers, in case it was hidden behind scratch usage. In a\n// `BehindScratch` scope, scratch is not used, and buffer pointers may be\n// changed. The current position reflects what has been written to the\n// destination and must not be changed.\nclass PushableBackwardWriter::BehindScratch {\n public:\n  explicit BehindScratch(\n      PushableBackwardWriter* context ABSL_ATTRIBUTE_LIFETIME_BOUND);\n\n  BehindScratch(BehindScratch&& that) = default;\n  BehindScratch& operator=(BehindScratch&&) = delete;\n\n  ~BehindScratch();\n\n private:\n  void Enter();\n  void Leave();\n\n  PushableBackwardWriter* context_;\n  std::unique_ptr<Scratch> scratch_;\n  size_t written_to_scratch_;\n};\n\n// Implementation details follow.\n\ninline PushableBackwardWriter::PushableBackwardWriter(\n    PushableBackwardWriter&& that) noexcept\n    : BackwardWriter(static_cast<BackwardWriter&&>(that)),\n      scratch_(std::move(that.scratch_)) {}\n\ninline PushableBackwardWriter& PushableBackwardWriter::operator=(\n    PushableBackwardWriter&& that) noexcept {\n  BackwardWriter::operator=(static_cast<BackwardWriter&&>(that));\n  scratch_ = std::move(that.scratch_);\n  return *this;\n}\n\ninline void PushableBackwardWriter::Reset(Closed) {\n  BackwardWriter::Reset(kClosed);\n  scratch_.reset();\n}\n\ninline void PushableBackwardWriter::Reset() {\n  BackwardWriter::Reset();\n  if (ABSL_PREDICT_FALSE(scratch_used())) scratch_->buffer.Clear();\n}\n\ninline bool PushableBackwardWriter::scratch_used() const {\n  return scratch_ != nullptr && !scratch_->buffer.empty();\n}\n\ninline PushableBackwardWriter::BehindScratch::BehindScratch(\n    PushableBackwardWriter* context ABSL_ATTRIBUTE_LIFETIME_BOUND)\n    : context_(context) {\n  if (ABSL_PREDICT_FALSE(context_->scratch_used())) Enter();\n}\n\ninline PushableBackwardWriter::BehindScratch::~BehindScratch() {\n  if (ABSL_PREDICT_FALSE(scratch_ != nullptr)) Leave();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_PUSHABLE_BACKWARD_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/pushable_writer.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/pushable_writer.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_utils.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/sized_shared_buffer.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid PushableWriter::Done() {\n  if (ABSL_PREDICT_TRUE(!scratch_used()) || ABSL_PREDICT_TRUE(SyncScratch())) {\n    DoneBehindScratch();\n  }\n  Writer::Done();\n  scratch_.reset();\n}\n\nvoid PushableWriter::OnFail() {\n  Writer::OnFail();\n  scratch_.reset();\n}\n\nvoid PushableWriter::DoneBehindScratch() {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::DoneBehindScratch(): \"\n         \"scratch used\";\n  FlushBehindScratch(FlushType::kFromObject);\n}\n\ninline bool PushableWriter::SyncScratch() {\n  RIEGELI_ASSERT(scratch_used())\n      << \"Failed precondition of PushableWriter::SyncScratch(): \"\n         \"scratch not used\";\n  RIEGELI_ASSERT_EQ(start(), scratch_->buffer.data())\n      << \"Failed invariant of PushableWriter: \"\n         \"scratch used but buffer pointers do not point to scratch\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), scratch_->buffer.size())\n      << \"Failed invariant of PushableWriter: \"\n         \"scratch used but buffer pointers do not point to scratch\";\n  const size_t length_to_write = start_to_cursor();\n  set_buffer(scratch_->original_start, scratch_->original_start_to_limit,\n             scratch_->original_start_to_cursor);\n  set_start_pos(start_pos() - start_to_cursor());\n  SizedSharedBuffer buffer = std::move(scratch_->buffer);\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Moving should have left the source SizedSharedBuffer cleared\";\n  const char* const data = buffer.data();\n  if (ABSL_PREDICT_FALSE(!Write(ExternalRef(\n          std::move(buffer), absl::string_view(data, length_to_write))))) {\n    return false;\n  }\n  RIEGELI_ASSERT(!scratch_used())\n      << \"WriteSlow(absl::string_view) must not start using scratch, \"\n         \"in particular if PushBehindScratch() calls ForcePushUsingScratch() \"\n         \"then WriteSlow(absl::string_view) must be overridden to avoid \"\n         \"indirectly calling ForcePushUsingScratch()\";\n  // Restore buffer allocation.\n  buffer.ClearAndShrink();\n  scratch_->buffer = std::move(buffer);\n  return true;\n}\n\nbool PushableWriter::PushSlow(size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    RIEGELI_ASSERT_EQ(start(), scratch_->buffer.data())\n        << \"Failed invariant of PushableWriter: \"\n           \"scratch used but buffer pointers do not point to scratch\";\n    RIEGELI_ASSERT_EQ(start_to_limit(), scratch_->buffer.size())\n        << \"Failed invariant of PushableWriter: \"\n           \"scratch used but buffer pointers do not point to scratch\";\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= min_length) return true;\n  }\n  if (ABSL_PREDICT_TRUE(min_length == 1)) {\n    return PushBehindScratch(recommended_length);\n  }\n  if (available() == 0) {\n    if (ABSL_PREDICT_FALSE(!PushBehindScratch(recommended_length))) {\n      return false;\n    }\n    if (available() >= min_length) return true;\n    if (ABSL_PREDICT_FALSE(scratch_used())) {\n      // `PushBehindScratch()` must have called `ForcePushUsingScratch()` but\n      // scratch is too small.\n      if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n      if (available() >= min_length) return true;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(scratch_ == nullptr)) {\n    scratch_ = std::make_unique<Scratch>();\n  }\n  const absl::Span<char> flat_buffer =\n      scratch_->buffer.AppendBuffer(min_length, recommended_length);\n  set_start_pos(pos());\n  scratch_->original_start = start();\n  scratch_->original_start_to_limit = start_to_limit();\n  scratch_->original_start_to_cursor = start_to_cursor();\n  set_buffer(flat_buffer.data(), flat_buffer.size());\n  return true;\n}\n\nbool PushableWriter::ForcePushUsingScratch() {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PushableWriter::ForcePushUsingScratch(): \"\n         \"some space available, nothing to do\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::ForcePushUsingScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(scratch_ == nullptr)) {\n    scratch_ = std::make_unique<Scratch>();\n  }\n  const absl::Span<char> flat_buffer = scratch_->buffer.AppendBuffer(1);\n  set_start_pos(pos());\n  scratch_->original_start = start();\n  scratch_->original_start_to_limit = start_to_limit();\n  scratch_->original_start_to_cursor = start_to_cursor();\n  set_buffer(flat_buffer.data(), flat_buffer.size());\n  return true;\n}\n\nbool PushableWriter::WriteBehindScratch(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of \"\n         \"PushableWriter::WriteBehindScratch(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableWriter::WriteBehindScratch(string_view): \"\n         \"scratch used\";\n  do {\n    const size_t available_length = available();\n    riegeli::null_safe_memcpy(cursor(), src.data(), available_length);\n    move_cursor(available_length);\n    src.remove_prefix(available_length);\n    if (ABSL_PREDICT_FALSE(!PushBehindScratch(src.size()))) return false;\n  } while (src.size() > available());\n  std::memcpy(cursor(), src.data(), src.size());\n  move_cursor(src.size());\n  return true;\n}\n\nbool PushableWriter::WriteBehindScratch(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of \"\n         \"PushableWriter::WriteBehindScratch(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableWriter::WriteBehindScratch(ExternalRef): \"\n         \"scratch used\";\n  return Write(absl::string_view(src));\n}\n\nbool PushableWriter::WriteBehindScratch(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Chain): \"\n         \"scratch used\";\n  for (const absl::string_view fragment : src.blocks()) {\n    if (ABSL_PREDICT_FALSE(!Write(fragment))) return false;\n  }\n  return true;\n}\n\nbool PushableWriter::WriteBehindScratch(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Chain&&): \"\n         \"scratch used\";\n  // Not `std::move(src)`: forward to `WriteBehindScratch(const Chain&)`.\n  return WriteBehindScratch(src);\n}\n\nbool PushableWriter::WriteBehindScratch(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Cord): \"\n         \"scratch used\";\n  if (const std::optional<absl::string_view> flat = src.TryFlat();\n      flat != std::nullopt) {\n    return Write(*flat);\n  }\n  for (const absl::string_view fragment : src.Chunks()) {\n    if (ABSL_PREDICT_FALSE(!Write(fragment))) return false;\n  }\n  return true;\n}\n\nbool PushableWriter::WriteBehindScratch(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Cord&&): \"\n         \"scratch used\";\n  // Not `std::move(src)`: forward to `WriteBehindScratch(const absl::Cord&)`.\n  return WriteBehindScratch(src);\n}\n\nbool PushableWriter::WriteBehindScratch(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of \"\n         \"PushableWriter::WriteBehindScratch(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableWriter::WriteBehindScratch(ByteFill): \"\n         \"scratch used\";\n  while (src.size() > available()) {\n    const size_t available_length = available();\n    riegeli::null_safe_memset(cursor(), src.fill(), available_length);\n    move_cursor(available_length);\n    src.Extract(available_length);\n    if (ABSL_PREDICT_FALSE(\n            !PushBehindScratch(SaturatingIntCast<size_t>(src.size())))) {\n      return false;\n    }\n  }\n  std::memset(cursor(), src.fill(), IntCast<size_t>(src.size()));\n  move_cursor(IntCast<size_t>(src.size()));\n  return true;\n}\n\nbool PushableWriter::FlushBehindScratch(FlushType flush_type) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::FlushBehindScratch(): \"\n         \"scratch used\";\n  return ok();\n}\n\nbool PushableWriter::SeekBehindScratch(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of PushableWriter::SeekBehindScratch(): \"\n         \"position unchanged, use Seek() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::SeekBehindScratch(): \"\n         \"scratch used\";\n  return Fail(absl::UnimplementedError(\"Writer::Seek() not supported\"));\n}\n\nstd::optional<Position> PushableWriter::SizeBehindScratch() {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::SizeBehindScratch(): \"\n         \"scratch used\";\n  Fail(absl::UnimplementedError(\"Writer::Size() not supported\"));\n  return std::nullopt;\n}\n\nbool PushableWriter::TruncateBehindScratch(Position new_size) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::TruncateBehindScratch(): \"\n         \"scratch used\";\n  return Fail(absl::UnimplementedError(\"Writer::Truncate() not supported\"));\n}\n\nReader* PushableWriter::ReadModeBehindScratch(Position initial_pos) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::ReadModeBehindScratch(): \"\n         \"scratch used\";\n  Fail(absl::UnimplementedError(\"Writer::ReadMode() not supported\"));\n  return nullptr;\n}\n\nbool PushableWriter::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of Writer::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size()) {\n      riegeli::null_safe_memcpy(cursor(), src.data(), src.size());\n      move_cursor(src.size());\n      return true;\n    }\n  }\n  return WriteBehindScratch(src);\n}\n\nbool PushableWriter::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size() && src.size() <= kMaxBytesToCopy) {\n      riegeli::null_safe_memcpy(cursor(), src.data(), src.size());\n      move_cursor(src.size());\n      return true;\n    }\n  }\n  return WriteBehindScratch(std::move(src));\n}\n\nbool PushableWriter::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size() && src.size() <= kMaxBytesToCopy) {\n      src.CopyTo(cursor());\n      move_cursor(src.size());\n      return true;\n    }\n  }\n  return WriteBehindScratch(src);\n}\n\nbool PushableWriter::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size() && src.size() <= kMaxBytesToCopy) {\n      src.CopyTo(cursor());\n      move_cursor(src.size());\n      return true;\n    }\n  }\n  return WriteBehindScratch(std::move(src));\n}\n\nbool PushableWriter::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size() && src.size() <= kMaxBytesToCopy) {\n      cord_internal::CopyCordToArray(src, cursor());\n      move_cursor(src.size());\n      return true;\n    }\n  }\n  return WriteBehindScratch(src);\n}\n\nbool PushableWriter::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size() && src.size() <= kMaxBytesToCopy) {\n      cord_internal::CopyCordToArray(src, cursor());\n      move_cursor(src.size());\n      return true;\n    }\n  }\n  return WriteBehindScratch(std::move(src));\n}\n\nbool PushableWriter::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n    if (available() >= src.size() && src.size() <= kMaxBytesToCopy) {\n      riegeli::null_safe_memset(cursor(), src.fill(),\n                                IntCast<size_t>(src.size()));\n      move_cursor(IntCast<size_t>(src.size()));\n      return true;\n    }\n  }\n  return WriteBehindScratch(src);\n}\n\nbool PushableWriter::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n  }\n  return FlushBehindScratch(flush_type);\n}\n\nbool PushableWriter::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of Writer::SeekSlow(): \"\n         \"position unchanged, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n  }\n  return SeekBehindScratch(new_pos);\n}\n\nstd::optional<Position> PushableWriter::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return std::nullopt;\n  }\n  return SizeBehindScratch();\n}\n\nbool PushableWriter::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return false;\n  }\n  return TruncateBehindScratch(new_size);\n}\n\nReader* PushableWriter::ReadModeImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(scratch_used())) {\n    if (ABSL_PREDICT_FALSE(!SyncScratch())) return nullptr;\n  }\n  return ReadModeBehindScratch(initial_pos);\n}\n\nvoid PushableWriter::BehindScratch::Enter() {\n  RIEGELI_ASSERT(context_->scratch_used())\n      << \"Failed precondition of PushableWriter::BehindScratch::Enter(): \"\n         \"scratch not used\";\n  RIEGELI_ASSERT_EQ(context_->start(), context_->scratch_->buffer.data())\n      << \"Failed invariant of PushableWriter: \"\n         \"scratch used but buffer pointers do not point to scratch\";\n  RIEGELI_ASSERT_EQ(context_->start_to_limit(),\n                    context_->scratch_->buffer.size())\n      << \"Failed invariant of PushableWriter: \"\n         \"scratch used but buffer pointers do not point to scratch\";\n  scratch_ = std::move(context_->scratch_);\n  written_to_scratch_ = context_->start_to_cursor();\n  context_->set_buffer(scratch_->original_start,\n                       scratch_->original_start_to_limit,\n                       scratch_->original_start_to_cursor);\n  context_->set_start_pos(context_->start_pos() - context_->start_to_cursor());\n}\n\nvoid PushableWriter::BehindScratch::Leave() {\n  RIEGELI_ASSERT_NE(scratch_, nullptr)\n      << \"Failed precondition of PushableWriter::BehindScratch::Leave(): \"\n         \"scratch not used\";\n  context_->set_start_pos(context_->pos());\n  scratch_->original_start = context_->start();\n  scratch_->original_start_to_limit = context_->start_to_limit();\n  scratch_->original_start_to_cursor = context_->start_to_cursor();\n  context_->set_buffer(const_cast<char*>(scratch_->buffer.data()),\n                       scratch_->buffer.size(), written_to_scratch_);\n  context_->scratch_ = std::move(scratch_);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/pushable_writer.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_PUSHABLE_WRITER_H_\n#define RIEGELI_BYTES_PUSHABLE_WRITER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/sized_shared_buffer.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nclass Reader;\n\n// Abstract class `PushableWriter` helps to implement\n// `Writer::PushSlow(min_length, recommended_length)` with `min_length > 1`.\n//\n// `PushableWriter` accumulates data to be pushed in a scratch buffer if needed.\nclass PushableWriter : public Writer {\n protected:\n  class BehindScratch;\n\n  using Writer::Writer;\n\n  PushableWriter(PushableWriter&& that) noexcept;\n  PushableWriter& operator=(PushableWriter&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `PushableWriter`. This\n  // avoids constructing a temporary `PushableWriter` and moving from it.\n  // Derived classes which redefine `Reset()` should include a call to\n  // `PushableWriter::Reset()`.\n  void Reset(Closed);\n  void Reset();\n\n  void Done() override;\n  void OnFail() override;\n\n  // Returns `true` if scratch is currently in use, which means that buffer\n  // pointers are temporarily unrelated to the destination. This is exposed for\n  // assertions.\n  bool scratch_used() const;\n\n  // `PushableWriter::{Done,FlushImpl}()` write the scratch if needed, and call\n  // `{Done,Flush}BehindScratch()` to close/flush the destination before buffer\n  // pointers are reset.\n  //\n  // For propagating `{Close,Flush}()` to dependencies, `{Done,FlushImpl}()`\n  // should be overridden to call `PushableWriter::{Done,FlushImpl}()` and then\n  // close/flush the dependencies.\n\n  // Implementation of `Done()`, called while scratch is not used, and only if\n  // writing the scratch succeeded. This is called before buffer pointers are\n  // reset.\n  //\n  // By default calls `FlushBehindScratch(FlushType::kFromObject)`, which by\n  // default does nothing.\n  //\n  // Precondition: `!scratch_used()`\n  virtual void DoneBehindScratch();\n\n  // Implementation of `PushSlow(1, recommended_length)`, called while scratch\n  // is not used.\n  //\n  // Preconditions:\n  //   `available() == 0`\n  //   `!scratch_used()`\n  virtual bool PushBehindScratch(size_t recommended_length) = 0;\n\n  // Force using scratch as the buffer.\n  //\n  // This can be used in the implementation of `PushBehindScratch()` if in some\n  // circumstances scratch should be used even when `min_length == 1`.\n  //\n  // These circumstances should be rare, otherwise performance would be poor\n  // because `Push()` would call `PushSlow()` too often.\n  //\n  // Warning: `WriteSlow(absl::string_view)` or `WriteSlow(Chain)` is\n  // called to stop using scratch by writing scratch contents, and the\n  // default implementation of `WriteSlow(absl::string_view)` calls\n  // `PushBehindScratch()`. This means that if `PushBehindScratch()` calls\n  // `ForcePushUsingScratch()`, then `WriteSlow(absl::string_view)` must be\n  // overridden to avoid indirectly calling `ForcePushUsingScratch()` or\n  // `PushSlow(min_length > 1)`.\n  //\n  // Preconditions:\n  //   `available() == 0`\n  //   `!scratch_used()`\n  //\n  // Always returns `true`.\n  //\n  // Postconditions:\n  //   `available() > 0`\n  //   `scratch_used()`\n  bool ForcePushUsingScratch();\n\n  // Implementation of `WriteSlow()`, `FlushImpl()`,`SeekSlow()`, `SizeImpl()`,\n  // `TruncateImpl()`, and `ReadModeImpl()`, called while scratch is not used.\n  //\n  // By default they are implemented analogously to the corresponding `Writer`\n  // functions.\n  //\n  // Preconditions:\n  //   like the corresponding `Writer` functions\n  //   `!scratch_used()`\n  virtual bool WriteBehindScratch(absl::string_view src);\n  virtual bool WriteBehindScratch(ExternalRef src);\n  virtual bool WriteBehindScratch(const Chain& src);\n  virtual bool WriteBehindScratch(Chain&& src);\n  virtual bool WriteBehindScratch(const absl::Cord& src);\n  virtual bool WriteBehindScratch(absl::Cord&& src);\n  virtual bool WriteBehindScratch(ByteFill src);\n  virtual bool FlushBehindScratch(FlushType flush_type);\n  virtual bool SeekBehindScratch(Position new_pos);\n  virtual std::optional<Position> SizeBehindScratch();\n  virtual bool TruncateBehindScratch(Position new_size);\n  virtual Reader* ReadModeBehindScratch(Position initial_pos);\n\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool FlushImpl(FlushType flush_type) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  bool TruncateImpl(Position new_size) override;\n  Reader* ReadModeImpl(Position initial_pos) override;\n\n private:\n  struct Scratch {\n    SizedSharedBuffer buffer;\n    char* original_start = nullptr;\n    size_t original_start_to_limit = 0;\n    size_t original_start_to_cursor = 0;\n  };\n\n  bool SyncScratch();\n\n  std::unique_ptr<Scratch> scratch_;\n\n  // Invariants if `scratch_used()`:\n  //   `start() == scratch_->buffer.data()`\n  //   `start_to_limit() == scratch_->buffer.size()`\n};\n\n// Helps to implement move constructor or move assignment if scratch is used.\n//\n// Moving the destination should be in scope of a `BehindScratch` object, unless\n// destination buffer pointers are known to remain unchanged during a move or\n// their change does not need to be reflected elsewhere.\n//\n// This temporarily reveals the relationship between the destination and the\n// buffer pointers, in case it was hidden behind scratch usage. In a\n// `BehindScratch` scope, scratch is not used, and buffer pointers may be\n// changed. The current position reflects what has been written to the\n// destination and must not be changed.\nclass PushableWriter::BehindScratch {\n public:\n  explicit BehindScratch(PushableWriter* context ABSL_ATTRIBUTE_LIFETIME_BOUND);\n\n  BehindScratch(BehindScratch&& that) = default;\n  BehindScratch& operator=(BehindScratch&&) = delete;\n\n  ~BehindScratch();\n\n private:\n  void Enter();\n  void Leave();\n\n  PushableWriter* context_;\n  std::unique_ptr<Scratch> scratch_;\n  size_t written_to_scratch_;\n};\n\n// Implementation details follow.\n\ninline PushableWriter::PushableWriter(PushableWriter&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)), scratch_(std::move(that.scratch_)) {}\n\ninline PushableWriter& PushableWriter::operator=(\n    PushableWriter&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  scratch_ = std::move(that.scratch_);\n  return *this;\n}\n\ninline void PushableWriter::Reset(Closed) {\n  Writer::Reset(kClosed);\n  scratch_.reset();\n}\n\ninline void PushableWriter::Reset() {\n  Writer::Reset();\n  if (ABSL_PREDICT_FALSE(scratch_used())) scratch_->buffer.Clear();\n}\n\ninline bool PushableWriter::scratch_used() const {\n  return scratch_ != nullptr && !scratch_->buffer.empty();\n}\n\ninline PushableWriter::BehindScratch::BehindScratch(\n    PushableWriter* context ABSL_ATTRIBUTE_LIFETIME_BOUND)\n    : context_(context) {\n  if (ABSL_PREDICT_FALSE(context_->scratch_used())) Enter();\n}\n\ninline PushableWriter::BehindScratch::~BehindScratch() {\n  if (ABSL_PREDICT_FALSE(scratch_ != nullptr)) Leave();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_PUSHABLE_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/read_all.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/read_all.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/resize_and_overwrite.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli::read_all_internal {\n\nnamespace {\n\nABSL_ATTRIBUTE_COLD absl::Status MaxLengthExceeded(Reader& src,\n                                                   Position max_length) {\n  return src.AnnotateStatus(absl::ResourceExhaustedError(\n      absl::StrCat(\"Maximum length exceeded: \", max_length)));\n}\n\nabsl::Status ReadAllImpl(Reader& src, absl::string_view& dest,\n                         size_t max_length) {\n  if (src.SupportsSize()) {\n    const std::optional<Position> size = src.Size();\n    if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n      dest = absl::string_view();\n      return src.status();\n    }\n    const Position remaining = SaturatingSub(*size, src.pos());\n    if (ABSL_PREDICT_FALSE(remaining > max_length)) {\n      if (ABSL_PREDICT_FALSE(!src.Read(max_length, dest))) {\n        if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n        return absl::OkStatus();\n      }\n      return MaxLengthExceeded(src, max_length);\n    }\n    if (ABSL_PREDICT_FALSE(!src.Read(IntCast<size_t>(remaining), dest))) {\n      if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n    }\n  } else {\n    do {\n      if (ABSL_PREDICT_FALSE(src.available() > max_length)) {\n        dest = absl::string_view(src.cursor(), max_length);\n        src.move_cursor(max_length);\n        return MaxLengthExceeded(src, max_length);\n      }\n    } while (src.Pull(src.available() + 1));\n    dest = absl::string_view(src.cursor(), src.available());\n    src.move_cursor(src.available());\n    if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n  }\n  return absl::OkStatus();\n}\n\nabsl::Status ReadAndAppendAllImpl(Reader& src, std::string& dest,\n                                  size_t max_length) {\n  if (src.SupportsSize()) {\n    const std::optional<Position> size = src.Size();\n    if (ABSL_PREDICT_FALSE(size == std::nullopt)) return src.status();\n    const Position remaining = SaturatingSub(*size, src.pos());\n    if (ABSL_PREDICT_FALSE(remaining > max_length)) {\n      if (ABSL_PREDICT_FALSE(!src.ReadAndAppend(max_length, dest))) {\n        if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n        return absl::OkStatus();\n      }\n      return MaxLengthExceeded(src, max_length);\n    }\n    if (ABSL_PREDICT_FALSE(\n            !src.ReadAndAppend(IntCast<size_t>(remaining), dest))) {\n      if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n    }\n  } else {\n    if (ABSL_PREDICT_FALSE(!src.Pull())) {\n      if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n      return absl::OkStatus();\n    }\n    size_t remaining_max_length = max_length;\n    const size_t old_size = dest.size();\n    if (src.available() < dest.capacity() - old_size) {\n      // Try to fill all remaining space in `dest`, to avoid copying through the\n      // `Chain` in case the remaining length is smaller.\n      const size_t length =\n          UnsignedMin(dest.capacity() - old_size, remaining_max_length);\n      bool read_ok;\n      size_t length_read;\n      absl::StringResizeAndOverwrite(\n          dest, old_size + length, [&](char* data, size_t size) {\n            read_ok = src.Read(size - old_size, data + old_size, &length_read);\n            return old_size + length_read;\n          });\n      if (!read_ok) {\n        if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n        return absl::OkStatus();\n      }\n      remaining_max_length -= length_read;\n    }\n    Chain buffer;\n    do {\n      if (ABSL_PREDICT_FALSE(src.available() > remaining_max_length)) {\n        src.ReadAndAppend(remaining_max_length, buffer);\n        std::move(buffer).AppendTo(dest);\n        return MaxLengthExceeded(src, max_length);\n      }\n      remaining_max_length -= src.available();\n      src.ReadAndAppend(src.available(), buffer);\n    } while (src.Pull());\n    std::move(buffer).AppendTo(dest);\n    if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n  }\n  return absl::OkStatus();\n}\n\nabsl::Status ReadAndAppendAllImpl(Reader& src, Chain& dest, size_t max_length) {\n  max_length =\n      UnsignedMin(max_length, std::numeric_limits<size_t>::max() - dest.size());\n  if (src.SupportsSize()) {\n    const std::optional<Position> size = src.Size();\n    if (ABSL_PREDICT_FALSE(size == std::nullopt)) return src.status();\n    const Position remaining = SaturatingSub(*size, src.pos());\n    if (ABSL_PREDICT_FALSE(remaining > max_length)) {\n      if (ABSL_PREDICT_FALSE(!src.ReadAndAppend(max_length, dest))) {\n        if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n        return absl::OkStatus();\n      }\n      return MaxLengthExceeded(src, max_length);\n    }\n    if (ABSL_PREDICT_FALSE(\n            !src.ReadAndAppend(IntCast<size_t>(remaining), dest))) {\n      if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n    }\n  } else {\n    size_t remaining_max_length = max_length;\n    do {\n      if (ABSL_PREDICT_FALSE(src.available() > remaining_max_length)) {\n        src.ReadAndAppend(remaining_max_length, dest);\n        return MaxLengthExceeded(src, max_length);\n      }\n      remaining_max_length -= src.available();\n      src.ReadAndAppend(src.available(), dest);\n    } while (src.Pull());\n    if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n  }\n  return absl::OkStatus();\n}\n\nabsl::Status ReadAndAppendAllImpl(Reader& src, absl::Cord& dest,\n                                  size_t max_length) {\n  max_length =\n      UnsignedMin(max_length, std::numeric_limits<size_t>::max() - dest.size());\n  if (src.SupportsSize()) {\n    const std::optional<Position> size = src.Size();\n    if (ABSL_PREDICT_FALSE(size == std::nullopt)) return src.status();\n    const Position remaining = SaturatingSub(*size, src.pos());\n    if (ABSL_PREDICT_FALSE(remaining > max_length)) {\n      if (ABSL_PREDICT_FALSE(!src.ReadAndAppend(max_length, dest))) {\n        if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n        return absl::OkStatus();\n      }\n      return MaxLengthExceeded(src, max_length);\n    }\n    if (ABSL_PREDICT_FALSE(\n            !src.ReadAndAppend(IntCast<size_t>(remaining), dest))) {\n      if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n    }\n  } else {\n    size_t remaining_max_length = max_length;\n    do {\n      if (ABSL_PREDICT_FALSE(src.available() > remaining_max_length)) {\n        src.ReadAndAppend(remaining_max_length, dest);\n        return MaxLengthExceeded(src, max_length);\n      }\n      remaining_max_length -= src.available();\n      src.ReadAndAppend(src.available(), dest);\n    } while (src.Pull());\n    if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n  }\n  return absl::OkStatus();\n}\n\n}  // namespace\n\nabsl::Status ReadAllImpl(Reader& src, absl::string_view& dest,\n                         size_t max_length, size_t* length_read) {\n  const absl::Status status = ReadAllImpl(src, dest, max_length);\n  if (length_read != nullptr) *length_read = dest.size();\n  return status;\n}\n\nabsl::Status ReadAllImpl(Reader& src, char* dest, size_t max_length,\n                         size_t* length_read) {\n  if (!src.Read(max_length, dest, length_read)) {\n    if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n    return absl::OkStatus();\n  }\n  if (ABSL_PREDICT_FALSE(src.Pull())) return MaxLengthExceeded(src, max_length);\n  return absl::OkStatus();\n}\n\nabsl::Status ReadAllImpl(Reader& src, std::string& dest, size_t max_length,\n                         size_t* length_read) {\n  dest.clear();\n  return ReadAndAppendAllImpl(src, dest, max_length, length_read);\n}\n\nabsl::Status ReadAllImpl(Reader& src, Chain& dest, size_t max_length,\n                         size_t* length_read) {\n  dest.Clear();\n  return ReadAndAppendAllImpl(src, dest, max_length, length_read);\n}\n\nabsl::Status ReadAllImpl(Reader& src, absl::Cord& dest, size_t max_length,\n                         size_t* length_read) {\n  dest.Clear();\n  return ReadAndAppendAllImpl(src, dest, max_length, length_read);\n}\n\nabsl::Status ReadAndAppendAllImpl(Reader& src, std::string& dest,\n                                  size_t max_length, size_t* length_read) {\n  if (length_read == nullptr) {\n    return ReadAndAppendAllImpl(src, dest, max_length);\n  }\n  const Position pos_before = src.pos();\n  const absl::Status status = ReadAndAppendAllImpl(src, dest, max_length);\n  RIEGELI_ASSERT_GE(src.pos(), pos_before)\n      << \"ReadAndAppendAllImpl(std::string&) decreased src.pos()\";\n  RIEGELI_ASSERT_LE(src.pos() - pos_before, max_length)\n      << \"ReadAndAppendAllImpl(std::string&) read more than requested\";\n  *length_read = IntCast<size_t>(src.pos() - pos_before);\n  return status;\n}\n\nabsl::Status ReadAndAppendAllImpl(Reader& src, Chain& dest, size_t max_length,\n                                  size_t* length_read) {\n  if (length_read == nullptr) {\n    return ReadAndAppendAllImpl(src, dest, max_length);\n  }\n  const Position pos_before = src.pos();\n  const absl::Status status = ReadAndAppendAllImpl(src, dest, max_length);\n  RIEGELI_ASSERT_GE(src.pos(), pos_before)\n      << \"ReadAndAppendAllImpl(Chain&) decreased src.pos()\";\n  RIEGELI_ASSERT_LE(src.pos() - pos_before, max_length)\n      << \"ReadAndAppendAllImpl(Chain&) read more than requested\";\n  *length_read = IntCast<size_t>(src.pos() - pos_before);\n  return status;\n}\n\nabsl::Status ReadAndAppendAllImpl(Reader& src, absl::Cord& dest,\n                                  size_t max_length, size_t* length_read) {\n  if (length_read == nullptr) {\n    return ReadAndAppendAllImpl(src, dest, max_length);\n  }\n  const Position pos_before = src.pos();\n  const absl::Status status = ReadAndAppendAllImpl(src, dest, max_length);\n  RIEGELI_ASSERT_GE(src.pos(), pos_before)\n      << \"ReadAndAppendAllImpl(absl::Cord&) decreased src.pos()\";\n  RIEGELI_ASSERT_LE(src.pos() - pos_before, max_length)\n      << \"ReadAndAppendAllImpl(absl::Cord&) read more than requested\";\n  *length_read = IntCast<size_t>(src.pos() - pos_before);\n  return status;\n}\n\n}  // namespace riegeli::read_all_internal\n"
  },
  {
    "path": "riegeli/bytes/read_all.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_READ_ALL_H_\n#define RIEGELI_BYTES_READ_ALL_H_\n\n#include <stddef.h>\n\n#include <limits>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nnamespace read_all_internal {\n\ntemplate <typename Work>\nusing StringViewCallResult =\n    decltype(std::declval<Work&&>()(std::declval<absl::string_view>()));\n\n}  // namespace read_all_internal\n\n// Combines creating a `Reader` (optionally), reading all remaining data to\n// `dest` (clearing any existing data in `dest`), and `VerifyEndAndClose()`\n// (if the `Reader` is owned).\n//\n// If `length_read != nullptr` then sets `*length_read` to the length read.\n// This is equal to the difference between `src.pos()` after and before the\n// call.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `Reader`. `Src` must support\n// `DependencyRef<Reader*, Src>`, e.g. `Reader&` (not owned),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `AnyRef<Reader*>` (maybe owned).\n//\n// Reading to `absl::string_view` is supported in two ways:\n//\n//  1. With `Src` being restricted to `Reader&`, i.e. not owned.\n//\n//     The `absl::string_view` is valid until the next non-const operation on\n//     the `Reader`.\n//\n//  2. With the `absl::string_view&` output parameter replaced with a function\n//     to be called with a parameter of type `absl::string_view`. The function\n//     is called with data read.\n//\n//     If the `Reader` is owned, it is closed after calling the function.\n//     This invalidates the `absl::string_view`.\n//\n//     For `T` being the result type of the function, the result type of\n//     `ReadAll()` generalizes `absl::StatusOr<T>` for types where that is not\n//     applicable:\n//      * `absl::StatusOr<const T>`           -> `absl::StatusOr<T>`\n//      * `absl::StatusOr<T&>`                -> `absl::StatusOr<T>`\n//      * `absl::StatusOr<T&&>`               -> `absl::StatusOr<T>`\n//      * `absl::StatusOr<void>`              -> `absl::Status`\n//      * `absl::StatusOr<absl::Status>`      -> `absl::Status`\n//      * `absl::StatusOr<absl::StatusOr<T>>` -> `absl::StatusOr<T>`\n//\n//     The function is called only if reading succeeded. If the function\n//     succeeds but closing fails, the result of the function is discarded.\nabsl::Status ReadAll(Reader& src, absl::string_view& dest,\n                     size_t max_length = std::numeric_limits<size_t>::max(),\n                     size_t* length_read = nullptr);\nabsl::Status ReadAll(Reader& src, absl::string_view& dest, size_t* length_read);\ntemplate <\n    typename Src, typename Work,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nStatusOrMakerT<read_all_internal::StringViewCallResult<Work>> ReadAll(\n    Src&& src, Work&& work,\n    size_t max_length = std::numeric_limits<size_t>::max(),\n    size_t* length_read = nullptr);\ntemplate <\n    typename Src, typename Work,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nStatusOrMakerT<read_all_internal::StringViewCallResult<Work>> ReadAll(\n    Src&& src, Work&& work, size_t* length_read);\n\n// Combines creating a `Reader` (optionally), reading all remaining data to\n// `dest` (clearing any existing data in `dest`), and `VerifyEndAndClose()`\n// (if the `Reader` is owned).\n//\n// If `length_read != nullptr` then sets `*length_read` to the length read.\n// This is equal to the difference between `src.pos()` after and before the\n// call.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `Reader`. `Src` must support\n// `DependencyRef<Reader*, Src>`, e.g. `Reader&` (not owned),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `AnyRef<Reader*>` (maybe owned).\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ReadAll(Src&& src, char* dest, size_t max_length,\n                     size_t* length_read);\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ReadAll(Src&& src, std::string& dest,\n                     size_t max_length = std::numeric_limits<size_t>::max(),\n                     size_t* length_read = nullptr);\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ReadAll(Src&& src, std::string& dest, size_t* length_read);\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ReadAll(Src&& src, Chain& dest,\n                     size_t max_length = std::numeric_limits<size_t>::max(),\n                     size_t* length_read = nullptr);\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ReadAll(Src&& src, Chain& dest, size_t* length_read);\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ReadAll(Src&& src, absl::Cord& dest,\n                     size_t max_length = std::numeric_limits<size_t>::max(),\n                     size_t* length_read = nullptr);\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ReadAll(Src&& src, absl::Cord& dest, size_t* length_read);\n\n// Combines creating a `Reader` (optionally), reading all remaining data to\n// `dest` (appending to any existing data in `dest`), and `VerifyEndAndClose()`\n// (if the `Reader` is owned).\n//\n// If `length_read != nullptr` then sets `*length_read` to the length read.\n// This is equal to the difference between `pos()` after and before the call.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `Reader`. `Src` must support\n// `DependencyRef<Reader*, Src>`, e.g. `Reader&` (not owned),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `AnyRef<Reader*>` (maybe owned).\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ReadAndAppendAll(\n    Src&& src, std::string& dest,\n    size_t max_length = std::numeric_limits<size_t>::max(),\n    size_t* length_read = nullptr);\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ReadAndAppendAll(Src&& src, std::string& dest,\n                              size_t* length_read);\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ReadAndAppendAll(\n    Src&& src, Chain& dest,\n    size_t max_length = std::numeric_limits<size_t>::max(),\n    size_t* length_read = nullptr);\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ReadAndAppendAll(Src&& src, Chain& dest, size_t* length_read);\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ReadAndAppendAll(\n    Src&& src, absl::Cord& dest,\n    size_t max_length = std::numeric_limits<size_t>::max(),\n    size_t* length_read = nullptr);\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ReadAndAppendAll(Src&& src, absl::Cord& dest, size_t* length_read);\n\n// Implementation details follow.\n\nnamespace read_all_internal {\n\nabsl::Status ReadAllImpl(Reader& src, absl::string_view& dest,\n                         size_t max_length, size_t* length_read);\nabsl::Status ReadAllImpl(Reader& src, char* dest, size_t max_length,\n                         size_t* length_read);\nabsl::Status ReadAllImpl(Reader& src, std::string& dest, size_t max_length,\n                         size_t* length_read);\nabsl::Status ReadAllImpl(Reader& src, Chain& dest, size_t max_length,\n                         size_t* length_read);\nabsl::Status ReadAllImpl(Reader& src, absl::Cord& dest, size_t max_length,\n                         size_t* length_read);\nabsl::Status ReadAndAppendAllImpl(Reader& src, std::string& dest,\n                                  size_t max_length, size_t* length_read);\nabsl::Status ReadAndAppendAllImpl(Reader& src, Chain& dest, size_t max_length,\n                                  size_t* length_read);\nabsl::Status ReadAndAppendAllImpl(Reader& src, absl::Cord& dest,\n                                  size_t max_length, size_t* length_read);\n\ntemplate <typename Src, typename Dest>\ninline absl::Status ReadAllInternal(Src&& src, Dest& dest, size_t max_length,\n                                    size_t* length_read) {\n  DependencyRef<Reader*, Src> src_dep(std::forward<Src>(src));\n  if (src_dep.IsOwning()) src_dep->SetReadAllHint(true);\n  absl::Status status = ReadAllImpl(*src_dep, dest, max_length, length_read);\n  if (src_dep.IsOwning()) {\n    if (ABSL_PREDICT_TRUE(status.ok())) src_dep->VerifyEnd();\n    if (ABSL_PREDICT_FALSE(!src_dep->Close())) status.Update(src_dep->status());\n  }\n  return status;\n}\n\ntemplate <typename Src, typename Dest>\ninline absl::Status ReadAndAppendAllInternal(Src&& src, Dest& dest,\n                                             size_t max_length,\n                                             size_t* length_read) {\n  DependencyRef<Reader*, Src> src_dep(std::forward<Src>(src));\n  if (src_dep.IsOwning()) src_dep->SetReadAllHint(true);\n  absl::Status status =\n      ReadAndAppendAllImpl(*src_dep, dest, max_length, length_read);\n  if (src_dep.IsOwning()) {\n    if (ABSL_PREDICT_TRUE(status.ok())) src_dep->VerifyEnd();\n    if (ABSL_PREDICT_FALSE(!src_dep->Close())) status.Update(src_dep->status());\n  }\n  return status;\n}\n\n}  // namespace read_all_internal\n\ninline absl::Status ReadAll(Reader& src, absl::string_view& dest,\n                            size_t max_length, size_t* length_read) {\n  return read_all_internal::ReadAllImpl(src, dest, max_length, length_read);\n}\n\ninline absl::Status ReadAll(Reader& src, absl::string_view& dest,\n                            size_t* length_read) {\n  return ReadAll(src, dest, std::numeric_limits<size_t>::max(), length_read);\n}\n\ntemplate <\n    typename Src, typename Work,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline StatusOrMakerT<read_all_internal::StringViewCallResult<Work>> ReadAll(\n    Src&& src, Work&& work, size_t max_length, size_t* length_read) {\n  using WorkResult = read_all_internal::StringViewCallResult<Work>;\n  DependencyRef<Reader*, Src> src_dep(std::forward<Src>(src));\n  if (src_dep.IsOwning()) src_dep->SetReadAllHint(true);\n  absl::string_view dest;\n  absl::Status status =\n      read_all_internal::ReadAllImpl(*src_dep, dest, max_length, length_read);\n  typename StatusOrMaker<WorkResult>::type result =\n      ABSL_PREDICT_FALSE(!status.ok())\n          ? StatusOrMaker<WorkResult>::FromStatus(std::move(status))\n          : StatusOrMaker<WorkResult>::FromWork(\n                [&]() -> WorkResult { return std::forward<Work>(work)(dest); });\n  if (src_dep.IsOwning()) {\n    if (ABSL_PREDICT_TRUE(result.ok())) src_dep->VerifyEnd();\n    if (ABSL_PREDICT_FALSE(!src_dep->Close())) {\n      StatusOrMaker<WorkResult>::Update(result, src_dep->status());\n    }\n  }\n  return result;\n}\n\ntemplate <\n    typename Src, typename Work,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline StatusOrMakerT<read_all_internal::StringViewCallResult<Work>> ReadAll(\n    Src&& src, Work&& work, size_t* length_read) {\n  return ReadAll(std::forward<Src>(src), std::forward<Work>(work),\n                 std::numeric_limits<size_t>::max(), length_read);\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\nabsl::Status ReadAll(Src&& src, char* dest, size_t max_length,\n                     size_t* length_read) {\n  DependencyRef<Reader*, Src> src_dep(std::forward<Src>(src));\n  if (src_dep.IsOwning()) src_dep->SetReadAllHint(true);\n  absl::Status status =\n      read_all_internal::ReadAllImpl(*src_dep, dest, max_length, length_read);\n  if (src_dep.IsOwning()) {\n    if (ABSL_PREDICT_TRUE(status.ok())) src_dep->VerifyEnd();\n    if (ABSL_PREDICT_FALSE(!src_dep->Close())) status.Update(src_dep->status());\n  }\n  return status;\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ReadAll(Src&& src, std::string& dest, size_t max_length,\n                            size_t* length_read) {\n  return read_all_internal::ReadAllInternal(std::forward<Src>(src), dest,\n                                            max_length, length_read);\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ReadAll(Src&& src, std::string& dest, size_t* length_read) {\n  return ReadAll(std::forward<Src>(src), dest,\n                 std::numeric_limits<size_t>::max(), length_read);\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ReadAll(Src&& src, Chain& dest, size_t max_length,\n                            size_t* length_read) {\n  return read_all_internal::ReadAllInternal(std::forward<Src>(src), dest,\n                                            max_length, length_read);\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ReadAll(Src&& src, Chain& dest, size_t* length_read) {\n  return ReadAll(std::forward<Src>(src), dest,\n                 std::numeric_limits<size_t>::max(), length_read);\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ReadAll(Src&& src, absl::Cord& dest, size_t max_length,\n                            size_t* length_read) {\n  return read_all_internal::ReadAllInternal(std::forward<Src>(src), dest,\n                                            max_length, length_read);\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ReadAll(Src&& src, absl::Cord& dest, size_t* length_read) {\n  return ReadAll(std::forward<Src>(src), dest,\n                 std::numeric_limits<size_t>::max(), length_read);\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ReadAndAppendAll(Src&& src, std::string& dest,\n                                     size_t max_length, size_t* length_read) {\n  return read_all_internal::ReadAndAppendAllInternal(\n      std::forward<Src>(src), dest, max_length, length_read);\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ReadAndAppendAll(Src&& src, std::string& dest,\n                                     size_t* length_read) {\n  return ReadAndAppendAll(std::forward<Src>(src), dest,\n                          std::numeric_limits<size_t>::max(), length_read);\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ReadAndAppendAll(Src&& src, Chain& dest, size_t max_length,\n                                     size_t* length_read) {\n  return read_all_internal::ReadAndAppendAllInternal(\n      std::forward<Src>(src), dest, max_length, length_read);\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ReadAndAppendAll(Src&& src, Chain& dest,\n                                     size_t* length_read) {\n  return ReadAndAppendAll(std::forward<Src>(src), dest,\n                          std::numeric_limits<size_t>::max(), length_read);\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ReadAndAppendAll(Src&& src, absl::Cord& dest,\n                                     size_t max_length, size_t* length_read) {\n  return read_all_internal::ReadAndAppendAllInternal(\n      std::forward<Src>(src), dest, max_length, length_read);\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ReadAndAppendAll(Src&& src, absl::Cord& dest,\n                                     size_t* length_read) {\n  return ReadAndAppendAll(std::forward<Src>(src), dest,\n                          std::numeric_limits<size_t>::max(), length_read);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_READ_ALL_H_\n"
  },
  {
    "path": "riegeli/bytes/reader.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/reader.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <limits>\n#include <memory>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/cord_buffer.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_utils.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/string_utils.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid Reader::VerifyEndImpl() {\n  if (ABSL_PREDICT_FALSE(Pull())) {\n    absl::Status status = absl::InvalidArgumentError(\"End of data expected\");\n    if (SupportsSize()) {\n      const std::optional<Position> size = Size();\n      if (size != std::nullopt) {\n        status = Annotate(status, absl::StrCat(\"remaining length: \",\n                                               SaturatingSub(*size, pos())));\n      }\n    }\n    Fail(std::move(status));\n  }\n}\n\nabsl::Status Reader::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) return Annotate(status, absl::StrCat(\"at byte \", pos()));\n  return status;\n}\n\nbool Reader::FailOverflow() {\n  return Fail(absl::ResourceExhaustedError(\"Reader position overflow\"));\n}\n\nbool Reader::Read(size_t length, std::string& dest, size_t* length_read) {\n  if (ABSL_PREDICT_TRUE(available() >= length)) {\n    // `std::string::assign()` checks for size overflow.\n    dest.assign(cursor(), length);\n    move_cursor(length);\n    if (length_read != nullptr) *length_read = length;\n    return true;\n  }\n  dest.clear();\n  if (length_read != nullptr) return ReadSlow(length, dest, *length_read);\n  return ReadSlow(length, dest);\n}\n\nbool Reader::Read(size_t length, Chain& dest, size_t* length_read) {\n  if (ABSL_PREDICT_TRUE(available() >= length && length <= kMaxBytesToCopy)) {\n    dest.Reset(absl::string_view(cursor(), length));\n    move_cursor(length);\n    if (length_read != nullptr) *length_read = length;\n    return true;\n  }\n  dest.Clear();\n  if (length_read != nullptr) return ReadSlow(length, dest, *length_read);\n  return ReadSlow(length, dest);\n}\n\nbool Reader::Read(size_t length, absl::Cord& dest, size_t* length_read) {\n  if (ABSL_PREDICT_TRUE(available() >= length && length <= kMaxBytesToCopy)) {\n    dest = absl::string_view(cursor(), length);\n    move_cursor(length);\n    if (length_read != nullptr) *length_read = length;\n    return true;\n  }\n  dest.Clear();\n  if (length_read != nullptr) return ReadSlow(length, dest, *length_read);\n  return ReadSlow(length, dest);\n}\n\nbool Reader::ReadAndAppend(size_t length, std::string& dest,\n                           size_t* length_read) {\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadAndAppend(string&): \"\n         \"string size overflow\";\n  if (ABSL_PREDICT_TRUE(available() >= length)) {\n    // `std::string::append()` checks for size overflow.\n    dest.append(cursor(), length);\n    move_cursor(length);\n    if (length_read != nullptr) *length_read = length;\n    return true;\n  }\n  RIEGELI_CHECK_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadAndAppend(string&): \"\n         \"string size overflow\";\n  if (length_read != nullptr) return ReadSlow(length, dest, *length_read);\n  return ReadSlow(length, dest);\n}\n\nbool Reader::ReadAndAppend(size_t length, Chain& dest, size_t* length_read) {\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadAndAppend(Chain&): \"\n         \"Chain size overflow\";\n  if (ABSL_PREDICT_TRUE(available() >= length && length <= kMaxBytesToCopy)) {\n    // `Chain::Append()` checks for size overflow.\n    dest.Append(absl::string_view(cursor(), length));\n    move_cursor(length);\n    if (length_read != nullptr) *length_read = length;\n    return true;\n  }\n  RIEGELI_CHECK_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadAndAppend(Chain&): \"\n         \"Chain size overflow\";\n  if (length_read != nullptr) return ReadSlow(length, dest, *length_read);\n  return ReadSlow(length, dest);\n}\n\nbool Reader::ReadAndAppend(size_t length, absl::Cord& dest,\n                           size_t* length_read) {\n  // `absl::Cord::Append()` does not check for size overflow.\n  RIEGELI_CHECK_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadAndAppend(Cord&): \"\n         \"Cord size overflow\";\n  if (ABSL_PREDICT_TRUE(available() >= length && length <= kMaxBytesToCopy)) {\n    dest.Append(absl::string_view(cursor(), length));\n    move_cursor(length);\n    if (length_read != nullptr) *length_read = length;\n    return true;\n  }\n  if (length_read != nullptr) return ReadSlow(length, dest, *length_read);\n  return ReadSlow(length, dest);\n}\n\nbool Reader::Copy(Position length, Writer& dest, Position* length_read) {\n  if (ABSL_PREDICT_TRUE(available() >= length && length <= kMaxBytesToCopy)) {\n    const absl::string_view data(cursor(), IntCast<size_t>(length));\n    move_cursor(IntCast<size_t>(length));\n    if (length_read != nullptr) *length_read = length;\n    return dest.Write(data);\n  }\n  if (length_read != nullptr) return CopySlow(length, dest, *length_read);\n  return CopySlow(length, dest);\n}\n\nbool Reader::Copy(size_t length, BackwardWriter& dest) {\n  if (ABSL_PREDICT_TRUE(available() >= length && length <= kMaxBytesToCopy)) {\n    const absl::string_view data(cursor(), length);\n    move_cursor(length);\n    return dest.Write(data);\n  }\n  return CopySlow(length, dest);\n}\n\nbool Reader::ReadSome(size_t max_length, std::string& dest,\n                      size_t* length_read) {\n  if (ABSL_PREDICT_TRUE(available() > 0) ||\n      ABSL_PREDICT_FALSE(max_length == 0)) {\n    max_length = UnsignedMin(max_length, available());\n    dest.assign(cursor(), max_length);\n    move_cursor(max_length);\n    if (length_read != nullptr) *length_read = max_length;\n    return true;\n  }\n  dest.clear();\n  if (length_read != nullptr) {\n    return ReadSomeSlow(max_length, dest, *length_read);\n  }\n  return ReadSomeSlow(max_length, dest);\n}\n\nbool Reader::ReadSome(size_t max_length, Chain& dest, size_t* length_read) {\n  if (ABSL_PREDICT_TRUE(available() > 0) ||\n      ABSL_PREDICT_FALSE(max_length == 0)) {\n    max_length = UnsignedMin(max_length, available());\n    if (ABSL_PREDICT_TRUE(max_length <= kMaxBytesToCopy)) {\n      dest.Reset(absl::string_view(cursor(), max_length));\n      move_cursor(max_length);\n      if (length_read != nullptr) *length_read = max_length;\n      return true;\n    }\n    dest.Clear();\n    if (length_read != nullptr) return ReadSlow(max_length, dest, *length_read);\n    return ReadSlow(max_length, dest);\n  }\n  dest.Clear();\n  if (length_read != nullptr) {\n    return ReadSomeSlow(max_length, dest, *length_read);\n  }\n  return ReadSomeSlow(max_length, dest);\n}\n\nbool Reader::ReadSome(size_t max_length, absl::Cord& dest,\n                      size_t* length_read) {\n  if (ABSL_PREDICT_TRUE(available() > 0) ||\n      ABSL_PREDICT_FALSE(max_length == 0)) {\n    max_length = UnsignedMin(max_length, available());\n    if (ABSL_PREDICT_TRUE(max_length <= kMaxBytesToCopy)) {\n      dest = absl::string_view(cursor(), max_length);\n      move_cursor(max_length);\n      if (length_read != nullptr) *length_read = max_length;\n      return true;\n    }\n    dest.Clear();\n    if (length_read != nullptr) return ReadSlow(max_length, dest, *length_read);\n    return ReadSlow(max_length, dest);\n  }\n  dest.Clear();\n  if (length_read != nullptr) {\n    return ReadSomeSlow(max_length, dest, *length_read);\n  }\n  return ReadSomeSlow(max_length, dest);\n}\n\nbool Reader::ReadAndAppendSome(size_t max_length, std::string& dest,\n                               size_t* length_read) {\n  RIEGELI_ASSERT_LE(max_length,\n                    std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadAndAppendSome(string&): \"\n         \"string size overflow\";\n  if (ABSL_PREDICT_TRUE(available() > 0) ||\n      ABSL_PREDICT_FALSE(max_length == 0)) {\n    max_length = UnsignedMin(max_length, available());\n    // `std::string::append()` checks for size overflow.\n    dest.append(cursor(), max_length);\n    move_cursor(max_length);\n    if (length_read != nullptr) *length_read = max_length;\n    return true;\n  }\n  RIEGELI_CHECK_LE(max_length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadAndAppendSome(string&): \"\n         \"string size overflow\";\n  if (length_read != nullptr) {\n    return ReadSomeSlow(max_length, dest, *length_read);\n  }\n  return ReadSomeSlow(max_length, dest);\n}\n\nbool Reader::ReadAndAppendSome(size_t max_length, Chain& dest,\n                               size_t* length_read) {\n  RIEGELI_ASSERT_LE(max_length,\n                    std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadAndAppendSome(Chain&): \"\n         \"Chain size overflow\";\n  if (ABSL_PREDICT_TRUE(available() > 0) ||\n      ABSL_PREDICT_FALSE(max_length == 0)) {\n    max_length = UnsignedMin(max_length, available());\n    if (ABSL_PREDICT_TRUE(max_length <= kMaxBytesToCopy)) {\n      // `Chain::Append()` checks for size overflow.\n      dest.Append(absl::string_view(cursor(), max_length));\n      move_cursor(max_length);\n      if (length_read != nullptr) *length_read = max_length;\n      return true;\n    }\n    RIEGELI_CHECK_LE(max_length,\n                     std::numeric_limits<size_t>::max() - dest.size())\n        << \"Failed precondition of Reader::ReadAndAppendSome(Chain&): \"\n           \"Chain size overflow\";\n    if (length_read != nullptr) return ReadSlow(max_length, dest, *length_read);\n    return ReadSlow(max_length, dest);\n  }\n  RIEGELI_CHECK_LE(max_length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadAndAppendSome(Chain&): \"\n         \"Chain size overflow\";\n  if (length_read != nullptr) {\n    return ReadSomeSlow(max_length, dest, *length_read);\n  }\n  return ReadSomeSlow(max_length, dest);\n}\n\nbool Reader::ReadAndAppendSome(size_t max_length, absl::Cord& dest,\n                               size_t* length_read) {\n  // `absl::Cord::Append()` does not check for size overflow.\n  RIEGELI_CHECK_LE(max_length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadAndAppendSome(Cord&): \"\n         \"Cord size overflow\";\n  if (ABSL_PREDICT_TRUE(available() > 0) ||\n      ABSL_PREDICT_FALSE(max_length == 0)) {\n    max_length = UnsignedMin(max_length, available());\n    if (ABSL_PREDICT_TRUE(max_length <= kMaxBytesToCopy)) {\n      dest.Append(absl::string_view(cursor(), max_length));\n      move_cursor(max_length);\n      if (length_read != nullptr) *length_read = max_length;\n      return true;\n    }\n    if (length_read != nullptr) return ReadSlow(max_length, dest, *length_read);\n    return ReadSlow(max_length, dest);\n  }\n  if (length_read != nullptr) {\n    return ReadSomeSlow(max_length, dest, *length_read);\n  }\n  return ReadSomeSlow(max_length, dest);\n}\n\nbool Reader::CopySome(size_t max_length, Writer& dest, size_t* length_read) {\n  if (ABSL_PREDICT_TRUE(available() > 0) ||\n      ABSL_PREDICT_FALSE(max_length == 0)) {\n    max_length = UnsignedMin(max_length, available());\n    if (ABSL_PREDICT_TRUE(max_length <= kMaxBytesToCopy)) {\n      const absl::string_view data(cursor(), max_length);\n      move_cursor(max_length);\n      if (length_read != nullptr) *length_read = max_length;\n      return dest.Write(data);\n    }\n    if (length_read != nullptr) {\n      Position length_read_pos;\n      const bool copy_ok = CopySlow(max_length, dest, length_read_pos);\n      *length_read = IntCast<size_t>(length_read_pos);\n      return copy_ok;\n    }\n    return CopySlow(max_length, dest);\n  }\n  if (length_read != nullptr) {\n    return CopySomeSlow(max_length, dest, *length_read);\n  }\n  return CopySomeSlow(max_length, dest);\n}\n\nbool Reader::ReadSlow(size_t length, char* dest) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of Reader::ReadSlow(char*): \"\n         \"enough data available, use Read(char*) instead\";\n  do {\n    const size_t available_length = available();\n    riegeli::null_safe_memcpy(dest, cursor(), available_length);\n    move_cursor(available_length);\n    dest += available_length;\n    length -= available_length;\n    if (ABSL_PREDICT_FALSE(!PullSlow(1, length))) return false;\n  } while (length > available());\n  std::memcpy(dest, cursor(), length);\n  move_cursor(length);\n  return true;\n}\n\nbool Reader::ReadSlow(size_t length, char* dest, size_t& length_read) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of Reader::ReadSlow(char*): \"\n         \"enough data available, use Read(char*) instead\";\n  const Position pos_before = pos();\n  const bool read_ok = ReadSlow(length, dest);\n  RIEGELI_ASSERT_GE(pos(), pos_before)\n      << \"Reader::ReadSlow(char*) decreased pos()\";\n  RIEGELI_ASSERT_LE(pos() - pos_before, length)\n      << \"Reader::ReadSlow(char*) read more than requested\";\n  if (ABSL_PREDICT_FALSE(!read_ok)) {\n    length_read = IntCast<size_t>(pos() - pos_before);\n    return false;\n  }\n  RIEGELI_ASSERT_EQ(pos() - pos_before, length)\n      << \"Reader::ReadSlow(char*) succeeded but read less than requested\";\n  length_read = length;\n  return true;\n}\n\nbool Reader::ReadSlow(size_t length, std::string& dest) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of Reader::ReadSlow(string&): \"\n         \"enough data available, use Read(string&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(string&): \"\n         \"string size overflow\";\n  const size_t old_size = dest.size();\n  bool read_ok;\n  riegeli::StringResizeAndOverwriteAmortized(\n      dest, old_size + length, [&](char* data, size_t size) {\n        size_t length_read;\n        read_ok = ReadSlow(size - old_size, data + old_size, length_read);\n        return old_size + length_read;\n      });\n  return read_ok;\n}\n\nbool Reader::ReadSlow(size_t length, std::string& dest, size_t& length_read) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of Reader::ReadSlow(string&): \"\n         \"enough data available, use Read(string&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(string&): \"\n         \"string size overflow\";\n  const Position pos_before = pos();\n  const bool read_ok = ReadSlow(length, dest);\n  RIEGELI_ASSERT_GE(pos(), pos_before)\n      << \"Reader::ReadSlow(string&) decreased pos()\";\n  RIEGELI_ASSERT_LE(pos() - pos_before, length)\n      << \"Reader::ReadSlow(string&) read more than requested\";\n  if (ABSL_PREDICT_FALSE(!read_ok)) {\n    length_read = IntCast<size_t>(pos() - pos_before);\n    return false;\n  }\n  RIEGELI_ASSERT_EQ(pos() - pos_before, length)\n      << \"Reader::ReadSlow(string&) succeeded but read less than requested\";\n  length_read = length;\n  return true;\n}\n\nbool Reader::ReadSlow(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"Chain size overflow\";\n  do {\n    const absl::Span<char> buffer = dest.AppendBuffer(1, length, length);\n    size_t length_read;\n    if (ABSL_PREDICT_FALSE(!Read(buffer.size(), buffer.data(), &length_read))) {\n      dest.RemoveSuffix(buffer.size() - length_read);\n      return false;\n    }\n    length -= length_read;\n  } while (length > 0);\n  return true;\n}\n\nbool Reader::ReadSlow(size_t length, Chain& dest, size_t& length_read) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"Chain size overflow\";\n  const Position pos_before = pos();\n  const bool read_ok = ReadSlow(length, dest);\n  RIEGELI_ASSERT_GE(pos(), pos_before)\n      << \"Reader::ReadSlow(Chain&) decreased pos()\";\n  RIEGELI_ASSERT_LE(pos() - pos_before, length)\n      << \"Reader::ReadSlow(Chain&) read more than requested\";\n  if (ABSL_PREDICT_FALSE(!read_ok)) {\n    length_read = IntCast<size_t>(pos() - pos_before);\n    return false;\n  }\n  RIEGELI_ASSERT_EQ(pos() - pos_before, length)\n      << \"Reader::ReadSlow(Chain&) succeeded but read less than requested\";\n  length_read = length;\n  return true;\n}\n\nbool Reader::ReadSlow(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"Cord size overflow\";\n  absl::CordBuffer buffer = dest.GetCustomAppendBuffer(\n      cord_internal::kCordBufferBlockSize, length, 1);\n  absl::Span<char> span = buffer.available_up_to(length);\n  if (buffer.capacity() < kDefaultMinBlockSize && length > span.size()) {\n    absl::CordBuffer new_buffer = absl::CordBuffer::CreateWithCustomLimit(\n        cord_internal::kCordBufferBlockSize, buffer.length() + length);\n    std::memcpy(new_buffer.data(), buffer.data(), buffer.length());\n    new_buffer.SetLength(buffer.length());\n    buffer = std::move(new_buffer);\n    span = buffer.available_up_to(length);\n  }\n  for (;;) {\n    size_t length_read;\n    const bool read_ok = Read(span.size(), span.data(), &length_read);\n    buffer.IncreaseLengthBy(length_read);\n    dest.Append(std::move(buffer));\n    if (ABSL_PREDICT_FALSE(!read_ok)) return false;\n    length -= length_read;\n    if (length == 0) return true;\n    buffer = absl::CordBuffer::CreateWithCustomLimit(\n        cord_internal::kCordBufferBlockSize, length);\n    span = buffer.available_up_to(length);\n  }\n}\n\nbool Reader::ReadSlow(size_t length, absl::Cord& dest, size_t& length_read) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"Cord size overflow\";\n  const Position pos_before = pos();\n  const bool read_ok = ReadSlow(length, dest);\n  RIEGELI_ASSERT_GE(pos(), pos_before)\n      << \"Reader::ReadSlow(Cord&) decreased pos()\";\n  RIEGELI_ASSERT_LE(pos() - pos_before, length)\n      << \"Reader::ReadSlow(Cord&) read more than requested\";\n  if (ABSL_PREDICT_FALSE(!read_ok)) {\n    length_read = IntCast<size_t>(pos() - pos_before);\n    return false;\n  }\n  RIEGELI_ASSERT_EQ(pos() - pos_before, length)\n      << \"Reader::ReadSlow(Cord&) succeeded but read less than requested\";\n  length_read = length;\n  return true;\n}\n\nbool Reader::CopySlow(Position length, Writer& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  while (length > available()) {\n    const absl::string_view data(cursor(), available());\n    move_cursor(data.size());\n    if (ABSL_PREDICT_FALSE(!dest.Write(data))) return false;\n    length -= data.size();\n    if (ABSL_PREDICT_FALSE(!PullSlow(1, length))) return false;\n  }\n  const absl::string_view data(cursor(), IntCast<size_t>(length));\n  move_cursor(IntCast<size_t>(length));\n  return dest.Write(data);\n}\n\nbool Reader::CopySlow(Position length, Writer& dest, Position& length_read) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  const Position pos_before = pos();\n  const bool copy_ok = CopySlow(length, dest);\n  RIEGELI_ASSERT_GE(pos(), pos_before)\n      << \"Reader::CopySlow(Writer&) decreased pos()\";\n  RIEGELI_ASSERT_LE(pos() - pos_before, length)\n      << \"Reader::CopySlow(Writer&) read more than requested\";\n  if (ABSL_PREDICT_FALSE(!copy_ok)) {\n    length_read = pos() - pos_before;\n    return false;\n  }\n  RIEGELI_ASSERT_EQ(pos() - pos_before, length)\n      << \"Reader::CopySlow(Writer&) succeeded but read less than requested\";\n  length_read = length;\n  return true;\n}\n\nbool Reader::CopySlow(size_t length, BackwardWriter& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(BackwardWriter&): \"\n         \"enough data available, use Copy(BackwardWriter&) instead\";\n  if (length <= available()) {\n    const absl::string_view data(cursor(), length);\n    move_cursor(length);\n    return dest.Write(data);\n  }\n  if (length <= kMaxBytesToCopy) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(length))) return false;\n    dest.move_cursor(length);\n    if (ABSL_PREDICT_FALSE(!ReadSlow(length, dest.cursor()))) {\n      dest.set_cursor(dest.cursor() + length);\n      return false;\n    }\n    return true;\n  }\n  Chain data;\n  if (ABSL_PREDICT_FALSE(!ReadSlow(length, data))) return false;\n  return dest.Write(std::move(data));\n}\n\nbool Reader::ReadSomeSlow(size_t max_length, char* dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"nothing to read, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"some data available, use ReadSome(char*) instead\";\n  if (ABSL_PREDICT_FALSE(!PullSlow(1, max_length))) return false;\n  max_length = UnsignedMin(max_length, available());\n  std::memcpy(dest, cursor(), max_length);\n  move_cursor(max_length);\n  return true;\n}\n\nbool Reader::ReadSomeSlow(size_t max_length, char* dest, size_t& length_read) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"nothing to read, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"some data available, use ReadSome(char*) instead\";\n  const Position pos_before = pos();\n  const bool read_ok = ReadSomeSlow(max_length, dest);\n  RIEGELI_ASSERT_GE(pos(), pos_before)\n      << \"Reader::ReadSomeSlow(char*) decreased pos()\";\n  RIEGELI_ASSERT_LE(pos() - pos_before, max_length)\n      << \"Reader::ReadSomeSlow(char*) read more than requested\";\n  length_read = IntCast<size_t>(pos() - pos_before);\n  if (!read_ok) {\n    RIEGELI_ASSERT_EQ(length_read, 0u)\n        << \"Reader::ReadSomeSlow(char*) failed but read some\";\n  } else {\n    RIEGELI_ASSERT_GT(length_read, 0u)\n        << \"Reader::ReadSomeSlow(char*) succeeded but read none\";\n  }\n  return read_ok;\n}\n\nbool Reader::ReadSomeSlow(size_t max_length, std::string& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(string&): \"\n         \"nothing to read, use ReadSome(string&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(string&): \"\n         \"some data available, use ReadSome(string&) instead\";\n  RIEGELI_ASSERT_LE(max_length,\n                    std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSomeSlow(string&): \"\n         \"string size overflow\";\n  const size_t old_size = dest.size();\n  bool read_ok;\n  riegeli::StringResizeAndOverwriteAmortized(\n      dest, old_size + max_length, [&](char* data, size_t size) {\n        size_t length_read;\n        read_ok = ReadSomeSlow(size - old_size, data + old_size, length_read);\n        return old_size + length_read;\n      });\n  return read_ok;\n}\n\nbool Reader::ReadSomeSlow(size_t max_length, std::string& dest,\n                          size_t& length_read) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(string&): \"\n         \"nothing to read, use ReadSome(string&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(string&): \"\n         \"some data available, use ReadSome(string&) instead\";\n  RIEGELI_ASSERT_LE(max_length,\n                    std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSomeSlow(string&): \"\n         \"string size overflow\";\n  const Position pos_before = pos();\n  const bool read_ok = ReadSomeSlow(max_length, dest);\n  RIEGELI_ASSERT_GE(pos(), pos_before)\n      << \"Reader::ReadSomeSlow(string&) decreased pos()\";\n  RIEGELI_ASSERT_LE(pos() - pos_before, max_length)\n      << \"Reader::ReadSomeSlow(string&) read more than requested\";\n  length_read = IntCast<size_t>(pos() - pos_before);\n  if (!read_ok) {\n    RIEGELI_ASSERT_EQ(length_read, 0u)\n        << \"Reader::ReadSomeSlow(string&) failed but read some\";\n  } else {\n    RIEGELI_ASSERT_GT(length_read, 0u)\n        << \"Reader::ReadSomeSlow(string&) succeeded but read none\";\n  }\n  return read_ok;\n}\n\nbool Reader::ReadSomeSlow(size_t max_length, Chain& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(Chain&): \"\n         \"nothing to read, use ReadSome(Chain&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(Chain&): \"\n         \"some data available, use ReadSome(Chain&) instead\";\n  RIEGELI_ASSERT_LE(max_length,\n                    std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSomeSlow(Chain&): \"\n         \"Chain size overflow\";\n  if (ABSL_PREDICT_FALSE(!PullSlow(1, max_length))) return false;\n  max_length = UnsignedMin(max_length, available());\n  // Should always succeed.\n  return ReadAndAppend(max_length, dest);\n}\n\nbool Reader::ReadSomeSlow(size_t max_length, Chain& dest, size_t& length_read) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(Chain&): \"\n         \"nothing to read, use ReadSome(Chain&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(Chain&): \"\n         \"some data available, use ReadSome(Chain&) instead\";\n  RIEGELI_ASSERT_LE(max_length,\n                    std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSomeSlow(Chain&): \"\n         \"Chain size overflow\";\n  const Position pos_before = pos();\n  const bool read_ok = ReadSomeSlow(max_length, dest);\n  RIEGELI_ASSERT_GE(pos(), pos_before)\n      << \"Reader::ReadSomeSlow(Chain&) decreased pos()\";\n  RIEGELI_ASSERT_LE(pos() - pos_before, max_length)\n      << \"Reader::ReadSomeSlow(Chain&) read more than requested\";\n  length_read = IntCast<size_t>(pos() - pos_before);\n  if (!read_ok) {\n    RIEGELI_ASSERT_EQ(length_read, 0u)\n        << \"Reader::ReadSomeSlow(Chain&) failed but read some\";\n  } else {\n    RIEGELI_ASSERT_GT(length_read, 0u)\n        << \"Reader::ReadSomeSlow(Chain&) succeeded but read none\";\n  }\n  return read_ok;\n}\n\nbool Reader::ReadSomeSlow(size_t max_length, absl::Cord& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(Cord&): \"\n         \"nothing to read, use ReadSome(Cord&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(Cord&): \"\n         \"some data available, use ReadSome(Cord&) instead\";\n  RIEGELI_ASSERT_LE(max_length,\n                    std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSomeSlow(Cord&): \"\n         \"Cord size overflow\";\n  if (ABSL_PREDICT_FALSE(!PullSlow(1, max_length))) return false;\n  max_length = UnsignedMin(max_length, available());\n  // Should always succeed.\n  return ReadAndAppend(max_length, dest);\n}\n\nbool Reader::ReadSomeSlow(size_t max_length, absl::Cord& dest,\n                          size_t& length_read) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(Cord&): \"\n         \"nothing to read, use ReadSome(Cord&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(Cord&): \"\n         \"some data available, use ReadSome(Cord&) instead\";\n  RIEGELI_ASSERT_LE(max_length,\n                    std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSomeSlow(Cord&): \"\n         \"Cord size overflow\";\n  const Position pos_before = pos();\n  const bool read_ok = ReadSomeSlow(max_length, dest);\n  RIEGELI_ASSERT_GE(pos(), pos_before)\n      << \"Reader::ReadSomeSlow(Cord&) decreased pos()\";\n  RIEGELI_ASSERT_LE(pos() - pos_before, max_length)\n      << \"Reader::ReadSomeSlow(Cord&) read more than requested\";\n  length_read = IntCast<size_t>(pos() - pos_before);\n  if (!read_ok) {\n    RIEGELI_ASSERT_EQ(length_read, 0u)\n        << \"Reader::ReadSomeSlow(Cord&) failed but read some\";\n  } else {\n    RIEGELI_ASSERT_GT(length_read, 0u)\n        << \"Reader::ReadSomeSlow(Cord&) succeeded but read none\";\n  }\n  return read_ok;\n}\n\nbool Reader::CopySomeSlow(size_t max_length, Writer& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"nothing to read, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"some data available, use CopySome(Writer&) instead\";\n  if (ABSL_PREDICT_FALSE(!PullSlow(1, max_length))) return false;\n  max_length = UnsignedMin(max_length, available());\n  if (available() >= max_length && max_length <= kMaxBytesToCopy) {\n    const absl::string_view data(cursor(), max_length);\n    move_cursor(max_length);\n    return dest.Write(data);\n  }\n  return CopySlow(max_length, dest);\n}\n\nbool Reader::CopySomeSlow(size_t max_length, Writer& dest,\n                          size_t& length_read) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"nothing to read, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"some data available, use CopySome(Writer&) instead\";\n  const Position pos_before = pos();\n  const bool copy_ok = CopySomeSlow(max_length, dest);\n  RIEGELI_ASSERT_GE(pos(), pos_before)\n      << \"Reader::CopySomeSlow(Writer&) decreased pos()\";\n  RIEGELI_ASSERT_LE(pos() - pos_before, max_length)\n      << \"Reader::CopySomeSlow(Writer&) read more than requested\";\n  length_read = IntCast<size_t>(pos() - pos_before);\n  if (!copy_ok) {\n    if (dest.ok()) {\n      RIEGELI_ASSERT_EQ(length_read, 0u)\n          << \"Reader::CopySomeSlow(Writer&) failed but read some\";\n    }\n  } else {\n    RIEGELI_ASSERT_GT(length_read, 0u)\n        << \"Reader::CopySomeSlow(Writer&) succeeded but read none\";\n  }\n  return copy_ok;\n}\n\nvoid Reader::ReadHintSlow(size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::ReadHintSlow(): \"\n         \"enough data available, use ReadHint() instead\";\n}\n\nbool Reader::SyncImpl(SyncType sync_type) { return ok(); }\n\nbool Reader::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of Reader::SeekSlow(): \"\n         \"position in the buffer, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(new_pos <= limit_pos())) {\n    return Fail(\n        absl::UnimplementedError(\"Reader::Seek() backwards not supported\"));\n  }\n  // Seeking forwards.\n  do {\n    move_cursor(available());\n    if (ABSL_PREDICT_FALSE(!PullSlow(1, 0))) return false;\n  } while (new_pos > limit_pos());\n  const Position available_length = limit_pos() - new_pos;\n  RIEGELI_ASSERT_LE(available_length, start_to_limit())\n      << \"Reader::PullSlow() skipped some data\";\n  set_cursor(limit() - available_length);\n  return true;\n}\n\nstd::optional<Position> Reader::SizeImpl() {\n  Fail(absl::UnimplementedError(\"Reader::Size() not supported\"));\n  return std::nullopt;\n}\n\nstd::unique_ptr<Reader> Reader::NewReaderImpl(Position initial_pos) {\n  Fail(absl::UnimplementedError(\"Reader::NewReader() not supported\"));\n  return nullptr;\n}\n\nstd::unique_ptr<Reader> Reader::NewReaderCurrentPosImpl() {\n  return NewReader(pos());\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/reader.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_READER_H_\n#define RIEGELI_BYTES_READER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/macros.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// Abstract class `Reader` reads sequences of bytes from a source. The nature of\n// the source depends on the particular class derived from `Reader`.\n//\n// A `Reader` object manages a buffer of data pulled from the source, which\n// amortizes the overhead of pulling data over multiple reads. Data can be read\n// directly from the buffer, and classes derived from `Reader` can avoid copying\n// by allocating the buffer in a way which fits the source, e.g. pointing it to\n// a fragment of the source itself.\n//\n// All `Reader`s support reading data sequentially and querying for the current\n// position. Some `Reader`s also support random access: changing the position\n// backwards for subsequent operations and querying for the total size of the\n// source.\nclass Reader : public Object {\n public:\n  // If `read_all_hint` is `true`, hints that all remaining bytes will be read\n  // sequentially, then `VerifyEndAndClose()` will be called. This also\n  // indicates that reading ahead more than immediately needed is acceptable,\n  // as if `ToleratesReadingAhead()` was `true`.\n  //\n  // This may improve performance and memory usage:\n  //  * More data than immediately needed may be read ahead, with\n  //    `ToleratesReadingAhead()` possibly becoming `true` even for reading from\n  //    an interactive stream.\n  //  * Larger buffer sizes may be used.\n  //  * This hint may be propagated to owned sources.\n  //  * Other consequences are possible.\n  //\n  // If the hint turns out to not match reality, nothing breaks, assuming that\n  // reading ahead more than immediately needed is acceptable.\n  //\n  // `SetReadAllHint()` is usually be called from the same abstraction layer\n  // which later calls `VerifyEndAndClose()`.\n  void SetReadAllHint(bool read_all_hint);\n\n  // Verifies that the source ends at the current position, failing the `Reader`\n  // with an `absl::InvalidArgumentError()` if not. Closes the `Reader`.\n  //\n  // This is an alternative to `Close()` if the presence of unread data at the\n  // current position should be treated as an error.\n  //\n  // If `*this` reads data from an owned source, such as a decompressor reading\n  // compressed data, then generally the source is verified too.\n  //\n  // Return values:\n  //  * `true`  - success (the source ends at the former current position)\n  //  * `false` - failure (the source does not end at the former current\n  //                       position or the `Reader` was not OK before closing)\n  bool VerifyEndAndClose();\n\n  // Verifies that the source ends at the current position, failing the `Reader`\n  // with an `absl::InvalidArgumentError()` if not.\n  //\n  // If `*this` reads data from an owned source, such as a decompressor reading\n  // compressed data, then generally the source is verified too.\n  void VerifyEnd();\n\n  // Ensures that enough data are available in the buffer: if less than\n  // `min_length` of data is available, pulls more data from the source, and\n  // points `cursor()` and `limit()` to data following the current position\n  // with length at least `min_length`, preferably `recommended_length`.\n  //\n  // The current position does not change with `Pull()`. It changes with e.g.\n  // `move_cursor()` and `Read()`.\n  //\n  // If `recommended_length < min_length`, `recommended_length` is assumed to be\n  // `min_length`.\n  //\n  // Return values:\n  //  * `true`                 - success (`available() >= min_length`)\n  //  * `false` (when `ok()`)  - source ends (`available() < min_length`)\n  //  * `false` (when `!ok()`) - failure (`available() < min_length`)\n  bool Pull(size_t min_length = 1, size_t recommended_length = 0);\n\n  // Buffer pointers. Data between `start()` and `limit()` are available for\n  // immediate reading, with `cursor()` pointing to the current position.\n  //\n  // Non-const member functions may change buffer pointers, including changing\n  // how much data around the current position are buffered.\n  //\n  // Invariants:\n  //   `start() <= cursor() <= limit()` (possibly all `nullptr`)\n  //   if `!is_open()` then `start() == cursor() == limit() == nullptr`\n  const char* start() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return start_; }\n  const char* cursor() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return cursor_; }\n  const char* limit() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return limit_; }\n\n  // Increments the value of `cursor()`. Does not change `start()` nor\n  // `limit()`. Call this during reading data under `cursor()` to indicate how\n  // much was read.\n  //\n  // Precondition: `length <= available()`\n  void move_cursor(size_t length);\n\n  // Sets the value of `cursor()`. Does not change `start()` nor `limit()`. Call\n  // this during reading data under `cursor()` to indicate how much was read, or\n  // to seek within the buffer.\n  //\n  // Precondition: `start() <= cursor <= limit()`\n  void set_cursor(const char* cursor);\n\n  // Returns the amount of data available in the buffer, between `cursor()` and\n  // `limit()`.\n  //\n  // It is possible that a `Reader` has a looming failure:\n  // `!ok() && available() > 0`. This means that the source failed but some data\n  // are already buffered and can be read before experiencing the failure.\n  //\n  // Invariant: if `!is_open()` then `available() == 0`\n  size_t available() const { return PtrDistance(cursor_, limit_); }\n\n  // Returns the buffer size, between `start()` and `limit()`.\n  size_t start_to_limit() const { return PtrDistance(start_, limit_); }\n\n  // Returns the amount of data read from the buffer, between `start()` and\n  // `cursor()`.\n  size_t start_to_cursor() const { return PtrDistance(start_, cursor_); }\n\n  // Reads a single byte from the buffer or the source.\n  //\n  // Return values:\n  //  * `true`                 - success (`dest` is set)\n  //  * `false` (when `ok()`)  - source ends (`dest` is undefined)\n  //  * `false` (when `!ok()`) - failure (`dest` is undefined)\n  bool ReadByte(uint8_t& dest);\n\n  // Reads a fixed number of bytes from the buffer and/or the source to `dest`,\n  // clearing any existing data in `dest`.\n  //\n  // `Read(absl::string_view&)` points `dest` to an array holding the data. The\n  // array is valid until the next non-const operation on the `Reader`.\n  //\n  // If `length_read != nullptr` then sets `*length_read` to the length read.\n  // This is equal to the difference between `pos()` after and before the call,\n  // and is equal to `length` if `Read()` returned `true`.\n  //\n  // Return values:\n  //  * `true`                 - success (`length` bytes read)\n  //  * `false` (when `ok()`)  - source ends (less than `length` bytes read)\n  //  * `false` (when `!ok()`) - failure (less than `length` bytes read)\n  bool Read(char& dest);\n  bool Read(size_t length, absl::string_view& dest,\n            size_t* length_read = nullptr);\n  bool Read(size_t length, char* dest, size_t* length_read = nullptr);\n  bool Read(size_t length, std::string& dest, size_t* length_read = nullptr);\n  bool Read(size_t length, Chain& dest, size_t* length_read = nullptr);\n  bool Read(size_t length, absl::Cord& dest, size_t* length_read = nullptr);\n\n  // Reads a fixed number of bytes from the buffer and/or the source to `dest`,\n  // appending to any existing data in `dest`.\n  //\n  // If `length_read != nullptr` then sets `*length_read` to the length read.\n  // This is equal to the difference between `pos()` or `dest.size()` after and\n  // before the call, and is equal to `length` if `ReadAndAppend()` returned\n  // `true`.\n  //\n  // Precondition for `ReadAndAppend(std::string&)`, `ReadAndAppend(Chain&)`,\n  // and `ReadAndAppend(absl::Cord&)`:\n  //   `length <= std::numeric_limits<size_t>::max() - dest->size()`\n  //\n  // Return values:\n  //  * `true`                 - success (`length` bytes read)\n  //  * `false` (when `ok()`)  - source ends (less than `length` bytes read)\n  //  * `false` (when `!ok()`) - failure (less than `length` bytes read)\n  bool ReadAndAppend(size_t length, std::string& dest,\n                     size_t* length_read = nullptr);\n  bool ReadAndAppend(size_t length, Chain& dest, size_t* length_read = nullptr);\n  bool ReadAndAppend(size_t length, absl::Cord& dest,\n                     size_t* length_read = nullptr);\n\n  // Reads a fixed number of bytes from the buffer and/or the source to `dest`.\n  //\n  // If `length_read != nullptr` then sets `*length_read` to the length read.\n  // This is equal to the difference between `pos()` after and before the call,\n  // and is equal to `length` if `Copy()` returned `true`.\n  //\n  // Return values:\n  //  * `true`                             - success (`length` bytes copied)\n  //  * `false` (when `dest.ok() && ok()`) - source ends (`Writer`: less than\n  //                                         `length` bytes copied;\n  //                                         `BackwardWriter`: nothing copied)\n  //  * `false` (when `!dest.ok()`)        - destination failed (data lost)\n  //  * `false` (when `!ok()`)             - source failed (`Writer`: less than\n  //                                         `length` bytes copied;\n  //                                         `BackwardWriter`: nothing copied)\n  bool Copy(Position length, Writer& dest, Position* length_read = nullptr);\n  bool Copy(size_t length, BackwardWriter& dest);\n\n  // Reads at most `max_length` bytes from the buffer and/or the source to\n  // `dest`, clearing any existing data in `dest`.\n  //\n  // In contrast to `Read()`, `ReadSome()` may read less than `max_length`\n  // before reaching the end of the source if less data are available earlier.\n  //\n  // `ReadSome(absl::string_view&)` points `dest` to an array holding the data.\n  // The array is valid until the next non-const operation on the `Reader`.\n  //\n  // If `length_read != nullptr` then sets `*length_read` to the length read.\n  // This is equal to the difference between `pos()` after and before the call.\n  //\n  // Return values:\n  //  * `true`                 - success (some bytes read or `max_length == 0`)\n  //  * `false` (when `ok()`)  - source ends\n  //                                     (no bytes read and `max_length > 0`)\n  //  * `false` (when `!ok()`) - failure (no bytes read and `max_length > 0`)\n  bool ReadSome(size_t max_length, absl::string_view& dest,\n                size_t* length_read = nullptr);\n  bool ReadSome(size_t max_length, char* dest, size_t* length_read = nullptr);\n  bool ReadSome(size_t max_length, std::string& dest,\n                size_t* length_read = nullptr);\n  bool ReadSome(size_t max_length, Chain& dest, size_t* length_read = nullptr);\n  bool ReadSome(size_t max_length, absl::Cord& dest,\n                size_t* length_read = nullptr);\n\n  // Reads at most `max_length` bytes from the buffer and/or the source to\n  // `dest`, appending to any existing data in `dest`.\n  //\n  // In contrast to `ReadAndAppend()`, `ReadAndAppendSome()` may read less than\n  // `max_length` before reaching the end of the source if less data are\n  // available earlier.\n  //\n  // If `length_read != nullptr` then sets `*length_read` to the length read.\n  // This is equal to the difference between `pos()` after and before the call.\n  //\n  // Precondition for `ReadAndAppendSome(std::string&)`,\n  // `ReadAndAppendSome(Chain&)`, and `ReadAndAppend(absl::Cord&)`:\n  //   `max_length == 0 || dest->size() < std::numeric_limits<size_t>::max()`\n  //\n  // Return values:\n  //  * `true`                 - success (some bytes read or `max_length == 0`)\n  //  * `false` (when `ok()`)  - source ends\n  //                                     (no bytes read and `max_length > 0`)\n  //  * `false` (when `!ok()`) - failure (no bytes read and `max_length > 0`)\n  bool ReadAndAppendSome(size_t max_length, std::string& dest,\n                         size_t* length_read = nullptr);\n  bool ReadAndAppendSome(size_t max_length, Chain& dest,\n                         size_t* length_read = nullptr);\n  bool ReadAndAppendSome(size_t max_length, absl::Cord& dest,\n                         size_t* length_read = nullptr);\n\n  // Reads at most `max_length` bytes from the buffer and/or the source to\n  // `dest`.\n  //\n  // In contrast to `Copy()`, `CopySome()` may read less than `max_length`\n  // before reaching the end of the source if less data are available earlier.\n  //\n  // If `length_read != nullptr` then sets `*length_read` to the length read.\n  // This is equal to the difference between `pos()` after and before the call.\n  //\n  // Return values:\n  //  * `true`                             - success (some bytes copied\n  //                                         or `max_length == 0`)\n  //  * `false` (when `dest.ok() && ok()`) - source ends (no bytes copied\n  //                                         and `max_length > 0`)\n  //  * `false` (when `!dest.ok()`)        - destination failed (no bytes copied\n  //                                         and `max_length > 0`)\n  //  * `false` (when `!ok()`)             - source failed (no bytes copied\n  //                                         and `max_length > 0`)\n  bool CopySome(size_t max_length, Writer& dest, size_t* length_read = nullptr);\n\n  // Hints that several consecutive `Pull()`, `Read()`, or `Copy()` calls will\n  // follow, reading this amount of data in total.\n  //\n  // This can make these calls faster by prefetching all the data at once into\n  // memory. In contrast to `Pull()`, the data are not necessarily flattened\n  // into a single array.\n  //\n  // `ReadHint()` ensures that at least `min_length`, preferably\n  // `recommended_length` is available in memory.\n  //\n  // If `recommended_length < min_length`, `recommended_length` is assumed to be\n  // `min_length`.\n  void ReadHint(size_t min_length = 1, size_t recommended_length = 0);\n\n  // Synchronizes the current position to the source and discards buffered data\n  // read from the source (if applicable).\n  //\n  // In contrast to `Close()`, keeps the possibility to read more data later.\n  // What exactly does it mean for the position to be synchronized depends on\n  // the source. If this is not applicable or not feasible, does nothing.\n  //\n  // The scope of objects to synchronize is specified by `sync_type`:\n  //  * `SyncType::kFromObject`  - Propagates synchronization through owned\n  //                               dependencies of the given reader.\n  //  * `SyncType::kFromProcess` - Propagates synchronization through all\n  //                               dependencies of the given reader.\n  //                               This is the default.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool Sync(SyncType sync_type = SyncType::kFromProcess);\n\n  // Returns `true` if reading ahead more than needed is known to be tolerable.\n  // This might not be the case e.g. for reading from an interactive stream,\n  // when it might be important to ensure returning with available data before\n  // waiting for more data.\n  //\n  // This can be used for optimizations, e.g. to fill a whole allocated buffer\n  // instead of a partial buffer, and thus avoid returning a `Chain` or `Cord`\n  // pointing to a partially empty block.\n  virtual bool ToleratesReadingAhead() { return false; }\n\n  // Returns the current position.\n  //\n  // This is often 0 after creating the `Reader`, but not necessarily if the\n  // `Reader` wraps another reader or input stream propagating its position.\n  //\n  // `pos()` is unchanged by `Close()`.\n  Position pos() const;\n\n  // Returns the position corresponding to `start()`,\n  // i.e. `pos() - start_to_cursor()`.\n  Position start_pos() const;\n\n  // Returns the position corresponding to `limit()`,\n  // i.e. `pos() + available()`.\n  Position limit_pos() const { return limit_pos_; }\n\n  // Returns `true` if this `Reader` supports efficient `Seek()`, `Skip()`,\n  // `Size()`, and also `NewReader()` if `SupportsNewReader()` is `true`.\n  //\n  // Invariant: if `SupportsRandomAccess()` then `SupportsRewind()`\n  //                                         and `SupportsSize()`\n  virtual bool SupportsRandomAccess() { return false; }\n\n  // Returns `true` if this `Reader` supports `Seek()` backwards (`Seek()`\n  // forwards is always supported).\n  //\n  // Even if `SupportsRewind()` is `true`, `Seek()` can be inefficient if\n  // `SupportsRandomAccess()` is `false`.\n  //\n  // Invariant: if `SupportsRandomAccess()` then `SupportsRewind()`\n  virtual bool SupportsRewind() { return SupportsRandomAccess(); }\n\n  // Sets the current position for subsequent operations.\n  //\n  // Return values:\n  //  * `true`                 - success (position is set to `new_pos`)\n  //  * `false` (when `ok()`)  - source ends before `new_pos`\n  //                             (position is set to the end)\n  //  * `false` (when `!ok()`) - failure\n  //\n  // `Seek()` forwards (or backwards but within the buffer) is always supported,\n  // although if `SupportsRandomAccess()` is `false`, then it is as inefficient\n  // as reading and discarding the intervening data.\n  //\n  // `Seek()` backwards is supported and efficient if `SupportsRandomAccess()`\n  // is `true`. Otherwise, if `SupportsRewind()` is `true`, `Seek()` backwards\n  // is as inefficient as seeking to 0, and then reading and discarding the\n  // intervening data. If `SupportsRewind()` is `false`, `Seek()` backwards is\n  // not supported.\n  bool Seek(Position new_pos);\n\n  // Increments the current position. Same as `Seek(pos() + length)` if there is\n  // no overflow.\n  //\n  // The current position might decrease if the source size decreased.\n  //\n  // If `length_skipped != nullptr` then sets `*length_skipped` to the length\n  // skipped. This is equal to the difference between `pos()` after and before\n  // the call (saturated to 0), and is equal to `length` if `Skip()` returned\n  // `true`.\n  //\n  // Return values:\n  //  * `true`                 - success (`length` bytes skipped)\n  //  * `false` (when `ok()`)  - source ends before skipping `length` bytes\n  //                             (position is set to the end)\n  //  * `false` (when `!ok()`) - failure\n  //\n  // `Skip()` is always supported, although if `SupportsRandomAccess()` is\n  // `false`, then it is as inefficient as reading and discarding the\n  // intervening data.\n  bool Skip(Position length, Position* length_skipped = nullptr);\n\n  // Returns `true` if this `Reader` supports `Size()`.\n  //\n  // Invariant: if `SupportsRandomAccess()` then `SupportsSize()`.\n  virtual bool SupportsSize() { return SupportsRandomAccess(); }\n\n  // Returns the size of the source, i.e. the position corresponding to its end.\n  //\n  // Returns `std::nullopt` on failure (`!ok()`).\n  //\n  // `Size()` is supported if `SupportsRandomAccess()` or `SupportsSize()` is\n  // `true`.\n  std::optional<Position> Size();\n\n  // Returns `true` if this `Reader` supports `NewReader()`.\n  virtual bool SupportsNewReader() { return false; }\n\n  // Returns a `Reader` which reads from the same source, but has an independent\n  // current position, starting from `initial_pos`, defaulting to `pos()`.\n  // The returned `Reader` can be used concurrently with this `Reader` and other\n  // siblings.\n  //\n  // If the source ends before `initial_pos`, the position of the new `Reader`\n  // is set to the end. The resulting `Reader` supports `Seek()` and\n  // `NewReader()`.\n  //\n  // The new `Reader` does not own the source, even if the this `Reader` does.\n  // The source of this `Reader` must not be changed until the new `Reader` is\n  // closed or no longer used.\n  //\n  // Returns `nullptr` on failure (`!ok()`).\n  //\n  // `NewReader()` is supported if `SupportsNewReader()` is `true`.\n  //\n  // If `SupportsNewReader()` is `true`, then `NewReader()` is effectively const\n  // and thread-safe. It may be called concurrently with other operations, even\n  // non-const ones.\n  //\n  // If `SupportsNewReader()` and `ok()` are `true`, then `NewReader()` does not\n  // return `nullptr`.\n  std::unique_ptr<Reader> NewReader(Position initial_pos) {\n    return NewReaderImpl(initial_pos);\n  }\n\n  // Like `NewReader(pos())`. It can be more efficient because for some classes\n  // the current buffer can be shared. It offers fewer thread safety guarantees.\n  //\n  // If `SupportsNewReader()` is `true`, then `NewReaderCurrentPos()` is\n  // effectively const but not thread-safe. It may be called concurrently\n  // with const operations and with other calls to `NewReader()` and\n  // `NewReaderCurrentPos()`, but not with non-const operations.\n  //\n  // If `SupportsNewReader()` and `ok()` are `true`, then\n  // `NewReaderCurrentPos()` does not return `nullptr`.\n  std::unique_ptr<Reader> NewReaderCurrentPos() {\n    return NewReaderCurrentPosImpl();\n  }\n  ABSL_DEPRECATE_AND_INLINE() std::unique_ptr<Reader> NewReader() {\n    return NewReaderCurrentPos();\n  }\n\n protected:\n  using Object::Object;\n\n  // Moves the part of the object defined in this class.\n  //\n  // Buffer pointers do not need to satisfy their invariants during this part of\n  // the move, here they are merely exchanged with `nullptr` and copied.\n  Reader(Reader&& that) noexcept;\n  Reader& operator=(Reader&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `Reader`. This avoids\n  // constructing a temporary `Reader` and moving from it. Derived classes which\n  // redefine `Reset()` should include a call to `Reader::Reset()`.\n  void Reset(Closed);\n  void Reset();\n\n  // `Reader` overrides `Object::Done()` to set buffer pointers to `nullptr`.\n  // Derived classes which override it further should include a call to\n  // `Reader::Done()`.\n  void Done() override;\n\n  // `Reader` overrides `Object::AnnotateStatusImpl()` to annotate the status\n  // with the current position.\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n\n  // Marks the `Reader` as failed with message \"Reader position overflow\".\n  // Always returns `false`.\n  //\n  // This can be called if `limit_pos()` would overflow.\n  ABSL_ATTRIBUTE_COLD bool FailOverflow();\n\n  // Implementation of `VerifyEnd()`.\n  //\n  // By default is implemented in terms of `Pull()`.\n  virtual void VerifyEndImpl();\n\n  // Implementation of `SetReadAllHint()`.\n  virtual void SetReadAllHintImpl(ABSL_ATTRIBUTE_UNUSED bool read_all_hint) {}\n\n  // Implementation of the slow part of `Pull()`.\n  //\n  // Precondition: `available() < min_length`\n  virtual bool PullSlow(size_t min_length, size_t recommended_length) = 0;\n\n  // Sets the values of:\n  //  * `start()`  - to `start`\n  //  * `cursor()` - to `start + start_to_cursor`\n  //  * `limit()`  - to `start + start_to_limit`\n  //\n  // Preconditions:\n  //   [`start`..`start + start_to_limit`) is a valid byte range\n  //   `start_to_cursor <= start_to_limit`\n  void set_buffer(const char* start = nullptr, size_t start_to_limit = 0,\n                  size_t start_to_cursor = 0);\n\n  // Implementations of the slow part of `Read()`, `ReadAndAppend()`, `Copy()`,\n  // `ReadSome()`, `ReadAndAppendSome()`, and `CopySome()`.\n  //\n  // `ReadSlow(std::string&)`, `ReadSlow(Chain&)`, `ReadSlow(absl::Cord&)`,\n  // `ReadSomeSlow(std::string&)`, `ReadSomeSlow(Chain&)`, and\n  // `ReadSomeSlow(absl::Cord&)` append to any existing data in `dest`.\n  //\n  // By default:\n  //  * `ReadSlow(char*)`, `CopySlow(Writer&)`, and `ReadSomeSlow(char*)`\n  //    are implemented in terms of `PullSlow()`\n  //  * `ReadSlow(std::string&)`, `ReadSlow(Chain&)`, and\n  //    `ReadSlow(absl::Cord&)` are implemented in terms of `ReadSlow(char*)`\n  //  * `CopySlow(BackwardWriter&)` is implemented in terms of `ReadSlow(char*)`\n  //    and `ReadSlow(Chain&)`\n  //  * `ReadSomeSlow(std::string&)` is implemented in terms of\n  //    `ReadSomeSlow(char*)`\n  //  * `ReadSomeSlow(Chain&)` is implemented in terms of `PullSlow()` and\n  //    `ReadSlow(Chain&)`\n  //  * `ReadSomeSlow(absl::Cord&)` is implemented in terms of `PullSlow()` and\n  //    `ReadSlow(absl::Cord&)`\n  //  * `CopySomeSlow(Writer&)` is implemented in terms of `PullSlow()` and\n  //    `CopySlow(Writer&)`\n  //\n  // Precondition for `ReadSlow(char*)` and `ReadSlow(std::string&)`:\n  //   `available() < length`\n  //\n  // Precondition for `ReadSlow(Chain&)`, `ReadSlow(absl::Cord&)`,\n  // `CopySlow(Writer&)`, and `CopySlow(BackwardWriter&)`:\n  //   `UnsignedMin(available(), kMaxBytesToCopy) < length`\n  //\n  // Preconditions for `ReadSomeSlow(char*)`, `ReadSomeSlow(std::string&)`,\n  // `ReadSomeSlow(Chain&)`, `ReadSomeSlow(absl::Cord&)`, and\n  // `CopySomeSlow(Writer&)`:\n  //   `max_length > 0`\n  //   `available() == 0`\n  //\n  // Additional precondition for `ReadSlow(std::string&)`, `ReadSlow(Chain&)`,\n  // and `ReadSlow(absl::Cord&)`:\n  //   `length <= std::numeric_limits<size_t>::max() - dest->size()`\n  //\n  // Additional precondition for `ReadSomeSlow(std::string&)`,\n  // `ReadSomeSlow(Chain&)`, and `ReadSomeSlow(absl::Cord&)`:\n  //   `max_length <= std::numeric_limits<size_t>::max() - dest->size()`\n  virtual bool ReadSlow(size_t length, char* dest);\n  bool ReadSlow(size_t length, char* dest, size_t& length_read);\n  bool ReadSlow(size_t length, std::string& dest);\n  bool ReadSlow(size_t length, std::string& dest, size_t& length_read);\n  virtual bool ReadSlow(size_t length, Chain& dest);\n  bool ReadSlow(size_t length, Chain& dest, size_t& length_read);\n  virtual bool ReadSlow(size_t length, absl::Cord& dest);\n  bool ReadSlow(size_t length, absl::Cord& dest, size_t& length_read);\n  virtual bool CopySlow(Position length, Writer& dest);\n  bool CopySlow(Position length, Writer& dest, Position& length_read);\n  virtual bool CopySlow(size_t length, BackwardWriter& dest);\n  virtual bool ReadSomeSlow(size_t max_length, char* dest);\n  bool ReadSomeSlow(size_t max_length, char* dest, size_t& length_read);\n  bool ReadSomeSlow(size_t max_length, std::string& dest);\n  bool ReadSomeSlow(size_t max_length, std::string& dest, size_t& length_read);\n  bool ReadSomeSlow(size_t max_length, Chain& dest);\n  bool ReadSomeSlow(size_t max_length, Chain& dest, size_t& length_read);\n  bool ReadSomeSlow(size_t max_length, absl::Cord& dest);\n  bool ReadSomeSlow(size_t max_length, absl::Cord& dest, size_t& length_read);\n  virtual bool CopySomeSlow(size_t max_length, Writer& dest);\n  bool CopySomeSlow(size_t max_length, Writer& dest, size_t& length_read);\n\n  // Implementation of the slow part of `ReadHint()`.\n  //\n  // By default does nothing.\n  //\n  // Precondition: `available() < min_length`\n  virtual void ReadHintSlow(size_t min_length, size_t recommended_length);\n\n  // Implementation of `Sync()`, except that the parameter is not defaulted,\n  // which is problematic for virtual functions.\n  //\n  // By default does nothing and returns `ok()`.\n  virtual bool SyncImpl(SyncType sync_type);\n\n  // Increments the value of `limit_pos()`.\n  void move_limit_pos(Position length);\n\n  // Sets the value of `limit_pos()`.\n  void set_limit_pos(Position limit_pos);\n\n  // Implementation of the slow part of `Seek()` and `Skip()`.\n  //\n  // By default seeking forwards is implemented in terms of `Pull()`, and\n  // seeking backwards fails.\n  //\n  // Precondition: `new_pos < start_pos() || new_pos > limit_pos()`\n  virtual bool SeekSlow(Position new_pos);\n\n  // Implementation of `Size()`.\n  //\n  // By default fails.\n  virtual std::optional<Position> SizeImpl();\n\n  virtual std::unique_ptr<Reader> NewReaderImpl(Position initial_pos);\n  virtual std::unique_ptr<Reader> NewReaderCurrentPosImpl();\n\n private:\n  const char* start_ = nullptr;\n  const char* cursor_ = nullptr;\n  const char* limit_ = nullptr;\n\n  // Source position corresponding to `limit_`.\n  //\n  // Invariant: `limit_pos_ >= start_to_limit()`\n  Position limit_pos_ = 0;\n};\n\n// Implementation details follow.\n\ninline Reader::Reader(Reader&& that) noexcept\n    : Object(static_cast<Object&&>(that)),\n      start_(std::exchange(that.start_, nullptr)),\n      cursor_(std::exchange(that.cursor_, nullptr)),\n      limit_(std::exchange(that.limit_, nullptr)),\n      limit_pos_(std::exchange(that.limit_pos_, 0)) {}\n\ninline Reader& Reader::operator=(Reader&& that) noexcept {\n  Object::operator=(static_cast<Object&&>(that));\n  start_ = std::exchange(that.start_, nullptr);\n  cursor_ = std::exchange(that.cursor_, nullptr);\n  limit_ = std::exchange(that.limit_, nullptr);\n  limit_pos_ = std::exchange(that.limit_pos_, 0);\n  return *this;\n}\n\ninline void Reader::Reset(Closed) {\n  Object::Reset(kClosed);\n  start_ = nullptr;\n  cursor_ = nullptr;\n  limit_ = nullptr;\n  limit_pos_ = 0;\n}\n\ninline void Reader::Reset() {\n  Object::Reset();\n  start_ = nullptr;\n  cursor_ = nullptr;\n  limit_ = nullptr;\n  limit_pos_ = 0;\n}\n\ninline void Reader::Done() {\n  limit_pos_ = pos();\n  set_buffer();\n}\n\ninline bool Reader::VerifyEndAndClose() {\n  VerifyEnd();\n  return Close();\n}\n\ninline void Reader::VerifyEnd() { VerifyEndImpl(); }\n\ninline void Reader::SetReadAllHint(bool read_all_hint) {\n  SetReadAllHintImpl(read_all_hint);\n}\n\ninline bool Reader::Pull(size_t min_length, size_t recommended_length) {\n  if (ABSL_PREDICT_TRUE(available() >= min_length)) return true;\n  if (ABSL_PREDICT_FALSE(!PullSlow(min_length, recommended_length))) {\n    return false;\n  }\n  RIEGELI_ASSERT_GE(available(), min_length)\n      << \"Failed postcondition of Reader::PullSlow(): \"\n         \"not enough data available\";\n  return true;\n}\n\ninline void Reader::move_cursor(size_t length) {\n  RIEGELI_ASSERT_LE(length, available())\n      << \"Failed precondition of Reader::move_cursor(): length out of range\";\n  cursor_ += length;\n}\n\ninline void Reader::set_cursor(const char* cursor) {\n  RIEGELI_ASSERT_GE(cursor, start())\n      << \"Failed precondition of Reader::set_cursor(): pointer out of range\";\n  RIEGELI_ASSERT_LE(cursor, limit())\n      << \"Failed precondition of Reader::set_cursor(): pointer out of range\";\n  cursor_ = cursor;\n}\n\ninline void Reader::set_buffer(const char* start, size_t start_to_limit,\n                               size_t start_to_cursor) {\n  RIEGELI_ASSERT_LE(start_to_cursor, start_to_limit)\n      << \"Failed precondition of Reader::set_buffer(): length out of range\";\n  start_ = start;\n  cursor_ = start + start_to_cursor;\n  limit_ = start + start_to_limit;\n}\n\ninline bool Reader::ReadByte(uint8_t& dest) {\n  if (ABSL_PREDICT_FALSE(!Pull())) return false;\n  dest = static_cast<uint8_t>(*cursor());\n  move_cursor(1);\n  return true;\n}\n\ninline bool Reader::Read(char& dest) {\n  if (ABSL_PREDICT_FALSE(!Pull())) return false;\n  dest = *cursor();\n  move_cursor(1);\n  return true;\n}\n\ninline bool Reader::Read(size_t length, absl::string_view& dest,\n                         size_t* length_read) {\n  const bool pull_ok = Pull(length);\n  if (ABSL_PREDICT_FALSE(!pull_ok)) length = available();\n  dest = absl::string_view(cursor(), length);\n  move_cursor(length);\n  if (length_read != nullptr) *length_read = length;\n  return pull_ok;\n}\n\ninline bool Reader::Read(size_t length, char* dest, size_t* length_read) {\n  if (ABSL_PREDICT_TRUE(available() >= length)) {\n    riegeli::null_safe_memcpy(dest, cursor(), length);\n    move_cursor(length);\n    if (length_read != nullptr) *length_read = length;\n    return true;\n  }\n  if (length_read != nullptr) return ReadSlow(length, dest, *length_read);\n  return ReadSlow(length, dest);\n}\n\ninline bool Reader::ReadSome(size_t max_length, absl::string_view& dest,\n                             size_t* length_read) {\n  const bool pull_ok =\n      ABSL_PREDICT_FALSE(max_length == 0) || Pull(1, max_length);\n  const size_t length = UnsignedMin(max_length, available());\n  dest = absl::string_view(cursor(), length);\n  move_cursor(length);\n  if (length_read != nullptr) *length_read = length;\n  return pull_ok;\n}\n\ninline bool Reader::ReadSome(size_t max_length, char* dest,\n                             size_t* length_read) {\n  if (ABSL_PREDICT_TRUE(available() > 0) ||\n      ABSL_PREDICT_FALSE(max_length == 0)) {\n    max_length = UnsignedMin(max_length, available());\n    riegeli::null_safe_memcpy(dest, cursor(), max_length);\n    move_cursor(max_length);\n    if (length_read != nullptr) *length_read = max_length;\n    return true;\n  }\n  if (length_read != nullptr) {\n    return ReadSomeSlow(max_length, dest, *length_read);\n  }\n  return ReadSomeSlow(max_length, dest);\n}\n\ninline void Reader::ReadHint(size_t min_length, size_t recommended_length) {\n  if (ABSL_PREDICT_TRUE(available() >= min_length)) return;\n  ReadHintSlow(min_length, recommended_length);\n}\n\ninline bool Reader::Sync(SyncType sync_type) { return SyncImpl(sync_type); }\n\ninline Position Reader::pos() const {\n  RIEGELI_ASSERT_GE(limit_pos_, start_to_limit())\n      << \"Failed invariant of Reader: negative position of buffer start\";\n  return limit_pos_ - available();\n}\n\ninline Position Reader::start_pos() const {\n  RIEGELI_ASSERT_GE(limit_pos_, start_to_limit())\n      << \"Failed invariant of Reader: negative position of buffer start\";\n  return limit_pos_ - start_to_limit();\n}\n\ninline void Reader::move_limit_pos(Position length) {\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<Position>::max() - limit_pos_)\n      << \"Failed precondition of Reader::move_limit_pos(): \"\n         \"position out of range\";\n  limit_pos_ += length;\n}\n\ninline void Reader::set_limit_pos(Position limit_pos) {\n  limit_pos_ = limit_pos;\n}\n\ninline bool Reader::Seek(Position new_pos) {\n  if (ABSL_PREDICT_TRUE(new_pos >= start_pos() && new_pos <= limit_pos())) {\n    set_cursor(limit() - (limit_pos() - new_pos));\n    return true;\n  }\n  return SeekSlow(new_pos);\n}\n\ninline bool Reader::Skip(Position length, Position* length_skipped) {\n  if (ABSL_PREDICT_TRUE(available() >= length)) {\n    move_cursor(IntCast<size_t>(length));\n    if (length_skipped != nullptr) *length_skipped = length;\n    return true;\n  }\n  if (length_skipped == nullptr) {\n    if (ABSL_PREDICT_FALSE(length >\n                           std::numeric_limits<Position>::max() - pos())) {\n      SeekSlow(std::numeric_limits<Position>::max());\n      return false;\n    }\n    return SeekSlow(pos() + length);\n  }\n  const Position pos_before = pos();\n  bool seek_ok;\n  if (ABSL_PREDICT_FALSE(length >\n                         std::numeric_limits<Position>::max() - pos_before)) {\n    SeekSlow(std::numeric_limits<Position>::max());\n    seek_ok = false;\n  } else {\n    seek_ok = SeekSlow(pos_before + length);\n  }\n  // `SeekSlow()` could have decreased `pos()` if the source decreased its\n  // size.\n  RIEGELI_ASSERT_LE(pos(), pos_before + length)\n      << \"Reader::SeekSlow() skipped more than requested\";\n  if (ABSL_PREDICT_FALSE(!seek_ok)) {\n    *length_skipped = SaturatingSub(pos(), pos_before);\n    return false;\n  }\n  RIEGELI_ASSERT_EQ(pos(), pos_before + length)\n      << \"Reader::SeekSlow() succeeded but skipped less than requested\";\n  *length_skipped = length;\n  return true;\n}\n\ninline std::optional<Position> Reader::Size() { return SizeImpl(); }\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/reader_cfile.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// Make `fopencookie()` and `off64_t` available.\n#if !defined(_GNU_SOURCE)\n#define _GNU_SOURCE\n#endif\n\n#include \"riegeli/bytes/reader_cfile.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n#include <stdio.h>\n#include <sys/types.h>\n\n#include <cerrno>\n#include <limits>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/errno_mapping.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/cfile_handle.h\"\n#include \"riegeli/bytes/path_ref.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli::cfile_internal {\n\nReaderCFileCookieBase::~ReaderCFileCookieBase() {}\n\ninline ssize_t ReaderCFileCookieBase::Read(char* dest, size_t length) {\n  Reader& reader = *SrcReader();\n  size_t length_read;\n  if (ABSL_PREDICT_FALSE(!reader.ReadSome(length, dest, &length_read) &&\n                         !reader.ok())) {\n    errno = StatusCodeToErrno(reader.status().code());\n    return -1;\n  }\n  return IntCast<ssize_t>(length_read);\n}\n\ninline std::optional<int64_t> ReaderCFileCookieBase::Seek(int64_t offset,\n                                                          int whence) {\n  Reader& reader = *SrcReader();\n  Position new_pos;\n  switch (whence) {\n    case SEEK_SET:\n      if (ABSL_PREDICT_FALSE(offset < 0)) {\n        errno = EINVAL;\n        return std::nullopt;\n      }\n      new_pos = IntCast<Position>(offset);\n      break;\n    case SEEK_CUR:\n      new_pos = reader.pos();\n      if (offset < 0) {\n        if (ABSL_PREDICT_FALSE(NegatingUnsignedCast(offset) > new_pos)) {\n          errno = EINVAL;\n          return std::nullopt;\n        }\n        new_pos -= NegatingUnsignedCast(offset);\n        if (ABSL_PREDICT_FALSE(new_pos >\n                               Position{std::numeric_limits<int64_t>::max()})) {\n          errno = EINVAL;\n          return std::nullopt;\n        }\n      } else {\n        if (ABSL_PREDICT_FALSE(\n                new_pos > Position{std::numeric_limits<int64_t>::max()} ||\n                IntCast<Position>(offset) >\n                    Position{std::numeric_limits<int64_t>::max()} - new_pos)) {\n          errno = EINVAL;\n          return std::nullopt;\n        }\n        new_pos += IntCast<Position>(offset);\n      }\n      break;\n    case SEEK_END: {\n      if (ABSL_PREDICT_FALSE(!reader.SupportsSize())) {\n        // Indicate that `fseek(SEEK_END)` is not supported.\n        errno = ESPIPE;\n        return std::nullopt;\n      }\n      const std::optional<Position> size = reader.Size();\n      if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n        errno = StatusCodeToErrno(reader.status().code());\n        return std::nullopt;\n      }\n      if (ABSL_PREDICT_FALSE(offset > 0 ||\n                             NegatingUnsignedCast(offset) > *size)) {\n        errno = EINVAL;\n        return std::nullopt;\n      }\n      new_pos = *size - NegatingUnsignedCast(offset);\n      if (ABSL_PREDICT_FALSE(new_pos >\n                             Position{std::numeric_limits<int64_t>::max()})) {\n        errno = EINVAL;\n        return std::nullopt;\n      }\n    } break;\n    default:\n      RIEGELI_ASSUME_UNREACHABLE() << \"Unknown seek origin: \" << whence;\n  }\n  if (new_pos >= reader.pos()) {\n    // Seeking forwards is supported even if random access is not.\n  } else if (ABSL_PREDICT_FALSE(!reader.SupportsRewind())) {\n    // Indicate that `fseek()` is not supported.\n    errno = ESPIPE;\n    return std::nullopt;\n  }\n  if (ABSL_PREDICT_FALSE(!reader.Seek(new_pos))) {\n    if (ABSL_PREDICT_FALSE(!reader.ok())) {\n      errno = StatusCodeToErrno(reader.status().code());\n    } else {\n      errno = EINVAL;\n    }\n    return std::nullopt;\n  }\n  return IntCast<int64_t>(new_pos);\n}\n\n// `extern \"C\"` sets the C calling convention for compatibility with\n// `fopencookie()`. `static` avoids making symbols public, as `extern \"C\"`\n// trumps anonymous namespace.\nextern \"C\" {\n\nstatic ssize_t ReaderCFileRead(void* cookie, char* buf, size_t size) {\n  return static_cast<ReaderCFileCookieBase*>(cookie)->Read(buf, size);\n}\n\nstatic int ReaderCFileSeek(void* cookie, off64_t* offset, int whence) {\n  const std::optional<int64_t> new_pos =\n      static_cast<ReaderCFileCookieBase*>(cookie)->Seek(\n          IntCast<int64_t>(*offset), whence);\n  if (ABSL_PREDICT_FALSE(new_pos == std::nullopt)) {\n    *offset = -1;\n    return -1;\n  }\n  *offset = IntCast<off64_t>(*new_pos);\n  return 0;\n}\n\nstatic int ReaderCFileClose(void* cookie) {\n  const int result = static_cast<ReaderCFileCookieBase*>(cookie)->Close();\n  delete static_cast<ReaderCFileCookieBase*>(cookie);\n  if (ABSL_PREDICT_FALSE(result != 0)) {\n    errno = result;\n    return -1;\n  }\n  return 0;\n}\n\n}  // extern \"C\"\n\nOwnedCFile ReaderCFileImpl(ReaderCFileCookieBase* cookie,\n                           std::string&& filename) {\n  return OwnedCFile(fopencookie(cookie, \"r\",\n                                {ReaderCFileRead, nullptr, ReaderCFileSeek,\n                                 ReaderCFileClose}),\n                    std::move(filename));\n}\n\n}  // namespace riegeli::cfile_internal\n"
  },
  {
    "path": "riegeli/bytes/reader_cfile.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_READER_CFILE_H_\n#define RIEGELI_BYTES_READER_CFILE_H_\n\n#include <stddef.h>\n#include <stdint.h>\n#include <stdio.h>\n#include <sys/types.h>\n\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/errno_mapping.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/bytes/cfile_handle.h\"\n#include \"riegeli/bytes/path_ref.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nclass ReaderCFileOptions {\n public:\n  ReaderCFileOptions() noexcept {}\n\n  // The filename assumed by the returned `OwnedCFile`.\n  //\n  // Default: \"<unspecified>\".\n  ReaderCFileOptions& set_filename(PathInitializer filename) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    riegeli::Reset(filename_, std::move(filename));\n    return *this;\n  }\n  ReaderCFileOptions&& set_filename(PathInitializer filename) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_filename(std::move(filename)));\n  }\n  std::string& filename() ABSL_ATTRIBUTE_LIFETIME_BOUND { return filename_; }\n  const std::string& filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return filename_;\n  }\n\n private:\n  std::string filename_ = \"<unspecified>\";\n};\n\n// A read-only `FILE` which reads data from a `Reader`.\n//\n// This requires `fopencookie()` to be supported by the C library.\n//\n// The `FILE` supports `fseek()` forwards in any case, and backwards if\n// `Reader::SupportsRewind()`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `Reader`. `Src` must support `Dependency<Reader*, Src>`,\n// e.g. `Reader*` (not owned), `ChainReader<>` (owned),\n// `std::unique_ptr<Reader>` (owned), `Any<Reader*>` (maybe owned).\n//\n// `src` supports `riegeli::Maker<Src>(args...)` to construct `Src` in-place.\n//\n// The `Reader` must not be accessed until the `FILE` is closed. Warning: this\n// includes implicit closing of all `FILE` objects which are still open at\n// program exit, hence if the `FILE` persists until program exit, then the\n// `Reader` must do so as well.\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetSupportsDependency<Reader*, Src>::value, int> = 0>\nOwnedCFile ReaderCFile(Src&& src,\n                       ReaderCFileOptions options = ReaderCFileOptions());\n\n// Implementation details follow.\n\nnamespace cfile_internal {\n\nclass ReaderCFileCookieBase {\n public:\n  virtual ~ReaderCFileCookieBase();\n\n  ssize_t Read(char* dest, size_t length);\n\n  // Use `int64_t` instead of `off64_t` to avoid a dependency on\n  // `#define _GNU_SOURCE` in a header.\n  std::optional<int64_t> Seek(int64_t offset, int whence);\n\n  // Returns 0 on success, or `errno` value on failure.\n  virtual int Close() = 0;\n\n protected:\n  ReaderCFileCookieBase() noexcept {}\n\n  ReaderCFileCookieBase(const ReaderCFileCookieBase&) = delete;\n  ReaderCFileCookieBase& operator=(const ReaderCFileCookieBase&) = delete;\n\n  virtual Reader* SrcReader() = 0;\n\n  void Initialize(Reader* reader);\n};\n\ninline void ReaderCFileCookieBase::Initialize(Reader* reader) {\n  RIEGELI_ASSERT_NE(reader, nullptr)\n      << \"Failed precondition of ReaderCFile(): null Reader pointer\";\n}\n\ntemplate <typename Src>\nclass ReaderCFileCookie : public ReaderCFileCookieBase {\n public:\n  explicit ReaderCFileCookie(Initializer<Src> src) : src_(std::move(src)) {\n    Initialize(src_.get());\n  }\n\n  int Close() override;\n\n protected:\n  Reader* SrcReader() override { return src_.get(); }\n\n private:\n  Dependency<Reader*, Src> src_;\n};\n\ntemplate <typename Src>\nint ReaderCFileCookie<Src>::Close() {\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      return StatusCodeToErrno(src_->status().code());\n    }\n  }\n  return 0;\n}\n\nOwnedCFile ReaderCFileImpl(ReaderCFileCookieBase* cookie,\n                           std::string&& filename);\n\n}  // namespace cfile_internal\n\ntemplate <typename Src,\n          std::enable_if_t<TargetSupportsDependency<Reader*, Src>::value, int>>\nOwnedCFile ReaderCFile(Src&& src, ReaderCFileOptions options) {\n  return cfile_internal::ReaderCFileImpl(\n      new cfile_internal::ReaderCFileCookie<TargetT<Src>>(\n          std::forward<Src>(src)),\n      std::move(options.filename()));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_READER_CFILE_H_\n"
  },
  {
    "path": "riegeli/bytes/reader_factory.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/reader_factory.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/base/thread_annotations.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/synchronization/mutex.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/pullable_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nclass ReaderFactoryBase::ConcurrentReader : public PullableReader {\n public:\n  explicit ConcurrentReader(Shared* shared, Position initial_pos);\n\n  ConcurrentReader(const ConcurrentReader&) = delete;\n  ConcurrentReader& operator=(const ConcurrentReader&) = delete;\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRandomAccess() override { return true; }\n  bool SupportsNewReader() override { return true; }\n\n protected:\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  bool PullBehindScratch(size_t recommended_length) override;\n  using PullableReader::ReadBehindScratch;\n  bool ReadBehindScratch(size_t length, char* dest) override;\n  bool ReadBehindScratch(size_t length, Chain& dest) override;\n  bool ReadBehindScratch(size_t length, absl::Cord& dest) override;\n  using PullableReader::CopyBehindScratch;\n  bool CopyBehindScratch(Position length, Writer& dest) override;\n  using PullableReader::ReadSomeBehindScratch;\n  bool ReadSomeBehindScratch(size_t max_length, char* dest) override;\n  using PullableReader::CopySomeBehindScratch;\n  bool CopySomeBehindScratch(size_t max_length, Writer& dest) override;\n  void ReadHintBehindScratch(size_t min_length,\n                             size_t recommended_length) override;\n  bool SyncBehindScratch(SyncType sync_type) override;\n  bool SeekBehindScratch(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  bool SyncPos() ABSL_SHARED_LOCKS_REQUIRED(shared_->mutex);\n  bool ReadSome() ABSL_SHARED_LOCKS_REQUIRED(shared_->mutex);\n\n  Shared* shared_;\n  ReadBufferSizer buffer_sizer_;\n  // Buffered data, read directly before the original position which is\n  // `start_pos() + (secondary_buffer_.size() - iter_.CharIndexInChain())`\n  // when scratch is not used.\n  Chain secondary_buffer_;\n  // Invariant: `iter_.chain() == (is_open() ? &secondary_buffer_ : nullptr)`\n  Chain::BlockIterator iter_;\n\n  // Invariants if `is_open()` and scratch is not used:\n  //   `start() == (iter_ == secondary_buffer_.blocks().cend() ? nullptr\n  //                                                           : iter_->data())`\n  //   `start_to_limit() ==\n  //        (iter_ == secondary_buffer_.blocks().cend() ? 0 : iter_->size())`\n};\n\ninline ReaderFactoryBase::ConcurrentReader::ConcurrentReader(\n    Shared* shared, Position initial_pos)\n    : shared_(shared),\n      buffer_sizer_(shared->buffer_options),\n      iter_(secondary_buffer_.blocks().cend()) {\n  set_limit_pos(initial_pos);\n  buffer_sizer_.BeginRun(limit_pos());\n}\n\nvoid ReaderFactoryBase::ConcurrentReader::Done() {\n  PullableReader::Done();\n  secondary_buffer_ = Chain();\n  iter_ = Chain::BlockIterator();\n}\n\nabsl::Status ReaderFactoryBase::ConcurrentReader::AnnotateStatusImpl(\n    absl::Status status) {\n  if (is_open()) {\n    absl::MutexLock lock(shared_->mutex);\n    shared_->reader->Seek(pos());\n    return shared_->reader->AnnotateStatus(std::move(status));\n  }\n  return status;\n}\n\ninline bool ReaderFactoryBase::ConcurrentReader::SyncPos() {\n  if (ABSL_PREDICT_FALSE(!shared_->reader->Seek(limit_pos()))) {\n    if (ABSL_PREDICT_FALSE(!shared_->reader->ok())) {\n      return FailWithoutAnnotation(shared_->reader->status());\n    }\n    return false;\n  }\n  return true;\n}\n\ninline bool ReaderFactoryBase::ConcurrentReader::ReadSome() {\n  if (!shared_->reader->ReadSome(buffer_sizer_.BufferLength(limit_pos()),\n                                 secondary_buffer_)) {\n    if (ABSL_PREDICT_FALSE(!shared_->reader->ok())) {\n      return FailWithoutAnnotation(shared_->reader->status());\n    }\n    return false;\n  }\n  iter_ = secondary_buffer_.blocks().cbegin();\n  return true;\n}\n\nvoid ReaderFactoryBase::ConcurrentReader::SetReadAllHintImpl(\n    bool read_all_hint) {\n  buffer_sizer_.set_read_all_hint(read_all_hint);\n}\n\nbool ReaderFactoryBase::ConcurrentReader::PullBehindScratch(\n    size_t recommended_length) {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"enough data available, use Pull() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"scratch used\";\n  if (iter_ != secondary_buffer_.blocks().cend()) ++iter_;\n  set_buffer();\n  for (;;) {\n    while (iter_ != secondary_buffer_.blocks().cend()) {\n      if (ABSL_PREDICT_TRUE(!iter_->empty())) {\n        set_buffer(iter_->data(), iter_->size());\n        move_limit_pos(available());\n        return true;\n      }\n      ++iter_;\n    }\n\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    secondary_buffer_.Clear();\n    iter_ = secondary_buffer_.blocks().cend();\n    absl::MutexLock lock(shared_->mutex);\n    if (ABSL_PREDICT_FALSE(!SyncPos())) return false;\n    if (ABSL_PREDICT_FALSE(!ReadSome())) return false;\n  }\n}\n\nbool ReaderFactoryBase::ConcurrentReader::ReadBehindScratch(size_t length,\n                                                            char* dest) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of PullableReader::ReadBehindScratch(char*): \"\n         \"enough data available, use Read(char*) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(char*): \"\n         \"scratch used\";\n  if (iter_ != secondary_buffer_.blocks().cend()) {\n    const size_t available_length = available();\n    riegeli::null_safe_memcpy(dest, cursor(), available_length);\n    move_cursor(available_length);\n    dest += available_length;\n    length -= available_length;\n    ++iter_;\n  }\n  set_buffer();\n  for (;;) {\n    while (iter_ != secondary_buffer_.blocks().cend()) {\n      move_limit_pos(iter_->size());\n      if (length <= iter_->size()) {\n        set_buffer(iter_->data(), iter_->size(), length);\n        std::memcpy(dest, start(), start_to_cursor());\n        return true;\n      }\n      std::memcpy(dest, iter_->data(), iter_->size());\n      dest += iter_->size();\n      length -= iter_->size();\n      ++iter_;\n    }\n\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    secondary_buffer_.Clear();\n    iter_ = secondary_buffer_.blocks().cend();\n    absl::MutexLock lock(shared_->mutex);\n    if (ABSL_PREDICT_FALSE(!SyncPos())) return false;\n    if (length >= buffer_sizer_.BufferLength(pos())) {\n      // Read directly to `dest`.\n      if (ABSL_PREDICT_FALSE(!shared_->reader->Read(length, dest))) {\n        set_limit_pos(shared_->reader->pos());\n        if (ABSL_PREDICT_FALSE(!shared_->reader->ok())) {\n          return FailWithoutAnnotation(shared_->reader->status());\n        }\n        return false;\n      }\n      move_limit_pos(length);\n      return true;\n    }\n    if (ABSL_PREDICT_FALSE(!ReadSome())) return false;\n  }\n}\n\nbool ReaderFactoryBase::ConcurrentReader::ReadBehindScratch(size_t length,\n                                                            Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"Chain size overflow\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Chain&): \"\n         \"scratch used\";\n  if (length <= available()) {\n    dest.Append(ExternalRef(*iter_, absl::string_view(cursor(), length)));\n    move_cursor(length);\n    return true;\n  }\n  if (iter_ != secondary_buffer_.blocks().cend()) {\n    dest.Append(ExternalRef(*iter_, absl::string_view(cursor(), available())));\n    length -= available();\n    ++iter_;\n  }\n  set_buffer();\n  for (;;) {\n    while (iter_ != secondary_buffer_.blocks().cend()) {\n      move_limit_pos(iter_->size());\n      if (length <= iter_->size()) {\n        set_buffer(iter_->data(), iter_->size(), length);\n        dest.Append(\n            ExternalRef(*iter_, absl::string_view(start(), start_to_cursor())));\n        return true;\n      }\n      dest.Append(*iter_);\n      length -= iter_->size();\n      ++iter_;\n    }\n\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    secondary_buffer_.Clear();\n    iter_ = secondary_buffer_.blocks().cend();\n    absl::MutexLock lock(shared_->mutex);\n    if (ABSL_PREDICT_FALSE(!SyncPos())) return false;\n    if (length >= buffer_sizer_.BufferLength(pos())) {\n      // Read directly to `dest`.\n      if (ABSL_PREDICT_FALSE(!shared_->reader->ReadAndAppend(length, dest))) {\n        set_limit_pos(shared_->reader->pos());\n        if (ABSL_PREDICT_FALSE(!shared_->reader->ok())) {\n          return FailWithoutAnnotation(shared_->reader->status());\n        }\n        return false;\n      }\n      move_limit_pos(length);\n      return true;\n    }\n    if (ABSL_PREDICT_FALSE(!ReadSome())) return false;\n  }\n}\n\nbool ReaderFactoryBase::ConcurrentReader::ReadBehindScratch(size_t length,\n                                                            absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"Chain size overflow\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadBehindScratch(Cord&): \"\n         \"scratch used\";\n  if (length <= available()) {\n    ExternalRef(*iter_, absl::string_view(cursor(), length)).AppendTo(dest);\n    move_cursor(length);\n    return true;\n  }\n  if (iter_ != secondary_buffer_.blocks().cend()) {\n    ExternalRef(*iter_, absl::string_view(cursor(), available()))\n        .AppendTo(dest);\n    length -= available();\n    ++iter_;\n  }\n  set_buffer();\n  for (;;) {\n    while (iter_ != secondary_buffer_.blocks().cend()) {\n      move_limit_pos(iter_->size());\n      if (length <= iter_->size()) {\n        set_buffer(iter_->data(), iter_->size(), length);\n        ExternalRef(*iter_, absl::string_view(start(), start_to_cursor()))\n            .AppendTo(dest);\n        return true;\n      }\n      ExternalRef(*iter_).AppendTo(dest);\n      length -= iter_->size();\n      ++iter_;\n    }\n\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    secondary_buffer_.Clear();\n    iter_ = secondary_buffer_.blocks().cend();\n    absl::MutexLock lock(shared_->mutex);\n    if (ABSL_PREDICT_FALSE(!SyncPos())) return false;\n    if (length >= buffer_sizer_.BufferLength(pos())) {\n      // Read directly to `dest`.\n      if (ABSL_PREDICT_FALSE(!shared_->reader->ReadAndAppend(length, dest))) {\n        set_limit_pos(shared_->reader->pos());\n        if (ABSL_PREDICT_FALSE(!shared_->reader->ok())) {\n          return FailWithoutAnnotation(shared_->reader->status());\n        }\n        return false;\n      }\n      move_limit_pos(length);\n      return true;\n    }\n    if (ABSL_PREDICT_FALSE(!ReadSome())) return false;\n  }\n}\n\nbool ReaderFactoryBase::ConcurrentReader::CopyBehindScratch(Position length,\n                                                            Writer& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of PullableReader::CopyBehindScratch(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::CopyBehindScratch(Writer&): \"\n         \"scratch used\";\n  if (length <= available()) {\n    const absl::string_view data(cursor(), length);\n    move_cursor(length);\n    return dest.Write(ExternalRef(*iter_, data));\n  }\n  if (iter_ != secondary_buffer_.blocks().cend()) {\n    if (ABSL_PREDICT_FALSE(!dest.Write(\n            ExternalRef(*iter_, absl::string_view(cursor(), available()))))) {\n      return false;\n    }\n    length -= available();\n    ++iter_;\n  }\n  set_buffer();\n  for (;;) {\n    while (iter_ != secondary_buffer_.blocks().cend()) {\n      move_limit_pos(iter_->size());\n      if (length <= iter_->size()) {\n        set_buffer(iter_->data(), iter_->size(), length);\n        return dest.Write(\n            ExternalRef(*iter_, absl::string_view(start(), start_to_cursor())));\n      }\n      if (ABSL_PREDICT_FALSE(!dest.Write(*iter_))) return false;\n      length -= iter_->size();\n      ++iter_;\n    }\n\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    secondary_buffer_.Clear();\n    iter_ = secondary_buffer_.blocks().cend();\n    set_buffer();\n    absl::MutexLock lock(shared_->mutex);\n    if (ABSL_PREDICT_FALSE(!SyncPos())) return false;\n    if (length >= buffer_sizer_.BufferLength(pos())) {\n      // Read directly to `dest`.\n      if (ABSL_PREDICT_FALSE(!shared_->reader->Copy(length, dest))) {\n        set_limit_pos(shared_->reader->pos());\n        if (ABSL_PREDICT_FALSE(!shared_->reader->ok())) {\n          return FailWithoutAnnotation(shared_->reader->status());\n        }\n        return false;\n      }\n      move_limit_pos(length);\n      return true;\n    }\n    if (ABSL_PREDICT_FALSE(!ReadSome())) return false;\n  }\n}\n\nbool ReaderFactoryBase::ConcurrentReader::ReadSomeBehindScratch(\n    size_t max_length, char* dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of PullableReader::ReadSomeBehindScratch(char*): \"\n         \"nothing to read, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PullableReader::ReadSomeBehindScratch(char*): \"\n         \"some data available, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadSomeBehindScratch(char*): \"\n         \"scratch used\";\n  if (iter_ != secondary_buffer_.blocks().cend()) ++iter_;\n  set_buffer();\n  for (;;) {\n    while (iter_ != secondary_buffer_.blocks().cend()) {\n      if (ABSL_PREDICT_TRUE(!iter_->empty())) {\n        max_length = UnsignedMin(max_length, iter_->size());\n        set_buffer(iter_->data(), iter_->size(), max_length);\n        move_limit_pos(start_to_limit());\n        std::memcpy(dest, start(), start_to_cursor());\n        return true;\n      }\n      ++iter_;\n    }\n\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    secondary_buffer_.Clear();\n    iter_ = secondary_buffer_.blocks().cend();\n    absl::MutexLock lock(shared_->mutex);\n    if (ABSL_PREDICT_FALSE(!SyncPos())) return false;\n    if (max_length >= buffer_sizer_.BufferLength(pos())) {\n      // Read directly to `dest`.\n      if (ABSL_PREDICT_FALSE(!shared_->reader->ReadSome(max_length, dest))) {\n        if (ABSL_PREDICT_FALSE(!shared_->reader->ok())) {\n          return FailWithoutAnnotation(shared_->reader->status());\n        }\n        return false;\n      }\n      set_limit_pos(shared_->reader->pos());\n      return true;\n    }\n    if (ABSL_PREDICT_FALSE(!ReadSome())) return false;\n  }\n}\n\nbool ReaderFactoryBase::ConcurrentReader::CopySomeBehindScratch(\n    size_t max_length, Writer& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of \"\n         \"PullableReader::CopySomeBehindScratch(Writer&): \"\n         \"nothing to read, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of \"\n         \"PullableReader::CopySomeBehindScratch(Writer&): \"\n         \"some data available, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PullableReader::CopySomeBehindScratch(Writer&): \"\n         \"scratch used\";\n  if (iter_ != secondary_buffer_.blocks().cend()) ++iter_;\n  set_buffer();\n  for (;;) {\n    while (iter_ != secondary_buffer_.blocks().cend()) {\n      if (ABSL_PREDICT_TRUE(!iter_->empty())) {\n        max_length = UnsignedMin(max_length, iter_->size());\n        set_buffer(iter_->data(), iter_->size(), max_length);\n        move_limit_pos(start_to_limit());\n        return dest.Write(\n            ExternalRef(*iter_, absl::string_view(start(), start_to_cursor())));\n      }\n      ++iter_;\n    }\n\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    secondary_buffer_.Clear();\n    iter_ = secondary_buffer_.blocks().cend();\n    absl::MutexLock lock(shared_->mutex);\n    if (ABSL_PREDICT_FALSE(!SyncPos())) return false;\n    if (max_length >= buffer_sizer_.BufferLength(pos())) {\n      // Copy directly to `dest`.\n      const bool copy_ok = shared_->reader->CopySome(max_length, dest);\n      set_limit_pos(shared_->reader->pos());\n      if (ABSL_PREDICT_FALSE(!copy_ok)) {\n        if (ABSL_PREDICT_FALSE(!shared_->reader->ok())) {\n          return FailWithoutAnnotation(shared_->reader->status());\n        }\n        return false;\n      }\n      return true;\n    }\n    if (ABSL_PREDICT_FALSE(!ReadSome())) return false;\n  }\n}\n\nvoid ReaderFactoryBase::ConcurrentReader::ReadHintBehindScratch(\n    size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of PullableReader::ReadHintBehindScratch(): \"\n         \"enough data available, use ReadHint() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::ReadHintBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  const size_t secondary_buffered_length =\n      secondary_buffer_.size() - iter_.CharIndexInChain(start_to_cursor());\n  if (secondary_buffered_length < min_length) {\n    set_limit_pos(pos());\n    set_buffer();\n    secondary_buffer_.RemovePrefix(secondary_buffer_.size() -\n                                   secondary_buffered_length);\n    const size_t min_length_to_read = min_length - secondary_buffered_length;\n    const size_t recommended_length_to_read =\n        UnsignedMax(recommended_length, min_length) - secondary_buffered_length;\n    {\n      absl::MutexLock lock(shared_->mutex);\n      if (ABSL_PREDICT_FALSE(!shared_->reader->Seek(\n              limit_pos() + secondary_buffered_length))) {\n        if (ABSL_PREDICT_FALSE(!shared_->reader->ok())) {\n          FailWithoutAnnotation(shared_->reader->status());\n        }\n      } else {\n        if (recommended_length_to_read > min_length_to_read) {\n          shared_->reader->ReadHint(min_length_to_read,\n                                    recommended_length_to_read);\n        }\n        if (ABSL_PREDICT_FALSE(!shared_->reader->ReadAndAppend(\n                min_length_to_read, secondary_buffer_))) {\n          if (ABSL_PREDICT_FALSE(!shared_->reader->ok())) {\n            FailWithoutAnnotation(shared_->reader->status());\n          }\n        }\n      }\n    }\n    iter_ = secondary_buffer_.blocks().cbegin();\n    if (iter_ != secondary_buffer_.blocks().cend()) {\n      set_buffer(iter_->data(), iter_->size());\n      move_limit_pos(available());\n    }\n  }\n}\n\nbool ReaderFactoryBase::ConcurrentReader::SyncBehindScratch(\n    SyncType sync_type) {\n  const Position new_pos = pos();\n  buffer_sizer_.EndRun(new_pos);\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  secondary_buffer_.Clear();\n  iter_ = secondary_buffer_.blocks().cend();\n  set_buffer();\n  set_limit_pos(new_pos);\n  buffer_sizer_.BeginRun(limit_pos());\n  if (sync_type == SyncType::kFromObject) return true;\n  absl::MutexLock lock(shared_->mutex);\n  return shared_->reader->Sync(sync_type);\n}\n\nbool ReaderFactoryBase::ConcurrentReader::ToleratesReadingAhead() {\n  if (buffer_sizer_.read_all_hint()) return true;\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  absl::MutexLock lock(shared_->mutex);\n  return shared_->reader->ToleratesReadingAhead();\n}\n\nbool ReaderFactoryBase::ConcurrentReader::SeekBehindScratch(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"scratch used\";\n  const Position secondary_buffer_begin =\n      start_pos() - iter_.CharIndexInChain();\n  const Position secondary_buffer_end =\n      secondary_buffer_begin + secondary_buffer_.size();\n  if (new_pos >= secondary_buffer_begin && new_pos <= secondary_buffer_end) {\n    // Seeking within `secondary_buffer_`.\n    if (new_pos == secondary_buffer_end) {\n      iter_ = secondary_buffer_.blocks().cend();\n      set_buffer();\n      set_limit_pos(secondary_buffer_end);\n      return true;\n    }\n    const Chain::BlockAndChar block_and_char =\n        secondary_buffer_.BlockAndCharIndex(\n            IntCast<size_t>(new_pos - secondary_buffer_begin));\n    iter_ = block_and_char.block_iter;\n    set_buffer(iter_->data(), iter_->size(), block_and_char.char_index);\n    set_limit_pos(new_pos + available());\n    return true;\n  }\n\n  buffer_sizer_.EndRun(pos());\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  secondary_buffer_.Clear();\n  iter_ = secondary_buffer_.blocks().cend();\n  set_buffer();\n  set_limit_pos(secondary_buffer_end);\n  if (new_pos > secondary_buffer_end) {\n    // Seeking forwards.\n    std::optional<Position> size;\n    {\n      absl::MutexLock lock(shared_->mutex);\n      size = shared_->reader->Size();\n      if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n        return FailWithoutAnnotation(shared_->reader->status());\n      }\n    }\n    if (ABSL_PREDICT_FALSE(new_pos > *size)) {\n      // Source ends.\n      set_limit_pos(*size);\n      buffer_sizer_.BeginRun(limit_pos());\n      return false;\n    }\n  }\n  set_limit_pos(new_pos);\n  buffer_sizer_.BeginRun(limit_pos());\n  return true;\n}\n\nstd::optional<Position> ReaderFactoryBase::ConcurrentReader::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  absl::MutexLock lock(shared_->mutex);\n  const std::optional<Position> size = shared_->reader->Size();\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n    FailWithoutAnnotation(shared_->reader->status());\n  }\n  return size;\n}\n\nstd::unique_ptr<Reader> ReaderFactoryBase::ConcurrentReader::NewReaderImpl(\n    Position initial_pos) {\n  return std::make_unique<ConcurrentReader>(shared_, initial_pos);\n}\n\nvoid ReaderFactoryBase::Initialize(BufferOptions buffer_options, Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of ReaderFactory: null Reader pointer\";\n  RIEGELI_ASSERT(src->SupportsRandomAccess())\n      << \"Failed precondition of ReaderFactory: \"\n         \"the original Reader does not support random access\";\n  initial_pos_ = src->pos();\n  if (!src->SupportsNewReader()) {\n    shared_ = std::make_unique<Shared>(buffer_options, src);\n  }\n  if (ABSL_PREDICT_FALSE(!src->ok())) FailWithoutAnnotation(src->status());\n}\n\nvoid ReaderFactoryBase::Done() { shared_.reset(); }\n\nabsl::Status ReaderFactoryBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    if (shared_ == nullptr) {\n      Reader& src = *SrcReader();\n      return src.AnnotateStatus(std::move(status));\n    } else {\n      absl::MutexLock lock(shared_->mutex);\n      return shared_->reader->AnnotateStatus(std::move(status));\n    }\n  }\n  return status;\n}\n\nstd::unique_ptr<Reader> ReaderFactoryBase::NewReader(Position initial_pos) const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  if (shared_ == nullptr) {\n    Reader& src = *SrcReader();\n    // `src.NewReaderCurrentPos()` is thread-safe in this context because the\n    // only operation on `src` is `NewReader()` or `NewReaderCurrentPos()`.\n    std::unique_ptr<Reader> reader = initial_pos == pos()\n                                         ? src.NewReaderCurrentPos()\n                                         : src.NewReader(initial_pos);\n    RIEGELI_ASSERT_NE(reader, nullptr)\n        << \"Failed postcondition of Reader::NewReader(): \"\n           \"returned null but Reader is ok() and SupportsNewReader()\";\n    return reader;\n  } else {\n    return std::make_unique<ConcurrentReader>(shared_.get(), initial_pos);\n  }\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/reader_factory.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_READER_FACTORY_H_\n#define RIEGELI_BYTES_READER_FACTORY_H_\n\n#include <memory>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/base/thread_annotations.h\"\n#include \"absl/status/status.h\"\n#include \"absl/synchronization/mutex.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/stable_dependency.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `ReaderFactory`.\nclass ReaderFactoryBase : public Object {\n public:\n  class Options : public BufferOptionsBase<Options> {};\n\n  // Returns the original `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the original position of the original `Reader`.\n  Position pos() const { return initial_pos_; }\n\n  // Returns a `Reader` which reads from the same source as the original\n  // `Reader`, but has an independent current position, starting from\n  // `initial_pos`, defaulting to `pos()`.\n  //\n  // If the source ends before `initial_pos`, the position of the new `Reader`\n  // is set to the end. The resulting `Reader` supports `Seek()` and\n  // `NewReader()`. Calling `NewReader()` on the new `Reader` is equivalent to\n  // calling it on this `ReaderFactory` again.\n  //\n  // The new `Reader` does not own the source, even if the original `Reader`\n  // does. The original `Reader` must not be accessed until the new `Reader` is\n  // closed or no longer used.\n  //\n  // In contrast to `Reader::NewReader()` and `Reader::NewReaderCurrentPos()`,\n  // `ReaderFactory::NewReader()` is unconditionally const and thread-safe,\n  // even with an implicit initial position (concurrency with other operations\n  // is not applicable because the original `Reader` must not be accessed\n  // directly). The optimization of sharing the current buffer is applicable\n  // as long as the initial position is implicit or matches `pos()`.\n  //\n  // If `ok()` is `true`, then `NewReader()` does not return `nullptr`.\n  std::unique_ptr<Reader> NewReader(Position initial_pos) const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  std::unique_ptr<Reader> NewReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return NewReader(pos());\n  }\n\n protected:\n  using Object::Object;\n\n  ReaderFactoryBase(ReaderFactoryBase&& that) noexcept;\n  ReaderFactoryBase& operator=(ReaderFactoryBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(BufferOptions buffer_options, Reader* src);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n\n private:\n  class ConcurrentReader;\n\n  struct Shared {\n    explicit Shared(BufferOptions buffer_options, Reader* reader)\n        : buffer_options(buffer_options), reader(reader) {}\n\n    BufferOptions buffer_options;\n    absl::Mutex mutex;\n    Reader* reader ABSL_GUARDED_BY(mutex);\n  };\n\n  Position initial_pos_ = 0;\n  // If `shared_ == nullptr`, then `!is_open()` or `Reader::NewReader()` is\n  // used. If `shared_ != nullptr`, then `ConcurrentReader` emulation is used.\n  std::unique_ptr<Shared> shared_;\n};\n\n// `ReaderFactory` exposes `Reader::NewReader()`, or provides its emulation\n// for `Reader` classes which do not support `NewReader()`. This allows for\n// interleaved or concurrent reading of several regions of the same source.\n//\n// The original `Reader` must support random access.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the original `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The original `Reader` must not be accessed until the `ReaderFactory` is\n// closed or no longer used.\ntemplate <typename Src = Reader*>\nclass ReaderFactory : public ReaderFactoryBase {\n public:\n  // Creates a closed `ReaderFactory`.\n  explicit ReaderFactory(Closed) noexcept : ReaderFactoryBase(kClosed) {}\n\n  // Will read from the original `Reader` provided by `src`.\n  explicit ReaderFactory(Initializer<Src> src, Options options = Options());\n\n  ReaderFactory(ReaderFactory&& that) = default;\n  ReaderFactory& operator=(ReaderFactory&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `ReaderFactory`. This\n  // avoids constructing a temporary `ReaderFactory` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the original `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n\n private:\n  // The object providing and possibly owning the original `Reader`.\n  StableDependency<Reader*, Src> src_;\n};\n\nexplicit ReaderFactory(Closed) -> ReaderFactory<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit ReaderFactory(Src&& src, ReaderFactoryBase::Options options =\n                                      ReaderFactoryBase::Options())\n    -> ReaderFactory<TargetT<Src>>;\n\n// Implementation details follow.\n\ninline ReaderFactoryBase::ReaderFactoryBase(ReaderFactoryBase&& that) noexcept\n    : Object(static_cast<Object&&>(that)),\n      initial_pos_(that.initial_pos_),\n      shared_(std::move(that.shared_)) {}\n\ninline ReaderFactoryBase& ReaderFactoryBase::operator=(\n    ReaderFactoryBase&& that) noexcept {\n  Object::operator=(static_cast<Object&&>(that));\n  initial_pos_ = that.initial_pos_;\n  shared_ = std::move(that.shared_);\n  return *this;\n}\n\ninline void ReaderFactoryBase::Reset(Closed) {\n  Object::Reset(kClosed);\n  initial_pos_ = 0;\n  shared_.reset();\n}\n\ninline void ReaderFactoryBase::Reset() {\n  Object::Reset();\n  // `initial_pos_` will be set by `Initialize()`.\n  shared_.reset();\n}\n\ntemplate <typename Src>\ninline ReaderFactory<Src>::ReaderFactory(Initializer<Src> src, Options options)\n    : src_(std::move(src)) {\n  Initialize(options.buffer_options(), src_.get());\n}\n\ntemplate <typename Src>\ninline void ReaderFactory<Src>::Reset(Closed) {\n  ReaderFactoryBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void ReaderFactory<Src>::Reset(Initializer<Src> src, Options options) {\n  ReaderFactoryBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(options.buffer_options(), src_.get());\n}\n\ntemplate <typename Src>\nvoid ReaderFactory<Src>::Done() {\n  ReaderFactoryBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(src_->status());\n    }\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_READER_FACTORY_H_\n"
  },
  {
    "path": "riegeli/bytes/reader_istream.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/reader_istream.h\"\n\n#include <stddef.h>\n\n#include <ios>\n#include <iosfwd>\n#include <limits>\n#include <optional>\n\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nnamespace stream_internal {\n\nclass ReaderStreambuf::BufferSync {\n public:\n  explicit BufferSync(ReaderStreambuf* streambuf) : streambuf_(streambuf) {\n    streambuf_->reader_->set_cursor(streambuf_->gptr());\n  }\n\n  BufferSync(const BufferSync&) = delete;\n  BufferSync& operator=(const BufferSync&) = delete;\n\n  ~BufferSync() {\n    streambuf_->setg(const_cast<char*>(streambuf_->reader_->start()),\n                     const_cast<char*>(streambuf_->reader_->cursor()),\n                     const_cast<char*>(streambuf_->reader_->limit()));\n  }\n  ReaderStreambuf* streambuf_;\n};\n\nvoid ReaderStreambuf::Fail() { state_.Fail(reader_->status()); }\n\nint ReaderStreambuf::sync() {\n  if (ABSL_PREDICT_FALSE(!ok())) return -1;\n  BufferSync buffer_sync(this);\n  if (ABSL_PREDICT_FALSE(!reader_->Sync())) {\n    Fail();\n    return -1;\n  }\n  return 0;\n}\n\nstd::streamsize ReaderStreambuf::showmanyc() {\n  if (ABSL_PREDICT_FALSE(!ok())) return -1;\n  BufferSync buffer_sync(this);\n  if (ABSL_PREDICT_FALSE(!reader_->Pull())) {\n    if (ABSL_PREDICT_FALSE(!reader_->ok())) Fail();\n    return -1;\n  }\n  return IntCast<std::streamsize>(reader_->available());\n}\n\nint ReaderStreambuf::underflow() {\n  if (ABSL_PREDICT_FALSE(!ok())) return traits_type::eof();\n  BufferSync buffer_sync(this);\n  if (ABSL_PREDICT_FALSE(!reader_->Pull())) {\n    if (ABSL_PREDICT_FALSE(!reader_->ok())) Fail();\n    return traits_type::eof();\n  }\n  return traits_type::to_int_type(*reader_->cursor());\n}\n\nstd::streamsize ReaderStreambuf::xsgetn(char* dest, std::streamsize length) {\n  RIEGELI_ASSERT_GE(length, 0)\n      << \"Failed precondition of streambuf::xsgetn(): negative length\";\n  if (ABSL_PREDICT_TRUE(length <= egptr() - gptr())) {\n    riegeli::null_safe_memcpy(dest, gptr(), IntCast<size_t>(length));\n    // Do not use `gbump()` because its parameter has type `int`.\n    setg(eback(), gptr() + length, egptr());\n    return length;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return 0;\n  BufferSync buffer_sync(this);\n  size_t length_read;\n  if (ABSL_PREDICT_FALSE(\n          !reader_->Read(IntCast<size_t>(length), dest, &length_read)) &&\n      ABSL_PREDICT_FALSE(!reader_->ok())) {\n    Fail();\n  }\n  return IntCast<std::streamsize>(length_read);\n}\n\nstd::streampos ReaderStreambuf::seekoff(std::streamoff off,\n                                        std::ios_base::seekdir dir,\n                                        std::ios_base::openmode which) {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::streampos(std::streamoff{-1});\n  BufferSync buffer_sync(this);\n  Position new_pos;\n  switch (dir) {\n    case std::ios_base::beg:\n      if (ABSL_PREDICT_FALSE(off < 0)) {\n        return std::streampos(std::streamoff{-1});\n      }\n      new_pos = IntCast<Position>(off);\n      break;\n    case std::ios_base::cur:\n      new_pos = reader_->pos();\n      if (off < 0) {\n        if (ABSL_PREDICT_FALSE(NegatingUnsignedCast(off) > new_pos)) {\n          return std::streampos(std::streamoff{-1});\n        }\n        new_pos -= NegatingUnsignedCast(off);\n        if (ABSL_PREDICT_FALSE(\n                new_pos >\n                Position{std::numeric_limits<std::streamoff>::max()})) {\n          return std::streampos(std::streamoff{-1});\n        }\n      } else {\n        if (ABSL_PREDICT_FALSE(\n                new_pos >\n                    Position{std::numeric_limits<std::streamoff>::max()} ||\n                IntCast<Position>(off) >\n                    Position{std::numeric_limits<std::streamoff>::max()} -\n                        new_pos)) {\n          return std::streampos(std::streamoff{-1});\n        }\n        new_pos += IntCast<Position>(off);\n      }\n      break;\n    case std::ios_base::end: {\n      if (ABSL_PREDICT_FALSE(!reader_->SupportsSize())) {\n        // Indicate that `seekoff(std::ios_base::end)` is not supported.\n        return std::streampos(std::streamoff{-1});\n      }\n      const std::optional<Position> size = reader_->Size();\n      if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n        Fail();\n        return std::streampos(std::streamoff{-1});\n      }\n      if (ABSL_PREDICT_FALSE(off > 0 || NegatingUnsignedCast(off) > *size)) {\n        return std::streampos(std::streamoff{-1});\n      }\n      new_pos = *size - NegatingUnsignedCast(off);\n      if (ABSL_PREDICT_FALSE(\n              new_pos > Position{std::numeric_limits<std::streamoff>::max()})) {\n        return std::streampos(std::streamoff{-1});\n      }\n    } break;\n    default:\n      RIEGELI_ASSUME_UNREACHABLE()\n          << \"Unknown seek direction: \" << static_cast<int>(dir);\n  }\n  if (new_pos >= reader_->pos()) {\n    // Seeking forwards is supported even if random access is not.\n  } else if (ABSL_PREDICT_FALSE(!reader_->SupportsRewind())) {\n    // Indicate that `seekoff()` is not supported.\n    return std::streampos(std::streamoff{-1});\n  }\n  if (ABSL_PREDICT_FALSE(!reader_->Seek(new_pos))) {\n    if (ABSL_PREDICT_FALSE(!reader_->ok())) Fail();\n    return std::streampos(std::streamoff{-1});\n  }\n  return std::streampos(IntCast<std::streamoff>(new_pos));\n}\n\nstd::streampos ReaderStreambuf::seekpos(std::streampos pos,\n                                        std::ios_base::openmode which) {\n  return seekoff(std::streamoff(pos), std::ios_base::beg, which);\n}\n\n}  // namespace stream_internal\n\nbool ReaderIStreamBase::close() {\n  Done();\n  return not_failed();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/reader_istream.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_READER_ISTREAM_H_\n#define RIEGELI_BYTES_READER_ISTREAM_H_\n\n#include <ios>\n#include <iosfwd>\n#include <istream>\n#include <streambuf>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nnamespace stream_internal {\n\nclass ReaderStreambuf : public std::streambuf {\n public:\n  explicit ReaderStreambuf(Closed) noexcept : state_(kClosed) {}\n\n  ReaderStreambuf() noexcept {}\n\n  ReaderStreambuf(ReaderStreambuf&& that) noexcept;\n  ReaderStreambuf& operator=(ReaderStreambuf&& that) noexcept;\n\n  void Initialize(Reader* src);\n  void MoveBegin();\n  void MoveEnd(Reader* src);\n  void Done();\n\n  bool ok() const { return state_.ok(); }\n  bool is_open() const { return state_.is_open(); }\n  bool not_failed() const { return state_.not_failed(); }\n  absl::Status status() const { return state_.status(); }\n  void MarkClosed() { state_.MarkClosed(); }\n  ABSL_ATTRIBUTE_COLD void Fail();\n\n protected:\n  int sync() override;\n  std::streamsize showmanyc() override;\n  int underflow() override;\n  std::streamsize xsgetn(char* dest, std::streamsize length) override;\n  std::streampos seekoff(std::streamoff off, std::ios_base::seekdir dir,\n                         std::ios_base::openmode which) override;\n  std::streampos seekpos(std::streampos pos,\n                         std::ios_base::openmode which) override;\n\n private:\n  class BufferSync;\n\n  ObjectState state_;\n  Reader* reader_ = nullptr;\n\n  // Invariants:\n  //   `eback() == (is_open() ? reader_->start() : nullptr)`\n  //   `egptr() == (is_open() ? reader_->limit() : nullptr)`\n};\n\n}  // namespace stream_internal\n\n// Template parameter independent part of `ReaderIStream`.\nclass ReaderIStreamBase : public std::istream {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n  };\n\n  // Returns the `Reader`. Unchanged by `close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // If `!is_open()`, does nothing. Otherwise:\n  //  * Synchronizes the current `ReaderIStream` position to the `Reader`.\n  //  * Closes the `Reader` if it is owned.\n  //\n  // Also, propagates `Reader` failures to `rdstate() & std::ios_base::badbit`\n  // (doing this during reading is not feasible without throwing exceptions).\n  //\n  // Returns `true` if the `Reader` did not fail, i.e. if it was OK just before\n  // becoming closed.\n  //\n  // Destroying or assigning to a `ReaderIStream` closes it implicitly, but an\n  // explicit `close()` call allows to detect failures (use `status()` for\n  // failure details).\n  bool close();\n\n  // Returns `true` if the `ReaderIStream` is OK, i.e. open and not failed.\n  bool ok() const { return streambuf_.ok(); }\n\n  // Returns `true` if the `ReaderIStream` is open, i.e. not closed.\n  bool is_open() const { return streambuf_.is_open(); }\n\n  // Returns `true` if the `ReaderIStream` is not failed.\n  bool not_failed() const { return streambuf_.not_failed(); }\n\n  // Returns an `absl::Status` describing the failure if the `ReaderIStream`\n  // is failed, or an `absl::FailedPreconditionError()` if the `ReaderIStream`\n  // is successfully closed, or `absl::OkStatus()` if the `ReaderIStream` is OK.\n  absl::Status status() const { return streambuf_.status(); }\n\n  // Supports `Dependency`.\n  friend MakerType<Closed> RiegeliDependencySentinel(ReaderIStreamBase*) {\n    return {kClosed};\n  }\n\n protected:\n  explicit ReaderIStreamBase(Closed) noexcept\n      : std::istream(&streambuf_), streambuf_(kClosed) {}\n\n  ReaderIStreamBase() noexcept : std::istream(&streambuf_) {}\n\n  ReaderIStreamBase(ReaderIStreamBase&& that) noexcept;\n  ReaderIStreamBase& operator=(ReaderIStreamBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(Reader* src);\n\n  virtual void Done() = 0;\n\n  stream_internal::ReaderStreambuf streambuf_;\n\n  // Invariant: `rdbuf() == &streambuf_`\n};\n\n// Adapts a `Reader` to a `std::istream`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `Reader`. `Src` must support `Dependency<Reader*, Src>`,\n// e.g. `Reader*` (not owned, default), `ChainReader<>` (owned),\n// `std::unique_ptr<Reader>` (owned), `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The `Reader` must not be accessed until the `ReaderIStream` is closed or no\n// longer used.\ntemplate <typename Src = Reader*>\nclass ReaderIStream : public ReaderIStreamBase {\n public:\n  // Creates a closed `ReaderIStream`.\n  explicit ReaderIStream(Closed) noexcept : ReaderIStreamBase(kClosed) {}\n\n  // Will read from the `Reader` provided by `src`.\n  explicit ReaderIStream(Initializer<Src> src, Options options = Options());\n\n  // These operations cannot be defaulted because `ReaderIStreamBase` virtually\n  // derives from `std::ios` which has these operations deleted.\n  ReaderIStream(ReaderIStream&& that) noexcept\n#if __cpp_concepts\n    requires std::is_move_constructible_v<Dependency<Reader*, Src>>\n#endif\n      : ReaderIStreamBase(static_cast<ReaderIStreamBase&&>(that)),\n        src_(std::move(that.src_), *this, that) {\n  }\n  ReaderIStream& operator=(ReaderIStream&& that) noexcept\n#if __cpp_concepts\n    requires(std::is_move_assignable_v<Dependency<Reader*, Src>>)\n#endif\n  {\n    ReaderIStreamBase::operator=(static_cast<ReaderIStreamBase&&>(that));\n    src_.Reset(std::move(that.src_), *this, that);\n    return *this;\n  }\n\n  ~ReaderIStream() override { Done(); }\n\n  // Makes `*this` equivalent to a newly constructed `ReaderIStream`. This\n  // avoids constructing a temporary `ReaderIStream` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the `Reader`. Unchanged by\n  // `close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the `Reader`.\n  MovingDependency<Reader*, Src, Mover> src_;\n};\n\nexplicit ReaderIStream(Closed) -> ReaderIStream<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit ReaderIStream(Src&& src, ReaderIStreamBase::Options options =\n                                      ReaderIStreamBase::Options())\n    -> ReaderIStream<TargetT<Src>>;\n\n// Implementation details follow.\n\nnamespace stream_internal {\n\ninline ReaderStreambuf::ReaderStreambuf(ReaderStreambuf&& that) noexcept\n    : std::streambuf(that),\n      state_(std::move(that.state_)),\n      reader_(that.reader_) {\n  that.setg(nullptr, nullptr, nullptr);\n}\n\ninline ReaderStreambuf& ReaderStreambuf::operator=(\n    ReaderStreambuf&& that) noexcept {\n  if (ABSL_PREDICT_TRUE(&that != this)) {\n    std::streambuf::operator=(that);\n    state_ = std::move(that.state_);\n    reader_ = that.reader_;\n    that.setg(nullptr, nullptr, nullptr);\n  }\n  return *this;\n}\n\ninline void ReaderStreambuf::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of ReaderStreambuf: null Reader pointer\";\n  reader_ = src;\n  setg(const_cast<char*>(reader_->start()),\n       const_cast<char*>(reader_->cursor()),\n       const_cast<char*>(reader_->limit()));\n  if (ABSL_PREDICT_FALSE(!reader_->ok()) && reader_->available() == 0) Fail();\n}\n\ninline void ReaderStreambuf::MoveBegin() {\n  // In a closed `ReaderIStream`, `ReaderIStream::src_ != nullptr`\n  // does not imply `ReaderStreambuf::reader_ != nullptr`, because\n  // `ReaderIStream::streambuf_` can be left uninitialized.\n  if (reader_ == nullptr) return;\n  reader_->set_cursor(gptr());\n}\n\ninline void ReaderStreambuf::MoveEnd(Reader* src) {\n  // In a closed `ReaderIStream`, `ReaderIStream::src_ != nullptr`\n  // does not imply `ReaderStreambuf::reader_ != nullptr`, because\n  // `ReaderIStream::streambuf_` can be left uninitialized.\n  if (reader_ == nullptr) return;\n  reader_ = src;\n  setg(const_cast<char*>(reader_->start()),\n       const_cast<char*>(reader_->cursor()),\n       const_cast<char*>(reader_->limit()));\n}\n\ninline void ReaderStreambuf::Done() {\n  reader_->set_cursor(gptr());\n  setg(nullptr, nullptr, nullptr);\n}\n\n}  // namespace stream_internal\n\ninline ReaderIStreamBase::ReaderIStreamBase(ReaderIStreamBase&& that) noexcept\n    : std::istream(static_cast<std::istream&&>(that)),\n      streambuf_(std::move(that.streambuf_)) {\n  set_rdbuf(&streambuf_);\n}\n\ninline ReaderIStreamBase& ReaderIStreamBase::operator=(\n    ReaderIStreamBase&& that) noexcept {\n  if (ABSL_PREDICT_TRUE(&that != this)) {\n    Done();\n    std::istream::operator=(static_cast<std::istream&&>(that));\n    streambuf_ = std::move(that.streambuf_);\n  }\n  return *this;\n}\n\ninline void ReaderIStreamBase::Reset(Closed) {\n  Done();\n  streambuf_ = stream_internal::ReaderStreambuf(kClosed);\n  init(&streambuf_);\n}\n\ninline void ReaderIStreamBase::Reset() {\n  Done();\n  streambuf_ = stream_internal::ReaderStreambuf();\n  init(&streambuf_);\n}\n\ninline void ReaderIStreamBase::Initialize(Reader* src) {\n  streambuf_.Initialize(src);\n  if (ABSL_PREDICT_FALSE(!streambuf_.ok())) setstate(std::ios_base::badbit);\n}\n\ntemplate <typename Src>\nclass ReaderIStream<Src>::Mover {\n public:\n  static auto member() { return &ReaderIStream::src_; }\n\n  explicit Mover(ReaderIStream& self) { self.streambuf_.MoveBegin(); }\n\n  void Done(ReaderIStream& self) { self.streambuf_.MoveEnd(self.src_.get()); }\n};\n\ntemplate <typename Src>\ninline ReaderIStream<Src>::ReaderIStream(Initializer<Src> src, Options options)\n    : src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void ReaderIStream<Src>::Reset(Closed) {\n  ReaderIStreamBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void ReaderIStream<Src>::Reset(Initializer<Src> src, Options options) {\n  ReaderIStreamBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\nvoid ReaderIStream<Src>::Done() {\n  if (ABSL_PREDICT_FALSE(!is_open())) return;\n  streambuf_.Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) streambuf_.Fail();\n  }\n  if (ABSL_PREDICT_FALSE(!streambuf_.ok())) setstate(std::ios_base::badbit);\n  streambuf_.MarkClosed();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_READER_ISTREAM_H_\n"
  },
  {
    "path": "riegeli/bytes/resizable_writer.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/resizable_writer.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_utils.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/string_reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid ResizableWriterBase::Done() {\n  ResizableWriterBase::FlushImpl(FlushType::kFromObject);\n  Writer::Done();\n  secondary_buffer_ = Chain();\n  associated_reader_.Reset();\n}\n\ninline void ResizableWriterBase::SyncSecondaryBuffer() {\n  set_start_pos(pos());\n  secondary_buffer_.RemoveSuffix(available(), options_);\n  set_buffer();\n}\n\ninline void ResizableWriterBase::MakeSecondaryBuffer(\n    size_t min_length, size_t recommended_length) {\n  const absl::Span<char> buffer = secondary_buffer_.AppendBuffer(\n      min_length, recommended_length, Chain::kAnyLength, options_);\n  set_buffer(buffer.data(), buffer.size());\n}\n\nvoid ResizableWriterBase::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (write_size_hint == std::nullopt || ABSL_PREDICT_FALSE(!ok())) return;\n  const size_t size_hint =\n      UnsignedMax(SaturatingAdd(IntCast<size_t>(pos()),\n                                SaturatingIntCast<size_t>(*write_size_hint)),\n                  written_size_);\n  if (!uses_secondary_buffer()) {\n    GrowDestAndMakeBuffer(size_hint);\n    return;\n  }\n  SyncSecondaryBuffer();\n  if (ABSL_PREDICT_FALSE(!GrowDestAndMakeBuffer(size_hint))) return;\n  secondary_buffer_.CopyTo(cursor() - secondary_buffer_.size());\n  secondary_buffer_.Clear();\n}\n\nbool ResizableWriterBase::PushSlow(size_t min_length,\n                                   size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(min_length > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    const size_t cursor_index = IntCast<size_t>(pos());\n    if (cursor_index == 0 || ABSL_PREDICT_FALSE(written_size_ > cursor_index)) {\n      // Allocate the first block directly in the destination. It is possible\n      // that it will not need to be copied if it turns out to be the only\n      // block, although this decision might cause it to remain wasteful if less\n      // data are written than space requested.\n      //\n      // Resize the destination also if data follow the current position,\n      // to make the data available for partial overwriting.\n      return GrowDestAndMakeBuffer(SaturatingAdd(\n          cursor_index, UnsignedMax(min_length, recommended_length)));\n    }\n    if (GrowDestUnderCapacityAndMakeBuffer(min_length)) return true;\n    set_start_pos(cursor_index);\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  MakeSecondaryBuffer(min_length, recommended_length);\n  return true;\n}\n\nbool ResizableWriterBase::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of Writer::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    const size_t cursor_index = IntCast<size_t>(pos());\n    if (cursor_index == 0) {\n      // Allocate the first block directly in the destination. It is possible\n      // that it will not need to be copied if it turns out to be the only\n      // block, although this decision might cause it to remain wasteful if less\n      // data are written than space requested.\n      if (ABSL_PREDICT_FALSE(\n              !GrowDestAndMakeBuffer(cursor_index + src.size()))) {\n        return false;\n      }\n      std::memcpy(cursor(), src.data(), src.size());\n      move_cursor(src.size());\n      return true;\n    }\n    if (GrowDestUnderCapacityAndMakeBuffer(src.size())) {\n      std::memcpy(cursor(), src.data(), src.size());\n      move_cursor(src.size());\n      return true;\n    }\n    set_start_pos(cursor_index);\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  secondary_buffer_.Append(src, options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool ResizableWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    if (GrowDestUnderCapacityAndMakeBuffer(src.size())) {\n      std::memcpy(cursor(), src.data(), src.size());\n      move_cursor(src.size());\n      return true;\n    }\n    set_start_pos(pos());\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  secondary_buffer_.Append(std::move(src), options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool ResizableWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    if (GrowDestUnderCapacityAndMakeBuffer(src.size())) {\n      src.CopyTo(cursor());\n      move_cursor(src.size());\n      return true;\n    }\n    set_start_pos(pos());\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  secondary_buffer_.Append(src, options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool ResizableWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    if (GrowDestUnderCapacityAndMakeBuffer(src.size())) {\n      src.CopyTo(cursor());\n      move_cursor(src.size());\n      return true;\n    }\n    set_start_pos(pos());\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  secondary_buffer_.Append(std::move(src), options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool ResizableWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    if (GrowDestUnderCapacityAndMakeBuffer(src.size())) {\n      cord_internal::CopyCordToArray(src, cursor());\n      move_cursor(src.size());\n      return true;\n    }\n    set_start_pos(pos());\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  secondary_buffer_.Append(src, options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool ResizableWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    if (GrowDestUnderCapacityAndMakeBuffer(src.size())) {\n      cord_internal::CopyCordToArray(src, cursor());\n      move_cursor(src.size());\n      return true;\n    }\n    set_start_pos(pos());\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  secondary_buffer_.Append(std::move(src), options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool ResizableWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    if (GrowDestUnderCapacityAndMakeBuffer(IntCast<size_t>(src.size()))) {\n      std::memset(cursor(), src.fill(), IntCast<size_t>(src.size()));\n      move_cursor(IntCast<size_t>(src.size()));\n      return true;\n    }\n    set_start_pos(pos());\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  src.AppendTo(secondary_buffer_, options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool ResizableWriterBase::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (!uses_secondary_buffer()) return ResizeDest();\n  SyncSecondaryBuffer();\n  if (ABSL_PREDICT_FALSE(!ResizeDest())) return false;\n  secondary_buffer_.CopyTo(cursor() - secondary_buffer_.size());\n  secondary_buffer_.Clear();\n  return true;\n}\n\nbool ResizableWriterBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of Writer::SeekSlow(): \"\n         \"position unchanged, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (new_pos > pos()) {\n    if (ABSL_PREDICT_FALSE(uses_secondary_buffer())) return false;\n    if (ABSL_PREDICT_FALSE(new_pos > used_size())) {\n      MakeDestBuffer(used_size());\n      return false;\n    }\n  } else {\n    if (uses_secondary_buffer()) {\n      SyncSecondaryBuffer();\n      if (ABSL_PREDICT_FALSE(!GrowDestAndMakeBuffer(IntCast<size_t>(pos())))) {\n        return false;\n      }\n      secondary_buffer_.CopyTo(cursor() - secondary_buffer_.size());\n      secondary_buffer_.Clear();\n    }\n    written_size_ = used_size();\n  }\n  MakeDestBuffer(IntCast<size_t>(new_pos));\n  return true;\n}\n\nstd::optional<Position> ResizableWriterBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  return used_size();\n}\n\nbool ResizableWriterBase::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (new_size > pos()) {\n    if (ABSL_PREDICT_FALSE(uses_secondary_buffer())) return false;\n    if (ABSL_PREDICT_FALSE(new_size > used_size())) {\n      MakeDestBuffer(used_size());\n      return false;\n    }\n  } else if (new_size > limit_pos() - secondary_buffer_.size()) {\n    secondary_buffer_.RemoveSuffix(\n        IntCast<size_t>(limit_pos()) - IntCast<size_t>(new_size), options_);\n    set_start_pos(new_size);\n    set_buffer();\n    return true;\n  } else {\n    secondary_buffer_.Clear();\n  }\n  written_size_ = 0;\n  MakeDestBuffer(IntCast<size_t>(new_size));\n  return true;\n}\n\nReader* ResizableWriterBase::ReadModeImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  if (!uses_secondary_buffer()) {\n    MakeDestBuffer(IntCast<size_t>(pos()));\n  } else {\n    SyncSecondaryBuffer();\n    if (ABSL_PREDICT_FALSE(!GrowDestAndMakeBuffer(IntCast<size_t>(pos())))) {\n      return nullptr;\n    }\n    secondary_buffer_.CopyTo(cursor() - secondary_buffer_.size());\n    secondary_buffer_.Clear();\n  }\n  StringReader<>* const reader =\n      associated_reader_.ResetReader(start(), used_dest_size());\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/resizable_writer.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_RESIZABLE_WRITER_H_\n#define RIEGELI_BYTES_RESIZABLE_WRITER_H_\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nclass Reader;\ntemplate <typename Src>\nclass StringReader;\n\n// Template parameter independent part of `ResizableWriter`.\nclass ResizableWriterBase : public Writer {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // If `false`, replaces existing contents of the destination, clearing it\n    // first.\n    //\n    // If `true`, appends to existing contents of the destination.\n    //\n    // Default: `false`.\n    Options& set_append(bool append) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      append_ = append;\n      return *this;\n    }\n    Options&& set_append(bool append) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_append(append));\n    }\n    bool append() const { return append_; }\n\n   private:\n    bool append_ = false;\n  };\n\n  bool SupportsRandomAccess() override { return true; }\n  bool SupportsReadMode() override { return true; }\n\n protected:\n  explicit ResizableWriterBase(Closed) noexcept : Writer(kClosed) {}\n\n  explicit ResizableWriterBase(BufferOptions buffer_options);\n\n  ResizableWriterBase(ResizableWriterBase&& that) noexcept;\n  ResizableWriterBase& operator=(ResizableWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options);\n  bool uses_secondary_buffer() const { return !secondary_buffer_.empty(); }\n\n  // Returns the amount of data written, either to the destination or to\n  // `secondary_buffer_`.\n  size_t used_size() const;\n\n  // Returns the amount of data written to the destination. Does not include\n  // data written to `secondary_buffer_`.\n  //\n  // Precondition: if `uses_secondary_buffer()` then `available() == 0`\n  size_t used_dest_size() const;\n\n  // Sets buffer pointers to the destination.\n  //\n  // Precondition: `!uses_secondary_buffer()`\n  virtual void MakeDestBuffer(size_t cursor_index) = 0;\n\n  // Sets the size of the destination to `used_size()`. Sets buffer pointers to\n  // the destination.\n  //\n  // Precondition: if `uses_secondary_buffer()` then `available() == 0`\n  virtual bool ResizeDest() = 0;\n\n  // Appends some uninitialized space to the destination to guarantee at least\n  // `new_size` of size. Sets buffer pointers to the destination.\n  //\n  // Precondition: if `uses_secondary_buffer()` then `available() == 0`\n  virtual bool GrowDestAndMakeBuffer(size_t new_size) = 0;\n\n  // Increases the size of the destination at least to `pos() + min_length`\n  // if this can be done without reallocation. New contents are unspecified.\n  // Sets buffer pointers to the destination.\n  //\n  // Returns `true` on success, or `false` if there was not enough space.\n  //\n  // Precondition: `!uses_secondary_buffer()`\n  virtual bool GrowDestUnderCapacityAndMakeBuffer(size_t min_length) = 0;\n\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool FlushImpl(FlushType flush_type) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  bool TruncateImpl(Position new_size) override;\n  Reader* ReadModeImpl(Position initial_pos) override;\n\n private:\n  // Discards uninitialized space from the end of `secondary_buffer_`, so that\n  // it contains only actual data written.\n  void SyncSecondaryBuffer();\n\n  // Appends uninitialized space to `secondary_buffer_`.\n  void MakeSecondaryBuffer(size_t min_length = 0,\n                           size_t recommended_length = 0);\n\n  // Move `secondary_buffer_`, adjusting buffer pointers if they point to it.\n  void MoveSecondaryBuffer(ResizableWriterBase& that);\n\n  Chain::Options options_;\n  // Buffered data which did not fit into the destination.\n  Chain secondary_buffer_;\n\n  // Size of written data is always `UnsignedMax(pos(), written_size_)`.\n  // This is used to determine the size after seeking backwards.\n  //\n  // Invariant: if `uses_secondary_buffer()` then `written_size_ == 0`.\n  size_t written_size_ = 0;\n\n  AssociatedReader<StringReader<absl::string_view>> associated_reader_;\n\n  // If `!uses_secondary_buffer()`, then the destination contains the data\n  // before the current position of length `pos()`, followed by the data after\n  // the current position of length `SaturatingSub(written_size_, pos())`,\n  // followed by free space of length\n  // `ResizableTraits::Size(*dest_) - UnsignedMax(pos(), written_size_)`.\n  //\n  // If `uses_secondary_buffer()`, then the destination contains the prefix of\n  // the data of length `limit_pos() - secondary_buffer_.size()` followed by\n  // free space, and `secondary_buffer_` contains the rest of the data of length\n  // `secondary_buffer_.size() - available()` followed by free space of length\n  // `available()`. In this case there is no data after the current position.\n  //\n  // Invariants if `ok()` (`dest_` is defined in `ResizableWriter`):\n  //   `!uses_secondary_buffer() &&\n  //    start() == ResizableTraits::Data(*dest_) &&\n  //    start_to_limit() == ResizableTraits::Size(*dest_) &&\n  //    start_pos() == 0` or\n  //       `uses_secondary_buffer() &&\n  //        limit() == secondary_buffer_.blocks().back().data() +\n  //                   secondary_buffer_.blocks().back().size()` or\n  //       `start() == nullptr`\n  //   `limit_pos() >= secondary_buffer_.size()`\n  //   `ResizableTraits::Size(*dest_) >= limit_pos() - secondary_buffer_.size()`\n};\n\n// A `Writer` which writes to a resizable array, resizing it as necessary.\n// It generalizes `StringWriter` to other objects with a flat representation.\n//\n// It supports `Seek()` and `ReadMode()`.\n//\n// The `ResizableTraits` template parameter specifies how the resizable is\n// represented. It must support at least the following static members:\n//\n// ```\n//   // The type of the resizable. It should be movable if\n//   // `!Dependency<Resizable*, Dest>::kIsStable` and the `ResizableWriter`\n//   // itself is being moved.\n//   using Resizable = ...;\n//\n//   // Returns the current data pointer.\n//   static char* Data(Resizable& dest);\n//\n//   // Returns the current size.\n//   static size_t Size(const Resizable& dest);\n//\n//   // If `true`, `Data(dest)` stays unchanged when a `Resizable` is moved.\n//   // `kIsStable` does not have to be defined, except if\n//   // `!Dependency<Resizable*, Dest>::kIsStable` and the `ResizableWriter`\n//   // itself is being moved.\n//   static constexpr bool kIsStable;\n//\n//   // Sets the size of `dest` to `new_size`.\n//   //\n//   // The prefix of data with `used_size` is preserved. Remaining contents are\n//   // unspecified. Returns `true` on success, or `false` on failure.\n//   //\n//   // The intent is to resize exactly to `new_size`, but the size reported by\n//   // `Size(dest)` can be larger than `new_size` if `dest` cannot represent\n//   // all sizes exactly.\n//   //\n//   // Preconditions:\n//   //   `used_size <= Size(dest)`\n//   //   `used_size <= new_size`\n//   static bool Resize(Resizable& dest, size_t new_size, size_t used_size);\n//\n//   // Increases the size of `dest` at least to `new_size`, or more to ensure\n//   // amortized constant time of reallocation, or more if this can be done\n//   // efficiently without reallocation.\n//   //\n//   // The prefix of data with `used_size` is preserved. Remaining contents are\n//   // unspecified. Returns `true` on success, or `false` on failure.\n//   //\n//   // This is usually equivalent to reserving `new_size`, and then calling\n//   // `GrowUnderCapacity()`, which can be assumed to always succeed after\n//   // reserving.\n//   //\n//   // Preconditions:\n//   //   `new_size > Size(dest)`\n//   //   `used_size <= Size(dest)`\n//   //   `used_size <= new_size`\n//   static bool Grow(Resizable& dest, size_t new_size, size_t used_size);\n//\n//   // Increases the size of `dest` at least to `new_size` if this can be done\n//   // without reallocation. New contents are unspecified.\n//   //\n//   // Returns `true` on success, or `false` if there was not enough space.\n//   //\n//   // If efficient resizing without filling new space is possible,\n//   // then it is recommended to resize to the capacity.\n//   // Otherwise, it is recommended to resize to approximately\n//   // `Size(dest) + UnsignedClamp(Size(dest), kDefaultMinBlockSize,\n//   //                             kDefaultMaxBlockSize)`,\n//   // but at least to `new_size` and at most to the capacity.\n//   //\n//   // Precondition: `new_size > Size(dest)`\n//   static bool GrowUnderCapacity(Resizable& dest, size_t new_size);\n// ```\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `ResizableTraits::Resizable` being written to. `Dest`\n// must support `Dependency<ResizableTraits::Resizable*, Dest>`, e.g.\n// `ResizableTraits::Resizable*` (not owned, default),\n// `ResizableTraits::Resizable` (owned),\n// `std::unique_ptr<ResizableTraits::Resizable>` (owned),\n// `Any<ResizableTraits::Resizable*>` (maybe owned).\n//\n// The `ResizableTraits::Resizable` must not be accessed until the\n// `ResizableWriter` is closed or no longer used, except that it is allowed\n// to read the `ResizableTraits::Resizable` immediately after `Flush()`.\ntemplate <typename ResizableTraits,\n          typename Dest = typename ResizableTraits::Resizable*>\nclass ResizableWriter : public ResizableWriterBase {\n public:\n  using Resizable = typename ResizableTraits::Resizable;\n\n  // Creates a closed `ResizableWriter`.\n  explicit ResizableWriter(Closed) noexcept : ResizableWriterBase(kClosed) {}\n\n  // Will write to the `Resizable` provided by `dest`.\n  explicit ResizableWriter(Initializer<Dest> dest, Options options = Options());\n\n  // Will write to an owned `Resizable` which can be accessed by `dest()`.\n  // This constructor is present only if `Dest` is `Resizable` which is\n  // default-constructible.\n  template <typename DependentDest = Dest,\n            std::enable_if_t<\n                std::conjunction_v<std::is_same<DependentDest, Resizable>,\n                                   std::is_default_constructible<Resizable>>,\n                int> = 0>\n  explicit ResizableWriter(Options options = Options());\n\n  ResizableWriter(ResizableWriter&& that) = default;\n  ResizableWriter& operator=(ResizableWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `ResizableWriter`. This\n  // avoids constructing a temporary `ResizableWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n  template <typename DependentDest = Dest,\n            std::enable_if_t<\n                std::conjunction_v<std::is_same<DependentDest, Resizable>,\n                                   std::is_default_constructible<Resizable>>,\n                int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Options options = Options());\n\n  // Returns the object providing and possibly owning the `Resizable` being\n  // written to. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n\n  // Returns the `Resizable` being written to. Unchanged by `Close()`.\n  Resizable* DestResizable() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.get();\n  }\n  Resizable& Digest() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    Flush();\n    return *DestResizable();\n  }\n\n protected:\n  void Initialize(Resizable* dest, bool append);\n\n  void MakeDestBuffer(size_t cursor_index) override;\n  bool ResizeDest() override;\n  bool GrowDestAndMakeBuffer(size_t new_size) override;\n  bool GrowDestUnderCapacityAndMakeBuffer(size_t min_length) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the `Resizable` being written\n  // to, with uninitialized space appended (possibly empty); `cursor()` points\n  // to the uninitialized space.\n  MovingDependency<Resizable*, Dest, Mover> dest_;\n};\n\n// `ResizableTraits` for `std::string` with an arbitrary allocator, i.e.\n// `std::basic_string<char, std::char_traits<char>, Alloc>`.\ntemplate <typename Alloc = std::allocator<char>>\nstruct StringResizableTraits {\n  using Resizable = std::basic_string<char, std::char_traits<char>, Alloc>;\n  static char* Data(Resizable& dest) { return dest.data(); }\n  static size_t Size(const Resizable& dest) { return dest.size(); }\n  static constexpr bool kIsStable = false;\n  static bool Resize(Resizable& dest, size_t new_size, size_t used_size) {\n    RIEGELI_ASSERT_LE(used_size, dest.size())\n        << \"Failed precondition of ResizableTraits::Resize(): \"\n           \"used size exceeds old size\";\n    RIEGELI_ASSERT_LE(used_size, new_size)\n        << \"Failed precondition of ResizableTraits::Resize(): \"\n           \"used size exceeds new size\";\n    Reserve(dest, new_size, used_size);\n    dest.resize(new_size);\n    return true;\n  }\n  static bool Grow(Resizable& dest, size_t new_size, size_t used_size) {\n    RIEGELI_ASSERT_GT(new_size, dest.size())\n        << \"Failed precondition of ResizableTraits::Grow(): \"\n           \"no need to grow\";\n    RIEGELI_ASSERT_LE(used_size, dest.size())\n        << \"Failed precondition of ResizableTraits::Grow(): \"\n           \"used size exceeds old size\";\n    RIEGELI_ASSERT_LE(used_size, new_size)\n        << \"Failed precondition of ResizableTraits::Grow(): \"\n           \"used size exceeds new size\";\n    Reserve(dest, new_size, used_size);\n    if (!GrowUnderCapacity(dest, new_size)) RIEGELI_ASSUME_UNREACHABLE();\n    return true;\n  }\n  static bool GrowUnderCapacity(Resizable& dest, size_t new_size) {\n    RIEGELI_ASSERT_GT(new_size, dest.size())\n        << \"Failed precondition of ResizableTraits::GrowUnderCapacity(): \"\n           \"no need to grow\";\n    if (new_size > dest.capacity()) return false;\n    new_size = UnsignedClamp(\n        dest.size() + UnsignedClamp(dest.size(), kDefaultMinBlockSize,\n                                    kDefaultMaxBlockSize),\n        new_size, dest.capacity());\n    dest.resize(new_size);\n    return true;\n  }\n\n private:\n  static void Reserve(Resizable& dest, size_t new_capacity, size_t used_size) {\n    if (new_capacity > dest.capacity()) {\n      dest.erase(used_size);\n      // Use `std::string().capacity()` instead of `Resizable().capacity()`\n      // because `Resizable` is not necessarily default-constructible. They are\n      // normally the same, and even if they are not, this is a matter of\n      // performance tuning, not correctness.\n      dest.reserve(dest.capacity() <= std::string().capacity()\n                       ? new_capacity\n                       : UnsignedClamp(dest.capacity() + dest.capacity() / 2,\n                                       new_capacity, dest.max_size()));\n    }\n    RIEGELI_ASSUME_GE(dest.capacity(), new_capacity);\n  }\n};\n\n// Implementation details follow.\n\ninline ResizableWriterBase::ResizableWriterBase(BufferOptions buffer_options)\n    : options_(Chain::Options()\n                   .set_min_block_size(buffer_options.min_buffer_size())\n                   .set_max_block_size(buffer_options.max_buffer_size())) {}\n\ninline ResizableWriterBase::ResizableWriterBase(\n    ResizableWriterBase&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)),\n      options_(that.options_),\n      written_size_(that.written_size_),\n      associated_reader_(std::move(that.associated_reader_)) {\n  MoveSecondaryBuffer(that);\n}\n\ninline ResizableWriterBase& ResizableWriterBase::operator=(\n    ResizableWriterBase&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  options_ = that.options_;\n  written_size_ = that.written_size_;\n  associated_reader_ = std::move(that.associated_reader_);\n  MoveSecondaryBuffer(that);\n  return *this;\n}\n\ninline void ResizableWriterBase::Reset(Closed) {\n  Writer::Reset(kClosed);\n  options_ = Chain::Options();\n  secondary_buffer_ = Chain();\n  written_size_ = 0;\n  associated_reader_.Reset();\n}\n\ninline void ResizableWriterBase::Reset(BufferOptions buffer_options) {\n  Writer::Reset();\n  options_ = Chain::Options()\n                 .set_min_block_size(buffer_options.min_buffer_size())\n                 .set_max_block_size(buffer_options.max_buffer_size());\n  secondary_buffer_.Clear();\n  written_size_ = 0;\n  associated_reader_.Reset();\n}\n\ninline void ResizableWriterBase::MoveSecondaryBuffer(\n    ResizableWriterBase& that) {\n  // Buffer pointers are already moved so `start()` is taken from `*this`.\n  // `secondary_buffer_` is not moved yet so `uses_secondary_buffer()` is called\n  // on `that`.\n  const bool uses_buffer = start() != nullptr && that.uses_secondary_buffer();\n  const size_t saved_start_to_limit = start_to_limit();\n  const size_t saved_start_to_cursor = start_to_cursor();\n  if (uses_buffer) {\n    RIEGELI_ASSERT(that.secondary_buffer_.blocks().back().data() +\n                       that.secondary_buffer_.blocks().back().size() ==\n                   limit())\n        << \"Failed invariant of ResizableWriter: \"\n           \"secondary buffer inconsistent with buffer pointers\";\n  }\n  secondary_buffer_ = std::move(that.secondary_buffer_);\n  if (uses_buffer) {\n    const absl::string_view last_block = secondary_buffer_.blocks().back();\n    set_buffer(const_cast<char*>(last_block.data() + last_block.size()) -\n                   saved_start_to_limit,\n               saved_start_to_limit, saved_start_to_cursor);\n  }\n}\n\ninline size_t ResizableWriterBase::used_size() const {\n  return UnsignedMax(IntCast<size_t>(pos()), written_size_);\n}\n\ninline size_t ResizableWriterBase::used_dest_size() const {\n  if (uses_secondary_buffer()) {\n    RIEGELI_ASSERT_EQ(available(), 0u)\n        << \"Failed precondition of ResizableWriterBase::used_dest_size(): \"\n        << \"secondary buffer has free space\";\n  }\n  RIEGELI_ASSERT_GE(used_size(), secondary_buffer_.size())\n      << \"Failed invariant of ResizableWriterBase: \"\n         \"negative destination size\";\n  return used_size() - secondary_buffer_.size();\n}\n\ntemplate <typename ResizableTraits, typename Dest>\nclass ResizableWriter<ResizableTraits, Dest>::Mover {\n public:\n  static auto member() { return &ResizableWriter::dest_; }\n\n  explicit Mover(ResizableWriter& self, ResizableWriter& that)\n      : uses_buffer_(!ResizableTraits::kIsStable && self.start() != nullptr &&\n                     !self.uses_secondary_buffer()),\n        start_to_cursor_(self.start_to_cursor()) {\n    if (uses_buffer_) {\n      RIEGELI_ASSERT_EQ(ResizableTraits::Data(*that.dest_), self.start())\n          << \"ResizableWriter destination changed unexpectedly\";\n      RIEGELI_ASSERT_EQ(ResizableTraits::Size(*that.dest_),\n                        self.start_to_limit())\n          << \"ResizableWriter destination changed unexpectedly\";\n    }\n  }\n\n  void Done(ResizableWriter& self) {\n    if (uses_buffer_) {\n      Resizable& dest = *self.dest_;\n      self.set_buffer(ResizableTraits::Data(dest), ResizableTraits::Size(dest),\n                      start_to_cursor_);\n    }\n  }\n\n private:\n  bool uses_buffer_;\n  size_t start_to_cursor_;\n};\n\ntemplate <typename ResizableTraits, typename Dest>\ninline ResizableWriter<ResizableTraits, Dest>::ResizableWriter(\n    Initializer<Dest> dest, Options options)\n    : ResizableWriterBase(options.buffer_options()), dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.append());\n}\n\ntemplate <typename ResizableTraits, typename Dest>\ntemplate <\n    typename DependentDest,\n    std::enable_if_t<\n        std::conjunction_v<\n            std::is_same<DependentDest, typename ResizableTraits::Resizable>,\n            std::is_default_constructible<typename ResizableTraits::Resizable>>,\n        int>>\ninline ResizableWriter<ResizableTraits, Dest>::ResizableWriter(Options options)\n    : ResizableWriter(riegeli::Maker(), std::move(options)) {}\n\ntemplate <typename ResizableTraits, typename Dest>\ninline void ResizableWriter<ResizableTraits, Dest>::Reset(Closed) {\n  ResizableWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename ResizableTraits, typename Dest>\ninline void ResizableWriter<ResizableTraits, Dest>::Reset(\n    Initializer<Dest> dest, Options options) {\n  ResizableWriterBase::Reset(options.buffer_options());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.append());\n}\n\ntemplate <typename ResizableTraits, typename Dest>\ntemplate <\n    typename DependentDest,\n    std::enable_if_t<\n        std::conjunction_v<\n            std::is_same<DependentDest, typename ResizableTraits::Resizable>,\n            std::is_default_constructible<typename ResizableTraits::Resizable>>,\n        int>>\ninline void ResizableWriter<ResizableTraits, Dest>::Reset(Options options) {\n  Reset(riegeli::Maker(), std::move(options));\n}\n\ntemplate <typename ResizableTraits, typename Dest>\ninline void ResizableWriter<ResizableTraits, Dest>::Initialize(Resizable* dest,\n                                                               bool append) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of ResizableWriter: null Resizable pointer\";\n  if (append) {\n    set_start_pos(ResizableTraits::Size(*dest));\n  } else {\n    set_buffer(ResizableTraits::Data(*dest), ResizableTraits::Size(*dest));\n  }\n}\n\ntemplate <typename ResizableTraits, typename Dest>\nvoid ResizableWriter<ResizableTraits, Dest>::MakeDestBuffer(\n    size_t cursor_index) {\n  RIEGELI_ASSERT(!uses_secondary_buffer())\n      << \"Failed precondition in ResizableWriter::MakeDestBuffer(): \"\n         \"secondary buffer is used\";\n  Resizable& dest = *dest_;\n  set_buffer(ResizableTraits::Data(dest), ResizableTraits::Size(dest),\n             cursor_index);\n  set_start_pos(0);\n}\n\ntemplate <typename ResizableTraits, typename Dest>\nbool ResizableWriter<ResizableTraits, Dest>::ResizeDest() {\n  if (uses_secondary_buffer()) {\n    RIEGELI_ASSERT_EQ(available(), 0u)\n        << \"Failed precondition of ResizableWriter::ResizeDest(): \"\n        << \"secondary buffer has free space\";\n  }\n  const size_t new_size = used_size();\n  const size_t cursor_index = IntCast<size_t>(pos());\n  if (ABSL_PREDICT_FALSE(\n          !ResizableTraits::Resize(*dest_, new_size, used_dest_size()))) {\n    return FailOverflow();\n  }\n  Resizable& dest = *dest_;\n  RIEGELI_ASSERT_GE(ResizableTraits::Size(dest), new_size)\n      << \"Failed postcondition of ResizableTraits::Resize(): \"\n         \"not resized to at least requested size\";\n  set_buffer(ResizableTraits::Data(dest), ResizableTraits::Size(dest),\n             cursor_index);\n  set_start_pos(0);\n  return true;\n}\n\ntemplate <typename ResizableTraits, typename Dest>\nbool ResizableWriter<ResizableTraits, Dest>::GrowDestAndMakeBuffer(\n    size_t new_size) {\n  if (uses_secondary_buffer()) {\n    RIEGELI_ASSERT_EQ(available(), 0u)\n        << \"Failed precondition of ResizableWriter::GrowDestAndMakeBuffer(): \"\n        << \"secondary buffer has free space\";\n  }\n  const size_t cursor_index = IntCast<size_t>(pos());\n  if (ABSL_PREDICT_TRUE(new_size > ResizableTraits::Size(*dest_))) {\n    if (ABSL_PREDICT_FALSE(\n            !ResizableTraits::Grow(*dest_, new_size, used_dest_size()))) {\n      return FailOverflow();\n    }\n  }\n  Resizable& dest = *dest_;\n  RIEGELI_ASSERT_GE(ResizableTraits::Size(dest), new_size)\n      << \"Failed postcondition of ResizableTraits::Grow(): \"\n         \"not resized to at least requested size\";\n  set_buffer(ResizableTraits::Data(dest), ResizableTraits::Size(dest),\n             cursor_index);\n  set_start_pos(0);\n  return true;\n}\n\ntemplate <typename ResizableTraits, typename Dest>\nbool ResizableWriter<ResizableTraits, Dest>::GrowDestUnderCapacityAndMakeBuffer(\n    size_t min_length) {\n  RIEGELI_ASSERT(!uses_secondary_buffer())\n      << \"Failed precondition in \"\n         \"ResizableWriter::GrowDestUnderCapacityAndMakeBuffer(): \"\n         \"secondary buffer is used\";\n  RIEGELI_ASSERT_LE(min_length,\n                    std::numeric_limits<size_t>::max() - IntCast<size_t>(pos()))\n      << \"Failed precondition of \"\n         \"ResizableWriter::GrowDestUnderCapacityAndMakeBuffer(): \"\n         \"Writer position overflow\";\n  const size_t cursor_index = IntCast<size_t>(pos());\n  const size_t new_size = cursor_index + min_length;\n  if (ABSL_PREDICT_TRUE(new_size > ResizableTraits::Size(*dest_))) {\n    if (!ResizableTraits::GrowUnderCapacity(*dest_, new_size)) return false;\n  }\n  Resizable& dest = *dest_;\n  RIEGELI_ASSERT_GE(ResizableTraits::Size(dest), new_size)\n      << \"Failed postcondition of ResizableTraits::GrowUnderCapacity(): \"\n         \"not resized to at least requested size\";\n  set_buffer(ResizableTraits::Data(dest), ResizableTraits::Size(dest),\n             cursor_index);\n  set_start_pos(0);\n  return true;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_RESIZABLE_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/restricted_chain_writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/restricted_chain_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid RestrictedChainWriter::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    RIEGELI_ASSERT_EQ(limit_pos(), dest_.size())\n        << \"RestrictedChainWriter destination changed unexpectedly\";\n    SyncBuffer();\n  }\n  Writer::Done();\n}\n\ninline void RestrictedChainWriter::SyncBuffer() {\n  set_start_pos(pos());\n  dest_.RemoveSuffix(available());\n  set_buffer();\n}\n\ninline void RestrictedChainWriter::MakeBuffer(size_t min_length,\n                                              size_t recommended_length) {\n  const absl::Span<char> buffer =\n      dest_.AppendBuffer(min_length, recommended_length, Chain::kAnyLength);\n  set_buffer(buffer.data(), buffer.size());\n}\n\nbool RestrictedChainWriter::PushSlow(size_t min_length,\n                                     size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  RIEGELI_ASSERT_EQ(limit_pos(), dest_.size())\n      << \"RestrictedChainWriter destination changed unexpectedly\";\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(min_length > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  MakeBuffer(min_length, recommended_length);\n  return true;\n}\n\nbool RestrictedChainWriter::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  RIEGELI_ASSERT_EQ(limit_pos(), dest_.size())\n      << \"RestrictedChainWriter destination changed unexpectedly\";\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  dest_.Append(std::move(src));\n  MakeBuffer();\n  return true;\n}\n\nbool RestrictedChainWriter::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  RIEGELI_ASSERT_EQ(limit_pos(), dest_.size())\n      << \"RestrictedChainWriter destination changed unexpectedly\";\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  dest_.Append(src);\n  MakeBuffer();\n  return true;\n}\n\nbool RestrictedChainWriter::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  RIEGELI_ASSERT_EQ(limit_pos(), dest_.size())\n      << \"RestrictedChainWriter destination changed unexpectedly\";\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  dest_.Append(std::move(src));\n  MakeBuffer();\n  return true;\n}\n\nbool RestrictedChainWriter::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  RIEGELI_ASSERT_EQ(limit_pos(), dest_.size())\n      << \"RestrictedChainWriter destination changed unexpectedly\";\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  dest_.Append(src);\n  MakeBuffer();\n  return true;\n}\n\nbool RestrictedChainWriter::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  RIEGELI_ASSERT_EQ(limit_pos(), dest_.size())\n      << \"RestrictedChainWriter destination changed unexpectedly\";\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  dest_.Append(std::move(src));\n  MakeBuffer();\n  return true;\n}\n\nbool RestrictedChainWriter::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  RIEGELI_ASSERT_EQ(limit_pos(), dest_.size())\n      << \"RestrictedChainWriter destination changed unexpectedly\";\n  SyncBuffer();\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  src.AppendTo(dest_);\n  MakeBuffer();\n  return true;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/restricted_chain_writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_RESTRICTED_CHAIN_WRITER_H_\n#define RIEGELI_BYTES_RESTRICTED_CHAIN_WRITER_H_\n\n#include <stddef.h>\n\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// A restricted version of `ChainWriter` with minimal dependencies.\n//\n// In comparison to `ChainWriter`, `RestrictedChainWriter` always owns the\n// destination `Chain` by value, and does not support appending, tuning block\n// sizes, effective `SetWriteSizeHint()`, effective `Flush()`, `Seek()`,\n// `Size()`, `Truncate()`, nor `ReadMode()`.\n//\n// It is intended to be used together with `WriterStringifySink` which needs\n// only writing.\nclass RestrictedChainWriter : public Writer {\n public:\n  // Creates a closed `RestrictedChainWriter`.\n  explicit RestrictedChainWriter(Closed) noexcept : Writer(kClosed) {}\n\n  // Will append to an owned `Chain` which can be accessed by `dest()`.\n  RestrictedChainWriter() = default;\n\n  RestrictedChainWriter(RestrictedChainWriter&& that) noexcept;\n  RestrictedChainWriter& operator=(RestrictedChainWriter&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `RestrictedChainWriter`.\n  // This avoids constructing a temporary `RestrictedChainWriter` and moving\n  // from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n\n  // Returns the `Chain` being written to. Unchanged by `Close()`.\n  Chain& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_; }\n  const Chain& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_; }\n\n protected:\n  void Done() override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n\n private:\n  // Discards uninitialized space from the end of `dest_`, so that it contains\n  // only actual data written.\n  void SyncBuffer();\n\n  // Appends uninitialized space to `dest_`.\n  void MakeBuffer(size_t min_length = 0, size_t recommended_length = 0);\n\n  // Moves `that.dest_` to `dest_`. Buffer pointers are already moved from\n  // `dest_` to `*this`; adjust them to match `dest_`.\n  void MoveDest(RestrictedChainWriter&& that);\n\n  // Invariants if `ok()`:\n  //   `limit() == nullptr || limit() == dest_.blocks().back().data() +\n  //                                     dest_.blocks().back().size()`\n  //   `limit_pos() == dest_.size()`\n  Chain dest_;\n};\n\n// Implementation details follow.\n\ninline RestrictedChainWriter::RestrictedChainWriter(\n    RestrictedChainWriter&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)) {\n  MoveDest(std::move(that));\n}\n\ninline RestrictedChainWriter& RestrictedChainWriter::operator=(\n    RestrictedChainWriter&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  MoveDest(std::move(that));\n  return *this;\n}\n\ninline void RestrictedChainWriter::Reset(Closed) {\n  Writer::Reset(kClosed);\n  dest_ = Chain();\n}\n\ninline void RestrictedChainWriter::Reset() {\n  Writer::Reset();\n  dest_.Clear();\n}\n\ninline void RestrictedChainWriter::MoveDest(RestrictedChainWriter&& that) {\n  const bool uses_buffer = start() != nullptr;\n  if (uses_buffer) {\n    RIEGELI_ASSERT(that.dest_.blocks().back().data() +\n                       that.dest_.blocks().back().size() ==\n                   limit())\n        << \"RestrictedChainWriter destination changed unexpectedly\";\n    RIEGELI_ASSERT_EQ(that.dest_.size(), limit_pos())\n        << \"RestrictedChainWriter destination changed unexpectedly\";\n  }\n  const size_t saved_start_to_cursor = start_to_cursor();\n  dest_ = std::move(that.dest_);\n  if (uses_buffer) {\n    const size_t buffer_size = dest_.size() - IntCast<size_t>(start_pos());\n    const absl::string_view last_block = dest_.blocks().back();\n    set_buffer(\n        const_cast<char*>(last_block.data() + last_block.size()) - buffer_size,\n        buffer_size, saved_start_to_cursor);\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_RESTRICTED_CHAIN_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/splitting_writer.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/splitting_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/cord_reader.h\"\n#include \"riegeli/bytes/pushable_writer.h\"\n#include \"riegeli/bytes/string_reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid SplittingWriterBase::DoneBehindScratch() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    Writer* shard = ShardWriter();\n    if (shard_is_open(shard)) {\n      SyncBuffer(*shard);\n      CloseShardInternal();\n    }\n  }\n}\n\nbool SplittingWriterBase::CloseShardImpl() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of SplittingWriterBase::CloseShardImpl()\";\n  RIEGELI_ASSERT(shard_is_open())\n      << \"Failed precondition of SplittingWriterBase::CloseShardImpl(): \"\n         \"shard already closed\";\n  Writer* shard = ShardWriter();\n  if (ABSL_PREDICT_FALSE(!shard->Close())) {\n    return FailWithoutAnnotation(AnnotateOverShard(shard->status()));\n  }\n  return true;\n}\n\ninline bool SplittingWriterBase::OpenShardInternal() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of SplittingWriterBase::OpenShardInternal()\";\n  RIEGELI_ASSERT(!shard_is_open())\n      << \"Failed precondition of SplittingWriterBase::OpenShardInternal(): \"\n         \"shard already opened\";\n  if (ABSL_PREDICT_FALSE(start_pos() == std::numeric_limits<Position>::max())) {\n    return FailOverflow();\n  }\n  const std::optional<Position> size_limit = OpenShardImpl();\n  if (ABSL_PREDICT_FALSE(size_limit == std::nullopt)) {\n    RIEGELI_ASSERT(!ok())\n        << \"Failed postcondition of SplittingWriterBase::OpenShardImpl(): \"\n           \"zero returned but SplittingWriterBase OK\";\n    return false;\n  }\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed postcondition of SplittingWriterBase::OpenShardImpl()\";\n  RIEGELI_ASSERT(shard_is_open())\n      << \"Failed postcondition of SplittingWriterBase::OpenShardImpl(): \"\n         \"shard not opened\";\n  shard_pos_limit_ = SaturatingAdd(start_pos(), *size_limit);\n  Writer* shard = ShardWriter();\n  Position limit_hint = shard_pos_limit_;\n  if (size_hint_ != std::nullopt) {\n    limit_hint = UnsignedMin(limit_hint, *size_hint_);\n  }\n  shard->SetWriteSizeHint(SaturatingSub(limit_hint, start_pos()));\n  return true;\n}\n\ninline bool SplittingWriterBase::CloseShardInternal() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of SplittingWriterBase::CloseShardInternal()\";\n  RIEGELI_ASSERT(shard_is_open())\n      << \"Failed precondition of SplittingWriterBase::CloseShardInternal(): \"\n         \"shard already closed\";\n  const bool close_shard_ok = CloseShardImpl();\n  RIEGELI_ASSERT(!shard_is_open())\n      << \"Failed postcondition of SplittingWriterBase::CloseShardImpl(): \"\n         \"shard not closed\";\n  if (ABSL_PREDICT_FALSE(!close_shard_ok)) {\n    RIEGELI_ASSERT(!ok())\n        << \"Failed postcondition of SplittingWriterBase::CloseShardImpl(): \"\n           \"SplittingWriterBase OK\";\n    return false;\n  }\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed postcondition of SplittingWriterBase::CloseShardImpl()\";\n  return true;\n}\n\nbool SplittingWriterBase::OpenShard() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of SplittingWriterBase::OpenShard()\";\n  RIEGELI_ASSERT(!shard_is_open())\n      << \"Failed precondition of SplittingWriterBase::OpenShard(): \"\n         \"shard already opened\";\n  if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n  Writer* shard = ShardWriter();\n  MakeBuffer(*shard);\n  return true;\n}\n\nbool SplittingWriterBase::CloseShard() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of SplittingWriterBase::CloseShard()\";\n  RIEGELI_ASSERT(shard_is_open())\n      << \"Failed precondition of SplittingWriterBase::CloseShard(): \"\n         \"shard already closed\";\n  Writer* shard = ShardWriter();\n  SyncBuffer(*shard);\n  return CloseShardInternal();\n}\n\nabsl::Status SplittingWriterBase::AnnotateStatusImpl(absl::Status status) {\n  Writer* shard = ShardWriter();\n  if (shard_is_open(shard)) status = shard->AnnotateStatus(std::move(status));\n  // The status might have been annotated by `*shard` with the position within\n  // the shard. Clarify that the current position is the position across shards\n  // instead of delegating to `PushableWriter::AnnotateStatusImpl()`.\n  return AnnotateOverShard(std::move(status));\n}\n\nabsl::Status SplittingWriterBase::AnnotateOverShard(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"across shards at byte \", pos()));\n  }\n  return status;\n}\n\nvoid SplittingWriterBase::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  if (write_size_hint == std::nullopt) {\n    size_hint_ = std::nullopt;\n  } else {\n    size_hint_ = SaturatingAdd(pos(), *write_size_hint);\n  }\n  Writer* shard = ShardWriter();\n  if (!shard_is_open(shard)) return;\n  BehindScratch behind_scratch(this);\n  SyncBuffer(*shard);\n  Position limit_hint = shard_pos_limit_;\n  if (size_hint_ != std::nullopt) {\n    limit_hint = UnsignedMin(limit_hint, *size_hint_);\n  }\n  shard->SetWriteSizeHint(SaturatingSub(limit_hint, start_pos()));\n  MakeBuffer(*shard);\n}\n\nbool SplittingWriterBase::PushBehindScratch(size_t recommended_length) {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PushableWriter::PushBehindScratch(): \"\n         \"some space available, use Push() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::PushBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  RIEGELI_ASSERT_LE(pos(), shard_pos_limit_)\n      << \"Failed invariant of SplittingWriter: \"\n         \"current position exceeds the shard limit\";\n  Writer* shard = ShardWriter();\n  if (!shard_is_open(shard)) {\n    if (!AllowEmptyShards()) return ForcePushUsingScratch();\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardWriter();\n  } else {\n    SyncBuffer(*shard);\n    while (start_pos() == shard_pos_limit_) {\n      if (ABSL_PREDICT_FALSE(!CloseShardInternal())) return false;\n      if (!AllowEmptyShards()) return ForcePushUsingScratch();\n      if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n      shard = ShardWriter();\n    }\n  }\n  const bool push_ok = shard->Push(1, recommended_length);\n  MakeBuffer(*shard);\n  return push_ok;\n}\n\nbool SplittingWriterBase::WriteBehindScratch(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of \"\n         \"PushableWriter::WriteBehindScratch(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableWriter::WriteBehindScratch(string_view): \"\n         \"scratch used\";\n  return WriteInternal<StringReader<const absl::string_view*>>(src);\n}\n\nbool SplittingWriterBase::WriteBehindScratch(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Chain): \"\n         \"scratch used\";\n  return WriteInternal<ChainReader<>>(src);\n}\n\nbool SplittingWriterBase::WriteBehindScratch(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Chain&&): \"\n         \"scratch used\";\n  return WriteInternal<ChainReader<>>(std::move(src));\n}\n\nbool SplittingWriterBase::WriteBehindScratch(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Cord): \"\n         \"scratch used\";\n  return WriteInternal<CordReader<>>(src);\n}\n\nbool SplittingWriterBase::WriteBehindScratch(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::WriteBehindScratch(Cord&&): \"\n         \"scratch used\";\n  return WriteInternal<CordReader<>>(std::move(src));\n}\n\ntemplate <typename SrcReader, typename Src>\ninline bool SplittingWriterBase::WriteInternal(Src&& src) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  RIEGELI_ASSERT_LE(pos(), shard_pos_limit_)\n      << \"Failed invariant of SplittingWriter: \"\n         \"current position exceeds the shard limit\";\n  Writer* shard = ShardWriter();\n  if (shard_is_open(shard)) {\n    SyncBuffer(*shard);\n  } else {\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardWriter();\n  }\n  Position length_to_write = shard_pos_limit_ - start_pos();\n  bool write_ok;\n  size_t length = src.size();\n  if (ABSL_PREDICT_TRUE(length <= length_to_write)) {\n    if (ABSL_PREDICT_FALSE(!shard->Write(std::forward<Src>(src)))) {\n      write_ok = false;\n    } else {\n      move_start_pos(length);\n      write_ok = true;\n    }\n  } else {\n    SrcReader reader(&src);\n    for (;;) {\n      if (ABSL_PREDICT_FALSE(!reader.Copy(length_to_write, *shard))) {\n        RIEGELI_ASSERT(!shard->ok()) << \"Reading failed\";\n        write_ok = false;\n        break;\n      }\n      move_start_pos(length_to_write);\n      length -= length_to_write;\n      if (length == 0) {\n        write_ok = true;\n        break;\n      }\n      if (ABSL_PREDICT_FALSE(!CloseShardInternal())) return false;\n      if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n      shard = ShardWriter();\n      length_to_write = UnsignedMin(length, shard_pos_limit_ - start_pos());\n    }\n  }\n  MakeBuffer(*shard);\n  return write_ok;\n}\n\nbool SplittingWriterBase::WriteBehindScratch(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of \"\n         \"PushableWriter::WriteBehindScratch(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of \"\n         \"PushableWriter::WriteBehindScratch(ByteFill): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  RIEGELI_ASSERT_LE(pos(), shard_pos_limit_)\n      << \"Failed invariant of SplittingWriter: \"\n         \"current position exceeds the shard limit\";\n  Writer* shard = ShardWriter();\n  if (shard_is_open(shard)) {\n    SyncBuffer(*shard);\n  } else {\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardWriter();\n  }\n  bool write_ok;\n  for (;;) {\n    const Position length_to_write =\n        UnsignedMin(src.size(), shard_pos_limit_ - start_pos());\n    if (ABSL_PREDICT_FALSE(!shard->Write(src.Extract(length_to_write)))) {\n      write_ok = false;\n      break;\n    }\n    move_start_pos(length_to_write);\n    if (src.empty()) {\n      write_ok = true;\n      break;\n    }\n    if (ABSL_PREDICT_FALSE(!CloseShardInternal())) return false;\n    if (ABSL_PREDICT_FALSE(!OpenShardInternal())) return false;\n    shard = ShardWriter();\n  }\n  MakeBuffer(*shard);\n  return write_ok;\n}\n\nbool SplittingWriterBase::FlushBehindScratch(FlushType flush_type) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::FlushBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  RIEGELI_ASSERT_LE(pos(), shard_pos_limit_)\n      << \"Failed invariant of SplittingWriter: \"\n         \"current position exceeds the shard limit\";\n  Writer* const shard = ShardWriter();\n  if (shard_is_open(shard)) {\n    SyncBuffer(*shard);\n    if (flush_type != FlushType::kFromObject) {\n      if (ABSL_PREDICT_FALSE(!shard->Flush(flush_type))) return false;\n    }\n    if (ABSL_PREDICT_FALSE(!CloseShardInternal())) return false;\n  }\n  return true;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/splitting_writer.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_SPLITTING_WRITER_H_\n#define RIEGELI_BYTES_SPLITTING_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pushable_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `SplittingWriter`.\nclass SplittingWriterBase : public PushableWriter {\n protected:\n  using PushableWriter::PushableWriter;\n\n  SplittingWriterBase(SplittingWriterBase&& that) noexcept;\n  SplittingWriterBase& operator=(SplittingWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n\n  void DoneBehindScratch() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n\n  // Returns the shard `Writer`.\n  virtual Writer* ShardWriter() ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n  virtual const Writer* ShardWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Opens the next shard as `shard()`. Or opens a temporary destination for\n  // shard data as `shard()`, to be moved to the final destination later.\n  //\n  // Preconditions:\n  //   `ok()`\n  //   `!shard_is_open()`\n  //\n  // Return values:\n  //  * size limit      - success (`ok()`, `shard_is_open()`)\n  //  * `std::nullopt` - failure (`!ok()`)\n  //\n  // When the size limit would be exceeded, the shard is closed and a new shard\n  // is opened.\n  //\n  // `OpenShardImpl()` must be overridden but should not be called directly\n  // because it does not synchronize buffer pointers of `*ShardWriter()` with\n  // `*this`. See `OpenShard()` for that.\n  virtual std::optional<Position> OpenShardImpl() = 0;\n\n  // Closes `shard()`. If `shard()` is a temporary destination for shard data,\n  // moves it to the final destination.\n  //\n  // Preconditions:\n  //   `ok()`\n  //   `shard_is_open()`\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`, `!shard_is_open()`)\n  //  * `false` - failure (`!ok()`, `!shard_is_open()`)\n  //\n  // The default implementation calls `shard_witer()->Close()` and propagates\n  // failures from that.\n  //\n  // `CloseShardImpl()` can be overridden but should not be called directly\n  // because it does not synchronize buffer pointers of `*this` with\n  // `*ShardWriter()`. See `CloseShard()` for that.\n  virtual bool CloseShardImpl();\n\n  // Calls `OpenShardImpl()` and synchronizes buffer pointers of\n  // `*ShardWriter()` with `*this`.\n  //\n  // Preconditions:\n  //   `ok()`\n  //   `!shard_is_open()`\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`, `shard_is_open()`)\n  //  * `false` - failure (`!ok()`)\n  bool OpenShard();\n\n  // Synchronizes buffer pointers of `*this` with `*ShardWriter()` and calls\n  // `CloseShardImpl()`.\n  //\n  // Preconditions:\n  //   `ok()`\n  //   `shard_is_open()`\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`, `!shard_is_open()`)\n  //  * `false` - failure (`!ok()`, `!shard_is_open()`)\n  bool CloseShard();\n\n  // Returns `true` if a shard is open.\n  //\n  // Same as `shard != nullptr && shard->is_open()`, with the default `shard` of\n  // `ShardWriter()`.\n  bool shard_is_open() const;\n  bool shard_is_open(const Writer* shard) const;\n\n  // Should return `true` if `OpenShardImpl()` and then `CloseShardImpl()` may\n  // be called with no data written to `shard()`. This makes `Push()` at a shard\n  // boundary more efficient, because the buffer will be created directly in the\n  // shard, but `OpenShardImpl()` and `CloseShardImpl()` must be able to deal\n  // with empty shards.\n  //\n  // Should return `false` if `OpenShardImpl()` may be called only if the shard\n  // will definitely have some data written before calling `CloseShardImpl()`.\n  // This makes `Push()` at a shard boundary less efficient, because the\n  // beginning of the data will be written to a scratch buffer and then copied\n  // to the shard, but `OpenShardImpl()` and `CloseShardImpl()` can assume that\n  // all shards are non-empty.\n  virtual bool AllowEmptyShards() { return false; }\n\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverShard(absl::Status status);\n\n  // Sets cursor of `shard` to cursor of `*this`. Sets buffer pointers of\n  // `*this` to `nullptr`.\n  void SyncBuffer(Writer& shard);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `shard`. Fails\n  // `*this` if `shard` failed.\n  void MakeBuffer(Writer& shard);\n\n  bool PushBehindScratch(size_t recommended_length) override;\n  bool WriteBehindScratch(absl::string_view src) override;\n  bool WriteBehindScratch(const Chain& src) override;\n  bool WriteBehindScratch(Chain&& src) override;\n  bool WriteBehindScratch(const absl::Cord& src) override;\n  bool WriteBehindScratch(absl::Cord&& src) override;\n  bool WriteBehindScratch(ByteFill src) override;\n\n  // Flushes the current shard if `flush_type != FlushType::kFromObject`.\n  // Then closes the current shard.\n  bool FlushBehindScratch(FlushType flush_type) override;\n\n private:\n  bool OpenShardInternal();\n  bool CloseShardInternal();\n\n  // This template is defined and used only in splitting_writer.cc.\n  template <typename SrcReader, typename Src>\n  bool WriteInternal(Src&& src);\n\n  std::optional<Position> size_hint_;\n\n  // The limit of `pos()` for data written to the current shard.\n  Position shard_pos_limit_ = 0;\n\n  // Invariants if `ok()` and scratch is not used:\n  //   `start() == (shard_is_open() ? ShardWriter()->cursor() : nullptr)`\n  //   `limit() <= (shard_is_open() ? ShardWriter()->limit() : nullptr)`\n  //   `pos() <= shard_pos_limit_`\n};\n\n// Abstract class of a `Writer` which splits data into multiple shards. When a\n// new shard is opened, the size limit of this shard is declared.\n//\n// The `Shard` template parameter specifies the type of the object providing and\n// possibly owning the shard `Writer`. `Shard` must support\n// `Dependency<Writer*, Shard>`, e.g. `Writer*` (not owned),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\ntemplate <typename Shard>\nclass SplittingWriter : public SplittingWriterBase {\n protected:\n  using SplittingWriterBase::SplittingWriterBase;\n\n  SplittingWriter(SplittingWriter&& that) = default;\n  SplittingWriter& operator=(SplittingWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `SplittingWriter`. This\n  // avoids constructing a temporary `SplittingWriter` and moving from it.\n  // Derived classes which override `Reset()` should include a call to\n  // `SplittingWriter::Reset()`.\n  void Reset(Closed);\n  void Reset();\n\n  void Done() override;\n\n  // Returns the object providing and possibly owning the shard `Writer`.\n  Shard& shard() ABSL_ATTRIBUTE_LIFETIME_BOUND { return shard_.manager(); }\n  const Shard& shard() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return shard_.manager();\n  }\n  Writer* ShardWriter() ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return shard_.get();\n  }\n  const Writer* ShardWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return shard_.get();\n  }\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the shard `Writer`.\n  MovingDependency<Writer*, Shard, Mover> shard_;\n};\n\n// Implementation details follow.\n\ninline SplittingWriterBase::SplittingWriterBase(\n    SplittingWriterBase&& that) noexcept\n    : PushableWriter(static_cast<PushableWriter&&>(that)),\n      size_hint_(that.size_hint_),\n      shard_pos_limit_(that.shard_pos_limit_) {}\n\ninline SplittingWriterBase& SplittingWriterBase::operator=(\n    SplittingWriterBase&& that) noexcept {\n  PushableWriter::operator=(static_cast<PushableWriter&&>(that));\n  size_hint_ = that.size_hint_;\n  shard_pos_limit_ = that.shard_pos_limit_;\n  return *this;\n}\n\ninline void SplittingWriterBase::Reset(Closed) {\n  PushableWriter::Reset(kClosed);\n  size_hint_ = std::nullopt;\n  shard_pos_limit_ = 0;\n}\n\ninline void SplittingWriterBase::Reset() {\n  PushableWriter::Reset();\n  size_hint_ = std::nullopt;\n  shard_pos_limit_ = 0;\n}\n\ninline bool SplittingWriterBase::shard_is_open() const {\n  return shard_is_open(ShardWriter());\n}\n\ninline bool SplittingWriterBase::shard_is_open(const Writer* shard) const {\n  return shard != nullptr && shard->is_open();\n}\n\ninline void SplittingWriterBase::SyncBuffer(Writer& shard) {\n  RIEGELI_ASSERT(shard_is_open(&shard))\n      << \"Failed precondition of SplittingWriterBase::SyncBuffer(): \"\n         \"shard is closed\";\n  shard.set_cursor(cursor());\n  move_start_pos(start_to_cursor());\n  set_buffer();\n}\n\ninline void SplittingWriterBase::MakeBuffer(Writer& shard) {\n  RIEGELI_ASSERT(shard_is_open(&shard))\n      << \"Failed precondition of SplittingWriterBase::MakeBuffer(): \"\n         \"shard is closed\";\n  RIEGELI_ASSERT_LE(start_pos(), shard_pos_limit_)\n      << \"Failed invariant of SplittingWriter: \"\n         \"current position exceeds the shard limit\";\n  set_buffer(shard.cursor(),\n             UnsignedMin(shard.available(), shard_pos_limit_ - start_pos()));\n  if (ABSL_PREDICT_FALSE(!shard.ok())) {\n    FailWithoutAnnotation(AnnotateOverShard(shard.status()));\n  }\n}\n\ntemplate <typename Shard>\nclass SplittingWriter<Shard>::Mover {\n public:\n  static auto member() { return &SplittingWriter::shard_; }\n\n  explicit Mover(SplittingWriter& self, SplittingWriter& that)\n      : behind_scratch_(&self), uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `shard_` is not moved yet so `shard_` is taken from `that`.\n    if (uses_buffer_) self.SyncBuffer(*that.shard_);\n  }\n\n  void Done(SplittingWriter& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.shard_);\n  }\n\n private:\n  BehindScratch behind_scratch_;\n  bool uses_buffer_;\n};\n\ntemplate <typename Shard>\ninline void SplittingWriter<Shard>::Reset(Closed) {\n  SplittingWriterBase::Reset(kClosed);\n  shard_.Reset();\n}\n\ntemplate <typename Shard>\ninline void SplittingWriter<Shard>::Reset() {\n  SplittingWriterBase::Reset();\n  shard_.Reset();\n}\n\ntemplate <typename Shard>\nvoid SplittingWriter<Shard>::Done() {\n  SplittingWriterBase::Done();\n  shard_.Reset();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_SPLITTING_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/std_io.cc",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/std_io.h\"\n\n#include <utility>\n\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/global.h\"\n#include \"riegeli/base/sized_shared_buffer.h\"\n#include \"riegeli/bytes/fd_reader.h\"\n#include \"riegeli/bytes/fd_writer.h\"\n\nnamespace riegeli {\n\nnamespace {\n\nint std_in_fd = 0;\nint std_out_fd = 1;\nint std_err_fd = 2;\n\nSizedSharedBuffer& StdInPending() {\n  return Global([] { return SizedSharedBuffer(); });\n}\n\n}  // namespace\n\nStdIn::StdIn(Options options) : FdReader(std_in_fd, std::move(options)) {\n  SizedSharedBuffer& pending = StdInPending();\n  if (!pending.empty()) RestoreBuffer(std::move(pending));\n}\n\nvoid StdIn::Reset(Options options) {\n  FdReader::Reset(std_in_fd, std::move(options));\n  SizedSharedBuffer& pending = StdInPending();\n  if (!pending.empty()) RestoreBuffer(std::move(pending));\n}\n\nvoid StdIn::Done() {\n  RIEGELI_ASSERT(StdInPending().empty())\n      << \"Multiple instances of StdIn in use at a time\";\n  if (available() > 0 && !SupportsRandomAccess()) StdInPending() = SaveBuffer();\n  FdReader::Done();\n}\n\nStdOut::StdOut(Options options) : FdWriter(std_out_fd, std::move(options)) {}\n\nvoid StdOut::Reset(Options options) {\n  FdWriter::Reset(std_out_fd, std::move(options));\n}\n\nStdErr::StdErr(Options options) : FdWriter(std_err_fd, std::move(options)) {}\n\nvoid StdErr::Reset(Options options) {\n  FdWriter::Reset(std_err_fd, std::move(options));\n}\n\nInjectedStdInFd::InjectedStdInFd(int fd)\n    : old_fd_(std::exchange(std_in_fd, fd)),\n      old_pending_(std::move(StdInPending())) {}\n\nInjectedStdInFd::~InjectedStdInFd() {\n  std_in_fd = old_fd_;\n  StdInPending() = std::move(old_pending_);\n}\n\nInjectedStdOutFd::InjectedStdOutFd(int fd)\n    : old_fd_(std::exchange(std_out_fd, fd)) {}\n\nInjectedStdOutFd::~InjectedStdOutFd() { std_out_fd = old_fd_; }\n\nInjectedStdErrFd::InjectedStdErrFd(int fd)\n    : old_fd_(std::exchange(std_err_fd, fd)) {}\n\nInjectedStdErrFd::~InjectedStdErrFd() { std_err_fd = old_fd_; }\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/std_io.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_STD_IO_H_\n#define RIEGELI_BYTES_STD_IO_H_\n\n#include \"absl/base/attributes.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/sized_shared_buffer.h\"\n#include \"riegeli/bytes/fd_handle.h\"\n#include \"riegeli/bytes/fd_reader.h\"\n#include \"riegeli/bytes/fd_writer.h\"\n\nnamespace riegeli {\n\n// A new `Reader` reading from standard input (by default from the same source\n// as fd 0, `std::cin`, and `stdin`).\n//\n// A `StdIn` must be explicitly closed or synced in order for its position to be\n// synchronized to the actual standard input. Closing a `StdIn` does not close\n// its file descriptor.\n//\n// Warning: synchronizing the position is feasible only if standard input\n// supports random access, otherwise standard input will have an unpredictable\n// amount of extra data consumed because of buffering. Nevertheless, closing\n// a `StdIn` and then creating another in the same process preserves these\n// pending data.\n//\n// At most one `StdIn` should be open at a time, and it should not be combined\n// with accessing standard input by other means.\nclass StdIn : public FdReader<UnownedFd> {\n public:\n  // Creates a closed `StdIn`.\n  explicit StdIn(Closed) noexcept : FdReader(kClosed) {}\n\n  // Will read from standard input.\n  explicit StdIn(Options options = Options());\n\n  StdIn(StdIn&& that) = default;\n  StdIn& operator=(StdIn&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `StdIn`. This avoids\n  // constructing a temporary `StdIn` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Options options = Options());\n\n protected:\n  void Done() override;\n};\n\n// A new `Writer` writing to standard output (by default to the same destination\n// as fd 1, `std::cout`, and `stdout`).\n//\n// In contrast to `std::cout` and `stdout`, `StdOut` is fully buffered (not line\n// buffered) even if it refers to an interactive device.\n//\n// A `StdOut` must be explicitly closed or flushed, and `Close()` or `Flush()`\n// must succeed, in order for its output to be guaranteed to be available in the\n// actual standard output. Closing a `StdOut` does not close its file\n// descriptor. Flushing a `StdOut` explicitly might be needed:\n//  * Before reading from standard input, so that output written so far appears\n//    before waiting for input.\n//  * Before writing to standard error, so that output written to different\n//    streams ultimately leading to the same destination appears in the correct\n//    order.\n//\n// At most one `StdOut` should be open at a time, and it should not be combined\n// with accessing standard output by other means at the same time. Switching\n// between means requires closing the old `StdOut` or flushing the object\n// becoming inactive (`std::cout.flush()`, `std::fflush(stdout)`) and may\n// require repositioning the object becoming active (`std::cout.seekp()`,\n// `std::fseek(stdout)`).\n//\n// As an alternative to `StdOut`, creating and later closing an\n// `OStreamWriter(&std::cout)` makes it easier to combine writing to a `Writer`\n// with accessing `std::cout`.\nclass StdOut : public FdWriter<UnownedFd> {\n public:\n  // Creates a closed `StdOut`.\n  explicit StdOut(Closed) noexcept : FdWriter(kClosed) {}\n\n  // Will write to standard output.\n  explicit StdOut(Options options = Options());\n\n  StdOut(StdOut&& that) = default;\n  StdOut& operator=(StdOut&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `StdOut`. This avoids\n  // constructing a temporary `StdOut` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Options options = Options());\n};\n\n// A new `Writer` writing to standard error (by default to the same destination\n// as fd 2, `std::cerr`, `std::clog`, and `stderr`).\n//\n// In contrast to `std::cerr` and `stderr`, `StdErr` is fully buffered (not\n// unbuffered).\n//\n// A `StdErr` must be explicitly closed or flushed, and `Close()` or `Flush()`\n// must succeed, in order for its output to be guaranteed to be available in the\n// actual standard error. Closing a `StdErr` does not close its file descriptor.\n// Flushing a `StdErr` explicitly might be needed after writing a complete\n// message, so that it appears promptly.\n//\n// At most one `StdErr` should be open at a time, and it should not be combined\n// with accessing standard error by other means at the same time. Switching\n// between means requires closing the old `StdErr` or flushing the object\n// becoming inactive (`std::clog.flush()`) and may require repositioning the\n// object becoming active (`std::cerr.seekp()`, `std::clog.seekp()`,\n// `std::fseek(stderr)`).\n//\n// As an alternative to `StdErr`, creating and later closing an\n// `OStreamWriter(&std::cerr)` makes it easier to combine writing to a `Writer`\n// with accessing `std::cerr`.\nclass StdErr : public FdWriter<UnownedFd> {\n public:\n  // Creates a closed `StdErr`.\n  explicit StdErr(Closed) noexcept : FdWriter(kClosed) {}\n\n  // Will write to standard error.\n  explicit StdErr(Options options = Options());\n\n  StdErr(StdErr&& that) = default;\n  StdErr& operator=(StdErr&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `StdErr`. This avoids\n  // constructing a temporary `StdErr` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Options options = Options());\n};\n\n// Sets file descriptors used by future instances of `Std{In,Out,Err}` in the\n// constructor. Restores the old value in the destructor. This affects global\n// state and is not thread safe.\n//\n// This is intended for testing of code which hardcodes usage of standard\n// streams. The preferred way of testing is to avoid redirecting standard\n// streams, by letting the code take explicit parameters specifying a `Reader`\n// or `Writer`, or their factory.\n//\n// Standard streams can also be redirected at a lower level, using `dup2()`.\n\nclass InjectedStdInFd {\n public:\n  explicit InjectedStdInFd(int fd);\n\n  InjectedStdInFd(const InjectedStdInFd&) = delete;\n  InjectedStdInFd& operator=(const InjectedStdInFd&) = delete;\n\n  ~InjectedStdInFd();\n\n private:\n  int old_fd_;\n  SizedSharedBuffer old_pending_;\n};\n\nclass InjectedStdOutFd {\n public:\n  explicit InjectedStdOutFd(int fd);\n\n  InjectedStdOutFd(const InjectedStdOutFd&) = delete;\n  InjectedStdOutFd& operator=(const InjectedStdOutFd&) = delete;\n\n  ~InjectedStdOutFd();\n\n private:\n  int old_fd_;\n};\n\nclass InjectedStdErrFd {\n public:\n  explicit InjectedStdErrFd(int fd);\n\n  InjectedStdErrFd(const InjectedStdErrFd&) = delete;\n  InjectedStdErrFd& operator=(const InjectedStdErrFd&) = delete;\n\n  ~InjectedStdErrFd();\n\n private:\n  int old_fd_;\n};\n\n// Implementation details follow.\n\ninline void StdIn::Reset(Closed) { FdReader::Reset(kClosed); }\n\ninline void StdOut::Reset(Closed) { FdWriter::Reset(kClosed); }\n\ninline void StdErr::Reset(Closed) { FdWriter::Reset(kClosed); }\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_STD_IO_H_\n"
  },
  {
    "path": "riegeli/bytes/string_reader.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/string_reader.h\"\n\n#include <stddef.h>\n\n#include <memory>\n#include <optional>\n\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nbool StringReaderBase::PullSlow(size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::PullSlow(): \"\n         \"enough data available, use Pull() instead\";\n  return false;\n}\n\nbool StringReaderBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of Reader::SeekSlow(): \"\n         \"position in the buffer, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  RIEGELI_ASSERT_EQ(start_pos(), 0u)\n      << \"Failed invariant of StringReader: non-zero position of buffer start\";\n  // Seeking forwards. Source ends.\n  set_cursor(limit());\n  return false;\n}\n\nstd::optional<Position> StringReaderBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  return limit_pos();\n}\n\nstd::unique_ptr<Reader> StringReaderBase::NewReaderImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point.\n  std::unique_ptr<Reader> reader =\n      std::make_unique<StringReader<>>(start(), start_to_limit());\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/string_reader.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_STRING_READER_H_\n#define RIEGELI_BYTES_STRING_READER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `StringReader`.\nclass StringReaderBase : public Reader {\n public:\n  // Returns the `std::string` or array being read from. Unchanged by `Close()`.\n  virtual absl::string_view SrcStringView() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool ToleratesReadingAhead() override { return true; }\n  bool SupportsRandomAccess() override { return true; }\n  bool SupportsNewReader() override { return true; }\n\n protected:\n  using Reader::Reader;\n\n  StringReaderBase(StringReaderBase&& that) noexcept;\n  StringReaderBase& operator=(StringReaderBase&& that) noexcept;\n\n  void Initialize(absl::string_view src);\n\n  bool PullSlow(size_t min_length, size_t recommended_length) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n  // Invariants if `is_open()`:\n  //   `start() == SrcStringView().data()`\n  //   `start_to_limit() == SrcStringView().size()`\n  //   `start_pos() == 0`\n};\n\n// A `Reader` which reads from a `std::string` or array.\n//\n// It supports random access and `NewReader()`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `std::string` or array being read from. `Src` must\n// support `Dependency<absl::string_view, Src>`, e.g.\n// `absl::string_view` (not owned, default), `const std::string*` (not owned),\n// `std::string` (owned), `Any<absl::string_view>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as\n// `absl::string_view` if there are no constructor arguments or if the first\n// constructor argument is an lvalue reference to a type convertible to\n// `absl::string_view` (to avoid unintended string copying) or to `const char*`\n// (to compute `std::strlen()` early), otherwise as `TargetT` of the type of the\n// first constructor argument.\n//\n// It might be better to use `ChainReader<Chain>` instead of\n// `StringReader<std::string>` to allow sharing the data (`Chain` blocks are\n// reference counted, `std::string` data have a single owner).\n//\n// The `std::string` or array must not be changed until the `StringReader` is\n// closed or no longer used.\ntemplate <typename Src = absl::string_view>\nclass StringReader : public StringReaderBase {\n public:\n  // Creates a closed `StringReader`.\n  explicit StringReader(Closed) noexcept : StringReaderBase(kClosed) {}\n\n  // Will read from the `std::string` or array provided by `src`.\n  explicit StringReader(Initializer<Src> src);\n\n  // Will read from an empty `absl::string_view`. This constructor is present\n  // only if `Src` is `absl::string_view`.\n  template <typename DependentSrc = Src,\n            std::enable_if_t<std::is_same_v<DependentSrc, absl::string_view>,\n                             int> = 0>\n  StringReader();\n\n  // Will read from `absl::string_view(src, size)`. This constructor is present\n  // only if `Src` is `absl::string_view`.\n  template <typename DependentSrc = Src,\n            std::enable_if_t<std::is_same_v<DependentSrc, absl::string_view>,\n                             int> = 0>\n  explicit StringReader(const char* src ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                        size_t size);\n\n  StringReader(StringReader&& that) = default;\n  StringReader& operator=(StringReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `StringReader`. This avoids\n  // constructing a temporary `StringReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src);\n  template <typename DependentSrc = Src,\n            std::enable_if_t<std::is_same_v<DependentSrc, absl::string_view>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n  template <typename DependentSrc = Src,\n            std::enable_if_t<std::is_same_v<DependentSrc, absl::string_view>,\n                             int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(const char* src, size_t size);\n\n  // Returns the object providing and possibly owning the `std::string` or array\n  // being read from. Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  absl::string_view SrcStringView() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the `std::string` or array being\n  // read from.\n  MovingDependency<absl::string_view, Src, Mover> src_;\n};\n\nexplicit StringReader(Closed) -> StringReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit StringReader(Src&& src) -> StringReader<std::conditional_t<\n    std::disjunction_v<\n        std::conjunction<std::is_lvalue_reference<Src>,\n                         std::is_convertible<Src, absl::string_view>>,\n        std::is_convertible<Src&&, const char*>>,\n    absl::string_view, TargetT<Src>>>;\nStringReader() -> StringReader<>;\nexplicit StringReader(const char* src, size_t size) -> StringReader<>;\n\n// Implementation details follow.\n\ninline StringReaderBase::StringReaderBase(StringReaderBase&& that) noexcept\n    : Reader(static_cast<Reader&&>(that)) {}\n\ninline StringReaderBase& StringReaderBase::operator=(\n    StringReaderBase&& that) noexcept {\n  Reader::operator=(static_cast<Reader&&>(that));\n  return *this;\n}\n\ninline void StringReaderBase::Initialize(absl::string_view src) {\n  set_buffer(src.data(), src.size());\n  move_limit_pos(available());\n}\n\ntemplate <typename Src>\nclass StringReader<Src>::Mover {\n public:\n  static auto member() { return &StringReader::src_; }\n\n  explicit Mover(StringReader& self, StringReader& that)\n      : uses_buffer_(self.start() != nullptr),\n        start_to_cursor_(self.start_to_cursor()) {\n    if (uses_buffer_) {\n      RIEGELI_ASSERT_EQ(that.src_.get().data(), self.start())\n          << \"StringReader source changed unexpectedly\";\n      RIEGELI_ASSERT_EQ(that.src_.get().size(), self.start_to_limit())\n          << \"StringReader source changed unexpectedly\";\n    }\n  }\n\n  void Done(StringReader& self) {\n    if (uses_buffer_) {\n      const absl::string_view src = self.src_.get();\n      self.set_buffer(src.data(), src.size(), start_to_cursor_);\n    }\n  }\n\n private:\n  bool uses_buffer_;\n  size_t start_to_cursor_;\n};\n\ntemplate <typename Src>\ninline StringReader<Src>::StringReader(Initializer<Src> src)\n    : src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ntemplate <\n    typename DependentSrc,\n    std::enable_if_t<std::is_same_v<DependentSrc, absl::string_view>, int>>\ninline StringReader<Src>::StringReader() : StringReader(absl::string_view()) {}\n\ntemplate <typename Src>\ntemplate <\n    typename DependentSrc,\n    std::enable_if_t<std::is_same_v<DependentSrc, absl::string_view>, int>>\ninline StringReader<Src>::StringReader(\n    const char* src ABSL_ATTRIBUTE_LIFETIME_BOUND, size_t size)\n    : StringReader(absl::string_view(src, size)) {}\n\ntemplate <typename Src>\ninline void StringReader<Src>::Reset(Closed) {\n  StringReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void StringReader<Src>::Reset(Initializer<Src> src) {\n  StringReaderBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ntemplate <\n    typename DependentSrc,\n    std::enable_if_t<std::is_same_v<DependentSrc, absl::string_view>, int>>\ninline void StringReader<Src>::Reset() {\n  Reset(absl::string_view());\n}\n\ntemplate <typename Src>\ntemplate <\n    typename DependentSrc,\n    std::enable_if_t<std::is_same_v<DependentSrc, absl::string_view>, int>>\ninline void StringReader<Src>::Reset(const char* src, size_t size) {\n  Reset(absl::string_view(src, size));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_STRING_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/string_writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/string_writer.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <limits>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_utils.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/string_utils.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/string_reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid StringWriterBase::Done() {\n  StringWriterBase::FlushImpl(FlushType::kFromObject);\n  Writer::Done();\n  secondary_buffer_ = Chain();\n  associated_reader_.Reset();\n}\n\ninline size_t StringWriterBase::used_size() const {\n  return UnsignedMax(IntCast<size_t>(pos()), written_size_);\n}\n\ninline size_t StringWriterBase::used_dest_size() const {\n  if (uses_secondary_buffer()) {\n    RIEGELI_ASSERT_EQ(available(), 0u)\n        << \"Failed precondition of StringWriterBase::used_dest_size(): \"\n        << \"secondary buffer has free space\";\n  }\n  RIEGELI_ASSERT_GE(used_size(), secondary_buffer_.size())\n      << \"Failed invariant of StringWriterBase: \"\n         \"negative destination size\";\n  return used_size() - secondary_buffer_.size();\n}\n\ninline void StringWriterBase::MakeDestBuffer(std::string& dest,\n                                             size_t cursor_index) {\n  RIEGELI_ASSERT(!uses_secondary_buffer())\n      << \"Failed precondition in StringWriterBase::MakeDestBuffer(): \"\n         \"secondary buffer is used\";\n  set_buffer(dest.data(), dest.size(), cursor_index);\n  set_start_pos(0);\n}\n\ninline void StringWriterBase::GrowDestAndMakeBuffer(std::string& dest,\n                                                    size_t cursor_index,\n                                                    size_t new_size) {\n  if (uses_secondary_buffer()) {\n    RIEGELI_ASSERT_EQ(available(), 0u)\n        << \"Failed precondition of StringWriterBase::GrowDestAndMakeBuffer(): \"\n        << \"secondary buffer has free space\";\n  }\n  const size_t old_size = dest.size();\n  if (ABSL_PREDICT_TRUE(new_size > old_size)) {\n    StringResizeAmortized(dest, new_size);\n    MarkPoisoned(dest.data() + old_size, new_size - old_size);\n  }\n  set_buffer(dest.data(), dest.size(), cursor_index);\n  set_start_pos(0);\n}\n\ninline bool StringWriterBase::GrowDestUnderCapacityAndMakeBuffer(\n    std::string& dest, size_t cursor_index, size_t min_length) {\n  RIEGELI_ASSERT(!uses_secondary_buffer())\n      << \"Failed precondition in \"\n         \"StringWriterBase::GrowDestUnderCapacityAndMakeBuffer(): \"\n         \"secondary buffer is used\";\n  RIEGELI_ASSERT_LE(min_length,\n                    std::numeric_limits<size_t>::max() - cursor_index)\n      << \"Failed precondition of \"\n         \"StringWriterBase::GrowDestUnderCapacityAndMakeBuffer(): \"\n         \"Writer position overflow\";\n  size_t new_size = cursor_index + min_length;\n  if (new_size > dest.capacity()) return false;\n  new_size = UnsignedClamp(\n      dest.size() + UnsignedClamp(dest.size(), kDefaultMinBlockSize,\n                                  kDefaultMaxBlockSize),\n      new_size, dest.capacity());\n  const size_t old_size = dest.size();\n  if (ABSL_PREDICT_TRUE(new_size > old_size)) {\n    dest.resize(new_size);\n    MarkPoisoned(dest.data() + old_size, new_size - old_size);\n  }\n  set_buffer(dest.data(), dest.size(), cursor_index);\n  set_start_pos(0);\n  return true;\n}\n\ninline void StringWriterBase::SyncSecondaryBuffer() {\n  set_start_pos(pos());\n  secondary_buffer_.RemoveSuffix(available(), options_);\n  set_buffer();\n}\n\ninline void StringWriterBase::MakeSecondaryBuffer(size_t min_length,\n                                                  size_t recommended_length) {\n  const absl::Span<char> buffer = secondary_buffer_.AppendBuffer(\n      min_length, recommended_length, Chain::kAnyLength, options_);\n  set_buffer(buffer.data(), buffer.size());\n}\n\nvoid StringWriterBase::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (write_size_hint == std::nullopt || ABSL_PREDICT_FALSE(!ok())) return;\n  std::string& dest = *DestString();\n  RIEGELI_ASSERT_GE(dest.size(), limit_pos() - secondary_buffer_.size())\n      << \"StringWriter destination changed unexpectedly\";\n  const size_t cursor_index = IntCast<size_t>(pos());\n  const size_t size_hint = UnsignedMax(\n      SaturatingAdd(cursor_index, SaturatingIntCast<size_t>(*write_size_hint)),\n      written_size_);\n  if (!uses_secondary_buffer()) {\n    GrowDestAndMakeBuffer(dest, cursor_index, size_hint);\n    return;\n  }\n  SyncSecondaryBuffer();\n  dest.erase(used_dest_size());\n  StringReserveAmortized(dest, size_hint);\n  std::move(secondary_buffer_).AppendTo(dest);\n  secondary_buffer_.Clear();\n  MakeDestBuffer(dest, cursor_index);\n}\n\nbool StringWriterBase::PushSlow(size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::string& dest = *DestString();\n  RIEGELI_ASSERT_GE(dest.size(), limit_pos() - secondary_buffer_.size())\n      << \"StringWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(min_length > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    const size_t cursor_index = IntCast<size_t>(pos());\n    if (cursor_index == 0 || ABSL_PREDICT_FALSE(written_size_ > cursor_index)) {\n      // Allocate the first block directly in `dest`. It is possible that it\n      // will not need to be copied if it turns out to be the only block,\n      // although this decision might cause it to remain wasteful if less data\n      // are written than space requested.\n      //\n      // Resize `dest` also if data follow the current position, to make the\n      // data available for partial overwriting.\n      const size_t size_hint = SaturatingAdd(\n          cursor_index, UnsignedMax(min_length, recommended_length));\n      StringReserveAmortized(dest, size_hint);\n    }\n    if (GrowDestUnderCapacityAndMakeBuffer(dest, cursor_index, min_length)) {\n      return true;\n    }\n    set_start_pos(cursor_index);\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  MakeSecondaryBuffer(min_length, recommended_length);\n  return true;\n}\n\nbool StringWriterBase::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of Writer::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::string& dest = *DestString();\n  RIEGELI_ASSERT_GE(dest.size(), limit_pos() - secondary_buffer_.size())\n      << \"StringWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    const size_t cursor_index = IntCast<size_t>(pos());\n    const size_t new_cursor_index = cursor_index + src.size();\n    if (cursor_index == 0) {\n      // Allocate the first block directly in `dest`. It is possible that it\n      // will not need to be copied if it turns out to be the only block,\n      // although this decision might cause it to remain wasteful if less data\n      // are written than space requested.\n      StringReserveAmortized(dest, new_cursor_index);\n    }\n    if (new_cursor_index <= dest.capacity()) {\n      if (ABSL_PREDICT_FALSE(new_cursor_index <= dest.size())) {\n        std::memcpy(dest.data() + cursor_index, src.data(), src.size());\n      } else {\n        dest.erase(cursor_index);\n        dest.append(src);\n      }\n      GrowDestUnderCapacityAndMakeBuffer(dest, new_cursor_index);\n      return true;\n    }\n    const size_t prefix_size = dest.capacity() - cursor_index;\n    dest.erase(cursor_index);\n    dest.append(src.data(), prefix_size);\n    src.remove_prefix(prefix_size);\n    set_start_pos(cursor_index + prefix_size);\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  secondary_buffer_.Append(src, options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool StringWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::string& dest = *DestString();\n  RIEGELI_ASSERT_GE(dest.size(), limit_pos() - secondary_buffer_.size())\n      << \"StringWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    const size_t cursor_index = IntCast<size_t>(pos());\n    const size_t new_cursor_index = cursor_index + src.size();\n    if (new_cursor_index <= dest.capacity()) {\n      if (ABSL_PREDICT_FALSE(new_cursor_index <= dest.size())) {\n        std::memcpy(dest.data() + cursor_index, src.data(), src.size());\n      } else {\n        dest.erase(cursor_index);\n        dest.append(absl::string_view(src));\n      }\n      GrowDestUnderCapacityAndMakeBuffer(dest, new_cursor_index);\n      return true;\n    }\n    set_start_pos(cursor_index);\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  secondary_buffer_.Append(std::move(src), options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool StringWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::string& dest = *DestString();\n  RIEGELI_ASSERT_GE(dest.size(), limit_pos() - secondary_buffer_.size())\n      << \"StringWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    const size_t cursor_index = IntCast<size_t>(pos());\n    const size_t new_cursor_index = cursor_index + src.size();\n    if (new_cursor_index <= dest.capacity()) {\n      if (ABSL_PREDICT_FALSE(new_cursor_index <= dest.size())) {\n        src.CopyTo(dest.data() + cursor_index);\n      } else {\n        dest.erase(cursor_index);\n        src.AppendTo(dest);\n      }\n      GrowDestUnderCapacityAndMakeBuffer(dest, new_cursor_index);\n      return true;\n    }\n    set_start_pos(cursor_index);\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  secondary_buffer_.Append(src, options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool StringWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::string& dest = *DestString();\n  RIEGELI_ASSERT_GE(dest.size(), limit_pos() - secondary_buffer_.size())\n      << \"StringWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    const size_t cursor_index = IntCast<size_t>(pos());\n    const size_t new_cursor_index = cursor_index + src.size();\n    if (new_cursor_index <= dest.capacity()) {\n      if (ABSL_PREDICT_FALSE(new_cursor_index <= dest.size())) {\n        src.CopyTo(dest.data() + cursor_index);\n      } else {\n        dest.erase(cursor_index);\n        std::move(src).AppendTo(dest);\n      }\n      GrowDestUnderCapacityAndMakeBuffer(dest, new_cursor_index);\n      return true;\n    }\n    set_start_pos(cursor_index);\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  secondary_buffer_.Append(std::move(src), options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool StringWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::string& dest = *DestString();\n  RIEGELI_ASSERT_GE(dest.size(), limit_pos() - secondary_buffer_.size())\n      << \"StringWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    const size_t cursor_index = IntCast<size_t>(pos());\n    const size_t new_cursor_index = cursor_index + src.size();\n    if (new_cursor_index <= dest.capacity()) {\n      if (ABSL_PREDICT_FALSE(new_cursor_index <= dest.size())) {\n        cord_internal::CopyCordToArray(src, dest.data() + cursor_index);\n      } else {\n        dest.erase(cursor_index);\n        absl::AppendCordToString(src, &dest);\n      }\n      GrowDestUnderCapacityAndMakeBuffer(dest, new_cursor_index);\n      return true;\n    }\n    set_start_pos(cursor_index);\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  secondary_buffer_.Append(src, options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool StringWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::string& dest = *DestString();\n  RIEGELI_ASSERT_GE(dest.size(), limit_pos() - secondary_buffer_.size())\n      << \"StringWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    const size_t cursor_index = IntCast<size_t>(pos());\n    const size_t new_cursor_index = cursor_index + src.size();\n    if (new_cursor_index <= dest.capacity()) {\n      if (ABSL_PREDICT_FALSE(new_cursor_index <= dest.size())) {\n        cord_internal::CopyCordToArray(src, dest.data() + cursor_index);\n      } else {\n        dest.erase(cursor_index);\n        absl::AppendCordToString(src, &dest);\n      }\n      GrowDestUnderCapacityAndMakeBuffer(dest, new_cursor_index);\n      return true;\n    }\n    set_start_pos(cursor_index);\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  secondary_buffer_.Append(std::move(src), options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool StringWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::string& dest = *DestString();\n  RIEGELI_ASSERT_GE(dest.size(), limit_pos() - secondary_buffer_.size())\n      << \"StringWriter destination changed unexpectedly\";\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  if (!uses_secondary_buffer()) {\n    const size_t cursor_index = IntCast<size_t>(pos());\n    const size_t new_cursor_index = cursor_index + IntCast<size_t>(src.size());\n    if (new_cursor_index <= dest.capacity()) {\n      if (ABSL_PREDICT_FALSE(new_cursor_index <= dest.size())) {\n        std::memset(dest.data() + cursor_index, src.fill(),\n                    IntCast<size_t>(src.size()));\n      } else {\n        dest.erase(cursor_index);\n        dest.append(IntCast<size_t>(src.size()), src.fill());\n      }\n      GrowDestUnderCapacityAndMakeBuffer(dest, new_cursor_index);\n      return true;\n    }\n    set_start_pos(cursor_index);\n    set_buffer();\n    written_size_ = 0;\n  } else {\n    SyncSecondaryBuffer();\n  }\n  move_start_pos(src.size());\n  src.AppendTo(secondary_buffer_, options_);\n  MakeSecondaryBuffer();\n  return true;\n}\n\nbool StringWriterBase::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::string& dest = *DestString();\n  RIEGELI_ASSERT_GE(dest.size(), limit_pos() - secondary_buffer_.size())\n      << \"StringWriter destination changed unexpectedly\";\n  const size_t cursor_index = IntCast<size_t>(pos());\n  if (!uses_secondary_buffer()) {\n    dest.erase(used_size());\n  } else {\n    SyncSecondaryBuffer();\n    dest.erase(used_dest_size());\n    std::move(secondary_buffer_).AppendTo(dest);\n    secondary_buffer_.Clear();\n  }\n  set_buffer();\n  set_start_pos(cursor_index);\n  return true;\n}\n\nbool StringWriterBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of Writer::SeekSlow(): \"\n         \"position unchanged, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::string& dest = *DestString();\n  RIEGELI_ASSERT_GE(dest.size(), limit_pos() - secondary_buffer_.size())\n      << \"StringWriter destination changed unexpectedly\";\n  if (new_pos > pos()) {\n    if (ABSL_PREDICT_FALSE(uses_secondary_buffer())) return false;\n    if (ABSL_PREDICT_FALSE(new_pos > used_size())) {\n      MakeDestBuffer(dest, used_size());\n      return false;\n    }\n  } else {\n    if (uses_secondary_buffer()) {\n      SyncSecondaryBuffer();\n      dest.erase(used_dest_size());\n      std::move(secondary_buffer_).AppendTo(dest);\n      secondary_buffer_.Clear();\n    }\n    written_size_ = used_size();\n  }\n  MakeDestBuffer(dest, IntCast<size_t>(new_pos));\n  return true;\n}\n\nstd::optional<Position> StringWriterBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  return used_size();\n}\n\nbool StringWriterBase::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  std::string& dest = *DestString();\n  RIEGELI_ASSERT_GE(dest.size(), limit_pos() - secondary_buffer_.size())\n      << \"StringWriter destination changed unexpectedly\";\n  if (new_size > pos()) {\n    if (ABSL_PREDICT_FALSE(uses_secondary_buffer())) return false;\n    if (ABSL_PREDICT_FALSE(new_size > used_size())) {\n      MakeDestBuffer(dest, used_size());\n      return false;\n    }\n  } else if (new_size > limit_pos() - secondary_buffer_.size()) {\n    secondary_buffer_.RemoveSuffix(\n        IntCast<size_t>(limit_pos()) - IntCast<size_t>(new_size), options_);\n    set_start_pos(new_size);\n    set_buffer();\n    return true;\n  } else {\n    secondary_buffer_.Clear();\n  }\n  written_size_ = 0;\n  MakeDestBuffer(dest, IntCast<size_t>(new_size));\n  return true;\n}\n\nReader* StringWriterBase::ReadModeImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  std::string& dest = *DestString();\n  RIEGELI_ASSERT_GE(dest.size(), limit_pos() - secondary_buffer_.size())\n      << \"StringWriter destination changed unexpectedly\";\n  if (uses_secondary_buffer()) {\n    SyncSecondaryBuffer();\n    dest.erase(used_dest_size());\n    std::move(secondary_buffer_).AppendTo(dest);\n    secondary_buffer_.Clear();\n  }\n  StringReader<>* const reader =\n      associated_reader_.ResetReader(dest.data(), used_size());\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/string_writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_STRING_WRITER_H_\n#define RIEGELI_BYTES_STRING_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nclass Reader;\ntemplate <typename Src>\nclass StringReader;\n\n// Template parameter independent part of `StringWriter`.\nclass StringWriterBase : public Writer {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // If `false`, replaces existing contents of the destination, clearing it\n    // first.\n    //\n    // If `true`, appends to existing contents of the destination.\n    //\n    // Default: `false`.\n    Options& set_append(bool append) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      append_ = append;\n      return *this;\n    }\n    Options&& set_append(bool append) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_append(append));\n    }\n    bool append() const { return append_; }\n\n   private:\n    bool append_ = false;\n  };\n\n  // Returns the `std::string` being written to. Unchanged by `Close()`.\n  virtual std::string* DestString() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n  std::string& Digest() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    Flush();\n    return *DestString();\n  }\n\n  bool SupportsRandomAccess() override { return true; }\n  bool SupportsReadMode() override { return true; }\n\n protected:\n  explicit StringWriterBase(Closed) noexcept : Writer(kClosed) {}\n\n  explicit StringWriterBase(BufferOptions buffer_options);\n\n  StringWriterBase(StringWriterBase&& that) noexcept;\n  StringWriterBase& operator=(StringWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options);\n  void Initialize(std::string* dest, bool append);\n  bool uses_secondary_buffer() const { return !secondary_buffer_.empty(); }\n\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool FlushImpl(FlushType flush_type) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  bool TruncateImpl(Position new_size) override;\n  Reader* ReadModeImpl(Position initial_pos) override;\n\n private:\n  // Returns the amount of data written, either to `*DestString()` or to\n  // `secondary_buffer_`.\n  size_t used_size() const;\n\n  // Returns the amount of data written to `*DestString()`. Does not include\n  // data written to `secondary_buffer_`.\n  //\n  // Precondition: if `uses_secondary_buffer()` then `available() == 0`\n  size_t used_dest_size() const;\n\n  // Sets buffer pointers to `dest`.\n  //\n  // Precondition: `!uses_secondary_buffer()`\n  void MakeDestBuffer(std::string& dest, size_t cursor_index);\n\n  // Appends some uninitialized space to `dest` to guarantee at least `new_size`\n  // of size. Sets buffer pointers to `dest`.\n  //\n  // Precondition: if `uses_secondary_buffer()` then `available() == 0`\n  void GrowDestAndMakeBuffer(std::string& dest, size_t cursor_index,\n                             size_t new_size);\n\n  // Increases the size of `dest` at least to `cursor_index + min_length`\n  // if this can be done without reallocation. New contents are unspecified.\n  // Sets buffer pointers to `dest`.\n  //\n  // Returns `true` on success, or `false` if there was not enough space.\n  //\n  // Precondition: `!uses_secondary_buffer()`\n  bool GrowDestUnderCapacityAndMakeBuffer(std::string& dest,\n                                          size_t cursor_index,\n                                          size_t min_length = 0);\n\n  // Discards uninitialized space from the end of `secondary_buffer_`, so that\n  // it contains only actual data written.\n  void SyncSecondaryBuffer();\n\n  // Appends uninitialized space to `secondary_buffer_`.\n  void MakeSecondaryBuffer(size_t min_length = 0,\n                           size_t recommended_length = 0);\n\n  // Move `secondary_buffer_`, adjusting buffer pointers if they point to it.\n  void MoveSecondaryBuffer(StringWriterBase& that);\n\n  Chain::Options options_;\n  // Buffered data which did not fit under `DestString()->capacity()`.\n  Chain secondary_buffer_;\n\n  // Size of written data is always `UnsignedMax(pos(), written_size_)`.\n  // This is used to determine the size after seeking backwards.\n  //\n  // Invariant: if `uses_secondary_buffer()` then `written_size_ == 0`.\n  size_t written_size_ = 0;\n\n  AssociatedReader<StringReader<absl::string_view>> associated_reader_;\n\n  // If `!uses_secondary_buffer()`, then `*DestString()` contains the data\n  // before the current position of length `pos()`, followed by the data after\n  // the current position of length `SaturatingSub(written_size_, pos())`,\n  // followed by free space of length\n  // `DestString()->size() - UnsignedMax(pos(), written_size_)`.\n  //\n  // If `uses_secondary_buffer()`, then `*DestString()` contains the prefix of\n  // the data of length `limit_pos() - secondary_buffer_.size()` followed by\n  // free space, and `secondary_buffer_` contains the rest of the data of length\n  // `secondary_buffer_.size() - available()` followed by free space of length\n  // `available()`. In this case there is no data after the current position.\n  //\n  // Invariants if `ok()`:\n  //   `!uses_secondary_buffer() &&\n  //    start() == DestString()->data() &&\n  //    start_to_limit() == DestString()->size() &&\n  //    start_pos() == 0` or\n  //       `uses_secondary_buffer() &&\n  //        limit() == secondary_buffer_.blocks().back().data() +\n  //                   secondary_buffer_.blocks().back().size()` or\n  //       `start() == nullptr`\n  //   `limit_pos() >= secondary_buffer_.size()`\n  //   `DestString()->size() >= limit_pos() - secondary_buffer_.size()`\n};\n\n// A `Writer` which writes to a `std::string`. If `Options::append()` is `false`\n// (the default), replaces existing contents of the `std::string`, clearing it\n// first. If `Options::append()` is `true`, appends to existing contents of the\n// `std::string`.\n//\n// It supports `Seek()` and `ReadMode()`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `std::string` being written to. `Dest` must support\n// `Dependency<std::string*, Dest>`, e.g. `std::string*` (not owned, default),\n// `std::string` (owned), `Any<std::string*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `std::string`\n// if there are no constructor arguments or the only argument is `Options`,\n// otherwise as `TargetT` of the type of the first constructor argument, except\n// that CTAD is deleted if the first constructor argument is a `std::string&`\n// or `const std::string&` (to avoid writing to an unintentionally separate copy\n// of an existing object).\n//\n// The `std::string` must not be accessed until the `StringWriter` is closed or\n// no longer used, except that it is allowed to read the `std::string`\n// immediately after `Flush()`.\n//\n// `VectorWriter` with `UninitializedVector<char>` or\n// `UninitializedInlinedVector<char, inlined_size>`, as well as\n// `CompactStringWriter`, is more efficient than `StringWriter`\n// because the destination can be resized with uninitialized space.\ntemplate <typename Dest = std::string*>\nclass StringWriter : public StringWriterBase {\n public:\n  // Creates a closed `StringWriter`.\n  explicit StringWriter(Closed) noexcept : StringWriterBase(kClosed) {}\n\n  // Will write to the `std::string` provided by `dest`.\n  explicit StringWriter(Initializer<Dest> dest, Options options = Options());\n\n  // Will write to an owned `std::string` which can be accessed by `dest()`.\n  // This constructor is present only if `Dest` is `std::string`.\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<std::is_same_v<DependentDest, std::string>, int> = 0>\n  explicit StringWriter(Options options = Options());\n\n  StringWriter(StringWriter&& that) = default;\n  StringWriter& operator=(StringWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `StringWriter`. This avoids\n  // constructing a temporary `StringWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<std::is_same_v<DependentDest, std::string>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Options options = Options());\n\n  // Returns the object providing and possibly owning the `std::string` being\n  // written to. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  std::string* DestString() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the `std::string` being written\n  // to, with uninitialized space appended (possibly empty); `cursor()` points\n  // to the uninitialized space.\n  MovingDependency<std::string*, Dest, Mover> dest_;\n};\n\nexplicit StringWriter(Closed) -> StringWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit StringWriter(Dest&& dest, StringWriterBase::Options options =\n                                       StringWriterBase::Options())\n    -> StringWriter<std::conditional_t<\n        std::conjunction_v<std::is_lvalue_reference<Dest>,\n                           std::is_convertible<std::remove_reference_t<Dest>*,\n                                               const std::string*>>,\n        DeleteCtad<Dest&&>, TargetT<Dest>>>;\nexplicit StringWriter(\n    StringWriterBase::Options options = StringWriterBase::Options())\n    -> StringWriter<std::string>;\n\n// Implementation details follow.\n\ninline StringWriterBase::StringWriterBase(BufferOptions buffer_options)\n    : options_(Chain::Options()\n                   .set_min_block_size(buffer_options.min_buffer_size())\n                   .set_max_block_size(buffer_options.max_buffer_size())) {}\n\ninline StringWriterBase::StringWriterBase(StringWriterBase&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)),\n      options_(that.options_),\n      written_size_(that.written_size_),\n      associated_reader_(std::move(that.associated_reader_)) {\n  MoveSecondaryBuffer(that);\n}\n\ninline StringWriterBase& StringWriterBase::operator=(\n    StringWriterBase&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  options_ = that.options_;\n  written_size_ = that.written_size_;\n  associated_reader_ = std::move(that.associated_reader_);\n  MoveSecondaryBuffer(that);\n  return *this;\n}\n\ninline void StringWriterBase::Reset(Closed) {\n  Writer::Reset(kClosed);\n  options_ = Chain::Options();\n  secondary_buffer_ = Chain();\n  written_size_ = 0;\n  associated_reader_.Reset();\n}\n\ninline void StringWriterBase::Reset(BufferOptions buffer_options) {\n  Writer::Reset();\n  options_ = Chain::Options()\n                 .set_min_block_size(buffer_options.min_buffer_size())\n                 .set_max_block_size(buffer_options.max_buffer_size());\n  secondary_buffer_.Clear();\n  written_size_ = 0;\n  associated_reader_.Reset();\n}\n\ninline void StringWriterBase::Initialize(std::string* dest, bool append) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of StringWriter: null string pointer\";\n  if (append) {\n    set_start_pos(dest->size());\n  } else {\n    set_buffer(dest->data(), dest->size());\n    MarkPoisoned(cursor(), available());\n  }\n}\n\ninline void StringWriterBase::MoveSecondaryBuffer(StringWriterBase& that) {\n  // Buffer pointers are already moved so `start()` is taken from `*this`.\n  // `secondary_buffer_` is not moved yet so `uses_secondary_buffer()` is called\n  // on `that`.\n  const bool uses_buffer = start() != nullptr && that.uses_secondary_buffer();\n  const size_t saved_start_to_limit = start_to_limit();\n  const size_t saved_start_to_cursor = start_to_cursor();\n  if (uses_buffer) {\n    RIEGELI_ASSERT(that.secondary_buffer_.blocks().back().data() +\n                       that.secondary_buffer_.blocks().back().size() ==\n                   limit())\n        << \"Failed invariant of StringWriter: \"\n           \"secondary buffer inconsistent with buffer pointers\";\n  }\n  secondary_buffer_ = std::move(that.secondary_buffer_);\n  if (uses_buffer) {\n    const absl::string_view last_block = secondary_buffer_.blocks().back();\n    set_buffer(const_cast<char*>(last_block.data() + last_block.size()) -\n                   saved_start_to_limit,\n               saved_start_to_limit, saved_start_to_cursor);\n  }\n}\n\ntemplate <typename Dest>\nclass StringWriter<Dest>::Mover {\n public:\n  static auto member() { return &StringWriter::dest_; }\n\n  explicit Mover(StringWriter& self, StringWriter& that)\n      : uses_buffer_(self.start() != nullptr && !self.uses_secondary_buffer()),\n        start_to_cursor_(self.start_to_cursor()) {\n    if (uses_buffer_) {\n      RIEGELI_ASSERT_EQ(that.dest_->data(), self.start())\n          << \"StringWriter destination changed unexpectedly\";\n      RIEGELI_ASSERT_EQ(that.dest_->size(), self.start_to_limit())\n          << \"StringWriter destination changed unexpectedly\";\n    }\n  }\n\n  void Done(StringWriter& self) {\n    if (uses_buffer_) {\n      std::string& dest = *self.dest_;\n      self.set_buffer(dest.data(), dest.size(), start_to_cursor_);\n    }\n  }\n\n private:\n  bool uses_buffer_;\n  size_t start_to_cursor_;\n};\n\ntemplate <typename Dest>\ninline StringWriter<Dest>::StringWriter(Initializer<Dest> dest, Options options)\n    : StringWriterBase(options.buffer_options()), dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.append());\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_same_v<DependentDest, std::string>, int>>\ninline StringWriter<Dest>::StringWriter(Options options)\n    : StringWriter(riegeli::Maker(), std::move(options)) {}\n\ntemplate <typename Dest>\ninline void StringWriter<Dest>::Reset(Closed) {\n  StringWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void StringWriter<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  StringWriterBase::Reset(options.buffer_options());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.append());\n}\n\ntemplate <typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_same_v<DependentDest, std::string>, int>>\ninline void StringWriter<Dest>::Reset(Options options) {\n  Reset(riegeli::Maker(), std::move(options));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_STRING_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/stringify.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_STRINGIFY_H_\n#define RIEGELI_BYTES_STRINGIFY_H_\n\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/numeric/int128.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/has_absl_stringify.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/write_int_internal.h\"\n\nnamespace riegeli {\n\nnamespace stringify_internal {\n\ntemplate <typename Src, typename Enable = void>\nstruct HasRiegeliStringifiedSize : std::false_type {};\n\ntemplate <typename Src>\nstruct HasRiegeliStringifiedSize<\n    Src, std::enable_if_t<std::is_convertible_v<\n             decltype(RiegeliStringifiedSize(std::declval<const Src&>())),\n             Position>>> : std::true_type {};\n\n}  // namespace stringify_internal\n\n// `riegeli::StringifiedSize()` of a stringifiable value returns the size of its\n// stringification as `Position` if easily known, otherwise it is only declared\n// as returning `void`.\n//\n// It has the same overloads as `Writer::Write()`, assuming that the parameter\n// is passed by const reference.\n//\n// To customize `riegeli::StringifiedSize()` for a class `Src` supporting\n// `AbslStringify()`, define a free function\n// `friend Position RiegeliStringifiedSize(const Src& src)` as a friend of `Src`\n// inside class definition or in the same namespace as `Src`, so that it can be\n// found via ADL. A function returning `void` is treated as absent.\n//\n// There is no need to define `RiegeliStringifiedSize()` for types convertible\n// to `BytesRef`, even if they support `AbslStringify()`.\ninline Position StringifiedSize(ABSL_ATTRIBUTE_UNUSED char src) { return 1; }\n#if __cpp_char8_t\ninline Position StringifiedSize(ABSL_ATTRIBUTE_UNUSED char8_t src) { return 1; }\n#endif\ninline Position StringifiedSize(BytesRef src) { return src.size(); }\nABSL_ATTRIBUTE_ALWAYS_INLINE inline Position StringifiedSize(const char* src) {\n  return absl::string_view(src).size();\n}\ninline Position StringifiedSize(const Chain& src) { return src.size(); }\ninline Position StringifiedSize(const absl::Cord& src) { return src.size(); }\ninline Position StringifiedSize(ByteFill src) { return src.size(); }\ninline Position StringifiedSize(signed char src) {\n  return write_int_internal::StringifiedSizeSigned(src);\n}\ninline Position StringifiedSize(unsigned char src) {\n  return write_int_internal::StringifiedSizeUnsigned(src);\n}\ninline Position StringifiedSize(short src) {\n  return write_int_internal::StringifiedSizeSigned(src);\n}\ninline Position StringifiedSize(unsigned short src) {\n  return write_int_internal::StringifiedSizeUnsigned(src);\n}\ninline Position StringifiedSize(int src) {\n  return write_int_internal::StringifiedSizeSigned(src);\n}\ninline Position StringifiedSize(unsigned src) {\n  return write_int_internal::StringifiedSizeUnsigned(src);\n}\ninline Position StringifiedSize(long src) {\n  return write_int_internal::StringifiedSizeSigned(src);\n}\ninline Position StringifiedSize(unsigned long src) {\n  return write_int_internal::StringifiedSizeUnsigned(src);\n}\ninline Position StringifiedSize(long long src) {\n  return write_int_internal::StringifiedSizeSigned(src);\n}\ninline Position StringifiedSize(unsigned long long src) {\n  return write_int_internal::StringifiedSizeUnsigned(src);\n}\ninline Position StringifiedSize(absl::int128 src) {\n  return write_int_internal::StringifiedSizeSigned(src);\n}\ninline Position StringifiedSize(absl::uint128 src) {\n  return write_int_internal::StringifiedSizeUnsigned(src);\n}\nvoid StringifiedSize(float);\nvoid StringifiedSize(double);\nvoid StringifiedSize(long double);\ntemplate <\n    typename Src,\n    std::enable_if_t<\n        std::conjunction_v<\n            absl::HasAbslStringify<Src>,\n            std::negation<std::is_convertible<const Src&, BytesRef>>,\n            std::negation<std::is_convertible<const Src&, const Chain&>>,\n            std::negation<std::is_convertible<const Src&, const absl::Cord&>>,\n            std::negation<std::is_convertible<const Src&, ByteFill>>>,\n        int> = 0>\ninline auto StringifiedSize(const Src& src) {\n  if constexpr (stringify_internal::HasRiegeliStringifiedSize<Src>::value) {\n    return RiegeliStringifiedSize(src);\n  }\n}\nvoid StringifiedSize(bool) = delete;\nvoid StringifiedSize(wchar_t) = delete;\nvoid StringifiedSize(char16_t) = delete;\nvoid StringifiedSize(char32_t) = delete;\n\nnamespace stringify_internal {\n\ntemplate <typename T, typename Enable = void>\nstruct IsStringifiableImpl : std::false_type {};\n\ntemplate <typename T>\nstruct IsStringifiableImpl<T, std::void_t<decltype(riegeli::StringifiedSize(\n                                  std::declval<const T&>()))>>\n    : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct HasStringifiedSizeImpl : std::false_type {};\n\ntemplate <typename T>\nstruct HasStringifiedSizeImpl<\n    T, std::enable_if_t<std::is_convertible_v<decltype(riegeli::StringifiedSize(\n                                                  std::declval<const T&>())),\n                                              Position>>> : std::true_type {};\n\n}  // namespace stringify_internal\n\n// `IsStringifiable` checks if the type is an appropriate argument for\n// `Writer::Write()`, `BackwardWriter::Write()`, `riegeli::Write()`, and e.g.\n// `riegeli::WriteLine()`.\ntemplate <typename... T>\nstruct IsStringifiable\n    : std::conjunction<stringify_internal::IsStringifiableImpl<T>...> {};\n\n// `HasStringifiedSize` checks if the type has `riegeli::StringifiedSize()`\n// defined returning the size.\ntemplate <typename... T>\nstruct HasStringifiedSize\n    : std::conjunction<stringify_internal::HasStringifiedSizeImpl<T>...> {};\n\n// `riegeli::StringifiedSize()` of multiple stringifiable values returns the\n// total size of their stringifications, interpreted as for\n// `riegeli::StringifiedSize()` with a single parameter.\ntemplate <typename... Srcs\n#if !__cpp_concepts\n          ,\n          std::enable_if_t<\n              std::conjunction_v<std::bool_constant<sizeof...(Srcs) != 1>,\n                                 IsStringifiable<Srcs...>>,\n              int> = 0\n#endif\n          >\ninline auto StringifiedSize(const Srcs&... srcs)\n#if __cpp_concepts\n    // For conjunctions, `requires` gives better error messages than\n    // `std::enable_if_t`, indicating the relevant argument.\n  requires(sizeof...(Srcs) != 1) && (IsStringifiable<Srcs>::value && ...)\n#endif\n{\n  if constexpr (HasStringifiedSize<Srcs...>::value) {\n    return (Position{0} + ... + riegeli::StringifiedSize(srcs));\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_STRINGIFY_H_\n"
  },
  {
    "path": "riegeli/bytes/stringify_writer.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_STRINGIFY_WRITER_H_\n#define RIEGELI_BYTES_STRINGIFY_WRITER_H_\n\n#include <limits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/prefix_limiting_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// A `Writer` which writes to a sink provided to `AbslStringify()`.\n//\n// The template parameter is the sink pointer type rather than the sink value\n// type for consistency with other templated `Writer` classes parameterized by\n// the type of the constructor argument. This must nevertheless be a pointer,\n// not an arbitrary type supporting `Dependency`.\n//\n// `Dest` must support `->Append(absl::string_view)`.\ntemplate <typename Dest>\nclass StringifyWriter : public BufferedWriter {\n public:\n  // Creates a closed `StringifyWriter`.\n  explicit StringifyWriter(Closed) noexcept : BufferedWriter(kClosed) {}\n\n  // Will write to `*dest`.\n  explicit StringifyWriter(Dest dest ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : dest_(std::move(RIEGELI_EVAL_ASSERT_NOTNULL(dest))) {}\n\n  StringifyWriter(StringifyWriter&& that) = default;\n  StringifyWriter& operator=(StringifyWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `StringifyWriter`. This\n  // avoids constructing a temporary `StringifyWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Dest dest);\n\n  // Returns a pointer to the sink being written to. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_; }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_; }\n\n protected:\n  bool WriteInternal(absl::string_view src) override;\n\n private:\n  Dest dest_{};\n};\n\n// Specialization of `StringifyWriter<WriterStringifySink*>` which\n// avoids wrapping a `Writer` in a `WriterStringifySink` and adapting it\n// back to a `Writer`.\ntemplate <>\nclass StringifyWriter<WriterStringifySink*> : public PrefixLimitingWriter<> {\n public:\n  // Creates a closed `StringifyWriter`.\n  explicit StringifyWriter(Closed) noexcept : PrefixLimitingWriter(kClosed) {}\n\n  // Will write to `*dest`.\n  explicit StringifyWriter(\n      WriterStringifySink* dest ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : PrefixLimitingWriter(RIEGELI_EVAL_ASSERT_NOTNULL(dest)->dest()),\n        dest_(dest) {}\n\n  StringifyWriter(StringifyWriter&& that) = default;\n  StringifyWriter& operator=(StringifyWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `StringifyWriter`. This\n  // avoids constructing a temporary `StringifyWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(WriterStringifySink* dest);\n\n  // Returns a pointer to the sink being written to. Unchanged by `Close()`.\n  WriterStringifySink*& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_; }\n  WriterStringifySink* const& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_;\n  }\n\n private:\n  WriterStringifySink* dest_{};\n};\n\nexplicit StringifyWriter(Closed) -> StringifyWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit StringifyWriter(Dest dest) -> StringifyWriter<Dest>;\n\n// Implementation details follow.\n\ntemplate <typename Dest>\ninline void StringifyWriter<Dest>::Reset(Closed) {\n  BufferedWriter::Reset(kClosed);\n  riegeli::Reset(dest_);\n}\n\ntemplate <typename Dest>\ninline void StringifyWriter<Dest>::Reset(Dest dest) {\n  BufferedWriter::Reset();\n  dest_ = std::move(dest);\n}\n\ntemplate <typename Dest>\nbool StringifyWriter<Dest>::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of BufferedWriter::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedWriter::WriteInternal()\";\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  dest_->Append(src);\n  move_start_pos(src.size());\n  return true;\n}\n\ninline void StringifyWriter<WriterStringifySink*>::Reset(Closed) {\n  PrefixLimitingWriter::Reset(kClosed);\n  dest_ = nullptr;\n}\n\ninline void StringifyWriter<WriterStringifySink*>::Reset(\n    WriterStringifySink* dest) {\n  PrefixLimitingWriter::Reset(dest->dest());\n  dest_ = dest;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_STRINGIFY_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/vector_writer.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_VECTOR_WRITER_H_\n#define RIEGELI_BYTES_VECTOR_WRITER_H_\n\n#include <stddef.h>\n\n#include <type_traits>\n\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/uninitialized_vector.h\"\n#include \"riegeli/bytes/resizable_writer.h\"\n\nnamespace riegeli {\n\nnamespace vector_writer_internal {\n\ntemplate <typename VectorType, typename Enable = void>\nstruct HasAllocatorType : std::false_type {};\n\ntemplate <typename VectorType>\nstruct HasAllocatorType<VectorType,\n                        std::void_t<typename VectorType::allocator_type>>\n    : std::true_type {};\n\n// `ResizableTraits` for `std::vector<T, Alloc>` including\n// `UninitializedVector<T>`, `absl::InlinedVector<T, inlined_size, Alloc>`\n// including `UninitializedInlinedVector<T, inlined_size>`, or a similar type.\n// Its value type must be trivially copyable, usually `char`.\n//\n// The vector type must support at least the following members:\n//\n// ```\n//   using value_type = ...;\n//\n//   value_type* data();\n//   size_t size() const;\n//   size_t max_size() const;\n//   size_t capacity() const;\n//\n//   iterator begin();\n//   iterator end();\n//\n//   void resize(size_t new_size);\n//   void reserve(size_t new_capacity);\n//   void erase(iterator first, iterator last);\n// ```\n//\n// Warning: byte contents are reinterpreted as values of type `T`, and the size\n// is rounded up to a multiple of the element type.\ntemplate <typename VectorType>\nstruct VectorResizableTraits {\n private:\n  using T = typename VectorType::value_type;\n\n public:\n  static_assert(std::is_trivially_copyable_v<T>,\n                \"Value type of the parameter of VectorResizableTraits must be \"\n                \"trivially copyable\");\n\n  using Resizable = VectorType;\n  static char* Data(Resizable& dest) {\n    return reinterpret_cast<char*>(dest.data());\n  }\n  static size_t Size(const Resizable& dest) { return dest.size() * sizeof(T); }\n  static constexpr bool kIsStable = true;\n  static bool Resize(Resizable& dest, size_t new_size, size_t used_size) {\n    RIEGELI_ASSERT_LE(used_size, dest.size() * sizeof(T))\n        << \"Failed precondition of ResizableTraits::Resize(): \"\n           \"used size exceeds old size\";\n    RIEGELI_ASSERT_LE(used_size, new_size)\n        << \"Failed precondition of ResizableTraits::Resize(): \"\n           \"used size exceeds new size\";\n    const size_t new_num_elements = SizeToNumElements(new_size);\n    Reserve(dest, new_num_elements, used_size);\n    dest.resize(new_num_elements);\n    return true;\n  }\n  static bool Grow(Resizable& dest, size_t new_size, size_t used_size) {\n    RIEGELI_ASSERT_GT(new_size, dest.size() * sizeof(T))\n        << \"Failed precondition of ResizableTraits::Grow(): \"\n           \"no need to grow\";\n    RIEGELI_ASSERT_LE(used_size, dest.size() * sizeof(T))\n        << \"Failed precondition of ResizableTraits::Grow(): \"\n           \"used size exceeds old size\";\n    RIEGELI_ASSERT_LE(used_size, new_size)\n        << \"Failed precondition of ResizableTraits::Grow(): \"\n           \"used size exceeds new size\";\n    const size_t new_num_elements = SizeToNumElements(new_size);\n    Reserve(dest, new_num_elements, used_size);\n    if (!GrowUnderCapacity(dest, new_size)) RIEGELI_ASSUME_UNREACHABLE();\n    return true;\n  }\n  static bool GrowUnderCapacity(Resizable& dest, size_t new_size) {\n    RIEGELI_ASSERT_GT(new_size, dest.size() * sizeof(T))\n        << \"Failed precondition of ResizableTraits::GrowUnderCapacity(): \"\n           \"no need to grow\";\n    const size_t new_num_elements = SizeToNumElements(new_size);\n    if (new_num_elements > dest.capacity()) return false;\n    if constexpr (HasAllocatorType<Resizable>::value) {\n      if constexpr (std::is_same_v<typename Resizable::allocator_type,\n                                   UninitializedAllocator<T>>) {\n        dest.resize(dest.capacity());\n        return true;\n      }\n    }\n    dest.resize(UnsignedClamp(\n        dest.size() + UnsignedClamp(dest.size(),\n                                    SizeToNumElements(kDefaultMinBlockSize),\n                                    SizeToNumElements(kDefaultMaxBlockSize)),\n        new_num_elements, dest.capacity()));\n    return true;\n  }\n\n private:\n  static size_t SizeToNumElements(size_t size) {\n    return size / sizeof(T) + (size % sizeof(T) == 0 ? 0 : 1);\n  }\n  static void Reserve(Resizable& dest, size_t new_num_elements,\n                      size_t used_size) {\n    if (new_num_elements > dest.capacity()) {\n      dest.erase(dest.begin() + SizeToNumElements(used_size), dest.end());\n      if constexpr (std::is_default_constructible_v<Resizable>) {\n        if (dest.capacity() <= Resizable().capacity()) {\n          dest.reserve(new_num_elements);\n          return;\n        }\n      }\n      dest.reserve(UnsignedClamp(dest.capacity() + dest.capacity() / 2,\n                                 new_num_elements, dest.max_size()));\n    }\n    RIEGELI_ASSUME_GE(dest.capacity(), new_num_elements);\n  }\n};\n\n}  // namespace vector_writer_internal\n\n// Template parameter independent part of `VectorWriter`.\nusing VectorWriterBase = ResizableWriterBase;\n\n// A `Writer` which writes to `std::vector<T, Alloc>` including\n// `UninitializedVector<T>`, `absl::InlinedVector<T, inlined_size, Alloc>`,\n// including `UninitializedInlinedVector<T, inlined_size>`, or a similar\n// type supported by `VectorResizableTraits`, called `VectorType` here.\n// Its value type must be trivially copyable, usually `char`.\n//\n// If `Options::append()` is `false` (the default), replaces existing contents\n// of the `VectorType`, clearing it first. If `Options::append()` is `true`,\n// appends to existing contents of the `VectorType`.\n//\n// It supports `Seek()` and `ReadMode()`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `VectorType` being written to. `Dest` must support\n// `Dependency<VectorType*, Dest>`, e.g. `VectorType*` (not owned, default),\n// `VectorType` (owned), `Any<VectorType*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as\n// `UninitializedVector<char>` if there are no constructor arguments or the only\n// argument is `Options`, otherwise as `TargetT` of the type of the first\n// constructor argument, except that CTAD is deleted if the first constructor\n// argument is a `VectorType&` or `const VectorType&` (to avoid writing to an\n// unintentionally separate copy of an existing object).\n//\n// The `VectorType` must not be accessed until the `VectorWriter` is closed or\n// no longer used, except that it is allowed to read the `VectorType`\n// immediately after `Flush()`.\n//\n// `VectorWriter` with `UninitializedVector<char>`\n// or `UninitializedInlinedVector<char, inlined_size>` is more efficient\n// than `StringWriter` because the destination can be resized with uninitialized\n// space. `CompactStringWriter` can also be used for this purpose.\ntemplate <typename Dest = UninitializedVector<char>*>\nclass VectorWriter\n    : public ResizableWriter<\n          vector_writer_internal::VectorResizableTraits<std::remove_pointer_t<\n              typename Dependency<void*, TargetT<Dest>>::Subhandle>>,\n          Dest> {\n public:\n  using VectorWriter::ResizableWriter::ResizableWriter;\n\n  VectorWriter(VectorWriter&& that) = default;\n  VectorWriter& operator=(VectorWriter&& that) = default;\n};\n\nexplicit VectorWriter(Closed) -> VectorWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit VectorWriter(Dest&& dest, VectorWriterBase::Options options =\n                                       VectorWriterBase::Options())\n    -> VectorWriter<std::conditional_t<\n        std::conjunction_v<\n            std::is_lvalue_reference<Dest>,\n            std::is_convertible<std::remove_reference_t<Dest>*,\n                                const std::remove_pointer_t<typename Dependency<\n                                    void*, TargetT<Dest>>::Subhandle>*>>,\n        DeleteCtad<Dest&&>, TargetT<Dest>>>;\nexplicit VectorWriter(\n    VectorWriterBase::Options options = VectorWriterBase::Options())\n    -> VectorWriter<UninitializedVector<char>>;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_VECTOR_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/wrapping_backward_writer.cc",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/wrapping_backward_writer.h\"\n\n#include <stddef.h>\n\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\nvoid WrappingBackwardWriterBase::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    BackwardWriter& dest = *DestWriter();\n    SyncBuffer(dest);\n  }\n  BackwardWriter::Done();\n}\n\nabsl::Status WrappingBackwardWriterBase::AnnotateStatusImpl(\n    absl::Status status) {\n  // Fully delegate annotations to `*DestWriter()`.\n  if (is_open()) {\n    BackwardWriter& dest = *DestWriter();\n    SyncBuffer(dest);\n    status = dest.AnnotateStatus(std::move(status));\n    MakeBuffer(dest);\n  }\n  return status;\n}\n\nbool WrappingBackwardWriterBase::PushSlow(size_t min_length,\n                                          size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of BackwardWriter::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  BackwardWriter& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool push_ok = dest.Push(min_length, recommended_length);\n  MakeBuffer(dest);\n  return push_ok;\n}\n\nbool WrappingBackwardWriterBase::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  return WriteInternal(src);\n}\n\nbool WrappingBackwardWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool WrappingBackwardWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  return WriteInternal(src);\n}\n\nbool WrappingBackwardWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool WrappingBackwardWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  return WriteInternal(src);\n}\n\nbool WrappingBackwardWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool WrappingBackwardWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of BackwardWriter::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  return WriteInternal(src);\n}\n\ntemplate <typename Src>\ninline bool WrappingBackwardWriterBase::WriteInternal(Src&& src) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  BackwardWriter& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool write_ok = dest.Write(std::forward<Src>(src));\n  MakeBuffer(dest);\n  return write_ok;\n}\n\nbool WrappingBackwardWriterBase::SupportsTruncate() {\n  BackwardWriter* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsTruncate();\n}\n\nbool WrappingBackwardWriterBase::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  BackwardWriter& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool truncate_ok = dest.Truncate(new_size);\n  MakeBuffer(dest);\n  return truncate_ok;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/wrapping_backward_writer.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_WRAPPING_BACKWARD_WRITER_H_\n#define RIEGELI_BYTES_WRAPPING_BACKWARD_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `WrappingBackwardWriter`.\nclass WrappingBackwardWriterBase : public BackwardWriter {\n public:\n  // Returns the original `BackwardWriter`. Unchanged by `Close()`.\n  virtual BackwardWriter* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsTruncate() override;\n\n protected:\n  using BackwardWriter::BackwardWriter;\n\n  WrappingBackwardWriterBase(WrappingBackwardWriterBase&& that) noexcept;\n  WrappingBackwardWriterBase& operator=(\n      WrappingBackwardWriterBase&& that) noexcept;\n\n  void Initialize(BackwardWriter* dest);\n\n  // Sets cursor of `dest` to cursor of `*this`.\n  void SyncBuffer(BackwardWriter& dest);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `dest`. Fails `*this`\n  // if `dest` failed.\n  void MakeBuffer(BackwardWriter& dest);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using BackwardWriter::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool TruncateImpl(Position new_size) override;\n\n private:\n  // This template is defined and used only in wrapping_backward_writer.cc.\n  template <typename Src>\n  bool WriteInternal(Src&& src);\n\n  // Invariants if `ok()`:\n  //   `start() == DestWriter()->start()`\n  //   `limit() == DestWriter()->limit()`\n  //   `start_pos() == DestWriter()->start_pos()`\n};\n\n// A `BackwardWriter` which juts writes to another `BackwardWriter`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the original `BackwardWriter`. `Dest` must support\n// `Dependency<BackwardWriter*, Dest>`, e.g.\n// `BackwardWriter*` (not owned, default),\n// `ChainBackwardWriter<>` (owned), `std::unique_ptr<BackwardWriter>` (owned),\n// `Any<BackwardWriter*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The original `BackwardWriter` must not be accessed until the\n// `WrappingBackwardWriter` is closed or no longer used, except that it is\n// allowed to read the destination of the original `BackwardWriter` immediately\n// after `Flush()`.\ntemplate <typename Dest = BackwardWriter*>\nclass WrappingBackwardWriter : public WrappingBackwardWriterBase {\n public:\n  // Creates a closed `WrappingBackwardWriter`.\n  explicit WrappingBackwardWriter(Closed) noexcept\n      : WrappingBackwardWriterBase(kClosed) {}\n\n  // Will write to the original `BackwardWriter` provided by `dest`.\n  explicit WrappingBackwardWriter(Initializer<Dest> dest);\n\n  WrappingBackwardWriter(WrappingBackwardWriter&& that) = default;\n  WrappingBackwardWriter& operator=(WrappingBackwardWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `WrappingBackwardWriter`.\n  // This avoids constructing a temporary `WrappingBackwardWriter` and moving\n  // from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest);\n\n  // Returns the object providing and possibly owning the original\n  // `BackwardWriter`. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  BackwardWriter* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the original `BackwardWriter`.\n  MovingDependency<BackwardWriter*, Dest, Mover> dest_;\n};\n\nexplicit WrappingBackwardWriter(Closed)\n    -> WrappingBackwardWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit WrappingBackwardWriter(Dest&& dest)\n    -> WrappingBackwardWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline WrappingBackwardWriterBase::WrappingBackwardWriterBase(\n    WrappingBackwardWriterBase&& that) noexcept\n    : BackwardWriter(static_cast<BackwardWriter&&>(that)) {}\n\ninline WrappingBackwardWriterBase& WrappingBackwardWriterBase::operator=(\n    WrappingBackwardWriterBase&& that) noexcept {\n  BackwardWriter::operator=(static_cast<BackwardWriter&&>(that));\n  return *this;\n}\n\ninline void WrappingBackwardWriterBase::Initialize(BackwardWriter* dest) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of WrappingBackwardWriter: \"\n         \"null BackwardWriter pointer\";\n  MakeBuffer(*dest);\n}\n\ninline void WrappingBackwardWriterBase::SyncBuffer(BackwardWriter& dest) {\n  dest.set_cursor(cursor());\n}\n\ninline void WrappingBackwardWriterBase::MakeBuffer(BackwardWriter& dest) {\n  set_buffer(dest.limit(), dest.start_to_limit(), dest.start_to_cursor());\n  set_start_pos(dest.start_pos());\n  if (ABSL_PREDICT_FALSE(!dest.ok())) FailWithoutAnnotation(dest.status());\n}\n\ntemplate <typename Dest>\nclass WrappingBackwardWriter<Dest>::Mover {\n public:\n  static auto member() { return &WrappingBackwardWriter::dest_; }\n\n  explicit Mover(WrappingBackwardWriter& self, WrappingBackwardWriter& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `dest_` is not moved yet so `dest_` is taken from `that`.\n    if (uses_buffer_) self.SyncBuffer(*that.dest_);\n  }\n\n  void Done(WrappingBackwardWriter& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.dest_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Dest>\ninline WrappingBackwardWriter<Dest>::WrappingBackwardWriter(\n    Initializer<Dest> dest)\n    : dest_(std::move(dest)) {\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\ninline void WrappingBackwardWriter<Dest>::Reset(Closed) {\n  WrappingBackwardWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void WrappingBackwardWriter<Dest>::Reset(Initializer<Dest> dest) {\n  WrappingBackwardWriterBase::Reset();\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\nvoid WrappingBackwardWriter<Dest>::Done() {\n  WrappingBackwardWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(dest_->status());\n    }\n  }\n}\n\ntemplate <typename Dest>\nvoid WrappingBackwardWriter<Dest>::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (dest_.IsOwning()) {\n    SyncBuffer(*dest_);\n    dest_->SetWriteSizeHint(write_size_hint);\n    MakeBuffer(*dest_);\n  }\n}\n\ntemplate <typename Dest>\nbool WrappingBackwardWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer(*dest_);\n  bool flush_ok = true;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    flush_ok = dest_->Flush(flush_type);\n  }\n  MakeBuffer(*dest_);\n  return flush_ok;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_WRAPPING_BACKWARD_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/wrapping_reader.cc",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/wrapping_reader.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid WrappingReaderBase::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    Reader& src = *SrcReader();\n    SyncBuffer(src);\n  }\n  Reader::Done();\n}\n\nabsl::Status WrappingReaderBase::AnnotateStatusImpl(absl::Status status) {\n  // Fully delegate annotations to `*SrcReader()`.\n  if (is_open()) {\n    Reader& src = *SrcReader();\n    SyncBuffer(src);\n    status = src.AnnotateStatus(std::move(status));\n    MakeBuffer(src);\n  }\n  return status;\n}\n\nbool WrappingReaderBase::PullSlow(size_t min_length,\n                                  size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::PullSlow(): \"\n         \"enough data available, use Pull() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool pull_ok = src.Pull(min_length, recommended_length);\n  MakeBuffer(src);\n  return pull_ok;\n}\n\nbool WrappingReaderBase::ReadSlow(size_t length, char* dest) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of Reader::ReadSlow(char*): \"\n         \"enough data available, use Read(char*) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool read_ok = src.Read(length, dest);\n  MakeBuffer(src);\n  return read_ok;\n}\n\nbool WrappingReaderBase::ReadSlow(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"Chain size overflow\";\n  return ReadInternal(length, dest);\n}\n\nbool WrappingReaderBase::ReadSlow(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"Cord size overflow\";\n  return ReadInternal(length, dest);\n}\n\ntemplate <typename Dest>\ninline bool WrappingReaderBase::ReadInternal(size_t length, Dest& dest) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool read_ok = src.ReadAndAppend(length, dest);\n  MakeBuffer(src);\n  return read_ok;\n}\n\nbool WrappingReaderBase::CopySlow(Position length, Writer& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool copy_ok = src.Copy(length, dest);\n  MakeBuffer(src);\n  return copy_ok;\n}\n\nbool WrappingReaderBase::CopySlow(size_t length, BackwardWriter& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(BackwardWriter&): \"\n         \"enough data available, use Copy(BackwardWriter&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool copy_ok = src.Copy(length, dest);\n  MakeBuffer(src);\n  return copy_ok;\n}\n\nbool WrappingReaderBase::ReadSomeSlow(size_t max_length, char* dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"nothing to read, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"some data available, use ReadSome(char*) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool read_ok = src.ReadSome(max_length, dest);\n  MakeBuffer(src);\n  return read_ok;\n}\n\nbool WrappingReaderBase::CopySomeSlow(size_t max_length, Writer& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"nothing to copy, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"some data available, use CopySome(Writer&) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool copy_ok = src.CopySome(max_length, dest);\n  MakeBuffer(src);\n  return copy_ok;\n}\n\nvoid WrappingReaderBase::ReadHintSlow(size_t min_length,\n                                      size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::ReadHintSlow(): \"\n         \"enough data available, use ReadHint() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  src.ReadHint(min_length, recommended_length);\n  MakeBuffer(src);\n}\n\nbool WrappingReaderBase::ToleratesReadingAhead() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->ToleratesReadingAhead();\n}\n\nbool WrappingReaderBase::SupportsRandomAccess() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRandomAccess();\n}\n\nbool WrappingReaderBase::SupportsRewind() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRewind();\n}\n\nbool WrappingReaderBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of Reader::SeekSlow(): \"\n         \"position in the buffer, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const bool seek_ok = src.Seek(new_pos);\n  MakeBuffer(src);\n  return seek_ok;\n}\n\nbool WrappingReaderBase::SupportsSize() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsSize();\n}\n\nstd::optional<Position> WrappingReaderBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  Reader& src = *SrcReader();\n  SyncBuffer(src);\n  const std::optional<Position> size = src.Size();\n  MakeBuffer(src);\n  return size;\n}\n\nbool WrappingReaderBase::SupportsNewReader() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsNewReader();\n}\n\nstd::unique_ptr<Reader> WrappingReaderBase::NewReaderImpl(\n    Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point\n  // if `SrcReader()->SupportsNewReader()`.\n  Reader& src = *SrcReader();\n  std::unique_ptr<Reader> reader = src.NewReader(initial_pos);\n  if (ABSL_PREDICT_FALSE(reader == nullptr)) {\n    FailWithoutAnnotation(src.status());\n  }\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/wrapping_reader.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_WRAPPING_READER_H_\n#define RIEGELI_BYTES_WRAPPING_READER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nclass BackwardWriter;\nclass Writer;\n\n// Template parameter independent part of `WrappingReader`.\nclass WrappingReaderBase : public Reader {\n public:\n  // Returns the original `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRandomAccess() override;\n  bool SupportsRewind() override;\n  bool SupportsSize() override;\n  bool SupportsNewReader() override;\n\n protected:\n  using Reader::Reader;\n\n  WrappingReaderBase(WrappingReaderBase&& that) noexcept;\n  WrappingReaderBase& operator=(WrappingReaderBase&& that) noexcept;\n\n  void Initialize(Reader* src);\n\n  // Sets cursor of `src` to cursor of `*this`.\n  void SyncBuffer(Reader& src);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `src`. Fails `*this`\n  // if `src` failed.\n  void MakeBuffer(Reader& src);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PullSlow(size_t min_length, size_t recommended_length) override;\n  using Reader::ReadSlow;\n  bool ReadSlow(size_t length, char* dest) override;\n  bool ReadSlow(size_t length, Chain& dest) override;\n  bool ReadSlow(size_t length, absl::Cord& dest) override;\n  using Reader::CopySlow;\n  bool CopySlow(Position length, Writer& dest) override;\n  bool CopySlow(size_t length, BackwardWriter& dest) override;\n  using Reader::ReadSomeSlow;\n  bool ReadSomeSlow(size_t max_length, char* dest) override;\n  using Reader::CopySomeSlow;\n  bool CopySomeSlow(size_t max_length, Writer& dest) override;\n  void ReadHintSlow(size_t min_length, size_t recommended_length) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  // This template is defined and used only in wrapping_reader.cc.\n  template <typename Dest>\n  bool ReadInternal(size_t length, Dest& dest);\n\n  // Invariants if `is_open()`:\n  //   `start() == SrcReader()->start()`\n  //   `limit() == SrcReader()->limit()`\n  //   `limit_pos() == SrcReader()->limit_pos()`\n};\n\n// A `Reader` which just reads from another `Reader`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the original `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The original `Reader` must not be accessed until the `WrappingReader` is\n// closed or no longer used.\ntemplate <typename Src = Reader*>\nclass WrappingReader : public WrappingReaderBase {\n public:\n  // Creates a closed `WrappingReader`.\n  explicit WrappingReader(Closed) noexcept : WrappingReaderBase(kClosed) {}\n\n  // Will read from the original `Reader` provided by `src`.\n  explicit WrappingReader(Initializer<Src> src);\n\n  WrappingReader(WrappingReader&& that) = default;\n  WrappingReader& operator=(WrappingReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `WrappingReader`. This\n  // avoids constructing a temporary `WrappingReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src);\n\n  // Returns the object providing and possibly owning the original `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  void VerifyEndImpl() override;\n  bool SyncImpl(SyncType sync_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the original `Reader`.\n  MovingDependency<Reader*, Src, Mover> src_;\n};\n\nexplicit WrappingReader(Closed) -> WrappingReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit WrappingReader(Src&& src) -> WrappingReader<TargetT<Src>>;\n\n// Implementation details follow.\n\ninline WrappingReaderBase::WrappingReaderBase(\n    WrappingReaderBase&& that) noexcept\n    : Reader(static_cast<Reader&&>(that)) {}\n\ninline WrappingReaderBase& WrappingReaderBase::operator=(\n    WrappingReaderBase&& that) noexcept {\n  Reader::operator=(static_cast<Reader&&>(that));\n  return *this;\n}\n\ninline void WrappingReaderBase::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of WrappingReader: null Reader pointer\";\n  MakeBuffer(*src);\n}\n\ninline void WrappingReaderBase::SyncBuffer(Reader& src) {\n  src.set_cursor(cursor());\n}\n\ninline void WrappingReaderBase::MakeBuffer(Reader& src) {\n  set_buffer(src.start(), src.start_to_limit(), src.start_to_cursor());\n  set_limit_pos(src.limit_pos());\n  if (ABSL_PREDICT_FALSE(!src.ok())) FailWithoutAnnotation(src.status());\n}\n\ntemplate <typename Src>\nclass WrappingReader<Src>::Mover {\n public:\n  static auto member() { return &WrappingReader::src_; }\n\n  explicit Mover(WrappingReader& self, WrappingReader& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `src_` is not moved yet so `src_` is taken from `that`.\n    if (uses_buffer_) self.SyncBuffer(*that.src_);\n  }\n\n  void Done(WrappingReader& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.src_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Src>\ninline WrappingReader<Src>::WrappingReader(Initializer<Src> src)\n    : src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void WrappingReader<Src>::Reset(Closed) {\n  WrappingReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void WrappingReader<Src>::Reset(Initializer<Src> src) {\n  WrappingReaderBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\nvoid WrappingReader<Src>::Done() {\n  WrappingReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(src_->status());\n    }\n  }\n}\n\ntemplate <typename Src>\nvoid WrappingReader<Src>::SetReadAllHintImpl(bool read_all_hint) {\n  WrappingReaderBase::SetReadAllHintImpl(read_all_hint);\n  if (src_.IsOwning()) {\n    SyncBuffer(*src_);\n    src_->SetReadAllHint(read_all_hint);\n    MakeBuffer(*src_);\n  }\n}\n\ntemplate <typename Src>\nvoid WrappingReader<Src>::VerifyEndImpl() {\n  if (!src_.IsOwning()) {\n    WrappingReaderBase::VerifyEndImpl();\n  } else if (ABSL_PREDICT_TRUE(ok())) {\n    SyncBuffer(*src_);\n    src_->VerifyEnd();\n    MakeBuffer(*src_);\n  }\n}\n\ntemplate <typename Src>\nbool WrappingReader<Src>::SyncImpl(SyncType sync_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer(*src_);\n  bool sync_ok = true;\n  if (sync_type != SyncType::kFromObject || src_.IsOwning()) {\n    sync_ok = src_->Sync(sync_type);\n  }\n  MakeBuffer(*src_);\n  return sync_ok;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_WRAPPING_READER_H_\n"
  },
  {
    "path": "riegeli/bytes/wrapping_writer.cc",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/wrapping_writer.h\"\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nvoid WrappingWriterBase::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    Writer& dest = *DestWriter();\n    SyncBuffer(dest);\n  }\n  Writer::Done();\n}\n\nabsl::Status WrappingWriterBase::AnnotateStatusImpl(absl::Status status) {\n  // Fully delegate annotations to `*DestWriter()`.\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    SyncBuffer(dest);\n    status = dest.AnnotateStatus(std::move(status));\n    MakeBuffer(dest);\n  }\n  return status;\n}\n\nbool WrappingWriterBase::PushSlow(size_t min_length,\n                                  size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool push_ok = dest.Push(min_length, recommended_length);\n  MakeBuffer(dest);\n  return push_ok;\n}\n\nbool WrappingWriterBase::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of Writer::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  return WriteInternal(src);\n}\n\nbool WrappingWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool WrappingWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  return WriteInternal(src);\n}\n\nbool WrappingWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool WrappingWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  return WriteInternal(src);\n}\n\nbool WrappingWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool WrappingWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  return WriteInternal(src);\n}\n\ntemplate <typename Src>\ninline bool WrappingWriterBase::WriteInternal(Src&& src) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool write_ok = dest.Write(std::forward<Src>(src));\n  MakeBuffer(dest);\n  return write_ok;\n}\n\nbool WrappingWriterBase::SupportsRandomAccess() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsRandomAccess();\n}\n\nbool WrappingWriterBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of Writer::SeekSlow(): \"\n         \"position unchanged, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool seek_ok = dest.Seek(new_pos);\n  MakeBuffer(dest);\n  return seek_ok;\n}\n\nstd::optional<Position> WrappingWriterBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  const std::optional<Position> size = dest.Size();\n  MakeBuffer(dest);\n  return size;\n}\n\nbool WrappingWriterBase::SupportsTruncate() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsTruncate();\n}\n\nbool WrappingWriterBase::TruncateImpl(Position new_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  const bool truncate_ok = dest.Truncate(new_size);\n  MakeBuffer(dest);\n  return truncate_ok;\n}\n\nbool WrappingWriterBase::SupportsReadMode() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsReadMode();\n}\n\nReader* WrappingWriterBase::ReadModeImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  Writer& dest = *DestWriter();\n  SyncBuffer(dest);\n  Reader* const reader = dest.ReadMode(initial_pos);\n  MakeBuffer(dest);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/wrapping_writer.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_WRAPPING_WRITER_H_\n#define RIEGELI_BYTES_WRAPPING_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nclass Reader;\n\n// Template parameter independent part of `WrappingWriter`.\nclass WrappingWriterBase : public Writer {\n public:\n  // Returns the original `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsRandomAccess() override;\n  bool SupportsTruncate() override;\n  bool SupportsReadMode() override;\n\n protected:\n  using Writer::Writer;\n\n  WrappingWriterBase(WrappingWriterBase&& that) noexcept;\n  WrappingWriterBase& operator=(WrappingWriterBase&& that) noexcept;\n\n  void Initialize(Writer* dest);\n\n  // Sets cursor of `dest` to cursor of `*this`.\n  void SyncBuffer(Writer& dest);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `dest`. Fails `*this`\n  // if `dest` failed.\n  void MakeBuffer(Writer& dest);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  bool TruncateImpl(Position new_size) override;\n  Reader* ReadModeImpl(Position initial_pos) override;\n\n private:\n  // This template is defined and used only in wrapping_writer.cc.\n  template <typename Src>\n  bool WriteInternal(Src&& src);\n\n  // Invariants if `ok()`:\n  //   `start() == DestWriter()->start()`\n  //   `limit() == DestWriter()->limit()`\n  //   `start_pos() == DestWriter()->start_pos()`\n};\n\n// A `Writer` which just writes to another `Writer`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the original `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The original `Writer` must not be accessed until the `WrappingWriter` is\n// closed or no longer used, except that it is allowed to read the destination\n// of the original `Writer` immediately after `Flush()`.\ntemplate <typename Dest = Writer*>\nclass WrappingWriter : public WrappingWriterBase {\n public:\n  // Creates a closed `WrappingWriter`.\n  explicit WrappingWriter(Closed) noexcept : WrappingWriterBase(kClosed) {}\n\n  // Will write to the original `Writer` provided by `dest`.\n  explicit WrappingWriter(Initializer<Dest> dest);\n\n  WrappingWriter(WrappingWriter&& that) = default;\n  WrappingWriter& operator=(WrappingWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `WrappingWriter`. This\n  // avoids constructing a temporary `WrappingWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest);\n\n  // Returns the object providing and possibly owning the original `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the original `Writer`.\n  MovingDependency<Writer*, Dest, Mover> dest_;\n};\n\nexplicit WrappingWriter(Closed) -> WrappingWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit WrappingWriter(Dest&& dest) -> WrappingWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline WrappingWriterBase::WrappingWriterBase(\n    WrappingWriterBase&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)) {}\n\ninline WrappingWriterBase& WrappingWriterBase::operator=(\n    WrappingWriterBase&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  return *this;\n}\n\ninline void WrappingWriterBase::Initialize(Writer* dest) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of WrappingWriter: null Writer pointer\";\n  MakeBuffer(*dest);\n}\n\ninline void WrappingWriterBase::SyncBuffer(Writer& dest) {\n  dest.set_cursor(cursor());\n}\n\ninline void WrappingWriterBase::MakeBuffer(Writer& dest) {\n  set_buffer(dest.start(), dest.start_to_limit(), dest.start_to_cursor());\n  set_start_pos(dest.start_pos());\n  if (ABSL_PREDICT_FALSE(!dest.ok())) FailWithoutAnnotation(dest.status());\n}\n\ntemplate <typename Dest>\nclass WrappingWriter<Dest>::Mover {\n public:\n  static auto member() { return &WrappingWriter::dest_; }\n\n  explicit Mover(WrappingWriter& self, WrappingWriter& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `dest_` is not moved yet so `dest_` is taken from `that`.\n    if (uses_buffer_) self.SyncBuffer(*that.dest_);\n  }\n\n  void Done(WrappingWriter& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.dest_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Dest>\ninline WrappingWriter<Dest>::WrappingWriter(Initializer<Dest> dest)\n    : dest_(std::move(dest)) {\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\ninline void WrappingWriter<Dest>::Reset(Closed) {\n  WrappingWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void WrappingWriter<Dest>::Reset(Initializer<Dest> dest) {\n  WrappingWriterBase::Reset();\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\nvoid WrappingWriter<Dest>::Done() {\n  WrappingWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(dest_->status());\n    }\n  }\n}\n\ntemplate <typename Dest>\nvoid WrappingWriter<Dest>::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (dest_.IsOwning()) {\n    SyncBuffer(*dest_);\n    dest_->SetWriteSizeHint(write_size_hint);\n    MakeBuffer(*dest_);\n  }\n}\n\ntemplate <typename Dest>\nbool WrappingWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer(*dest_);\n  bool flush_ok = true;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    flush_ok = dest_->Flush(flush_type);\n  }\n  MakeBuffer(*dest_);\n  return flush_ok;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_WRAPPING_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/write.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_WRITE_H_\n#define RIEGELI_BYTES_WRITE_H_\n\n#include <stddef.h>\n\n#include <tuple>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/stringify.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// Combines creating a `Writer` / `BackwardWriter` (optionally), calling\n// `Write()`, and `Close()` (if the `Writer` / `BackwardWriter` is owned).\n//\n// The last argument is the destination of some type `Dest`. The remaining\n// arguments are the values.\n//\n// `Dest` specifies the type of the object providing and possibly owning the\n// `Writer` / `BackwardWriter`. `Dest` must support\n// `DependencyRef<Writer*, Dest>`, e.g. `Writer&` (not owned),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `AnyRef<Writer*>` (maybe owned). Analogously for `BackwardWriter`.\n\ntemplate <\n    typename... Args,\n    std::enable_if_t<\n        std::conjunction_v<\n            TargetRefSupportsDependency<Writer*, GetTypeFromEndT<1, Args...>>,\n            TupleElementsSatisfy<RemoveTypesFromEndT<1, Args&&...>,\n                                 IsStringifiable>>,\n        int> = 0>\nabsl::Status Write(Args&&... args);\ntemplate <\n    typename... Args,\n    std::enable_if_t<std::conjunction_v<\n                         TargetRefSupportsDependency<\n                             BackwardWriter*, GetTypeFromEndT<1, Args...>>,\n                         TupleElementsSatisfy<RemoveTypesFromEndT<1, Args&&...>,\n                                              IsStringifiable>>,\n                     int> = 0>\nabsl::Status Write(Args&&... args);\n\n// Implementation details follow.\n\nnamespace write_internal {\n\ntemplate <typename... Srcs, typename Dest>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline absl::Status WriteInternal(\n    ABSL_ATTRIBUTE_UNUSED std::tuple<Srcs...> srcs, Dest&& dest) {\n  DependencyRef<Writer*, Dest> dest_dep(std::forward<Dest>(dest));\n  if constexpr (HasStringifiedSize<Srcs...>::value) {\n    if (dest_dep.IsOwning()) {\n      dest_dep->SetWriteSizeHint(std::apply(\n          [](const Srcs&... srcs) { return riegeli::StringifiedSize(srcs...); },\n          srcs));\n    }\n  }\n  absl::Status status;\n  if (ABSL_PREDICT_FALSE(!std::apply(\n          [&](Srcs&&... srcs) {\n            return dest_dep->Write(std::forward<Srcs>(srcs)...);\n          },\n          std::move(srcs)))) {\n    status = dest_dep->status();\n  }\n  if (dest_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_dep->Close())) {\n      status.Update(dest_dep->status());\n    }\n  }\n  return status;\n}\n\ntemplate <typename... Srcs, typename Dest>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline absl::Status BackwardWriteInternal(\n    ABSL_ATTRIBUTE_UNUSED std::tuple<Srcs...> srcs, Dest&& dest) {\n  DependencyRef<BackwardWriter*, Dest> dest_dep(std::forward<Dest>(dest));\n  if constexpr (HasStringifiedSize<Srcs...>::value) {\n    if (dest_dep.IsOwning()) {\n      dest_dep->SetWriteSizeHint(std::apply(\n          [](const Srcs&... srcs) { return riegeli::StringifiedSize(srcs...); },\n          srcs));\n    }\n  }\n  absl::Status status;\n  if (ABSL_PREDICT_FALSE(!std::apply(\n          [&](Srcs&&... srcs) {\n            return dest_dep->Write(std::forward<Srcs>(srcs)...);\n          },\n          std::move(srcs)))) {\n    status = dest_dep->status();\n  }\n  if (dest_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_dep->Close())) {\n      status.Update(dest_dep->status());\n    }\n  }\n  return status;\n}\n\n}  // namespace write_internal\n\ntemplate <\n    typename... Args,\n    std::enable_if_t<\n        std::conjunction_v<\n            TargetRefSupportsDependency<Writer*, GetTypeFromEndT<1, Args...>>,\n            TupleElementsSatisfy<RemoveTypesFromEndT<1, Args&&...>,\n                                 IsStringifiable>>,\n        int>>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline absl::Status Write(Args&&... args) {\n  return write_internal::WriteInternal(\n      RemoveFromEnd<1>(std::forward<Args>(args)...),\n      GetFromEnd<1>(std::forward<Args>(args)...));\n}\n\ntemplate <\n    typename... Args,\n    std::enable_if_t<std::conjunction_v<\n                         TargetRefSupportsDependency<\n                             BackwardWriter*, GetTypeFromEndT<1, Args...>>,\n                         TupleElementsSatisfy<RemoveTypesFromEndT<1, Args&&...>,\n                                              IsStringifiable>>,\n                     int>>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline absl::Status Write(Args&&... args) {\n  return write_internal::BackwardWriteInternal(\n      RemoveFromEnd<1>(std::forward<Args>(args)...),\n      GetFromEnd<1>(std::forward<Args>(args)...));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_WRITE_H_\n"
  },
  {
    "path": "riegeli/bytes/write_int_internal.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/write_int_internal.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <array>\n#include <cstring>\n#include <limits>\n#include <type_traits>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/numeric/bits.h\"\n#include \"absl/numeric/int128.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n\nnamespace riegeli::write_int_internal {\n\nnamespace {\n\n// `WriteDec{1,2}Impl()` write a fixed number of digits.\n\ninline char* WriteDec1Impl(uint32_t src, char* dest) {\n  RIEGELI_ASSERT_LT(src, 10u)\n      << \"Failed precondition of WriteDec1Impl(): value too large\";\n  *dest = '0' + static_cast<char>(src);\n  return dest + 1;\n}\n\ninline char* WriteDec2Impl(uint32_t src, char* dest) {\n  RIEGELI_ASSERT_LT(src, 100u)\n      << \"Failed precondition of WriteDec2Impl(): value too large\";\n  static constexpr char kTwoDigits[100][2] = {\n      {'0', '0'}, {'0', '1'}, {'0', '2'}, {'0', '3'}, {'0', '4'}, {'0', '5'},\n      {'0', '6'}, {'0', '7'}, {'0', '8'}, {'0', '9'}, {'1', '0'}, {'1', '1'},\n      {'1', '2'}, {'1', '3'}, {'1', '4'}, {'1', '5'}, {'1', '6'}, {'1', '7'},\n      {'1', '8'}, {'1', '9'}, {'2', '0'}, {'2', '1'}, {'2', '2'}, {'2', '3'},\n      {'2', '4'}, {'2', '5'}, {'2', '6'}, {'2', '7'}, {'2', '8'}, {'2', '9'},\n      {'3', '0'}, {'3', '1'}, {'3', '2'}, {'3', '3'}, {'3', '4'}, {'3', '5'},\n      {'3', '6'}, {'3', '7'}, {'3', '8'}, {'3', '9'}, {'4', '0'}, {'4', '1'},\n      {'4', '2'}, {'4', '3'}, {'4', '4'}, {'4', '5'}, {'4', '6'}, {'4', '7'},\n      {'4', '8'}, {'4', '9'}, {'5', '0'}, {'5', '1'}, {'5', '2'}, {'5', '3'},\n      {'5', '4'}, {'5', '5'}, {'5', '6'}, {'5', '7'}, {'5', '8'}, {'5', '9'},\n      {'6', '0'}, {'6', '1'}, {'6', '2'}, {'6', '3'}, {'6', '4'}, {'6', '5'},\n      {'6', '6'}, {'6', '7'}, {'6', '8'}, {'6', '9'}, {'7', '0'}, {'7', '1'},\n      {'7', '2'}, {'7', '3'}, {'7', '4'}, {'7', '5'}, {'7', '6'}, {'7', '7'},\n      {'7', '8'}, {'7', '9'}, {'8', '0'}, {'8', '1'}, {'8', '2'}, {'8', '3'},\n      {'8', '4'}, {'8', '5'}, {'8', '6'}, {'8', '7'}, {'8', '8'}, {'8', '9'},\n      {'9', '0'}, {'9', '1'}, {'9', '2'}, {'9', '3'}, {'9', '4'}, {'9', '5'},\n      {'9', '6'}, {'9', '7'}, {'9', '8'}, {'9', '9'}};\n  std::memcpy(dest, kTwoDigits[src], 2);\n  return dest + 2;\n}\n\n// `WriteDecImpl()` writes at least `width` digits.\n\n// Inline to optimize for a constant `width`.\nABSL_ATTRIBUTE_ALWAYS_INLINE\ninline char* WriteDecImpl(uint32_t src, char* dest, size_t width) {\n  uint32_t digits;\n\n  if (src < 100 && width <= 2) {\n    if (src >= 10 || width == 2) goto write_2_digits;\n    return WriteDec1Impl(src, dest);\n  }\n  if (src < 10'000 && width <= 4) {\n    if (src >= 1'000 || width == 4) goto write_4_digits;\n    digits = src / 100;\n    src %= 100;\n    dest = WriteDec1Impl(digits, dest);\n    goto write_2_digits;\n  }\n  if (src < 1'000'000 && width <= 6) {\n    if (src >= 100'000 || width == 6) goto write_6_digits;\n    digits = src / 10'000;\n    src %= 10'000;\n    dest = WriteDec1Impl(digits, dest);\n    goto write_4_digits;\n  }\n  if (src < 100'000'000 && width <= 8) {\n    if (src >= 10'000'000 || width == 8) goto write_8_digits;\n    digits = src / 1'000'000;\n    src %= 1'000'000;\n    dest = WriteDec1Impl(digits, dest);\n    goto write_6_digits;\n  }\n  if (src < 1'000'000'000 && width <= 9) {\n    digits = src / 100'000'000;\n    src %= 100'000'000;\n    dest = WriteDec1Impl(digits, dest);\n    goto write_8_digits;\n  }\n\n  if (width > 10) {\n    // Redundant condition suppresses gcc warning `-Wstringop-overflow`.\n    std::memset(dest, '0', width > 10 ? width - 10 : 0);\n    dest += width - 10;\n  }\n  digits = src / 100'000'000;\n  src %= 100'000'000;\n  dest = WriteDec2Impl(digits, dest);\nwrite_8_digits:\n  digits = src / 1'000'000;\n  src %= 1'000'000;\n  dest = WriteDec2Impl(digits, dest);\nwrite_6_digits:\n  digits = src / 10'000;\n  src %= 10'000;\n  dest = WriteDec2Impl(digits, dest);\nwrite_4_digits:\n  digits = src / 100;\n  src %= 100;\n  dest = WriteDec2Impl(digits, dest);\nwrite_2_digits:\n  return WriteDec2Impl(src, dest);\n}\n\n// Inline to optimize for a constant `width`.\nABSL_ATTRIBUTE_ALWAYS_INLINE\ninline char* WriteDecImpl(uint64_t src, char* dest, size_t width) {\n  if (src <= std::numeric_limits<uint32_t>::max()) {\n    return WriteDecImpl(IntCast<uint32_t>(src), dest, width);\n  }\n  // `src` needs at least 10 digits.\n  if (src >= 10'000'000'000 || width > 10) {\n    // `src` needs more than 10 digits.\n    const uint64_t over_10_digits = src / 10'000'000'000;\n    src %= 10'000'000'000;\n    dest = WriteDecImpl(IntCast<uint32_t>(over_10_digits), dest,\n                        SaturatingSub(width, size_t{10}));\n  }\n  // Now `src < 1e10`. Write `src` using exactly 10 digits. Leading zeros are\n  // needed for the case where the original `src` needed more than 10 digits or\n  // `width > 10`.\n  const uint32_t digits = IntCast<uint32_t>(src / 100'000'000);\n  src %= 100'000'000;\n  dest = WriteDec2Impl(digits, dest);\n  return WriteDecImpl(IntCast<uint32_t>(src), dest, 8);\n}\n\n// Inline to optimize for a constant `width`.\nABSL_ATTRIBUTE_ALWAYS_INLINE\ninline char* WriteDecImpl(absl::uint128 src, char* dest, size_t width) {\n  if (src <= std::numeric_limits<uint64_t>::max()) {\n    return WriteDecImpl(IntCast<uint64_t>(src), dest, width);\n  }\n  // `src` needs at least 20 digits.\n  constexpr absl::uint128 k1e20 = absl::MakeUint128(5, 0x6bc75e2d63100000);\n  RIEGELI_ASSERT_EQ(\n      k1e20, absl::uint128(10'000'000'000) * absl::uint128(10'000'000'000));\n  if (src >= k1e20 || width > 20) {\n    // `src` needs more than 20 digits.\n    const absl::uint128 over_20_digits = src / k1e20;\n    src %= k1e20;\n    dest = WriteDecImpl(IntCast<uint64_t>(over_20_digits), dest,\n                        SaturatingSub(width, size_t{20}));\n  }\n  // Now `src < 1e20`. Write `src` using exactly 20 digits. Leading zeros are\n  // needed for the case where the original `src` needed more than 20 digits or\n  // `width > 20`.\n  const uint32_t digits = IntCast<uint32_t>(src / 1'000'000'000'000'000'000);\n  src %= 1'000'000'000'000'000'000;\n  dest = WriteDec2Impl(digits, dest);\n  return WriteDecImpl(IntCast<uint64_t>(src), dest, 18);\n}\n\n}  // namespace\n\nchar* WriteDec(uint32_t src, char* dest) { return WriteDecImpl(src, dest, 0); }\n\nchar* WriteDec(uint64_t src, char* dest) { return WriteDecImpl(src, dest, 0); }\n\nchar* WriteDec(absl::uint128 src, char* dest) {\n  return WriteDecImpl(src, dest, 0);\n}\n\nchar* WriteDec(int32_t src, char* dest) {\n  uint32_t abs_value;\n  if (src >= 0) {\n    abs_value = UnsignedCast(src);\n  } else {\n    *dest = '-';\n    ++dest;\n    abs_value = NegatingUnsignedCast(src);\n  }\n  return WriteDec(abs_value, dest);\n}\n\nchar* WriteDec(int64_t src, char* dest) {\n  uint64_t abs_value;\n  if (src >= 0) {\n    abs_value = UnsignedCast(src);\n  } else {\n    *dest = '-';\n    ++dest;\n    abs_value = NegatingUnsignedCast(src);\n  }\n  return WriteDec(abs_value, dest);\n}\n\nchar* WriteDec(absl::int128 src, char* dest) {\n  absl::uint128 abs_value;\n  if (src >= 0) {\n    abs_value = UnsignedCast(src);\n  } else {\n    *dest = '-';\n    ++dest;\n    abs_value = NegatingUnsignedCast(src);\n  }\n  return WriteDec(abs_value, dest);\n}\n\nchar* WriteDec(uint32_t src, char* dest, size_t width) {\n  return WriteDecImpl(src, dest, width);\n}\n\nchar* WriteDec(uint64_t src, char* dest, size_t width) {\n  return WriteDecImpl(src, dest, width);\n}\n\nchar* WriteDec(absl::uint128 src, char* dest, size_t width) {\n  return WriteDecImpl(src, dest, width);\n}\n\nchar* WriteDec(int32_t src, char* dest, size_t width) {\n  uint32_t abs_value;\n  if (src >= 0) {\n    abs_value = UnsignedCast(src);\n  } else {\n    *dest = '-';\n    ++dest;\n    abs_value = NegatingUnsignedCast(src);\n    width = SaturatingSub(width, size_t{1});\n  }\n  return WriteDec(abs_value, dest, width);\n}\n\nchar* WriteDec(int64_t src, char* dest, size_t width) {\n  uint64_t abs_value;\n  if (src >= 0) {\n    abs_value = UnsignedCast(src);\n  } else {\n    *dest = '-';\n    ++dest;\n    abs_value = NegatingUnsignedCast(src);\n    width = SaturatingSub(width, size_t{1});\n  }\n  return WriteDec(abs_value, dest, width);\n}\n\nchar* WriteDec(absl::int128 src, char* dest, size_t width) {\n  absl::uint128 abs_value;\n  if (src >= 0) {\n    abs_value = UnsignedCast(src);\n  } else {\n    *dest = '-';\n    ++dest;\n    abs_value = NegatingUnsignedCast(src);\n    width = SaturatingSub(width, size_t{1});\n  }\n  return WriteDec(abs_value, dest, width);\n}\n\nchar* WriteDecBackward(uint32_t src, char* dest) {\n  while (src >= 100) {\n    const uint32_t digits = src % 100;\n    src /= 100;\n    dest -= 2;\n    WriteDec2Impl(digits, dest);\n  }\n  if (src >= 10) {\n    dest -= 2;\n    WriteDec2Impl(src, dest);\n  } else {\n    --dest;\n    WriteDec1Impl(src, dest);\n  }\n  return dest;\n}\n\nchar* WriteDecBackward(uint64_t src, char* dest) {\n  while (src > std::numeric_limits<uint32_t>::max()) {\n    const uint32_t digits = IntCast<uint32_t>(src % 100);\n    src /= 100;\n    dest -= 2;\n    WriteDec2Impl(digits, dest);\n  }\n  return WriteDecBackward(IntCast<uint32_t>(src), dest);\n}\n\nchar* WriteDecBackward(absl::uint128 src, char* dest) {\n  while (src > std::numeric_limits<uint64_t>::max()) {\n    const uint32_t digits = IntCast<uint32_t>(src % 100);\n    src /= 100;\n    dest -= 2;\n    WriteDec2Impl(digits, dest);\n  }\n  return WriteDecBackward(IntCast<uint64_t>(src), dest);\n}\n\nchar* WriteDecBackward(int32_t src, char* dest) {\n  if (src >= 0) {\n    return WriteDecBackward(UnsignedCast(src), dest);\n  } else {\n    dest = WriteDecBackward(NegatingUnsignedCast(src), dest);\n    --dest;\n    *dest = '-';\n    return dest;\n  }\n}\n\nchar* WriteDecBackward(int64_t src, char* dest) {\n  if (src >= 0) {\n    return WriteDecBackward(UnsignedCast(src), dest);\n  } else {\n    dest = WriteDecBackward(NegatingUnsignedCast(src), dest);\n    --dest;\n    *dest = '-';\n    return dest;\n  }\n}\n\nchar* WriteDecBackward(absl::int128 src, char* dest) {\n  if (src >= 0) {\n    return WriteDecBackward(UnsignedCast(src), dest);\n  } else {\n    dest = WriteDecBackward(NegatingUnsignedCast(src), dest);\n    --dest;\n    *dest = '-';\n    return dest;\n  }\n}\n\n// For `StringifiedSize(value)`, there is an entry for each\n// `absl::bit_width(value) / 2`.\ntemplate <typename T>\nstruct StringifiedSizeEntry {\n  using Size = std::conditional_t<(sizeof(T) < sizeof(size_t)), T, size_t>;\n  Size size;    // Number of digits at the upper bound of this entry.\n  T threshold;  // If `value < threshold`, there is one digit less.\n};\n\ntemplate <typename T>\nconstexpr size_t kNumStringifiedSizeEntries =\n    std::numeric_limits<T>::digits / 2 + 1;\n\ntemplate <typename T>\nconstexpr std::array<StringifiedSizeEntry<T>, kNumStringifiedSizeEntries<T>>\nMakeStringifiedSizeEntries() {\n  // Default construction of `entries` should not be needed, but without that\n  // GCC rejects this code in constexpr context for some reason.\n  std::array<StringifiedSizeEntry<T>, kNumStringifiedSizeEntries<T>> entries{};\n  typename StringifiedSizeEntry<T>::Size size = 1;\n  T threshold = 1;\n  T upper_bound = 3;\n  for (StringifiedSizeEntry<T>& entry : entries) {\n    entry.size = size;\n    entry.threshold = threshold;\n    if (size < std::numeric_limits<T>::digits10 + 1) {\n      // Some `absl::uint128` operations are not constexpr,\n      // hence multiplication is avoided.\n      upper_bound = (upper_bound << 2) + 3;\n      const T next_threshold = (threshold << 3) + (threshold << 1);\n      if (upper_bound >= next_threshold) {\n        ++size;\n        threshold = next_threshold;\n      }\n    }\n  }\n  entries[0].threshold = T{0};  // 0 has one digit.\n  return entries;\n}\n\nsize_t StringifiedSize(uint32_t src) {\n  static constexpr std::array<StringifiedSizeEntry<uint32_t>,\n                              kNumStringifiedSizeEntries<uint32_t>>\n      kEntries = MakeStringifiedSizeEntries<uint32_t>();\n  const size_t width = IntCast<size_t>(absl::bit_width(src));\n  const StringifiedSizeEntry<uint32_t>& entry = kEntries[width / 2];\n  return size_t{entry.size} - (src < entry.threshold ? 1 : 0);\n}\n\nsize_t StringifiedSize(uint64_t src) {\n  static constexpr std::array<StringifiedSizeEntry<uint64_t>,\n                              kNumStringifiedSizeEntries<uint64_t>>\n      kEntries = MakeStringifiedSizeEntries<uint64_t>();\n  const size_t width = IntCast<size_t>(absl::bit_width(src));\n  const StringifiedSizeEntry<uint64_t>& entry = kEntries[width / 2];\n  return size_t{entry.size} - (src < entry.threshold ? 1 : 0);\n}\n\nsize_t StringifiedSize(absl::uint128 src) {\n  static constexpr std::array<StringifiedSizeEntry<absl::uint128>,\n                              kNumStringifiedSizeEntries<absl::uint128>>\n      kEntries = MakeStringifiedSizeEntries<absl::uint128>();\n  const size_t width =\n      IntCast<size_t>(absl::Uint128High64(src) == 0\n                          ? absl::bit_width(absl::Uint128Low64(src))\n                          : absl::bit_width(absl::Uint128High64(src)) + 64);\n  const StringifiedSizeEntry<absl::uint128>& entry = kEntries[width / 2];\n  return size_t{entry.size} - (src < entry.threshold ? 1 : 0);\n}\n\nsize_t StringifiedSize(int32_t src) {\n  if (src >= 0) {\n    return StringifiedSize(UnsignedCast(src));\n  } else {\n    return StringifiedSize(NegatingUnsignedCast(src)) + 1;\n  }\n}\n\nsize_t StringifiedSize(int64_t src) {\n  if (src >= 0) {\n    return StringifiedSize(UnsignedCast(src));\n  } else {\n    return StringifiedSize(NegatingUnsignedCast(src)) + 1;\n  }\n}\n\nsize_t StringifiedSize(absl::int128 src) {\n  if (src >= 0) {\n    return StringifiedSize(UnsignedCast(src));\n  } else {\n    return StringifiedSize(NegatingUnsignedCast(src)) + 1;\n  }\n}\n\n}  // namespace riegeli::write_int_internal\n"
  },
  {
    "path": "riegeli/bytes/write_int_internal.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_WRITE_INT_INTERNAL_H_\n#define RIEGELI_BYTES_WRITE_INT_INTERNAL_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <type_traits>\n\n#include \"absl/numeric/int128.h\"\n#include \"riegeli/base/arithmetic.h\"\n\nnamespace riegeli::write_int_internal {\n\ntemplate <typename T, typename Target, typename Enable = void>\nstruct FitsIn;\n\ntemplate <typename T, typename Target>\nstruct FitsIn<T, Target,\n              std::enable_if_t<std::disjunction_v<\n                  std::conjunction<IsUnsignedInt<T>, IsUnsignedInt<Target>>,\n                  std::conjunction<IsSignedInt<T>, IsSignedInt<Target>>>>>\n    : std::bool_constant<(std::numeric_limits<T>::max() <=\n                          std::numeric_limits<Target>::max())> {};\n\n// `WriteDec()` with no width parameter writes no leading zeros, except for 0\n// itself.\nchar* WriteDec(uint32_t src, char* dest);\nchar* WriteDec(uint64_t src, char* dest);\nchar* WriteDec(absl::uint128 src, char* dest);\nchar* WriteDec(int32_t src, char* dest);\nchar* WriteDec(int64_t src, char* dest);\nchar* WriteDec(absl::int128 src, char* dest);\n\n// `WriteDecUnsigned()` with no width parameter writes no leading zeros, except\n// for 0 itself.\n\ntemplate <typename T, std::enable_if_t<FitsIn<T, uint32_t>::value, int> = 0>\ninline char* WriteDecUnsigned(T src, char* dest) {\n  return WriteDec(IntCast<uint32_t>(src), dest);\n}\n\ntemplate <typename T, std::enable_if_t<\n                          std::conjunction_v<std::negation<FitsIn<T, uint32_t>>,\n                                             FitsIn<T, uint64_t>>,\n                          int> = 0>\ninline char* WriteDecUnsigned(T src, char* dest) {\n  return WriteDec(IntCast<uint64_t>(src), dest);\n}\n\ntemplate <typename T, std::enable_if_t<\n                          std::conjunction_v<std::negation<FitsIn<T, uint64_t>>,\n                                             FitsIn<T, absl::uint128>>,\n                          int> = 0>\ninline char* WriteDecUnsigned(T src, char* dest) {\n  return WriteDec(IntCast<absl::uint128>(src), dest);\n}\n\n// `WriteDecSigned()` with no width parameter writes no leading zeros, except\n// for 0 itself.\n\ntemplate <typename T, std::enable_if_t<FitsIn<T, int32_t>::value, int> = 0>\ninline char* WriteDecSigned(T src, char* dest) {\n  return WriteDec(IntCast<int32_t>(src), dest);\n}\n\ntemplate <typename T,\n          std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, int32_t>>,\n                                              FitsIn<T, int64_t>>,\n                           int> = 0>\ninline char* WriteDecSigned(T src, char* dest) {\n  return WriteDec(IntCast<int64_t>(src), dest);\n}\n\ntemplate <typename T,\n          std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, int64_t>>,\n                                              FitsIn<T, absl::int128>>,\n                           int> = 0>\ninline char* WriteDecSigned(T src, char* dest) {\n  return WriteDec(IntCast<absl::int128>(src), dest);\n}\n\n// `WriteDec()` with a width parameter writes at least `width` characters.\nchar* WriteDec(uint32_t src, char* dest, size_t width);\nchar* WriteDec(uint64_t src, char* dest, size_t width);\nchar* WriteDec(absl::uint128 src, char* dest, size_t width);\nchar* WriteDec(int32_t src, char* dest, size_t width);\nchar* WriteDec(int64_t src, char* dest, size_t width);\nchar* WriteDec(absl::int128 src, char* dest, size_t width);\n\n// `WriteDecUnsigned()` with a width parameter writes at least `width`\n// characters.\n\ntemplate <typename T, std::enable_if_t<FitsIn<T, uint32_t>::value, int> = 0>\ninline char* WriteDecUnsigned(T src, char* dest, size_t width) {\n  return width <= 1 ? WriteDec(IntCast<uint32_t>(src), dest)\n                    : WriteDec(IntCast<uint32_t>(src), dest, width);\n}\n\ntemplate <typename T, std::enable_if_t<\n                          std::conjunction_v<std::negation<FitsIn<T, uint32_t>>,\n                                             FitsIn<T, uint64_t>>,\n                          int> = 0>\ninline char* WriteDecUnsigned(T src, char* dest, size_t width) {\n  return width <= 1 ? WriteDec(IntCast<uint64_t>(src), dest)\n                    : WriteDec(IntCast<uint64_t>(src), dest, width);\n}\n\ntemplate <typename T, std::enable_if_t<\n                          std::conjunction_v<std::negation<FitsIn<T, uint64_t>>,\n                                             FitsIn<T, absl::uint128>>,\n                          int> = 0>\ninline char* WriteDecUnsigned(T src, char* dest, size_t width) {\n  return width <= 1 ? WriteDec(IntCast<absl::uint128>(src), dest)\n                    : WriteDec(IntCast<absl::uint128>(src), dest, width);\n}\n\n// `WriteDecSigned()` with a width parameter writes at least `width` characters.\n\ntemplate <typename T, std::enable_if_t<FitsIn<T, int32_t>::value, int> = 0>\ninline char* WriteDecSigned(T src, char* dest, size_t width) {\n  return width <= 1 ? WriteDec(IntCast<int32_t>(src), dest)\n                    : WriteDec(IntCast<int32_t>(src), dest, width);\n}\n\ntemplate <typename T,\n          std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, int32_t>>,\n                                              FitsIn<T, int64_t>>,\n                           int> = 0>\ninline char* WriteDecSigned(T src, char* dest, size_t width) {\n  return width <= 1 ? WriteDec(IntCast<int64_t>(src), dest)\n                    : WriteDec(IntCast<int64_t>(src), dest, width);\n}\n\ntemplate <typename T,\n          std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, int64_t>>,\n                                              FitsIn<T, absl::int128>>,\n                           int> = 0>\ninline char* WriteDecSigned(T src, char* dest, size_t width) {\n  return width <= 1 ? WriteDec(IntCast<absl::int128>(src), dest)\n                    : WriteDec(IntCast<absl::int128>(src), dest, width);\n}\n\n// `WriteDecBackward()` writes no leading zeros, except for 0 itself.\nchar* WriteDecBackward(uint32_t src, char* dest);\nchar* WriteDecBackward(uint64_t src, char* dest);\nchar* WriteDecBackward(absl::uint128 src, char* dest);\nchar* WriteDecBackward(int32_t src, char* dest);\nchar* WriteDecBackward(int64_t src, char* dest);\nchar* WriteDecBackward(absl::int128 src, char* dest);\n\n// `WriteDecUnsignedBackward()` writes no leading zeros, except for 0 itself.\n\ntemplate <typename T, std::enable_if_t<FitsIn<T, uint32_t>::value, int> = 0>\ninline char* WriteDecUnsignedBackward(T src, char* dest) {\n  return WriteDecBackward(IntCast<uint32_t>(src), dest);\n}\n\ntemplate <typename T, std::enable_if_t<\n                          std::conjunction_v<std::negation<FitsIn<T, uint32_t>>,\n                                             FitsIn<T, uint64_t>>,\n                          int> = 0>\ninline char* WriteDecUnsignedBackward(T src, char* dest) {\n  return WriteDecBackward(IntCast<uint64_t>(src), dest);\n}\n\ntemplate <typename T, std::enable_if_t<\n                          std::conjunction_v<std::negation<FitsIn<T, uint64_t>>,\n                                             FitsIn<T, absl::uint128>>,\n                          int> = 0>\ninline char* WriteDecUnsignedBackward(T src, char* dest) {\n  return WriteDecBackward(IntCast<absl::uint128>(src), dest);\n}\n\n// `WriteDecSignedBackward()` writes no leading zeros, except for 0 itself.\n\ntemplate <typename T, std::enable_if_t<FitsIn<T, int32_t>::value, int> = 0>\ninline char* WriteDecSignedBackward(T src, char* dest) {\n  return WriteDecBackward(IntCast<int32_t>(src), dest);\n}\n\ntemplate <typename T,\n          std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, int32_t>>,\n                                              FitsIn<T, int64_t>>,\n                           int> = 0>\ninline char* WriteDecSignedBackward(T src, char* dest) {\n  return WriteDecBackward(IntCast<int64_t>(src), dest);\n}\n\ntemplate <typename T,\n          std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, int64_t>>,\n                                              FitsIn<T, absl::int128>>,\n                           int> = 0>\ninline char* WriteDecSignedBackward(T src, char* dest) {\n  return WriteDecBackward(IntCast<absl::int128>(src), dest);\n}\n\nsize_t StringifiedSize(uint32_t src);\nsize_t StringifiedSize(uint64_t src);\nsize_t StringifiedSize(absl::uint128 src);\nsize_t StringifiedSize(int32_t src);\nsize_t StringifiedSize(int64_t src);\nsize_t StringifiedSize(absl::int128 src);\n\ntemplate <typename T, std::enable_if_t<FitsIn<T, uint32_t>::value, int> = 0>\ninline size_t StringifiedSizeUnsigned(T src) {\n  return StringifiedSize(IntCast<uint32_t>(src));\n}\n\ntemplate <typename T, std::enable_if_t<\n                          std::conjunction_v<std::negation<FitsIn<T, uint32_t>>,\n                                             FitsIn<T, uint64_t>>,\n                          int> = 0>\ninline size_t StringifiedSizeUnsigned(T src) {\n  return StringifiedSize(IntCast<uint64_t>(src));\n}\n\ntemplate <typename T, std::enable_if_t<\n                          std::conjunction_v<std::negation<FitsIn<T, uint64_t>>,\n                                             FitsIn<T, absl::uint128>>,\n                          int> = 0>\ninline size_t StringifiedSizeUnsigned(T src) {\n  return StringifiedSize(IntCast<absl::uint128>(src));\n}\n\ntemplate <typename T, std::enable_if_t<FitsIn<T, int32_t>::value, int> = 0>\ninline size_t StringifiedSizeSigned(T src) {\n  return StringifiedSize(IntCast<int32_t>(src));\n}\n\ntemplate <typename T,\n          std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, int32_t>>,\n                                              FitsIn<T, int64_t>>,\n                           int> = 0>\ninline size_t StringifiedSizeSigned(T src) {\n  return StringifiedSize(IntCast<int64_t>(src));\n}\n\ntemplate <typename T,\n          std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, int64_t>>,\n                                              FitsIn<T, absl::int128>>,\n                           int> = 0>\ninline size_t StringifiedSizeSigned(T src) {\n  return StringifiedSize(IntCast<absl::int128>(src));\n}\n\n}  // namespace riegeli::write_int_internal\n\n#endif  // RIEGELI_BYTES_WRITE_INT_INTERNAL_H_\n"
  },
  {
    "path": "riegeli/bytes/writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/writer.h\"\n\n#include <stddef.h>\n\n#include <cmath>\n#include <cstring>\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/int128.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/str_format.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_utils.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nnamespace {\n\ntemplate <typename T>\ninline bool WriteUnsigned(T src, Writer& dest) {\n  // `digits10` is rounded down, `kMaxNumDigits` is rounded up, hence `+ 1`.\n  constexpr size_t kMaxNumDigits = std::numeric_limits<T>::digits10 + 1;\n  if (ABSL_PREDICT_FALSE(!dest.Push(kMaxNumDigits))) return false;\n  dest.set_cursor(write_int_internal::WriteDecUnsigned(src, dest.cursor()));\n  return true;\n}\n\ntemplate <typename T>\ninline bool WriteSigned(T src, Writer& dest) {\n  // `digits10` is rounded down, `kMaxNumDigits` is rounded up, hence `+ 1`.\n  constexpr size_t kMaxNumDigits = std::numeric_limits<T>::digits10 + 1;\n  // `+ 1` for the minus sign.\n  if (ABSL_PREDICT_FALSE(!dest.Push(kMaxNumDigits + 1))) return false;\n  dest.set_cursor(write_int_internal::WriteDecSigned(src, dest.cursor()));\n  return true;\n}\n\n}  // namespace\n\nvoid Writer::OnFail() { set_buffer(start()); }\n\nabsl::Status Writer::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) return Annotate(status, absl::StrCat(\"at byte \", pos()));\n  return status;\n}\n\nbool Writer::FailOverflow() {\n  return Fail(absl::ResourceExhaustedError(\"Writer position overflow\"));\n}\n\nbool Writer::Write(const Chain& src) {\n#ifdef MEMORY_SANITIZER\n  for (const absl::string_view fragment : src.blocks()) {\n    AssertInitialized(fragment.data(), fragment.size());\n  }\n#endif\n  if (ABSL_PREDICT_TRUE(available() >= src.size() &&\n                        src.size() <= kMaxBytesToCopy)) {\n    src.CopyTo(cursor());\n    move_cursor(src.size());\n    return true;\n  }\n  AssertInitialized(start(), start_to_cursor());\n  return WriteSlow(src);\n}\n\nbool Writer::Write(Chain&& src) {\n#ifdef MEMORY_SANITIZER\n  for (const absl::string_view fragment : src.blocks()) {\n    AssertInitialized(fragment.data(), fragment.size());\n  }\n#endif\n  if (ABSL_PREDICT_TRUE(available() >= src.size() &&\n                        src.size() <= kMaxBytesToCopy)) {\n    src.CopyTo(cursor());\n    move_cursor(src.size());\n    return true;\n  }\n  AssertInitialized(start(), start_to_cursor());\n  return WriteSlow(std::move(src));\n}\n\nbool Writer::Write(const absl::Cord& src) {\n#ifdef MEMORY_SANITIZER\n  for (const absl::string_view fragment : src.Chunks()) {\n    AssertInitialized(fragment.data(), fragment.size());\n  }\n#endif\n  if (ABSL_PREDICT_TRUE(available() >= src.size() &&\n                        src.size() <= kMaxBytesToCopy)) {\n    cord_internal::CopyCordToArray(src, cursor());\n    move_cursor(src.size());\n    return true;\n  }\n  AssertInitialized(start(), start_to_cursor());\n  return WriteSlow(src);\n}\n\nbool Writer::Write(absl::Cord&& src) {\n#ifdef MEMORY_SANITIZER\n  for (const absl::string_view fragment : src.Chunks()) {\n    AssertInitialized(fragment.data(), fragment.size());\n  }\n#endif\n  if (ABSL_PREDICT_TRUE(available() >= src.size() &&\n                        src.size() <= kMaxBytesToCopy)) {\n    cord_internal::CopyCordToArray(src, cursor());\n    move_cursor(src.size());\n    return true;\n  }\n  AssertInitialized(start(), start_to_cursor());\n  return WriteSlow(std::move(src));\n}\n\nbool Writer::Write(signed char src) { return WriteSigned(src, *this); }\n\nbool Writer::Write(unsigned char src) { return WriteUnsigned(src, *this); }\n\nbool Writer::Write(short src) { return WriteSigned(src, *this); }\n\nbool Writer::Write(unsigned short src) { return WriteUnsigned(src, *this); }\n\nbool Writer::Write(int src) { return WriteSigned(src, *this); }\n\nbool Writer::Write(unsigned src) { return WriteUnsigned(src, *this); }\n\nbool Writer::Write(long src) { return WriteSigned(src, *this); }\n\nbool Writer::Write(unsigned long src) { return WriteUnsigned(src, *this); }\n\nbool Writer::Write(long long src) { return WriteSigned(src, *this); }\n\nbool Writer::Write(unsigned long long src) { return WriteUnsigned(src, *this); }\n\nbool Writer::Write(absl::int128 src) { return WriteSigned(src, *this); }\n\nbool Writer::Write(absl::uint128 src) { return WriteUnsigned(src, *this); }\n\n// TODO: Optimize implementations below.\nbool Writer::Write(float src) { return Write(absl::StrCat(src)); }\n\nbool Writer::Write(double src) { return Write(absl::StrCat(src)); }\n\nbool Writer::Write(long double src) {\n  absl::Format(this, \"%g\",\n               // Consistently use \"nan\", never \"-nan\".\n               ABSL_PREDICT_FALSE(std::isnan(src))\n                   ? std::numeric_limits<long double>::quiet_NaN()\n                   : src);\n  return ok();\n}\n\nbool Writer::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of Writer::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  do {\n    const size_t available_length = available();\n    riegeli::null_safe_memcpy(cursor(), src.data(), available_length);\n    move_cursor(available_length);\n    src.remove_prefix(available_length);\n    if (ABSL_PREDICT_FALSE(!PushSlow(1, src.size()))) return false;\n  } while (src.size() > available());\n  std::memcpy(cursor(), src.data(), src.size());\n  move_cursor(src.size());\n  return true;\n}\n\nbool Writer::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  return Write(absl::string_view(src));\n}\n\nbool Writer::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  for (const absl::string_view fragment : src.blocks()) {\n    if (ABSL_PREDICT_FALSE(!Write(fragment))) return false;\n  }\n  return true;\n}\n\nbool Writer::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  // Not `std::move(src)`: forward to `WriteSlow(const Chain&)`.\n  return WriteSlow(src);\n}\n\nbool Writer::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (const std::optional<absl::string_view> flat = src.TryFlat();\n      flat != std::nullopt) {\n    return Write(*flat);\n  }\n  for (const absl::string_view fragment : src.Chunks()) {\n    if (ABSL_PREDICT_FALSE(!Write(fragment))) return false;\n  }\n  return true;\n}\n\nbool Writer::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  // Not `std::move(src)`: forward to `WriteSlow(const absl::Cord&)`.\n  return WriteSlow(src);\n}\n\nbool Writer::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  while (src.size() > available()) {\n    const size_t available_length = available();\n    riegeli::null_safe_memset(cursor(), src.fill(), available_length);\n    move_cursor(available_length);\n    src.Extract(available_length);\n    if (ABSL_PREDICT_FALSE(!Push(1, SaturatingIntCast<size_t>(src.size())))) {\n      return false;\n    }\n  }\n  std::memset(cursor(), src.fill(), IntCast<size_t>(src.size()));\n  move_cursor(IntCast<size_t>(src.size()));\n  return true;\n}\n\nbool Writer::FlushImpl(FlushType flush_type) { return ok(); }\n\nbool Writer::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT_NE(new_pos, pos())\n      << \"Failed precondition of Writer::SeekSlow(): \"\n         \"position unchanged, use Seek() instead\";\n  return Fail(absl::UnimplementedError(\"Writer::Seek() not supported\"));\n}\n\nstd::optional<Position> Writer::SizeImpl() {\n  Fail(absl::UnimplementedError(\"Writer::Size() not supported\"));\n  return std::nullopt;\n}\n\nbool Writer::TruncateImpl(Position new_size) {\n  return Fail(absl::UnimplementedError(\"Writer::Truncate() not supported\"));\n}\n\nReader* Writer::ReadModeImpl(Position initial_pos) {\n  Fail(absl::UnimplementedError(\"Writer::ReadMode() not supported\"));\n  return nullptr;\n}\n\nnamespace writer_internal {\n\nvoid DeleteReader(Reader* reader) { delete reader; }\n\n}  // namespace writer_internal\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_WRITER_H_\n#define RIEGELI_BYTES_WRITER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/int128.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/has_absl_stringify.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/stringify.h\"\n\nnamespace riegeli {\n\nclass Reader;\nclass Writer;\n\n// A sink for `AbslStringify()` which appends to a `Writer`.\nclass WriterStringifySink {\n public:\n  // Creates a dummy `WriterStringifySink`. It must not be used.\n  WriterStringifySink() = default;\n\n  // Will write to `*dest`.\n  explicit WriterStringifySink(Writer* dest ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : dest_(RIEGELI_EVAL_ASSERT_NOTNULL(dest)) {}\n\n  WriterStringifySink(WriterStringifySink&& that) = default;\n  WriterStringifySink& operator=(WriterStringifySink&& that) = default;\n\n  // Returns the `Writer` being written to.\n  Writer* dest() const { return dest_; }\n\n  void Append(size_t length, char fill);\n  void Append(absl::string_view src);\n  friend void AbslFormatFlush(WriterStringifySink* dest,\n                              absl::string_view src) {\n    dest->Append(src);\n  }\n\n private:\n  Writer* dest_ = nullptr;\n};\n\n// Abstract class `Writer` writes sequences of bytes to a destination. The\n// nature of the destination depends on the particular class derived from\n// `Writer`.\n//\n// A `Writer` object manages a buffer of data to be pushed to the destination,\n// which amortizes the overhead of pushing data over multiple writes. Data can\n// be written directly into the buffer, and classes derived from `Writer` can\n// avoid copying by allocating the buffer in a way which fits the destination,\n// e.g. pointing it to a fragment of the destination itself.\n//\n// All `Writer`s support writing data sequentially and querying for the current\n// position. Some `Writer`s also support random access: changing the position\n// for subsequent operations and querying for the total size of the destination.\n// Some `Writer`s also support truncation; this is implied by supporting random\n// access.\n//\n// A `Writer` must be explicitly closed, and `Close()` must succeed, in order\n// for its output to be guaranteed to be available in the destination.\nclass Writer : public Object {\n public:\n  // The same as `Object::Close()`.\n  //\n  // The implementation in this class adds an assertion.\n  bool Close();\n\n  // If `write_size_hint` is not `std::nullopt`, hints that this amount of data\n  // will be written sequentially from the current position, then `Close()` will\n  // be called.\n  //\n  // This may improve performance and memory usage:\n  //  * Larger buffer sizes may be used before reaching the size hint, and\n  //    a smaller buffer size may be used when reaching the size hint.\n  //  * This hint may be propagated to owned destinations.\n  //  * Other consequences are possible.\n  //\n  // If the hint turns out to not match reality, nothing breaks. It is better if\n  // `write_size_hint` is slightly too large than slightly too small.\n  //\n  // `SetWriteSizeHint()` is usually be called from the same abstraction layer\n  // which later calls `Close()`.\n  void SetWriteSizeHint(std::optional<Position> write_size_hint);\n\n  // Ensures that enough space is available in the buffer: if less than\n  // `min_length` of space is available, pushes previously written data to the\n  // destination, and points `cursor()` and `limit()` to space following the\n  // current position with length at least `min_length`, preferably\n  // `recommended_length`.\n  //\n  // The current position does not change with `Push()`. It changes with e.g.\n  // `move_cursor()` and `Write()`.\n  //\n  // If `recommended_length < min_length`, `recommended_length` is assumed to be\n  // `min_length`.\n  //\n  // Return values:\n  //  * `true`  - success (`available() >= min_length`)\n  //  * `false` - failure (`available() < min_length`, `!ok()`)\n  bool Push(size_t min_length = 1, size_t recommended_length = 0);\n\n  // Buffer pointers. Space between `start()` and `limit()` is available for\n  // immediate writing data to it, with `cursor()` pointing to the current\n  // position.\n  //\n  // Memory after the address to which `cursor()` is eventually moved must not\n  // be clobbered.\n  //\n  // Non-const member functions may change buffer pointers, including changing\n  // how much data around the current position are buffered.\n  //\n  // Invariants:\n  //   `start() <= cursor() <= limit()` (possibly all `nullptr`)\n  //   if `!ok()` then `start() == cursor() == limit() == nullptr`\n  char* start() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return start_; }\n  char* cursor() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return cursor_; }\n  char* limit() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return limit_; }\n\n  // Increments the value of `cursor()`. Does not change `start()` nor\n  // `limit()`. Call this during writing data under `cursor()` to indicate how\n  // much was written.\n  //\n  // Precondition: `length <= available()`\n  void move_cursor(size_t length);\n\n  // Sets the value of `cursor()`. Does not change `start()` nor `limit()`. Call\n  // this during writing data under `cursor()` to indicate how much was written.\n  //\n  // Preconditions: `start() <= cursor <= limit()`\n  void set_cursor(char* cursor);\n\n  // Returns the amount of space available in the buffer, between `cursor()` and\n  // `limit()`.\n  //\n  // Invariant: if `!ok()` then `available() == 0`\n  size_t available() const { return PtrDistance(cursor_, limit_); }\n\n  // Returns the buffer size, between `start()` and `limit()`.\n  //\n  // Invariant: if `!ok()` then `start_to_limit() == 0`\n  size_t start_to_limit() const { return PtrDistance(start_, limit_); }\n\n  // Returns the amount of data written to the buffer, between `start()` and\n  // `cursor()`.\n  //\n  // Invariant: if `!ok()` then `start_to_cursor() == 0`\n  size_t start_to_cursor() const { return PtrDistance(start_, cursor_); }\n\n  // Writes a single byte to the buffer or the destination.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool WriteByte(uint8_t src) { return Write(static_cast<char>(src)); }\n\n  // Writes a fixed number of bytes from `src` to the buffer and/or the\n  // destination.\n  //\n  // Return values:\n  //  * `true`  - success (`src.size()` bytes written)\n  //  * `false` - failure (less than `src.size()` bytes written, `!ok()`)\n  bool Write(char src);\n#if __cpp_char8_t\n  bool Write(char8_t src) { return Write(static_cast<char>(src)); }\n#endif\n  bool Write(BytesRef src);\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  bool Write(const char* src) { return Write(absl::string_view(src)); }\n  bool Write(ExternalRef src);\n  template <typename Src,\n            std::enable_if_t<SupportsExternalRefWhole<Src>::value, int> = 0>\n  bool Write(Src&& src);\n  bool Write(const Chain& src);\n  bool Write(Chain&& src);\n  bool Write(const absl::Cord& src);\n  bool Write(absl::Cord&& src);\n  bool Write(ByteFill src);\n\n  // Writes a stringified value to the buffer and/or the destination.\n  //\n  // Return values:\n  //  * `true`  - success\n  //  * `false` - failure (`!ok()`)\n  bool Write(signed char src);\n  bool Write(unsigned char src);\n  bool Write(short src);\n  bool Write(unsigned short src);\n  bool Write(int src);\n  bool Write(unsigned src);\n  bool Write(long src);\n  bool Write(unsigned long src);\n  bool Write(long long src);\n  bool Write(unsigned long long src);\n  bool Write(absl::int128 src);\n  bool Write(absl::uint128 src);\n  bool Write(float src);\n  bool Write(double src);\n  bool Write(long double src);\n  template <\n      typename Src,\n      std::enable_if_t<\n          std::conjunction_v<\n              absl::HasAbslStringify<Src>,\n              std::negation<std::is_convertible<Src&&, BytesRef>>,\n              std::negation<std::is_convertible<Src&&, const Chain&>>,\n              std::negation<std::is_convertible<Src&&, const absl::Cord&>>,\n              std::negation<std::is_convertible<Src&&, ByteFill>>>,\n          int> = 0>\n  bool Write(Src&& src);\n\n  // Other integer types are is not supported. Delete overloads to avoid\n  // implicit conversions.\n  bool Write(bool src) = delete;\n  bool Write(wchar_t src) = delete;\n  bool Write(char16_t src) = delete;\n  bool Write(char32_t src) = delete;\n\n  // Writes stringified values to the buffer and/or the destination.\n  //\n  // Return values:\n  //  * `true`  - success\n  //  * `false` - failure (`!ok()`)\n  template <typename... Srcs\n#if !__cpp_concepts\n            ,\n            std::enable_if_t<\n                std::conjunction_v<std::bool_constant<sizeof...(Srcs) != 1>,\n                                   IsStringifiable<Srcs...>>,\n                int> = 0\n#endif\n            >\n  bool Write(Srcs&&... srcs)\n#if __cpp_concepts\n      // For conjunctions, `requires` gives better error messages than\n      // `std::enable_if_t`, indicating the relevant argument.\n    requires(sizeof...(Srcs) != 1) && (IsStringifiable<Srcs>::value && ...)\n#endif\n  ;\n\n  // Pushes buffered data to the destination.\n  //\n  // This makes data written so far visible, but in contrast to `Close()`,\n  // keeps the possibility to write more data later. What exactly does it mean\n  // for data to be visible depends on the destination. If this is not\n  // applicable or not feasible, does nothing.\n  //\n  // The scope of objects to flush and the intended data durability (without a\n  // guarantee) are specified by `flush_type`:\n  //  * `FlushType::kFromObject`  - Makes data written so far visible in other\n  //                                objects, propagating flushing through owned\n  //                                dependencies of the given writer.\n  //  * `FlushType::kFromProcess` - Makes data written so far visible outside\n  //                                the process, propagating flushing through\n  //                                dependencies of the given writer.\n  //                                This is the default.\n  //  * `FlushType::kFromMachine` - Makes data written so far visible outside\n  //                                the process and durable in case of operating\n  //                                system crash, propagating flushing through\n  //                                dependencies of the given writer.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool Flush(FlushType flush_type = FlushType::kFromProcess);\n\n  // Returns the current position.\n  //\n  // This is not necessarily 0 after creating the `Writer` if it appends to a\n  // destination with existing contents, or if the `Writer` wraps another writer\n  // or output stream propagating its position.\n  //\n  // This may decrease when the `Writer` becomes not OK (due to buffering,\n  // previously written but unflushed data may be lost).\n  //\n  // `pos()` is unchanged by a successful `Close()`.\n  Position pos() const;\n\n  // Returns the position corresponding to `start()`,\n  // i.e. `pos() - start_to_cursor()`.\n  Position start_pos() const { return start_pos_; }\n\n  // Returns the position corresponding to `limit()`,\n  // i.e. `pos() + available()`.\n  Position limit_pos() const;\n\n  // Returns `true` if this Writer supports `Seek()` to other positions\n  // (`Seek()` to the current position is always supported) and `Size()`.\n  virtual bool SupportsRandomAccess() { return false; }\n\n  // Sets the current position for subsequent operations.\n  //\n  // Return values:\n  //  * `true`                 - success (position is set to `new_pos`)\n  //  * `false` (when `ok()`)  - destination ends before `new_pos`\n  //                             (position is set to end)\n  //  * `false` (when `!ok()`) - failure\n  //\n  // `Seek()` to the current position is always supported.\n  //\n  // `Seek()` to other positions is supported if `SupportsRandomAccess()` is\n  // `true`.\n  bool Seek(Position new_pos);\n\n  // Returns the size of the destination, i.e. the position corresponding to its\n  // end.\n  //\n  // Returns `std::nullopt` on failure (`!ok()`).\n  //\n  // `Size()` is supported if `SupportsRandomAccess()` is `true`.\n  std::optional<Position> Size();\n\n  // Returns `true` if this `Writer` supports `Truncate()`.\n  virtual bool SupportsTruncate() { return SupportsRandomAccess(); }\n\n  // Discards the part of the destination after the given position. Sets the\n  // current position to the new end.\n  //\n  // Return values:\n  //  * `true`                 - success (destination truncated, `ok()`)\n  //  * `false` (when `ok()`)  - destination is smaller than `new_size`\n  //                             (position is set to end)\n  //  * `false` (when `!ok()`) - failure\n  //\n  // `Truncate()` is supported if `SupportsTruncate()` is `true`.\n  bool Truncate(Position new_size);\n\n  // Returns `true` if this `Writer` supports `ReadMode()`.\n  virtual bool SupportsReadMode() { return false; }\n\n  // Switches from writing to reading.\n  //\n  // Returns a `Reader` which reads from the current contents of the destination\n  // of this `Writer`, starting from `initial_pos`.\n  //\n  // If the source ends before `initial_pos`, the position of the new `Reader`\n  // is set to the end. The resulting `Reader` `SupportsRewind()`. If this\n  // `Writer` `SupportsRandomAccess()`, the resulting `Reader` also\n  // `SupportsRandomAccess()`.\n  //\n  // When this `Writer` is used again, its position is the same as before\n  // `ReadMode()` was called, and the `Reader` becomes invalid.\n  //\n  // The returned `Reader` is owned by this `Writer`. The `Reader` does not own\n  // its source, even if this `Writer` owns its destination.\n  //\n  // Returns `nullptr` on failure (`!ok()`).\n  //\n  // `ReadMode()` is supported if `SupportsReadMode()` is `true`.\n  Reader* ReadMode(Position initial_pos) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Supports `absl::Format(&writer, format, args...)`.\n  friend void AbslFormatFlush(Writer* dest, absl::string_view src) {\n    dest->Write(src);\n  }\n\n protected:\n  using Object::Object;\n\n  // Moves the part of the object defined in this class.\n  //\n  // Buffer pointers do not need to satisfy their invariants during this part of\n  // the move, here they are merely exchanged with `nullptr` and copied.\n  Writer(Writer&& that) noexcept;\n  Writer& operator=(Writer&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `Writer`. This avoids\n  // constructing a temporary `Writer` and moving from it. Derived classes which\n  // redefine `Reset()` should include a call to `Writer::Reset()`.\n  void Reset(Closed);\n  void Reset();\n\n  // `Writer` overrides `Object::Done()` to set buffer pointers to `nullptr`.\n  // Derived classes which override it further should include a call to\n  // `Writer::Done()`.\n  void Done() override;\n\n  // `Writer` overrides `Object::OnFail()` to set buffer pointers to `nullptr`.\n  // Derived classes which override it further should include a call to\n  // `Writer::OnFail()`.\n  //\n  // `pos()` decreases by `start_to_cursor()` to indicate that any buffered\n  // data have been lost.\n  ABSL_ATTRIBUTE_COLD void OnFail() override;\n\n  // `Writer` overrides `Object::AnnotateStatusImpl()` to annotate the status\n  // with the current position.\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n\n  // Marks the `Writer` as failed with message \"Writer position overflow\".\n  // Always returns `false`.\n  //\n  // This can be called if the destination would exceed its maximum possible\n  // size or if `start_pos()` would overflow.\n  ABSL_ATTRIBUTE_COLD bool FailOverflow();\n\n  // Implementation of `SetWriteSizeHint()`.\n  virtual void SetWriteSizeHintImpl(\n      ABSL_ATTRIBUTE_UNUSED std::optional<Position> write_size_hint) {}\n\n  // Implementation of the slow part of `Push()`.\n  //\n  // Precondition: `available() < min_length`\n  virtual bool PushSlow(size_t min_length, size_t recommended_length) = 0;\n\n  // Sets the values of:\n  //  * `start()`  - to `start`\n  //  * `cursor()` - to `start + start_to_cursor`\n  //  * `limit()`  - to `start + start_to_limit`\n  //\n  // Preconditions:\n  //   [`start`..`start + start_to_limit`) is a valid byte range\n  //   `start_to_cursor <= start_to_limit`\n  void set_buffer(char* start = nullptr, size_t start_to_limit = 0,\n                  size_t start_to_cursor = 0);\n\n  // Implementation of the slow part of `Write()`.\n  //\n  // By default:\n  //  * `WriteSlow(absl::string_view)` and `WriteSlow(ByteFill)` are\n  //     implemented in terms of `PushSlow()`\n  //  * `WriteSlow(ExternalRef)`, `WriteSlow(const Chain&)`, and\n  //    `WriteSlow(const absl::Cord&)` are implemented in terms of\n  //    `WriteSlow(absl::string_view)`\n  //  * `WriteSlow(Chain&&)` is implemented in terms of\n  //    `WriteSlow(const Chain&)`\n  //  * `WriteSlow(absl::Cord&&)` is implemented in terms of\n  //    `WriteSlow(const absl::Cord&)`\n  //\n  // Precondition for `WriteSlow(absl::string_view)`:\n  //   `available() < src.size()`\n  //\n  // Precondition for `WriteSlow(ExternalRef)`, `WriteSlow(const Chain&)`,\n  // `WriteSlow(Chain&&)`, `WriteSlow(const absl::Cord&)`,\n  // `WriteSlow(absl::Cord&&), and `WriteSlow(ByteFill)`:\n  //   `UnsignedMin(available(), kMaxBytesToCopy) < src.size()`\n  virtual bool WriteSlow(absl::string_view src);\n  virtual bool WriteSlow(ExternalRef src);\n  virtual bool WriteSlow(const Chain& src);\n  virtual bool WriteSlow(Chain&& src);\n  virtual bool WriteSlow(const absl::Cord& src);\n  virtual bool WriteSlow(absl::Cord&& src);\n  virtual bool WriteSlow(ByteFill src);\n\n  // Implementation of `Flush()`, except that the parameter is not defaulted,\n  // which is problematic for virtual functions.\n  //\n  // By default does nothing and returns `ok()`.\n  virtual bool FlushImpl(FlushType flush_type);\n\n  // Increments the value of `start_pos()`.\n  void move_start_pos(Position length);\n\n  // Sets the value of `start_pos()`.\n  void set_start_pos(Position start_pos);\n\n  // Implementation of the slow part of `Seek()`.\n  //\n  // By default fails.\n  //\n  // Precondition: `new_pos != pos()`\n  virtual bool SeekSlow(Position new_pos);\n\n  // Implementation of `Size()`.\n  //\n  // By default fails.\n  virtual std::optional<Position> SizeImpl();\n\n  // Implementation of `Truncate()`.\n  //\n  // By default fails.\n  virtual bool TruncateImpl(Position new_size);\n\n  // Implementation of `ReadMode()`.\n  //\n  // By default fails.\n  virtual Reader* ReadModeImpl(Position initial_pos);\n\n private:\n  char* start_ = nullptr;\n  char* cursor_ = nullptr;\n  char* limit_ = nullptr;\n\n  // Destination position corresponding to `start_`.\n  //\n  // Invariant:\n  //   `start_pos_ <= std::numeric_limits<Position>::max() - start_to_limit()`\n  Position start_pos_ = 0;\n};\n\n// Helps to implement `ReadMode()`. Stores a lazily created `Reader` of the\n// given concrete type.\n//\n// The `ReaderClass` type does not have to be complete, except in a context when\n// `ResetReader()` or `reader()` is called. This allows to have only a forward\n// declaration of `ReaderClass` in the header which defines the containing\n// `Writer` class.\ntemplate <typename ReaderClass>\nclass AssociatedReader {\n public:\n  AssociatedReader() = default;\n\n  AssociatedReader(AssociatedReader&& that) noexcept;\n  AssociatedReader& operator=(AssociatedReader&& that) noexcept;\n\n  ~AssociatedReader();\n\n  // Destroys the contained `Reader`.\n  void Reset();\n\n  // Creates or resets the `ReaderClass` with the given arguments of the\n  // constructor or `Reset()`. `ReaderClass` must be complete.\n  template <typename... Args>\n  ReaderClass* ResetReader(Args&&... args);\n\n  // Returns the `ReaderClass` pointer, or `nullptr` if it was not created yet.\n  // `ReaderClass` must be complete.\n  ReaderClass* reader() const;\n\n private:\n  static void Delete(Reader* reader);\n\n  Reader* reader_ = nullptr;\n};\n\n// Implementation details follow.\n\ninline void WriterStringifySink::Append(size_t length, char fill) {\n  dest_->Write(ByteFill(length, fill));\n}\n\ninline void WriterStringifySink::Append(absl::string_view src) {\n  dest_->Write(src);\n}\n\ninline Writer::Writer(Writer&& that) noexcept\n    : Object(static_cast<Object&&>(that)),\n      start_(std::exchange(that.start_, nullptr)),\n      cursor_(std::exchange(that.cursor_, nullptr)),\n      limit_(std::exchange(that.limit_, nullptr)),\n      start_pos_(std::exchange(that.start_pos_, 0)) {}\n\ninline Writer& Writer::operator=(Writer&& that) noexcept {\n  Object::operator=(static_cast<Object&&>(that));\n  start_ = std::exchange(that.start_, nullptr);\n  cursor_ = std::exchange(that.cursor_, nullptr);\n  limit_ = std::exchange(that.limit_, nullptr);\n  start_pos_ = std::exchange(that.start_pos_, 0);\n  return *this;\n}\n\ninline void Writer::Reset(Closed) {\n  Object::Reset(kClosed);\n  start_ = nullptr;\n  cursor_ = nullptr;\n  limit_ = nullptr;\n  start_pos_ = 0;\n}\n\ninline void Writer::Reset() {\n  Object::Reset();\n  start_ = nullptr;\n  cursor_ = nullptr;\n  limit_ = nullptr;\n  start_pos_ = 0;\n}\n\ninline bool Writer::Close() {\n  AssertInitialized(start(), start_to_cursor());\n  return Object::Close();\n}\n\ninline void Writer::Done() {\n  start_pos_ = pos();\n  set_buffer();\n}\n\ninline void Writer::SetWriteSizeHint(std::optional<Position> write_size_hint) {\n  AssertInitialized(start(), start_to_cursor());\n  SetWriteSizeHintImpl(write_size_hint);\n}\n\ninline bool Writer::Push(size_t min_length, size_t recommended_length) {\n  if (ABSL_PREDICT_TRUE(available() >= min_length)) return true;\n  AssertInitialized(start(), start_to_cursor());\n  if (ABSL_PREDICT_FALSE(!PushSlow(min_length, recommended_length))) {\n    return false;\n  }\n  RIEGELI_ASSERT_GE(available(), min_length)\n      << \"Failed postcondition of Writer::PushSlow(): \"\n         \"not enough space available\";\n  return true;\n}\n\ninline void Writer::move_cursor(size_t length) {\n  RIEGELI_ASSERT_LE(length, available())\n      << \"Failed precondition of Writer::move_cursor(): length out of range\";\n  cursor_ += length;\n}\n\ninline void Writer::set_cursor(char* cursor) {\n  RIEGELI_ASSERT_GE(cursor, start())\n      << \"Failed precondition of Writer::set_cursor(): pointer out of range\";\n  RIEGELI_ASSERT_LE(cursor, limit())\n      << \"Failed precondition of Writer::set_cursor(): pointer out of range\";\n  cursor_ = cursor;\n}\n\ninline void Writer::set_buffer(char* start, size_t start_to_limit,\n                               size_t start_to_cursor) {\n  RIEGELI_ASSERT_LE(start_to_cursor, start_to_limit)\n      << \"Failed precondition of Writer::set_buffer(): length out of range\";\n  start_ = start;\n  cursor_ = start + start_to_cursor;\n  limit_ = start + start_to_limit;\n}\n\ninline bool Writer::Write(char src) {\n  if (ABSL_PREDICT_FALSE(!Push())) return false;\n  *cursor() = src;\n  move_cursor(1);\n  return true;\n}\n\ninline bool Writer::Write(BytesRef src) {\n  AssertInitialized(src.data(), src.size());\n  if (ABSL_PREDICT_TRUE(available() >= src.size())) {\n    riegeli::null_safe_memcpy(cursor(), src.data(), src.size());\n    move_cursor(src.size());\n    return true;\n  }\n  AssertInitialized(start(), start_to_cursor());\n  return WriteSlow(src);\n}\n\ninline bool Writer::Write(ExternalRef src) {\n  if (ABSL_PREDICT_TRUE(available() >= src.size() &&\n                        src.size() <= kMaxBytesToCopy)) {\n    riegeli::null_safe_memcpy(cursor(), src.data(), src.size());\n    move_cursor(src.size());\n    return true;\n  }\n  AssertInitialized(start(), start_to_cursor());\n  return WriteSlow(std::move(src));\n}\n\ntemplate <typename Src,\n          std::enable_if_t<SupportsExternalRefWhole<Src>::value, int>>\ninline bool Writer::Write(Src&& src) {\n  return Write(ExternalRef(std::forward<Src>(src)));\n}\n\ninline bool Writer::Write(ByteFill src) {\n  if (ABSL_PREDICT_TRUE(available() >= src.size() &&\n                        src.size() <= kMaxBytesToCopy)) {\n    riegeli::null_safe_memset(cursor(), src.fill(),\n                              IntCast<size_t>(src.size()));\n    move_cursor(IntCast<size_t>(src.size()));\n    return true;\n  }\n  AssertInitialized(start(), start_to_cursor());\n  return WriteSlow(src);\n}\n\ntemplate <typename Src,\n          std::enable_if_t<\n              std::conjunction_v<\n                  absl::HasAbslStringify<Src>,\n                  std::negation<std::is_convertible<Src&&, BytesRef>>,\n                  std::negation<std::is_convertible<Src&&, const Chain&>>,\n                  std::negation<std::is_convertible<Src&&, const absl::Cord&>>,\n                  std::negation<std::is_convertible<Src&&, ByteFill>>>,\n              int>>\nbool Writer::Write(Src&& src) {\n  WriterStringifySink sink(this);\n  AbslStringify(sink, std::forward<Src>(src));\n  return ok();\n}\n\ntemplate <typename... Srcs\n#if !__cpp_concepts\n          ,\n          std::enable_if_t<\n              std::conjunction_v<std::bool_constant<sizeof...(Srcs) != 1>,\n                                 IsStringifiable<Srcs...>>,\n              int>\n#endif\n          >\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool Writer::Write(Srcs&&... srcs)\n#if __cpp_concepts\n  requires(sizeof...(Srcs) != 1) && (IsStringifiable<Srcs>::value && ...)\n#endif\n{\n  return (Write(std::forward<Srcs>(srcs)) && ...);\n}\n\ninline bool Writer::Flush(FlushType flush_type) {\n  AssertInitialized(start(), start_to_cursor());\n  return FlushImpl(flush_type);\n}\n\ninline Position Writer::pos() const {\n  RIEGELI_ASSERT_LE(start_pos_,\n                    std::numeric_limits<Position>::max() - start_to_limit())\n      << \"Failed invariant of Writer: position of buffer limit overflow\";\n  return start_pos_ + start_to_cursor();\n}\n\ninline Position Writer::limit_pos() const {\n  RIEGELI_ASSERT_LE(start_pos_,\n                    std::numeric_limits<Position>::max() - start_to_limit())\n      << \"Failed invariant of Writer: position of buffer limit overflow\";\n  return start_pos_ + start_to_limit();\n}\n\ninline void Writer::move_start_pos(Position length) {\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<Position>::max() - start_pos_)\n      << \"Failed precondition of Writer::move_start_pos(): \"\n         \"position out of range\";\n  start_pos_ += length;\n}\n\ninline void Writer::set_start_pos(Position start_pos) {\n  start_pos_ = start_pos;\n}\n\ninline bool Writer::Seek(Position new_pos) {\n  if (ABSL_PREDICT_TRUE(new_pos == pos())) return true;\n  AssertInitialized(start(), start_to_cursor());\n  return SeekSlow(new_pos);\n}\n\ninline std::optional<Position> Writer::Size() {\n  AssertInitialized(start(), start_to_cursor());\n  return SizeImpl();\n}\n\ninline bool Writer::Truncate(Position new_size) {\n  AssertInitialized(start(), start_to_cursor());\n  return TruncateImpl(new_size);\n}\n\ninline Reader* Writer::ReadMode(Position initial_pos)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  AssertInitialized(start(), start_to_cursor());\n  return ReadModeImpl(initial_pos);\n}\n\nnamespace writer_internal {\n\n// Does `delete reader`. This is defined in a separate file because `Reader`\n// might be incomplete here.\nvoid DeleteReader(Reader* reader);\n\n}  // namespace writer_internal\n\ntemplate <typename ReaderClass>\ninline AssociatedReader<ReaderClass>::AssociatedReader(\n    AssociatedReader&& that) noexcept\n    : reader_(std::exchange(that.reader_, nullptr)) {}\n\ntemplate <typename ReaderClass>\ninline AssociatedReader<ReaderClass>& AssociatedReader<ReaderClass>::operator=(\n    AssociatedReader&& that) noexcept {\n  Delete(std::exchange(reader_, std::exchange(that.reader_, nullptr)));\n  return *this;\n}\n\ntemplate <typename ReaderClass>\ninline AssociatedReader<ReaderClass>::~AssociatedReader() {\n  Delete(reader_);\n}\n\ntemplate <typename ReaderClass>\ninline void AssociatedReader<ReaderClass>::Reset() {\n  Delete(std::exchange(reader_, nullptr));\n}\n\ntemplate <typename ReaderClass>\ntemplate <typename... Args>\ninline ReaderClass* AssociatedReader<ReaderClass>::ResetReader(Args&&... args) {\n  if (ABSL_PREDICT_FALSE(reader_ == nullptr)) {\n    reader_ = new ReaderClass(std::forward<Args>(args)...);\n  } else {\n    riegeli::Reset(*reader(), std::forward<Args>(args)...);\n  }\n  return reader();\n}\n\ntemplate <typename ReaderClass>\nReaderClass* AssociatedReader<ReaderClass>::reader() const {\n  return static_cast<ReaderClass*>(reader_);\n}\n\ntemplate <typename ReaderClass>\nvoid AssociatedReader<ReaderClass>::Delete(Reader* reader) {\n  if (ABSL_PREDICT_FALSE(reader != nullptr)) {\n    writer_internal::DeleteReader(reader);\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_WRITER_H_\n"
  },
  {
    "path": "riegeli/bytes/writer_cfile.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// Make `fopencookie()` and `off64_t` available.\n#if !defined(_GNU_SOURCE)\n#define _GNU_SOURCE\n#endif\n\n#include \"riegeli/bytes/writer_cfile.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n#include <stdio.h>\n#include <sys/types.h>\n\n#include <cerrno>\n#include <limits>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/dynamic_annotations.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/errno_mapping.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/cfile_handle.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli::cfile_internal {\n\nWriterCFileCookieBase::~WriterCFileCookieBase() {}\n\nvoid WriterCFileCookieBase::Initialize(Writer* writer) {\n  RIEGELI_ASSERT_NE(writer, nullptr)\n      << \"Failed precondition of WriterCFile(): null Writer pointer\";\n  if (flush_type_ != std::nullopt) writer->Flush(*flush_type_);\n}\n\ninline const char* WriterCFileCookieBase::OpenMode() {\n  Writer& writer = *DestWriter();\n  return writer.SupportsReadMode() && writer.SupportsRandomAccess() ? \"w+\"\n                                                                    : \"w\";\n}\n\ninline ssize_t WriterCFileCookieBase::Read(char* dest, size_t length) {\n  if (ABSL_PREDICT_FALSE(reader_ == nullptr)) {\n    Writer& writer = *DestWriter();\n    const Position pos = writer.pos();\n    reader_ = writer.ReadMode(pos);\n    if (ABSL_PREDICT_FALSE(reader_ == nullptr)) {\n      errno = StatusCodeToErrno(writer.status().code());\n      return -1;\n    }\n    if (ABSL_PREDICT_FALSE(reader_->pos() != pos)) {\n      if (ABSL_PREDICT_FALSE(!reader_->ok())) {\n        errno = StatusCodeToErrno(reader_->status().code());\n      } else {\n        errno = EINVAL;\n      }\n      return -1;\n    }\n  }\n  size_t length_read;\n  if (ABSL_PREDICT_FALSE(!reader_->ReadSome(length, dest, &length_read) &&\n                         !reader_->ok())) {\n    errno = StatusCodeToErrno(reader_->status().code());\n    return -1;\n  }\n  return IntCast<ssize_t>(length_read);\n}\n\ninline ssize_t WriterCFileCookieBase::Write(const char* src, size_t length) {\n  // Msan does not properly track initialization performed by precompiled\n  // libraries. The data to write might have been composed by e.g. `fprintf()`.\n  ABSL_ANNOTATE_MEMORY_IS_INITIALIZED(src, length);\n  Writer& writer = *DestWriter();\n  if (ABSL_PREDICT_FALSE(reader_ != nullptr)) {\n    const Position pos = reader_->pos();\n    reader_ = nullptr;\n    if (ABSL_PREDICT_FALSE(!writer.Seek(pos))) {\n      if (ABSL_PREDICT_FALSE(!writer.ok())) {\n        errno = StatusCodeToErrno(writer.status().code());\n      } else {\n        errno = EINVAL;\n      }\n      return 0;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!writer.Write(absl::string_view(src, length)))) {\n    errno = StatusCodeToErrno(writer.status().code());\n    return 0;\n  }\n  if (flush_type_ != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(!writer.Flush(*flush_type_))) {\n      errno = StatusCodeToErrno(writer.status().code());\n      return 0;\n    }\n  }\n  return IntCast<ssize_t>(length);\n}\n\ninline std::optional<int64_t> WriterCFileCookieBase::Seek(int64_t offset,\n                                                          int whence) {\n  Writer& writer = *DestWriter();\n  Position new_pos;\n  switch (whence) {\n    case SEEK_SET:\n      if (ABSL_PREDICT_FALSE(offset < 0)) {\n        errno = EINVAL;\n        return std::nullopt;\n      }\n      new_pos = IntCast<Position>(offset);\n      break;\n    case SEEK_CUR:\n      new_pos = reader_ != nullptr ? reader_->pos() : writer.pos();\n      if (offset < 0) {\n        if (ABSL_PREDICT_FALSE(NegatingUnsignedCast(offset) > new_pos)) {\n          errno = EINVAL;\n          return std::nullopt;\n        }\n        new_pos -= NegatingUnsignedCast(offset);\n        if (ABSL_PREDICT_FALSE(new_pos >\n                               Position{std::numeric_limits<int64_t>::max()})) {\n          errno = EINVAL;\n          return std::nullopt;\n        }\n      } else {\n        if (ABSL_PREDICT_FALSE(\n                new_pos > Position{std::numeric_limits<int64_t>::max()} ||\n                IntCast<Position>(offset) >\n                    Position{std::numeric_limits<int64_t>::max()} - new_pos)) {\n          errno = EINVAL;\n          return std::nullopt;\n        }\n        new_pos += IntCast<Position>(offset);\n      }\n      break;\n    case SEEK_END: {\n      if (ABSL_PREDICT_FALSE(!writer.SupportsRandomAccess())) {\n        // Indicate that `fseek(SEEK_END)` is not supported.\n        errno = ESPIPE;\n        return std::nullopt;\n      }\n      std::optional<Position> size;\n      if (reader_ != nullptr) {\n        RIEGELI_ASSERT(reader_->SupportsSize())\n            << \"Failed postcondition of Writer::ReadMode(): \"\n               \"!Reader::SupportsSize() even though \"\n               \"Writer::SupportsRandomAccess()\";\n        size = reader_->Size();\n        if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n          errno = StatusCodeToErrno(reader_->status().code());\n          return std::nullopt;\n        }\n      } else {\n        size = writer.Size();\n        if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n          errno = StatusCodeToErrno(writer.status().code());\n          return std::nullopt;\n        }\n      }\n      if (ABSL_PREDICT_FALSE(offset > 0 ||\n                             NegatingUnsignedCast(offset) > *size)) {\n        errno = EINVAL;\n        return std::nullopt;\n      }\n      new_pos = *size - NegatingUnsignedCast(offset);\n      if (ABSL_PREDICT_FALSE(new_pos >\n                             Position{std::numeric_limits<int64_t>::max()})) {\n        errno = EINVAL;\n        return std::nullopt;\n      }\n    } break;\n    default:\n      RIEGELI_ASSUME_UNREACHABLE() << \"Unknown seek origin: \" << whence;\n  }\n  if (new_pos == (reader_ != nullptr ? reader_->pos() : writer.pos())) {\n    // Seeking to the current position is supported even if random access is\n    // not.\n    return IntCast<int64_t>(new_pos);\n  }\n  if (reader_ != nullptr) {\n    RIEGELI_ASSERT(reader_->SupportsRewind())\n        << \"Failed postcondition of Writer::ReadMode(): \"\n           \"!Reader::SupportsRewind()\";\n    if (ABSL_PREDICT_FALSE(!reader_->Seek(IntCast<Position>(new_pos)))) {\n      if (ABSL_PREDICT_FALSE(!reader_->ok())) {\n        errno = StatusCodeToErrno(reader_->status().code());\n      } else {\n        errno = EINVAL;\n      }\n      return std::nullopt;\n    }\n  } else {\n    if (ABSL_PREDICT_FALSE(!writer.SupportsRandomAccess())) {\n      // Indicate that `fseek()` is not supported.\n      errno = ESPIPE;\n      return std::nullopt;\n    }\n    if (ABSL_PREDICT_FALSE(!writer.Seek(IntCast<Position>(new_pos)))) {\n      if (ABSL_PREDICT_FALSE(!writer.ok())) {\n        errno = StatusCodeToErrno(writer.status().code());\n      } else {\n        errno = EINVAL;\n      }\n      return std::nullopt;\n    }\n  }\n  return IntCast<int64_t>(new_pos);\n}\n\n// `extern \"C\"` sets the C calling convention for compatibility with\n// `fopencookie()`. `static` avoids making symbols public, as `extern \"C\"`\n// trumps anonymous namespace.\nextern \"C\" {\n\nstatic ssize_t WriterCFileRead(void* cookie, char* buf, size_t size) {\n  return static_cast<WriterCFileCookieBase*>(cookie)->Read(buf, size);\n}\n\nstatic ssize_t WriterCFileWrite(void* cookie, const char* buf, size_t size) {\n  return static_cast<WriterCFileCookieBase*>(cookie)->Write(buf, size);\n}\n\nstatic int WriterCFileSeek(void* cookie, off64_t* offset, int whence) {\n  const std::optional<int64_t> new_pos =\n      static_cast<WriterCFileCookieBase*>(cookie)->Seek(\n          IntCast<int64_t>(*offset), whence);\n  if (ABSL_PREDICT_FALSE(new_pos == std::nullopt)) {\n    *offset = -1;\n    return -1;\n  }\n  *offset = IntCast<off64_t>(*new_pos);\n  return 0;\n}\n\nstatic int WriterCFileClose(void* cookie) {\n  const int result = static_cast<WriterCFileCookieBase*>(cookie)->Close();\n  delete static_cast<WriterCFileCookieBase*>(cookie);\n  if (ABSL_PREDICT_FALSE(result != 0)) {\n    errno = result;\n    return -1;\n  }\n  return 0;\n}\n\n}  // extern \"C\"\n\nOwnedCFile WriterCFileImpl(WriterCFileCookieBase* cookie,\n                           std::string&& filename) {\n  return OwnedCFile(fopencookie(cookie, cookie->OpenMode(),\n                                {WriterCFileRead, WriterCFileWrite,\n                                 WriterCFileSeek, WriterCFileClose}),\n                    std::move(filename));\n}\n\n}  // namespace riegeli::cfile_internal\n"
  },
  {
    "path": "riegeli/bytes/writer_cfile.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_WRITER_CFILE_H_\n#define RIEGELI_BYTES_WRITER_CFILE_H_\n\n#include <stddef.h>\n#include <stdint.h>\n#include <stdio.h>\n#include <sys/types.h>\n\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/errno_mapping.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/cfile_handle.h\"\n#include \"riegeli/bytes/path_ref.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nclass Reader;\n\nclass WriterCFileOptions {\n public:\n  WriterCFileOptions() noexcept {}\n\n  // The filename assumed by the returned `OwnedCFile`.\n  //\n  // Default: \"<unspecified>\".\n  WriterCFileOptions& set_filename(PathInitializer filename) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    riegeli::Reset(filename_, std::move(filename));\n    return *this;\n  }\n  WriterCFileOptions&& set_filename(PathInitializer filename) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_filename(std::move(filename)));\n  }\n  std::string& filename() ABSL_ATTRIBUTE_LIFETIME_BOUND { return filename_; }\n  const std::string& filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return filename_;\n  }\n\n  // If `std::nullopt`, `fflush()` pushes data buffered in the `FILE` to the\n  // `Writer`, but does not call `Writer::Flush()`. There is no way to trigger\n  // `Writer::Flush()` from the `FILE`.\n  //\n  // If a `FlushType`, pushing data buffered in the `FILE` to the `Writer` calls\n  // `Writer::Flush()`. This is done by `fflush()`, but also when the `FILE`\n  // buffer is full. This degrades performance, but `fflush()` works as\n  // expected.\n  //\n  // There is no way for `WriterCFile()` to distinguish `fflush()` from the\n  // `FILE` buffer being full, hence this option.\n  //\n  // Default: `std::nullopt`.\n  WriterCFileOptions& set_flush_type(std::optional<FlushType> flush_type) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    flush_type_ = flush_type;\n    return *this;\n  }\n  WriterCFileOptions&& set_flush_type(std::optional<FlushType> flush_type) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_flush_type(flush_type));\n  }\n  std::optional<FlushType> flush_type() const { return flush_type_; }\n\n private:\n  std::string filename_ = \"<unspecified>\";\n  std::optional<FlushType> flush_type_;\n};\n\n// A `FILE` which writes data to a `Writer`.\n//\n// This requires `fopencookie()` to be supported by the C library.\n//\n// The `FILE` supports reading and writing if `Writer::SupportsReadMode()`.\n// Otherwise the `FILE` is write-only.\n//\n// The `FILE` supports `fseek()` if `Writer::SupportsRandomAccess()` or\n// `Writer::SupportsReadMode()`, which implies `Reader::SupportsRewind()`.\n// Seeking to the current position is always supported.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// `dest` supports `riegeli::Maker<Dest>(args...)` to construct `Dest` in-place.\n//\n// The `Writer` must not be accessed until the `FILE` is closed. Warning: this\n// includes implicit closing of all `FILE` objects which are still open at\n// program exit, hence if the `FILE` persists until program exit, then the\n// `Writer` must do so as well.\ntemplate <\n    typename Dest,\n    std::enable_if_t<TargetSupportsDependency<Writer*, Dest>::value, int> = 0>\nOwnedCFile WriterCFile(Dest&& dest,\n                       WriterCFileOptions options = WriterCFileOptions());\n\n// Implementation details follow.\n\nnamespace cfile_internal {\n\nclass WriterCFileCookieBase {\n public:\n  virtual ~WriterCFileCookieBase();\n\n  const char* OpenMode();\n\n  ssize_t Read(char* dest, size_t length);\n\n  ssize_t Write(const char* src, size_t length);\n\n  // Use `int64_t` instead of `off64_t` to avoid a dependency on\n  // `#define _GNU_SOURCE` in a header.\n  std::optional<int64_t> Seek(int64_t offset, int whence);\n\n  // Returns 0 on success, or `errno` value on failure.\n  virtual int Close() = 0;\n\n protected:\n  explicit WriterCFileCookieBase(std::optional<FlushType> flush_type) noexcept\n      : flush_type_(flush_type) {}\n\n  WriterCFileCookieBase(const WriterCFileCookieBase&) = delete;\n  WriterCFileCookieBase& operator=(const WriterCFileCookieBase&) = delete;\n\n  virtual Writer* DestWriter() = 0;\n\n  void Initialize(Writer* writer);\n\n  std::optional<FlushType> flush_type_;\n  // If `nullptr`, `*DestWriter()` was used last time. If not `nullptr`,\n  // `*reader_` was used last time.\n  Reader* reader_ = nullptr;\n};\n\ntemplate <typename Dest>\nclass WriterCFileCookie : public WriterCFileCookieBase {\n public:\n  explicit WriterCFileCookie(Initializer<Dest> dest,\n                             std::optional<FlushType> flush_type)\n      : WriterCFileCookieBase(flush_type), dest_(std::move(dest)) {\n    Initialize(dest_.get());\n  }\n\n  int Close() override;\n\n protected:\n  Writer* DestWriter() override { return dest_.get(); }\n\n private:\n  Dependency<Writer*, Dest> dest_;\n};\n\ntemplate <typename Dest>\nint WriterCFileCookie<Dest>::Close() {\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      return StatusCodeToErrno(dest_->status().code());\n    }\n  }\n  return 0;\n}\n\nOwnedCFile WriterCFileImpl(WriterCFileCookieBase* cookie,\n                           std::string&& filename);\n\n}  // namespace cfile_internal\n\ntemplate <typename Dest,\n          std::enable_if_t<TargetSupportsDependency<Writer*, Dest>::value, int>>\nOwnedCFile WriterCFile(Dest&& dest, WriterCFileOptions options) {\n  return cfile_internal::WriterCFileImpl(\n      new cfile_internal::WriterCFileCookie<TargetT<Dest>>(\n          std::forward<Dest>(dest), options.flush_type()),\n      std::move(options.filename()));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_WRITER_CFILE_H_\n"
  },
  {
    "path": "riegeli/bytes/writer_ostream.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bytes/writer_ostream.h\"\n\n#include <stddef.h>\n\n#include <ios>\n#include <iosfwd>\n#include <limits>\n#include <optional>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nnamespace stream_internal {\n\nclass WriterStreambuf::BufferSync {\n public:\n  explicit BufferSync(WriterStreambuf* streambuf) : streambuf_(streambuf) {\n    if (streambuf_->reader_ != nullptr) {\n      streambuf_->reader_->set_cursor(streambuf_->gptr());\n    } else {\n      streambuf_->writer_->set_cursor(streambuf_->pptr());\n    }\n  }\n\n  BufferSync(const BufferSync&) = delete;\n  BufferSync& operator=(const BufferSync&) = delete;\n\n  ~BufferSync() {\n    if (streambuf_->reader_ != nullptr) {\n      streambuf_->setg(const_cast<char*>(streambuf_->reader_->start()),\n                       const_cast<char*>(streambuf_->reader_->cursor()),\n                       const_cast<char*>(streambuf_->reader_->limit()));\n    } else {\n      streambuf_->setp(streambuf_->writer_->cursor(),\n                       streambuf_->writer_->limit());\n    }\n  }\n  WriterStreambuf* streambuf_;\n};\n\nstd::optional<Position> WriterStreambuf::MoveBegin() {\n  // In a closed `WriterOstream`, `WriterOstream::writer_ != nullptr`\n  // does not imply `WriterStreambuf::writer_ != nullptr`, because\n  // `WriterOstream::streambuf_` can be left uninitialized.\n  if (writer_ == nullptr) return std::nullopt;\n  if (reader_ != nullptr) {\n    reader_->set_cursor(gptr());\n    return reader_->pos();\n  } else {\n    writer_->set_cursor(pptr());\n    return std::nullopt;\n  }\n}\n\nvoid WriterStreambuf::MoveEnd(Writer* dest,\n                              std::optional<Position> reader_pos) {\n  // In a closed `WriterOStream`, `WriterOStream::writer_ != nullptr`\n  // does not imply `WriterStreambuf::writer_ != nullptr`, because\n  // `WriterOStream::streambuf_` can be left uninitialized.\n  if (writer_ == nullptr) return;\n  writer_ = dest;\n  if (reader_pos != std::nullopt) {\n    reader_ = writer_->ReadMode(*reader_pos);\n    if (ABSL_PREDICT_FALSE(reader_ == nullptr)) {\n      FailWriter();\n      setg(nullptr, nullptr, nullptr);\n      return;\n    }\n    if (ABSL_PREDICT_FALSE(reader_->pos() != *reader_pos)) {\n      if (!reader_->ok()) {\n        FailReader();\n      } else {\n        state_.Fail(absl::OutOfRangeError(absl::StrCat(\n            \"Current read position out of range for reading: position \",\n            *reader_pos, \" > stream size \", reader_->pos())));\n      }\n      setg(nullptr, nullptr, nullptr);\n      return;\n    }\n    setg(const_cast<char*>(reader_->start()),\n         const_cast<char*>(reader_->cursor()),\n         const_cast<char*>(reader_->limit()));\n  } else {\n    setp(writer_->cursor(), writer_->limit());\n  }\n}\n\nvoid WriterStreambuf::Done() {\n  if (reader_ != nullptr) {\n    reader_->set_cursor(gptr());\n    setg(nullptr, nullptr, nullptr);\n  } else {\n    writer_->set_cursor(pptr());\n    setp(nullptr, nullptr);\n  }\n}\n\nvoid WriterStreambuf::FailReader() { state_.Fail(reader_->status()); }\n\nvoid WriterStreambuf::FailWriter() { state_.Fail(writer_->status()); }\n\nbool WriterStreambuf::ReadMode() {\n  if (ABSL_PREDICT_FALSE(reader_ == nullptr)) {\n    setp(nullptr, nullptr);\n    const Position new_pos = writer_->pos();\n    reader_ = writer_->ReadMode(new_pos);\n    if (ABSL_PREDICT_FALSE(reader_ == nullptr)) {\n      FailWriter();\n      return false;\n    }\n    if (ABSL_PREDICT_FALSE(reader_->pos() != new_pos)) {\n      if (!reader_->ok()) {\n        FailReader();\n      } else {\n        state_.Fail(absl::OutOfRangeError(absl::StrCat(\n            \"Current write position out of range for reading: position \",\n            new_pos, \" > stream size \", reader_->pos())));\n      }\n      return false;\n    }\n  }\n  return true;\n}\n\nbool WriterStreambuf::WriteMode() {\n  if (ABSL_PREDICT_FALSE(reader_ != nullptr)) {\n    setg(nullptr, nullptr, nullptr);\n    const Position new_pos = reader_->pos();\n    reader_ = nullptr;\n    if (ABSL_PREDICT_FALSE(!writer_->Seek(new_pos))) {\n      if (!writer_->ok()) {\n        FailWriter();\n      } else {\n        state_.Fail(absl::OutOfRangeError(absl::StrCat(\n            \"Current read position out of range for writing: position \",\n            new_pos, \" > stream size \", writer_->pos())));\n      }\n      return false;\n    }\n  }\n  return true;\n}\n\nint WriterStreambuf::sync() {\n  if (ABSL_PREDICT_FALSE(!ok())) return -1;\n  BufferSync buffer_sync(this);\n  if (reader_ != nullptr) {\n    if (ABSL_PREDICT_FALSE(!reader_->Sync())) {\n      FailReader();\n      return -1;\n    }\n  } else {\n    if (ABSL_PREDICT_FALSE(!writer_->Flush())) {\n      FailWriter();\n      return -1;\n    }\n  }\n  return 0;\n}\n\nstd::streamsize WriterStreambuf::showmanyc() {\n  if (ABSL_PREDICT_FALSE(!ok())) return -1;\n  BufferSync buffer_sync(this);\n  if (reader_ == nullptr && ABSL_PREDICT_FALSE(!writer_->SupportsReadMode())) {\n    // Indicate that reading is not supported.\n    return -1;\n  }\n  if (ABSL_PREDICT_FALSE(!ReadMode())) return -1;\n  if (ABSL_PREDICT_FALSE(!reader_->Pull())) {\n    if (ABSL_PREDICT_FALSE(!reader_->ok())) FailReader();\n    return -1;\n  }\n  return IntCast<std::streamsize>(reader_->available());\n}\n\nint WriterStreambuf::underflow() {\n  if (ABSL_PREDICT_FALSE(!ok())) return traits_type::eof();\n  BufferSync buffer_sync(this);\n  if (reader_ == nullptr && ABSL_PREDICT_FALSE(!writer_->SupportsReadMode())) {\n    // Indicate that reading is not supported.\n    return traits_type::eof();\n  }\n  if (ABSL_PREDICT_FALSE(!ReadMode())) return traits_type::eof();\n  if (ABSL_PREDICT_FALSE(!reader_->Pull())) {\n    if (ABSL_PREDICT_FALSE(!reader_->ok())) FailReader();\n    return traits_type::eof();\n  }\n  return traits_type::to_int_type(*reader_->cursor());\n}\n\nstd::streamsize WriterStreambuf::xsgetn(char* dest, std::streamsize length) {\n  RIEGELI_ASSERT_GE(length, 0)\n      << \"Failed precondition of streambuf::xsgetn(): negative length\";\n  if (ABSL_PREDICT_TRUE(length <= egptr() - gptr())) {\n    riegeli::null_safe_memcpy(dest, gptr(), IntCast<size_t>(length));\n    // Do not use `gbump()` because its parameter has type `int`.\n    setg(eback(), gptr() + length, egptr());\n    return length;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return 0;\n  BufferSync buffer_sync(this);\n  if (reader_ == nullptr && ABSL_PREDICT_FALSE(!writer_->SupportsReadMode())) {\n    // Indicate that reading is not supported.\n    return 0;\n  }\n  if (ABSL_PREDICT_FALSE(!ReadMode())) return 0;\n  size_t length_read;\n  if (ABSL_PREDICT_FALSE(\n          !reader_->Read(IntCast<size_t>(length), dest, &length_read)) &&\n      ABSL_PREDICT_FALSE(!reader_->ok())) {\n    FailReader();\n  }\n  return IntCast<std::streamsize>(length_read);\n}\n\nint WriterStreambuf::overflow(int src) {\n  if (ABSL_PREDICT_FALSE(!ok())) return traits_type::eof();\n  BufferSync buffer_sync(this);\n  if (ABSL_PREDICT_FALSE(!WriteMode())) return traits_type::eof();\n  if (ABSL_PREDICT_FALSE(!writer_->Push())) {\n    FailWriter();\n    return traits_type::eof();\n  }\n  if (src != traits_type::eof()) {\n    *writer_->cursor() = traits_type::to_char_type(src);\n    writer_->move_cursor(1);\n  }\n  return traits_type::not_eof(src);\n}\n\nstd::streamsize WriterStreambuf::xsputn(const char* src,\n                                        std::streamsize length) {\n  RIEGELI_ASSERT_GE(length, 0)\n      << \"Failed precondition of streambuf::xsputn(): negative length\";\n  if (ABSL_PREDICT_TRUE(length <= epptr() - pptr())) {\n    riegeli::null_safe_memcpy(pptr(), src, IntCast<size_t>(length));\n    // Do not use `pbump()` because its parameter has type `int`.\n    setp(pptr() + length, epptr());\n    return length;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return 0;\n  BufferSync buffer_sync(this);\n  if (ABSL_PREDICT_FALSE(!WriteMode())) return 0;\n  const Position pos_before = writer_->pos();\n  if (ABSL_PREDICT_FALSE(\n          !writer_->Write(absl::string_view(src, IntCast<size_t>(length))))) {\n    FailWriter();\n    // `Write()` could have decreased `pos()` on failure.\n    const Position length_written = SaturatingSub(writer_->pos(), pos_before);\n    RIEGELI_ASSERT_LE(length_written, IntCast<size_t>(length))\n        << \"Writer::Write(absl::string_view) wrote more than requested\";\n    return IntCast<std::streamsize>(length_written);\n  }\n  return length;\n}\n\nstd::streampos WriterStreambuf::seekoff(std::streamoff off,\n                                        std::ios_base::seekdir dir,\n                                        std::ios_base::openmode which) {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::streampos(std::streamoff{-1});\n  BufferSync buffer_sync(this);\n  Position new_pos;\n  switch (dir) {\n    case std::ios_base::beg:\n      if (ABSL_PREDICT_FALSE(off < 0)) {\n        return std::streampos(std::streamoff{-1});\n      }\n      new_pos = IntCast<Position>(off);\n      break;\n    case std::ios_base::cur:\n      new_pos = reader_ != nullptr ? reader_->pos() : writer_->pos();\n      if (off < 0) {\n        if (ABSL_PREDICT_FALSE(NegatingUnsignedCast(off) > new_pos)) {\n          return std::streampos(std::streamoff{-1});\n        }\n        new_pos -= NegatingUnsignedCast(off);\n        if (ABSL_PREDICT_FALSE(\n                new_pos >\n                Position{std::numeric_limits<std::streamoff>::max()})) {\n          return std::streampos(std::streamoff{-1});\n        }\n      } else {\n        if (ABSL_PREDICT_FALSE(\n                new_pos >\n                    Position{std::numeric_limits<std::streamoff>::max()} ||\n                IntCast<Position>(off) >\n                    Position{std::numeric_limits<std::streamoff>::max()} -\n                        new_pos)) {\n          return std::streampos(std::streamoff{-1});\n        }\n        new_pos += IntCast<Position>(off);\n      }\n      break;\n    case std::ios_base::end: {\n      std::optional<Position> size;\n      if (reader_ != nullptr) {\n        if (ABSL_PREDICT_FALSE(!reader_->SupportsSize())) {\n          // Indicate that `seekoff(std::ios_base::end)` is not supported.\n          return std::streampos(std::streamoff{-1});\n        }\n        size = reader_->Size();\n        if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n          FailReader();\n          return std::streampos(std::streamoff{-1});\n        }\n      } else {\n        if (ABSL_PREDICT_FALSE(!writer_->SupportsRandomAccess())) {\n          // Indicate that `seekoff(std::ios_base::end)` is not supported.\n          return std::streampos(std::streamoff{-1});\n        }\n        size = writer_->Size();\n        if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n          FailWriter();\n          return std::streampos(std::streamoff{-1});\n        }\n      }\n      if (ABSL_PREDICT_FALSE(off > 0 || NegatingUnsignedCast(off) > *size)) {\n        return std::streampos(std::streamoff{-1});\n      }\n      new_pos = *size - NegatingUnsignedCast(off);\n      if (ABSL_PREDICT_FALSE(\n              new_pos > Position{std::numeric_limits<std::streamoff>::max()})) {\n        return std::streampos(std::streamoff{-1});\n      }\n    } break;\n    default:\n      RIEGELI_ASSUME_UNREACHABLE()\n          << \"Unknown seek direction: \" << static_cast<int>(dir);\n  }\n  if ((which & std::ios_base::in) != 0) {\n    // Switch to read mode.\n    bool seek_ok;\n    if (ABSL_PREDICT_FALSE(reader_ == nullptr)) {\n      setp(nullptr, nullptr);\n      reader_ = writer_->ReadMode(new_pos);\n      if (ABSL_PREDICT_FALSE(reader_ == nullptr)) {\n        FailWriter();\n        return std::streampos(std::streamoff{-1});\n      }\n      seek_ok = reader_->pos() == new_pos;\n    } else {\n      RIEGELI_ASSERT(reader_->SupportsRewind())\n          << \"Failed postcondition of Writer::ReadMode(): \"\n             \"!Reader::SupportsRewind()\";\n      seek_ok = reader_->Seek(new_pos);\n    }\n    if (ABSL_PREDICT_FALSE(!seek_ok)) {\n      if (ABSL_PREDICT_FALSE(!reader_->ok())) FailReader();\n      return std::streampos(std::streamoff{-1});\n    }\n  } else {\n    // Switch to write mode.\n    if (ABSL_PREDICT_FALSE(reader_ != nullptr)) {\n      setg(nullptr, nullptr, nullptr);\n      reader_ = nullptr;\n    }\n    if (new_pos == writer_->pos()) {\n      // Seeking to the current position is supported even if random access is\n      // not.\n    } else {\n      if (ABSL_PREDICT_FALSE(!writer_->SupportsRandomAccess())) {\n        // Indicate that `seekoff()` is not supported.\n        return std::streampos(std::streamoff{-1});\n      }\n      if (ABSL_PREDICT_FALSE(!writer_->Seek(new_pos))) {\n        if (ABSL_PREDICT_FALSE(!writer_->ok())) FailWriter();\n        return std::streampos(std::streamoff{-1});\n      }\n    }\n  }\n  return std::streampos(IntCast<std::streamoff>(new_pos));\n}\n\nstd::streampos WriterStreambuf::seekpos(std::streampos pos,\n                                        std::ios_base::openmode which) {\n  return seekoff(std::streamoff(pos), std::ios_base::beg, which);\n}\n\n}  // namespace stream_internal\n\nbool WriterOStreamBase::close() {\n  Done();\n  return not_failed();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bytes/writer_ostream.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BYTES_WRITER_OSTREAM_H_\n#define RIEGELI_BYTES_WRITER_OSTREAM_H_\n\n#include <ios>\n#include <iosfwd>\n#include <istream>\n#include <optional>\n#include <streambuf>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nclass Reader;\n\nnamespace stream_internal {\n\nclass WriterStreambuf : public std::streambuf {\n public:\n  explicit WriterStreambuf(Closed) noexcept : state_(kClosed) {}\n\n  WriterStreambuf() noexcept {}\n\n  WriterStreambuf(WriterStreambuf&& that) noexcept;\n  WriterStreambuf& operator=(WriterStreambuf&& that) noexcept;\n\n  void Initialize(Writer* dest);\n  std::optional<Position> MoveBegin();\n  void MoveEnd(Writer* dest, std::optional<Position> reader_pos);\n  void Done();\n\n  bool ok() const { return state_.ok(); }\n  bool is_open() const { return state_.is_open(); }\n  bool not_failed() const { return state_.not_failed(); }\n  absl::Status status() const { return state_.status(); }\n  void MarkClosed() { state_.MarkClosed(); }\n  ABSL_ATTRIBUTE_COLD void FailReader();\n  ABSL_ATTRIBUTE_COLD void FailWriter();\n\n protected:\n  int sync() override;\n  std::streamsize showmanyc() override;\n  int underflow() override;\n  std::streamsize xsgetn(char* dest, std::streamsize length) override;\n  int overflow(int src) override;\n  std::streamsize xsputn(const char* src, std::streamsize length) override;\n  std::streampos seekoff(std::streamoff off, std::ios_base::seekdir dir,\n                         std::ios_base::openmode which) override;\n  std::streampos seekpos(std::streampos pos,\n                         std::ios_base::openmode which) override;\n\n private:\n  class BufferSync;\n\n  bool ReadMode();\n  bool WriteMode();\n\n  ObjectState state_;\n  Writer* writer_ = nullptr;\n  // If `nullptr`, `*writer_` was used last time. If not `nullptr`, `*reader_`\n  // was used last time.\n  Reader* reader_ = nullptr;\n\n  // Invariants:\n  //   `is_open() && reader_ == nullptr ? pbase() >= writer_->start()\n  //                                    : pbase() == nullptr`\n  //   `epptr() == (is_open() && reader_ == nullptr ? writer_->limit()\n  //                                                : nullptr)`\n  //   `eback() == (is_open() && reader_ != nullptr ? reader_->start()\n  //                                                : nullptr)`\n  //   `egptr() == (is_open() && reader_ != nullptr ? reader_->limit()\n  //                                                : nullptr)`\n};\n\n}  // namespace stream_internal\n\n// Template parameter independent part of `WriterOStream`.\nclass WriterOStreamBase : public std::iostream {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n  };\n\n  // Returns the `Writer`. Unchanged by `close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // If `!is_open()`, does nothing. Otherwise:\n  //  * Synchronizes the current `WriterOStream` position to the `Writer`.\n  //  * Closes the `Writer` if it is owned.\n  //\n  // Returns `true` if the `Writer` did not fail, i.e. if it was OK just before\n  // becoming closed.\n  //\n  // Destroying or assigning to a `WriterOStream` closes it implicitly, but an\n  // explicit `close()` call allows to detect failures (use `status()` for\n  // failure details).\n  bool close();\n\n  // Returns `true` if the `WriterOStream` is OK, i.e. open and not failed.\n  bool ok() const { return streambuf_.ok(); }\n\n  // Returns `true` if the `WriterOStream` is open, i.e. not closed.\n  bool is_open() const { return streambuf_.is_open(); }\n\n  // Returns `true` if the `WriterOStream` is not failed.\n  bool not_failed() const { return streambuf_.not_failed(); }\n\n  // Returns an `absl::Status` describing the failure if the `WriterOStream`\n  // is failed, or an `absl::FailedPreconditionError()` if the `WriterOStream`\n  // is successfully closed, or `absl::OkStatus()` if the `WriterOStream` is OK.\n  absl::Status status() const { return streambuf_.status(); }\n\n  // Supports `Dependency`.\n  friend MakerType<Closed> RiegeliDependencySentinel(WriterOStreamBase*) {\n    return {kClosed};\n  }\n\n protected:\n  explicit WriterOStreamBase(Closed) noexcept\n      : std::iostream(&streambuf_), streambuf_(kClosed) {}\n\n  WriterOStreamBase() noexcept : std::iostream(&streambuf_) {}\n\n  WriterOStreamBase(WriterOStreamBase&& that) noexcept;\n  WriterOStreamBase& operator=(WriterOStreamBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(Writer* dest);\n\n  virtual void Done() = 0;\n\n  stream_internal::WriterStreambuf streambuf_;\n\n  // Invariant: `rdbuf() == &streambuf_`\n};\n\n// Adapts a `Writer` to a `std::iostream`.\n//\n// The `std::iostream` supports reading and writing if\n// `Writer::SupportsReadMode()`, with a single position maintained for both\n// reading and writing. Otherwise the `std::iostream` is write-only, and only\n// the `std::ostream` aspect of it is functional.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The `Writer` must not be accessed until the `WriterOStream` is closed or no\n// longer used, except that it is allowed to read the destination of the\n// `Writer` immediately after `flush()`.\n//\n// Destroying or assigning to a `WriterOStream` closes it first.\ntemplate <typename Dest = Writer*>\nclass WriterOStream : public WriterOStreamBase {\n public:\n  // Creates a closed `WriterOStream`.\n  explicit WriterOStream(Closed) noexcept : WriterOStreamBase(kClosed) {}\n\n  // Will write to the `Writer` provided by `dest`.\n  explicit WriterOStream(Initializer<Dest> dest, Options options = Options());\n\n  // These operations cannot be defaulted because `WriterOStreamBase` virtually\n  // derives from `std::ios` which has these operations deleted.\n  WriterOStream(WriterOStream&& that) noexcept\n#if __cpp_concepts\n    requires std::is_move_constructible_v<Dependency<Writer*, Dest>>\n#endif\n      : WriterOStreamBase(static_cast<WriterOStreamBase&&>(that)),\n        dest_(std::move(that.dest_), *this, that) {\n  }\n  WriterOStream& operator=(WriterOStream&& that) noexcept\n#if __cpp_concepts\n    requires(std::is_move_assignable_v<Dependency<Writer*, Dest>>)\n#endif\n  {\n    WriterOStreamBase::operator=(static_cast<WriterOStreamBase&&>(that));\n    dest_.Reset(std::move(that.dest_), *this, that);\n    return *this;\n  }\n\n  ~WriterOStream() override { Done(); }\n\n  // Makes `*this` equivalent to a newly constructed `WriterOStream`. This\n  // avoids constructing a temporary `WriterOStream` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the `Writer`. Unchanged by\n  // `close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the `Writer`.\n  MovingDependency<Writer*, Dest, Mover> dest_;\n};\n\nexplicit WriterOStream(Closed) -> WriterOStream<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit WriterOStream(Dest&& dest, WriterOStreamBase::Options options =\n                                        WriterOStreamBase::Options())\n    -> WriterOStream<TargetT<Dest>>;\n\n// Implementation details follow.\n\nnamespace stream_internal {\n\ninline WriterStreambuf::WriterStreambuf(WriterStreambuf&& that) noexcept\n    : std::streambuf(that),\n      state_(std::move(that.state_)),\n      writer_(that.writer_),\n      reader_(that.reader_) {\n  that.setg(nullptr, nullptr, nullptr);\n  that.setp(nullptr, nullptr);\n}\n\ninline WriterStreambuf& WriterStreambuf::operator=(\n    WriterStreambuf&& that) noexcept {\n  if (ABSL_PREDICT_TRUE(&that != this)) {\n    std::streambuf::operator=(that);\n    state_ = std::move(that.state_);\n    writer_ = that.writer_;\n    reader_ = that.reader_;\n    that.setg(nullptr, nullptr, nullptr);\n    that.setp(nullptr, nullptr);\n  }\n  return *this;\n}\n\ninline void WriterStreambuf::Initialize(Writer* dest) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of WriterStreambuf: null Writer pointer\";\n  writer_ = dest;\n  setp(writer_->cursor(), writer_->limit());\n  if (ABSL_PREDICT_FALSE(!writer_->ok())) FailWriter();\n}\n\n}  // namespace stream_internal\n\ninline WriterOStreamBase::WriterOStreamBase(WriterOStreamBase&& that) noexcept\n    : std::iostream(static_cast<std::iostream&&>(that)),\n      streambuf_(std::move(that.streambuf_)) {\n  set_rdbuf(&streambuf_);\n}\n\ninline WriterOStreamBase& WriterOStreamBase::operator=(\n    WriterOStreamBase&& that) noexcept {\n  if (ABSL_PREDICT_TRUE(&that != this)) {\n    Done();\n    std::iostream::operator=(static_cast<std::iostream&&>(that));\n    streambuf_ = std::move(that.streambuf_);\n  }\n  return *this;\n}\n\ninline void WriterOStreamBase::Reset(Closed) {\n  Done();\n  streambuf_ = stream_internal::WriterStreambuf(kClosed);\n  init(&streambuf_);\n}\n\ninline void WriterOStreamBase::Reset() {\n  Done();\n  streambuf_ = stream_internal::WriterStreambuf();\n  init(&streambuf_);\n}\n\ninline void WriterOStreamBase::Initialize(Writer* dest) {\n  streambuf_.Initialize(dest);\n  if (ABSL_PREDICT_FALSE(!streambuf_.ok())) setstate(std::ios_base::badbit);\n}\n\ntemplate <typename Dest>\nclass WriterOStream<Dest>::Mover {\n public:\n  explicit Mover(WriterOStream& self)\n      : reader_pos_(self.streambuf_.MoveBegin()) {}\n\n  void Done(WriterOStream& self) {\n    self.streambuf_.MoveEnd(self.dest_.get(), reader_pos_);\n  }\n\n private:\n  std::optional<Position> reader_pos_;\n};\n\ntemplate <typename Dest>\ninline WriterOStream<Dest>::WriterOStream(Initializer<Dest> dest,\n                                          Options options)\n    : dest_(std::move(dest)) {\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\ninline void WriterOStream<Dest>::Reset(Closed) {\n  WriterOStreamBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void WriterOStream<Dest>::Reset(Initializer<Dest> dest,\n                                       Options options) {\n  WriterOStreamBase::Reset();\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\nvoid WriterOStream<Dest>::Done() {\n  if (ABSL_PREDICT_FALSE(!is_open())) return;\n  streambuf_.Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) streambuf_.FailWriter();\n  }\n  if (ABSL_PREDICT_FALSE(!streambuf_.ok())) setstate(std::ios_base::badbit);\n  streambuf_.MarkClosed();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BYTES_WRITER_OSTREAM_H_\n"
  },
  {
    "path": "riegeli/bzip2/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"bzip2_reader\",\n    srcs = [\"bzip2_reader.cc\"],\n    hdrs = [\"bzip2_reader.h\"],\n    deps = [\n        \":bzip2_error\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_reader\",\n        \"//riegeli/bytes:reader\",\n        \"@bzip2//:bz2\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"bzip2_writer\",\n    srcs = [\"bzip2_writer.cc\"],\n    hdrs = [\"bzip2_writer.h\"],\n    deps = [\n        \":bzip2_error\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_writer\",\n        \"//riegeli/bytes:writer\",\n        \"@bzip2//:bz2\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"bzip2_error\",\n    srcs = [\"bzip2_error.cc\"],\n    hdrs = [\"bzip2_error.h\"],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \"//riegeli/base:assert\",\n        \"@bzip2//:bz2\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/bzip2/bzip2_error.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bzip2/bzip2_error.h\"\n\n#include <string>\n\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"bzlib.h\"\n#include \"riegeli/base/assert.h\"\n\nnamespace riegeli::bzip2_internal {\n\nabsl::Status Bzip2ErrorToStatus(absl::string_view operation, int bzlib_code) {\n  absl::StatusCode code;\n  switch (bzlib_code) {\n    case BZ_OK:\n    case BZ_RUN_OK:\n    case BZ_FLUSH_OK:\n    case BZ_FINISH_OK:\n      return absl::OkStatus();\n    case BZ_DATA_ERROR:\n    case BZ_DATA_ERROR_MAGIC:\n      code = absl::StatusCode::kInvalidArgument;\n      break;\n    case BZ_MEM_ERROR:\n      code = absl::StatusCode::kResourceExhausted;\n      break;\n    default:\n      // Should not happen.\n      code = absl::StatusCode::kInternal;\n      break;\n  }\n  std::string message = absl::StrCat(operation, \" failed\");\n  absl::string_view details;\n  switch (bzlib_code) {\n    case BZ_OK:\n    case BZ_RUN_OK:\n    case BZ_FLUSH_OK:\n    case BZ_FINISH_OK:\n      RIEGELI_ASSUME_UNREACHABLE() << \"Handled before switch\";\n    case BZ_STREAM_END:\n      details = \"stream end\";\n      break;\n    case BZ_SEQUENCE_ERROR:\n      details = \"sequence error\";\n      break;\n    case BZ_PARAM_ERROR:\n      details = \"parameter error\";\n      break;\n    case BZ_MEM_ERROR:\n      details = \"memory error\";\n      break;\n    case BZ_DATA_ERROR:\n      details = \"data error\";\n      break;\n    case BZ_DATA_ERROR_MAGIC:\n      details = \"data error (magic)\";\n      break;\n    case BZ_IO_ERROR:\n      details = \"I/O error\";\n      break;\n    case BZ_UNEXPECTED_EOF:\n      details = \"unexpected EOF\";\n      break;\n    case BZ_OUTBUFF_FULL:\n      details = \"output buffer full\";\n      break;\n    case BZ_CONFIG_ERROR:\n      details = \"config error\";\n      break;\n    default:\n      absl::StrAppend(&message, \": unknown bzlib error code: \", bzlib_code);\n      break;\n  }\n  if (!details.empty()) absl::StrAppend(&message, \": \", details);\n  return absl::Status(code, message);\n}\n\n}  // namespace riegeli::bzip2_internal\n"
  },
  {
    "path": "riegeli/bzip2/bzip2_error.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BZIP2_BZIP2_ERROR_H_\n#define RIEGELI_BZIP2_BZIP2_ERROR_H_\n\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n\nnamespace riegeli::bzip2_internal {\n\nabsl::Status Bzip2ErrorToStatus(absl::string_view operation, int bzlib_code);\n\n}  // namespace riegeli::bzip2_internal\n\n#endif  // RIEGELI_BZIP2_BZIP2_ERROR_H_\n"
  },
  {
    "path": "riegeli/bzip2/bzip2_reader.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bzip2/bzip2_reader.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"bzlib.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bzip2/bzip2_error.h\"\n\nnamespace riegeli {\n\nvoid Bzip2ReaderBase::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of Bzip2Reader: null Reader pointer\";\n  if (ABSL_PREDICT_FALSE(!src->ok()) && src->available() == 0) {\n    FailWithoutAnnotation(AnnotateOverSrc(src->status()));\n    return;\n  }\n  initial_compressed_pos_ = src->pos();\n  InitializeDecompressor();\n}\n\ninline void Bzip2ReaderBase::InitializeDecompressor() {\n  decompressor_ = riegeli::Maker<bz_stream>();\n  const int bzlib_code = BZ2_bzDecompressInit(decompressor_.get(), 0, 0);\n  if (ABSL_PREDICT_FALSE(bzlib_code != BZ_OK)) {\n    delete decompressor_.release();  // Skip `BZ2_bzDecompressEnd()`.\n    FailOperation(\"BZ2_bzDecompressInit()\", bzlib_code);\n  }\n}\n\nvoid Bzip2ReaderBase::Done() {\n  if (ABSL_PREDICT_FALSE(truncated_)) {\n    Reader& src = *SrcReader();\n    FailWithoutAnnotation(AnnotateOverSrc(src.AnnotateStatus(\n        absl::InvalidArgumentError(\"Truncated Bzip2-compressed stream\"))));\n  }\n  BufferedReader::Done();\n  decompressor_.reset();\n}\n\ninline bool Bzip2ReaderBase::FailOperation(absl::string_view operation,\n                                           int bzlib_code) {\n  RIEGELI_ASSERT(bzlib_code != BZ_OK && bzlib_code != BZ_RUN_OK &&\n                 bzlib_code != BZ_FLUSH_OK && bzlib_code != BZ_FINISH_OK)\n      << \"Failed precondition of Bzip2ReaderBase::FailOperation(): \"\n         \"bzlib error code not failed\";\n  RIEGELI_ASSERT(is_open())\n      << \"Failed precondition of Bzip2ReaderBase::FailOperation(): \"\n         \"Object closed\";\n  return Fail(bzip2_internal::Bzip2ErrorToStatus(operation, bzlib_code));\n}\n\nabsl::Status Bzip2ReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    if (ABSL_PREDICT_FALSE(truncated_)) {\n      status = Annotate(status, \"reading truncated Bzip2-compressed stream\");\n    }\n    Reader& src = *SrcReader();\n    status = src.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `src` with the compressed position.\n  // Clarify that the current position is the uncompressed position instead of\n  // delegating to `BufferedReader::AnnotateStatusImpl()`.\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status Bzip2ReaderBase::AnnotateOverSrc(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nbool Bzip2ReaderBase::ReadInternal(size_t min_length, size_t max_length,\n                                   char* dest) {\n  RIEGELI_ASSERT_GT(min_length, 0u)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"nothing to read\";\n  RIEGELI_ASSERT_GE(max_length, min_length)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"max_length < min_length\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::ReadInternal()\";\n  Reader& src = *SrcReader();\n  truncated_ = false;\n  max_length = UnsignedMin(max_length,\n                           std::numeric_limits<Position>::max() - limit_pos());\n  decompressor_->next_out = dest;\n  for (;;) {\n    decompressor_->avail_out = SaturatingIntCast<unsigned int>(\n        PtrDistance(decompressor_->next_out, dest + max_length));\n    decompressor_->next_in = const_cast<char*>(src.cursor());\n    decompressor_->avail_in = SaturatingIntCast<unsigned int>(src.available());\n    if (decompressor_->avail_in > 0) stream_had_data_ = true;\n    int bzlib_code = BZ2_bzDecompress(decompressor_.get());\n    src.set_cursor(decompressor_->next_in);\n    const size_t length_read = PtrDistance(dest, decompressor_->next_out);\n    switch (bzlib_code) {\n      case BZ_OK:\n        if (length_read >= min_length) break;\n        if (ABSL_PREDICT_FALSE(decompressor_->avail_in > 0)) {\n          RIEGELI_ASSERT_EQ(decompressor_->avail_out, 0u)\n              << \"BZ2_bzDecompress() returned but there are still input data \"\n                 \"and output space\";\n          RIEGELI_ASSERT_EQ(length_read,\n                            std::numeric_limits<Position>::max() - limit_pos())\n              << \"The position does not overflow but the output buffer is \"\n                 \"full, while less than min_length was output, which is \"\n                 \"impossible because the buffer has size max_length which is \"\n                 \"at least min_length if the position does not overflow\";\n          move_limit_pos(length_read);\n          return FailOverflow();\n        }\n        if (ABSL_PREDICT_FALSE(!src.Pull())) {\n          move_limit_pos(length_read);\n          if (ABSL_PREDICT_FALSE(!src.ok())) {\n            return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n          }\n          if (ABSL_PREDICT_FALSE(!concatenate_ || stream_had_data_)) {\n            truncated_ = true;\n          }\n          return false;\n        }\n        continue;\n      case BZ_STREAM_END:\n        if (concatenate_) {\n          bzlib_code = BZ2_bzDecompressEnd(decompressor_.get());\n          if (ABSL_PREDICT_FALSE(bzlib_code != BZ_OK)) {\n            FailOperation(\"BZ2_bzDecompressEnd()\", bzlib_code);\n            break;\n          }\n          // Not needed for `libbz2` but needed for `libbz2_rs_sys`.\n          decompressor_->bzalloc = nullptr;\n          decompressor_->bzfree = nullptr;\n          decompressor_->opaque = nullptr;\n          bzlib_code = BZ2_bzDecompressInit(decompressor_.get(), 0, 0);\n          if (ABSL_PREDICT_FALSE(bzlib_code != BZ_OK)) {\n            delete decompressor_.release();  // Skip `BZ2_bzDecompressEnd()`.\n            FailOperation(\"BZ2_bzDecompressInit()\", bzlib_code);\n            break;\n          }\n          stream_had_data_ = false;\n          if (length_read >= min_length) break;\n          continue;\n        }\n        decompressor_.reset();\n        move_limit_pos(length_read);\n        // Avoid `BufferedReader` allocating another buffer.\n        set_exact_size(limit_pos());\n        return length_read >= min_length;\n      default:\n        FailOperation(\"BZ2_bzDecompress()\", bzlib_code);\n        break;\n    }\n    move_limit_pos(length_read);\n    return length_read >= min_length;\n  }\n}\n\nvoid Bzip2ReaderBase::ExactSizeReached() {\n  if (decompressor_ == nullptr) return;\n  char buffer[1];\n  if (ABSL_PREDICT_FALSE(Bzip2ReaderBase::ReadInternal(1, 1, buffer))) {\n    decompressor_.reset();\n    Fail(absl::FailedPreconditionError(\n        \"Uncompressed size reached but more data can be decompressed, \"\n        \"which implies that seeking back and reading again encountered \"\n        \"changed Bzip2-compressed data\"));\n  }\n}\n\nbool Bzip2ReaderBase::ToleratesReadingAhead() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->ToleratesReadingAhead();\n}\n\nbool Bzip2ReaderBase::SupportsRewind() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRewind();\n}\n\nbool Bzip2ReaderBase::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (new_pos <= limit_pos()) {\n    // Seeking backwards.\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    Reader& src = *SrcReader();\n    truncated_ = false;\n    stream_had_data_ = false;\n    set_buffer();\n    set_limit_pos(0);\n    decompressor_.reset();\n    if (ABSL_PREDICT_FALSE(!src.Seek(initial_compressed_pos_))) {\n      return FailWithoutAnnotation(AnnotateOverSrc(src.StatusOrAnnotate(\n          absl::DataLossError(\"Bzip2-compressed stream got truncated\"))));\n    }\n    InitializeDecompressor();\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    if (new_pos == 0) return true;\n  }\n  return BufferedReader::SeekBehindBuffer(new_pos);\n}\n\nbool Bzip2ReaderBase::SupportsNewReader() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsNewReader();\n}\n\nstd::unique_ptr<Reader> Bzip2ReaderBase::NewReaderImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point\n  // if `SrcReader()->SupportsNewReader()`.\n  Reader& src = *SrcReader();\n  std::unique_ptr<Reader> compressed_reader =\n      src.NewReader(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    return nullptr;\n  }\n  std::unique_ptr<Reader> reader =\n      std::make_unique<Bzip2Reader<std::unique_ptr<Reader>>>(\n          std::move(compressed_reader),\n          Bzip2ReaderBase::Options()\n              .set_concatenate(concatenate_)\n              .set_buffer_options(buffer_options()));\n  reader->Seek(initial_pos);\n  return reader;\n}\n\nnamespace {\n\ninline bool GetByte(Reader& src, size_t offset, uint8_t& value) {\n  if (!src.Pull(offset + 1, 8)) return false;\n  value = static_cast<uint8_t>(src.cursor()[offset]);\n  return true;\n}\n\n}  // namespace\n\nbool RecognizeBzip2(Reader& src) {\n  // Based on https://github.com/dsnet/compress/blob/master/doc/bzip2-format.pdf\n  uint8_t byte;\n  // HeaderMagic: \"BZh1\" to \"BZh9\".\n  if (!(GetByte(src, 0, byte) && byte == 'B' && GetByte(src, 1, byte) &&\n        byte == 'Z' && GetByte(src, 2, byte) && byte == 'h' &&\n        GetByte(src, 3, byte) && byte >= '1' && byte <= '9' &&\n        GetByte(src, 4, byte))) {\n    return false;\n  }\n  switch (byte) {\n    case 0x31:\n      // BlockMagic: 0x31, 0x41, 0x59, 0x26.\n      return GetByte(src, 5, byte) && byte == 0x41 && GetByte(src, 6, byte) &&\n             byte == 0x59 && GetByte(src, 7, byte) && byte == 0x26;\n    case 0x17:\n      // FooterMagic: 0x17, 0x72, 0x45, 0x38.\n      return GetByte(src, 5, byte) && byte == 0x72 && GetByte(src, 6, byte) &&\n             byte == 0x45 && GetByte(src, 7, byte) && byte == 0x38;\n    default:\n      return false;\n  }\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bzip2/bzip2_reader.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BZIP2_BZIP2_READER_H_\n#define RIEGELI_BZIP2_BZIP2_READER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"bzlib.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `Bzip2Reader`.\nclass Bzip2ReaderBase : public BufferedReader {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // If `true`, concatenated compressed streams are decoded to concatenation\n    // of their decompressed contents. An empty compressed stream is decoded to\n    // empty decompressed contents.\n    //\n    // If `false`, exactly one compressed stream is consumed.\n    //\n    // Default: `false`.\n    Options& set_concatenate(bool concatenate) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      concatenate_ = concatenate;\n      return *this;\n    }\n    Options&& set_concatenate(bool concatenate) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_concatenate(concatenate));\n    }\n    bool concatenate() const { return concatenate_; }\n\n   private:\n    bool concatenate_ = false;\n  };\n\n  // Returns the compressed `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns `true` if the source is truncated (without a clean end of the\n  // compressed stream) at the current position. In such case, if the source\n  // does not grow, `Close()` will fail.\n  //\n  // Precondition: `Options::concatenate()` was `false`.\n  bool truncated() const {\n    RIEGELI_ASSERT(!concatenate_)\n        << \"Failed precondition of Bzip2ReaderBase::truncated(): \"\n           \"Options::concatenate() is true\";\n    return truncated_ && available() == 0;\n  }\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRewind() override;\n  bool SupportsNewReader() override;\n\n protected:\n  explicit Bzip2ReaderBase(Closed) noexcept : BufferedReader(kClosed) {}\n\n  explicit Bzip2ReaderBase(BufferOptions buffer_options, bool concatenate);\n\n  Bzip2ReaderBase(Bzip2ReaderBase&& that) noexcept;\n  Bzip2ReaderBase& operator=(Bzip2ReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, bool concatenate);\n  void Initialize(Reader* src);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool ReadInternal(size_t min_length, size_t max_length, char* dest) override;\n  void ExactSizeReached() override;\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  struct BZStreamDeleter {\n    void operator()(bz_stream* ptr) const {\n      const int bzlib_code = BZ2_bzDecompressEnd(ptr);\n      RIEGELI_ASSERT_EQ(bzlib_code, BZ_OK)\n          << \"BZ2_bzDecompressEnd() failed: \" << bzlib_code;\n      delete ptr;\n    }\n  };\n\n  void InitializeDecompressor();\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation,\n                                         int bzlib_code);\n\n  bool concatenate_ = false;\n  // If `true`, the source is truncated (without a clean end of the compressed\n  // stream) at the current position. If the source does not grow, `Close()`\n  // will fail.\n  Position initial_compressed_pos_ = 0;\n  bool truncated_ = false;\n  // If `true`, some compressed data from the current stream were processed.\n  // If `concatenate_` and `!stream_had_data_`, an end of the source is\n  // legitimate, it does not imply that the source is truncated.\n  bool stream_had_data_ = false;\n  // If `ok()` but `decompressor_ == nullptr` then all data have been\n  // decompressed, `exact_size() == limit_pos()`, and `ReadInternal()` must not\n  // be called again.\n  std::unique_ptr<bz_stream, BZStreamDeleter> decompressor_;\n};\n\n// A `Reader` which decompresses data with Bzip2 after getting it from another\n// `Reader`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Reader` must not be accessed until the `Bzip2Reader` is\n// closed or no longer used.\ntemplate <typename Src = Reader*>\nclass Bzip2Reader : public Bzip2ReaderBase {\n public:\n  // Creates a closed `Bzip2Reader`.\n  explicit Bzip2Reader(Closed) noexcept : Bzip2ReaderBase(kClosed) {}\n\n  // Will read from the compressed `Reader` provided by `src`.\n  explicit Bzip2Reader(Initializer<Src> src, Options options = Options());\n\n  Bzip2Reader(Bzip2Reader&&) = default;\n  Bzip2Reader& operator=(Bzip2Reader&&) = default;\n\n  // Makes `*this` equivalent to a newly constructed `Bzip2Reader`. This avoids\n  // constructing a temporary `Bzip2Reader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  void VerifyEndImpl() override;\n\n private:\n  // The object providing and possibly owning the compressed `Reader`.\n  Dependency<Reader*, Src> src_;\n};\n\nexplicit Bzip2Reader(Closed) -> Bzip2Reader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit Bzip2Reader(\n    Src&& src, Bzip2ReaderBase::Options options = Bzip2ReaderBase::Options())\n    -> Bzip2Reader<TargetT<Src>>;\n\n// Returns `true` if the data look like they have been Bzip2-compressed.\n//\n// The current position of `src` is unchanged.\nbool RecognizeBzip2(Reader& src);\n\n// Implementation details follow.\n\ninline Bzip2ReaderBase::Bzip2ReaderBase(BufferOptions buffer_options,\n                                        bool concatenate)\n    : BufferedReader(buffer_options), concatenate_(concatenate) {}\n\ninline Bzip2ReaderBase::Bzip2ReaderBase(Bzip2ReaderBase&& that) noexcept\n    : BufferedReader(static_cast<BufferedReader&&>(that)),\n      concatenate_(that.concatenate_),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      truncated_(that.truncated_),\n      stream_had_data_(that.stream_had_data_),\n      decompressor_(std::move(that.decompressor_)) {}\n\ninline Bzip2ReaderBase& Bzip2ReaderBase::operator=(\n    Bzip2ReaderBase&& that) noexcept {\n  BufferedReader::operator=(static_cast<BufferedReader&&>(that));\n  concatenate_ = that.concatenate_;\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  truncated_ = that.truncated_;\n  stream_had_data_ = that.stream_had_data_;\n  decompressor_ = std::move(that.decompressor_);\n  return *this;\n}\n\ninline void Bzip2ReaderBase::Reset(Closed) {\n  BufferedReader::Reset(kClosed);\n  concatenate_ = false;\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  stream_had_data_ = false;\n  decompressor_.reset();\n}\n\ninline void Bzip2ReaderBase::Reset(BufferOptions buffer_options,\n                                   bool concatenate) {\n  BufferedReader::Reset(buffer_options);\n  concatenate_ = concatenate;\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  stream_had_data_ = false;\n  decompressor_.reset();\n}\n\ntemplate <typename Src>\ninline Bzip2Reader<Src>::Bzip2Reader(Initializer<Src> src, Options options)\n    : Bzip2ReaderBase(options.buffer_options(), options.concatenate()),\n      src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void Bzip2Reader<Src>::Reset(Closed) {\n  Bzip2ReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void Bzip2Reader<Src>::Reset(Initializer<Src> src, Options options) {\n  Bzip2ReaderBase::Reset(options.buffer_options(), options.concatenate());\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\nvoid Bzip2Reader<Src>::Done() {\n  Bzip2ReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(AnnotateOverSrc(src_->status()));\n    }\n  }\n}\n\ntemplate <typename Src>\nvoid Bzip2Reader<Src>::SetReadAllHintImpl(bool read_all_hint) {\n  Bzip2ReaderBase::SetReadAllHintImpl(read_all_hint);\n  if (src_.IsOwning()) src_->SetReadAllHint(read_all_hint);\n}\n\ntemplate <typename Src>\nvoid Bzip2Reader<Src>::VerifyEndImpl() {\n  Bzip2ReaderBase::VerifyEndImpl();\n  if (src_.IsOwning() && ABSL_PREDICT_TRUE(ok())) src_->VerifyEnd();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BZIP2_BZIP2_READER_H_\n"
  },
  {
    "path": "riegeli/bzip2/bzip2_writer.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/bzip2/bzip2_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"bzlib.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/bzip2/bzip2_error.h\"\n\nnamespace riegeli {\n\nvoid Bzip2WriterBase::Initialize(Writer* dest, int compression_level) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of Bzip2Writer: null Writer pointer\";\n  if (ABSL_PREDICT_FALSE(!dest->ok())) {\n    FailWithoutAnnotation(AnnotateOverDest(dest->status()));\n    return;\n  }\n  initial_compressed_pos_ = dest->pos();\n  compressor_ = riegeli::Maker<bz_stream>();\n  const int bzlib_code =\n      BZ2_bzCompressInit(compressor_.get(), compression_level, 0, 0);\n  if (ABSL_PREDICT_FALSE(bzlib_code != BZ_OK)) {\n    delete compressor_.release();  // Skip `BZ2_bzCompressEnd()`.\n    FailOperation(\"BZ2_bzCompressInit()\", bzlib_code);\n  }\n}\n\nvoid Bzip2WriterBase::DoneBehindBuffer(absl::string_view src) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::DoneBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  Writer& dest = *DestWriter();\n  WriteInternal(src, dest, BZ_FINISH);\n}\n\nvoid Bzip2WriterBase::Done() {\n  BufferedWriter::Done();\n  compressor_.reset();\n}\n\ninline bool Bzip2WriterBase::FailOperation(absl::string_view operation,\n                                           int bzlib_code) {\n  RIEGELI_ASSERT(bzlib_code != BZ_OK && bzlib_code != BZ_RUN_OK &&\n                 bzlib_code != BZ_FLUSH_OK && bzlib_code != BZ_FINISH_OK)\n      << \"Failed precondition of Bzip2WriterBase::FailOperation(): \"\n         \"bzlib error code not failed\";\n  RIEGELI_ASSERT(is_open())\n      << \"Failed precondition of Bzip2WriterBase::FailOperation(): \"\n         \"Object closed\";\n  return Fail(bzip2_internal::Bzip2ErrorToStatus(operation, bzlib_code));\n}\n\nabsl::Status Bzip2WriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    status = dest.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `dest` with the compressed\n  // position. Clarify that the current position is the uncompressed position\n  // instead of delegating to `BufferedWriter::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status Bzip2WriterBase::AnnotateOverDest(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nbool Bzip2WriterBase::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of BufferedWriter::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedWriter::WriteInternal()\";\n  Writer& dest = *DestWriter();\n  return WriteInternal(src, dest, BZ_RUN);\n}\n\ninline bool Bzip2WriterBase::WriteInternal(absl::string_view src, Writer& dest,\n                                           int flush) {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of Bzip2WriterBase::WriteInternal()\";\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  compressor_->next_in = const_cast<char*>(src.data());\n  for (;;) {\n    // If no progress was made, e.g. `compressor_->avail_out == 0` but\n    // `BZ2_bzCompress()` wants to output compressed data, then\n    // `BZ2_bzCompress()` returns `BZ_PARAM_ERROR`, so `dest.Push()` first.\n    if (ABSL_PREDICT_FALSE(!dest.Push())) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    }\n    size_t avail_in =\n        PtrDistance(compressor_->next_in, src.data() + src.size());\n    int action = flush;\n    if (ABSL_PREDICT_FALSE(avail_in >\n                           std::numeric_limits<unsigned int>::max())) {\n      avail_in = size_t{std::numeric_limits<unsigned int>::max()};\n      action = BZ_RUN;\n    }\n    compressor_->avail_in = IntCast<unsigned int>(avail_in);\n    compressor_->next_out = dest.cursor();\n    compressor_->avail_out = SaturatingIntCast<unsigned int>(dest.available());\n    const int bzlib_code = BZ2_bzCompress(compressor_.get(), action);\n    dest.set_cursor(compressor_->next_out);\n    const size_t length_written = PtrDistance(src.data(), compressor_->next_in);\n    switch (bzlib_code) {\n      case BZ_RUN_OK:\n        if (length_written < src.size()) {\n          RIEGELI_ASSERT(compressor_->avail_in == 0 ||\n                         compressor_->avail_out == 0)\n              << \"BZ2_bzCompress() returned but there are still input data \"\n                 \"and output space\";\n          continue;\n        }\n        break;\n      case BZ_FLUSH_OK:\n      case BZ_FINISH_OK:\n        RIEGELI_ASSERT_EQ(compressor_->avail_out, 0u)\n            << \"BZ2_bzCompress() is \"\n            << (bzlib_code == BZ_FLUSH_OK ? \"flushing\" : \"finishing\")\n            << \" but there is still output space\";\n        continue;\n      case BZ_STREAM_END:\n        break;\n      default:\n        return FailOperation(\"BZ2_bzCompress()\", bzlib_code);\n    }\n    RIEGELI_ASSERT_EQ(length_written, src.size())\n        << \"BZ2_bzCompress() returned but there are still input data\";\n    move_start_pos(length_written);\n    return true;\n  }\n}\n\nbool Bzip2WriterBase::FlushBehindBuffer(absl::string_view src,\n                                        FlushType flush_type) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::FlushBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  return WriteInternal(src, dest, BZ_FLUSH);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/bzip2/bzip2_writer.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_BZIP2_BZIP2_WRITER_H_\n#define RIEGELI_BZIP2_BZIP2_WRITER_H_\n\n#include <memory>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"bzlib.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `Bzip2Writer`.\nclass Bzip2WriterBase : public BufferedWriter {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // Tunes the tradeoff between compression density and compression speed\n    // (higher = better density but slower).\n    //\n    // `compression_level` must be between `kMinCompressionLevel` (1) and\n    // `kMaxCompressionLevel` (9). Default: `kDefaultCompressionLevel` (9).\n    static constexpr int kMinCompressionLevel = 1;\n    static constexpr int kMaxCompressionLevel = 9;\n    static constexpr int kDefaultCompressionLevel = 9;\n    Options& set_compression_level(int compression_level) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(compression_level, kMinCompressionLevel)\n          << \"Failed precondition of \"\n             \"Bzip2WriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      RIEGELI_ASSERT_LE(compression_level, kMaxCompressionLevel)\n          << \"Failed precondition of \"\n             \"Bzip2WriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      compression_level_ = compression_level;\n      return *this;\n    }\n    Options&& set_compression_level(int compression_level) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_compression_level(compression_level));\n    }\n    int compression_level() const { return compression_level_; }\n\n   private:\n    int compression_level_ = kDefaultCompressionLevel;\n  };\n\n  // Returns the compressed `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n protected:\n  explicit Bzip2WriterBase(Closed) noexcept : BufferedWriter(kClosed) {}\n\n  explicit Bzip2WriterBase(BufferOptions buffer_options);\n\n  Bzip2WriterBase(Bzip2WriterBase&& that) noexcept;\n  Bzip2WriterBase& operator=(Bzip2WriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options);\n  void Initialize(Writer* dest, int compression_level);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  void DoneBehindBuffer(absl::string_view src) override;\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool WriteInternal(absl::string_view src) override;\n  bool FlushBehindBuffer(absl::string_view src, FlushType flush_type) override;\n\n private:\n  struct BZStreamDeleter {\n    void operator()(bz_stream* ptr) const {\n      const int bzlib_code = BZ2_bzCompressEnd(ptr);\n      RIEGELI_ASSERT_EQ(bzlib_code, BZ_OK)\n          << \"BZ2_bzCompressEnd() failed: \" << bzlib_code;\n      delete ptr;\n    }\n  };\n\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation,\n                                         int bzlib_code);\n  bool WriteInternal(absl::string_view src, Writer& dest, int flush);\n\n  Position initial_compressed_pos_ = 0;\n  std::unique_ptr<bz_stream, BZStreamDeleter> compressor_;\n};\n\n// A `Writer` which compresses data with Bzip2 before passing it to another\n// `Writer`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Writer` must not be accessed until the `Bzip2Writer` is\n// closed or no longer used.\n//\n// Because Bzip2 blocks are bit-aligned, `Flush()` is not effective (the effect\n// is usually delayed by one block) and `ReadMode()` is not supported.\ntemplate <typename Dest = Writer*>\nclass Bzip2Writer : public Bzip2WriterBase {\n public:\n  // Creates a closed `Bzip2Writer`.\n  explicit Bzip2Writer(Closed) noexcept : Bzip2WriterBase(kClosed) {}\n\n  // Will write to the compressed `Writer` provided by `dest`.\n  explicit Bzip2Writer(Initializer<Dest> dest, Options options = Options());\n\n  Bzip2Writer(Bzip2Writer&& that) = default;\n  Bzip2Writer& operator=(Bzip2Writer&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `Bzip2Writer`. This avoids\n  // constructing a temporary `Bzip2Writer` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  // The object providing and possibly owning the compressed `Writer`.\n  Dependency<Writer*, Dest> dest_;\n};\n\nexplicit Bzip2Writer(Closed) -> Bzip2Writer<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit Bzip2Writer(\n    Dest&& dest, Bzip2WriterBase::Options options = Bzip2WriterBase::Options())\n    -> Bzip2Writer<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline Bzip2WriterBase::Bzip2WriterBase(BufferOptions buffer_options)\n    : BufferedWriter(buffer_options) {}\n\ninline Bzip2WriterBase::Bzip2WriterBase(Bzip2WriterBase&& that) noexcept\n    : BufferedWriter(static_cast<BufferedWriter&&>(that)),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      compressor_(std::move(that.compressor_)) {}\n\ninline Bzip2WriterBase& Bzip2WriterBase::operator=(\n    Bzip2WriterBase&& that) noexcept {\n  BufferedWriter::operator=(static_cast<BufferedWriter&&>(that));\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  compressor_ = std::move(that.compressor_);\n  return *this;\n}\n\ninline void Bzip2WriterBase::Reset(Closed) {\n  BufferedWriter::Reset(kClosed);\n  initial_compressed_pos_ = 0;\n  compressor_.reset();\n}\n\ninline void Bzip2WriterBase::Reset(BufferOptions buffer_options) {\n  BufferedWriter::Reset(buffer_options);\n  initial_compressed_pos_ = 0;\n  compressor_.reset();\n}\n\ntemplate <typename Dest>\ninline Bzip2Writer<Dest>::Bzip2Writer(Initializer<Dest> dest, Options options)\n    : Bzip2WriterBase(options.buffer_options()), dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.compression_level());\n}\n\ntemplate <typename Dest>\ninline void Bzip2Writer<Dest>::Reset(Closed) {\n  Bzip2WriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void Bzip2Writer<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  Bzip2WriterBase::Reset(options.buffer_options());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.compression_level());\n}\n\ntemplate <typename Dest>\nvoid Bzip2Writer<Dest>::Done() {\n  Bzip2WriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <typename Dest>\nbool Bzip2Writer<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!Bzip2WriterBase::FlushImpl(flush_type))) return false;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Flush(flush_type))) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n  return true;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_BZIP2_BZIP2_WRITER_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"chunk_encoder\",\n    srcs = [\"chunk_encoder.cc\"],\n    hdrs = [\"chunk_encoder.h\"],\n    deps = [\n        \":constants\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:object\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/messages:serialize_message\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_protobuf//:protobuf_lite\",\n    ],\n)\n\ncc_library(\n    name = \"chunk_decoder\",\n    srcs = [\"chunk_decoder.cc\"],\n    hdrs = [\"chunk_decoder.h\"],\n    deps = [\n        \":chunk\",\n        \":constants\",\n        \":field_projection\",\n        \":simple_decoder\",\n        \":transpose_decoder\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/bytes:array_backward_writer\",\n        \"//riegeli/bytes:chain_backward_writer\",\n        \"//riegeli/bytes:chain_reader\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/messages:parse_message\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n        \"@com_google_protobuf//:protobuf_lite\",\n    ],\n)\n\ncc_library(\n    name = \"constants\",\n    hdrs = [\"constants.h\"],\n)\n\ncc_library(\n    name = \"chunk\",\n    srcs = [\"chunk.cc\"],\n    hdrs = [\"chunk.h\"],\n    deps = [\n        \":constants\",\n        \":hash\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/endian:endian_reading\",\n        \"//riegeli/endian:endian_writing\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"hash\",\n    srcs = [\"hash.cc\"],\n    hdrs = [\"hash.h\"],\n    deps = [\n        \"//riegeli/base:chain\",\n        \"@com_google_absl//absl/container:fixed_array\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@highwayhash//:hh_types\",\n        \"@highwayhash//:highwayhash_dynamic\",\n        \"@highwayhash//:instruction_sets\",\n    ],\n)\n\ncc_library(\n    name = \"compressor_options\",\n    srcs = [\"compressor_options.cc\"],\n    hdrs = [\"compressor_options.h\"],\n    deps = [\n        \":constants\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:options_parser\",\n        \"//riegeli/brotli:brotli_writer\",\n        \"//riegeli/snappy:snappy_writer\",\n        \"//riegeli/zstd:zstd_writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"compressor\",\n    srcs = [\"compressor.cc\"],\n    hdrs = [\"compressor.h\"],\n    deps = [\n        \":brotli_encoder_selection\",\n        \":compressor_options\",\n        \":constants\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:chain_writer\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/snappy:snappy_writer\",\n        \"//riegeli/varint:varint_writing\",\n        \"//riegeli/zstd:zstd_writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n    ],\n)\n\ncc_library(\n    name = \"decompressor\",\n    srcs = [\"decompressor.cc\"],\n    hdrs = [\"decompressor.h\"],\n    deps = [\n        \":constants\",\n        \"//riegeli/base:any\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/brotli:brotli_reader\",\n        \"//riegeli/bytes:chain_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/snappy:snappy_reader\",\n        \"//riegeli/varint:varint_reading\",\n        \"//riegeli/zstd:zstd_reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n    ],\n)\n\ncc_library(\n    name = \"brotli_encoder_selection\",\n    srcs = [\"brotli_encoder_selection.cc\"],\n    hdrs = [\"brotli_encoder_selection.h\"],\n    deps = [\n        \":compressor_options\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/brotli:brotli_writer\",\n        \"//riegeli/bytes:chain_writer\",\n        \"//riegeli/bytes:null_writer\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n    ],\n)\n\ncc_library(\n    name = \"simple_encoder\",\n    srcs = [\"simple_encoder.cc\"],\n    hdrs = [\"simple_encoder.h\"],\n    deps = [\n        \":chunk_encoder\",\n        \":compressor\",\n        \":compressor_options\",\n        \":constants\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/messages:serialize_message\",\n        \"//riegeli/varint:varint_writing\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_protobuf//:protobuf_lite\",\n    ],\n)\n\ncc_library(\n    name = \"simple_decoder\",\n    srcs = [\"simple_decoder.cc\"],\n    hdrs = [\"simple_decoder.h\"],\n    deps = [\n        \":constants\",\n        \":decompressor\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/varint:varint_reading\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n    ],\n)\n\ncc_library(\n    name = \"transpose_encoder\",\n    srcs = [\"transpose_encoder.cc\"],\n    hdrs = [\"transpose_encoder.h\"],\n    deps = [\n        \":chunk_encoder\",\n        \":compressor\",\n        \":compressor_options\",\n        \":constants\",\n        \":transpose_internal\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:backward_writer\",\n        \"//riegeli/bytes:chain_backward_writer\",\n        \"//riegeli/bytes:chain_reader\",\n        \"//riegeli/bytes:chain_writer\",\n        \"//riegeli/bytes:cord_reader\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:string_reader\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/messages:message_wire_format\",\n        \"//riegeli/varint:varint_reading\",\n        \"//riegeli/varint:varint_writing\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/container:flat_hash_map\",\n        \"@com_google_absl//absl/container:inlined_vector\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"transpose_decoder\",\n    srcs = [\"transpose_decoder.cc\"],\n    hdrs = [\"transpose_decoder.h\"],\n    deps = [\n        \":constants\",\n        \":decompressor\",\n        \":field_projection\",\n        \":transpose_internal\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:global\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/bytes:backward_writer\",\n        \"//riegeli/bytes:chain_reader\",\n        \"//riegeli/bytes:limiting_backward_writer\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:string_reader\",\n        \"//riegeli/messages:message_wire_format\",\n        \"//riegeli/varint:varint_reading\",\n        \"//riegeli/varint:varint_writing\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/container:fixed_array\",\n        \"@com_google_absl//absl/container:flat_hash_map\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"transpose_internal\",\n    hdrs = [\"transpose_internal.h\"],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \"//riegeli/base:assert\",\n        \"//riegeli/messages:message_wire_format\",\n        \"//riegeli/varint:varint_writing\",\n    ],\n)\n\ncc_library(\n    name = \"field_projection\",\n    hdrs = [\"field_projection.h\"],\n    deps = [\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:initializer\",\n        \"@com_google_absl//absl/container:inlined_vector\",\n    ],\n)\n\ncc_library(\n    name = \"deferred_encoder\",\n    srcs = [\"deferred_encoder.cc\"],\n    hdrs = [\"deferred_encoder.h\"],\n    deps = [\n        \":chunk_encoder\",\n        \":constants\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/bytes:chain_writer\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/messages:serialize_message\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_protobuf//:protobuf_lite\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/chunk_encoding/README.md",
    "content": "# Purpose\n\nRiegeli/transpose transforms protocol buffer byte streams into a custom data\nformat that can be compressed around 20% more densely. The additional transform\nslows down both compression and decompression by around 50%. Often this is still\na desirable trade-off, but in the end it depends on the compressed data and\nother system requirements such as latency vs. resource use.\n\n# Detailed design\n\nTransposition of a set of protocol buffers means that we associate a container\nwith each tag. Then all the values corresponding to a specific tag are stored in\nthe container associated with it. Invocation of a general purpose compression\nalgorithm on the concatenation of these containers offers better compression\nratios than it's invocation on the concatenation of the original binary encoding\nof the protocol buffers.\n"
  },
  {
    "path": "riegeli/chunk_encoding/brotli_encoder_selection.cc",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/chunk_encoding/brotli_encoder_selection.h\"\n\n#include <memory>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/brotli/brotli_writer.h\"\n#include \"riegeli/bytes/chain_writer.h\"\n#include \"riegeli/bytes/null_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/compressor_options.h\"\n\nnamespace riegeli::chunk_encoding_internal {\n\nABSL_ATTRIBUTE_WEAK std::unique_ptr<Writer> NewBrotliWriter(\n    Chain* compressed, const CompressorOptions& compressor_options,\n    ABSL_ATTRIBUTE_UNUSED const RecyclingPoolOptions& recycling_pool_options) {\n  switch (compressor_options.brotli_encoder()) {\n    case BrotliEncoder::kRBrotliOrCBrotli:\n    case BrotliEncoder::kCBrotli:\n      return NewCBrotliWriter(compressed, compressor_options);\n    case BrotliEncoder::kRBrotli: {\n      std::unique_ptr<Writer> writer = std::make_unique<riegeli::NullWriter>();\n      writer->Fail(absl::UnimplementedError(\"Rust Brotli not available\"));\n      return writer;\n    }\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown Brotli encoder: \"\n      << static_cast<int>(compressor_options.brotli_encoder());\n}\n\nstd::unique_ptr<Writer> NewCBrotliWriter(\n    Chain* compressed, const CompressorOptions& compressor_options) {\n  return std::make_unique<BrotliWriter<ChainWriter<>>>(\n      riegeli::Maker(compressed),\n      BrotliWriterBase::Options()\n          .set_compression_level(compressor_options.compression_level())\n          .set_window_log(compressor_options.brotli_window_log()));\n}\n\n}  // namespace riegeli::chunk_encoding_internal\n"
  },
  {
    "path": "riegeli/chunk_encoding/brotli_encoder_selection.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_BROTLI_ENCODER_SELECTION_H_\n#define RIEGELI_CHUNK_ENCODING_BROTLI_ENCODER_SELECTION_H_\n\n#include <memory>\n\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/compressor_options.h\"\n\nnamespace riegeli::chunk_encoding_internal {\n\n// Creates a `Writer` which compresses data with Brotli and writes them to\n// `compressed`.\n//\n// The encoder implementation is determined by\n// `compressor_options.brotli_encoder()`.\n//\n// This is a weak function. Its default definition supports only C Brotli.\n// It can be overridden to support also Rust Brotli.\nstd::unique_ptr<Writer> NewBrotliWriter(\n    Chain* compressed, const CompressorOptions& compressor_options,\n    const RecyclingPoolOptions& recycling_pool_options);\n\n// Support for `NewBrotliWriter()`: uses C Brotli, ignores\n// `compressor_options.brotli_encoder()`.\nstd::unique_ptr<Writer> NewCBrotliWriter(\n    Chain* compressed, const CompressorOptions& compressor_options);\n\n}  // namespace riegeli::chunk_encoding_internal\n\n#endif  // RIEGELI_CHUNK_ENCODING_BROTLI_ENCODER_SELECTION_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/chunk.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/chunk_encoding/chunk.h\"\n\n#include <stdint.h>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/chunk_encoding/hash.h\"\n\nnamespace riegeli {\n\nChunkHeader::ChunkHeader(const Chain& data, ChunkType chunk_type,\n                         uint64_t num_records, uint64_t decoded_data_size) {\n  RIEGELI_ASSERT_LE(num_records, kMaxNumRecords)\n      << \"Failed precondition of ChunkHeader::ChunkHeader(): \"\n         \"number of records out of range\";\n  set_data_size(data.size());\n  set_data_hash(chunk_encoding_internal::Hash(data));\n  set_chunk_type_and_num_records(chunk_type, num_records);\n  set_decoded_data_size(decoded_data_size);\n  set_header_hash(computed_header_hash());\n}\n\nuint64_t ChunkHeader::computed_header_hash() const {\n  return chunk_encoding_internal::Hash(\n      absl::string_view(bytes() + sizeof(uint64_t), size() - sizeof(uint64_t)));\n}\n\nbool Chunk::WriteTo(Writer& dest) const {\n  if (ABSL_PREDICT_FALSE(\n          !dest.Write(absl::string_view(header.bytes(), header.size())))) {\n    return false;\n  }\n  return dest.Write(data);\n}\n\nbool Chunk::ReadFrom(Reader& src) {\n  if (ABSL_PREDICT_FALSE(!src.Read(header.size(), header.bytes()))) {\n    data.Clear();\n    return false;\n  }\n  return src.Read(header.data_size(), data);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/chunk_encoding/chunk.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_CHUNK_H_\n#define RIEGELI_CHUNK_ENCODING_CHUNK_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include \"absl/base/attributes.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/endian/endian_reading.h\"\n#include \"riegeli/endian/endian_writing.h\"\n\nnamespace riegeli {\n\nclass ChunkHeader {\n public:\n  ChunkHeader() = default;\n\n  explicit ChunkHeader(const Chain& data, ChunkType chunk_type,\n                       uint64_t num_records, uint64_t decoded_data_size);\n\n  ChunkHeader(const ChunkHeader& that) = default;\n  ChunkHeader& operator=(const ChunkHeader& that) = default;\n\n  char* bytes() { return bytes_; }\n  const char* bytes() const { return bytes_; }\n  static constexpr size_t size() { return sizeof(bytes_); }\n\n  uint64_t computed_header_hash() const;\n  uint64_t stored_header_hash() const {\n    return ReadLittleEndian<uint64_t>(bytes_);\n  }\n  uint64_t data_size() const {\n    return ReadLittleEndian<uint64_t>(bytes_ + sizeof(uint64_t));\n  }\n  uint64_t data_hash() const {\n    return ReadLittleEndian<uint64_t>(bytes_ + 2 * sizeof(uint64_t));\n  }\n  ChunkType chunk_type() const {\n    return static_cast<ChunkType>(\n        ReadLittleEndian<uint64_t>(bytes_ + 3 * sizeof(uint64_t)) & 0xff);\n  }\n  uint64_t num_records() const {\n    return ReadLittleEndian<uint64_t>(bytes_ + 3 * sizeof(uint64_t)) >> 8;\n  }\n  uint64_t decoded_data_size() const {\n    return ReadLittleEndian<uint64_t>(bytes_ + 4 * sizeof(uint64_t));\n  }\n\n private:\n  void set_header_hash(uint64_t value) {\n    WriteLittleEndian<uint64_t>(value, bytes_);\n  }\n  void set_data_size(uint64_t value) {\n    WriteLittleEndian<uint64_t>(value, bytes_ + sizeof(uint64_t));\n  }\n  void set_data_hash(uint64_t value) {\n    WriteLittleEndian<uint64_t>(value, bytes_ + 2 * sizeof(uint64_t));\n  }\n  void set_chunk_type_and_num_records(ChunkType chunk_type,\n                                      uint64_t num_records) {\n    RIEGELI_ASSERT_LE(num_records, kMaxNumRecords)\n        << \"Failed precondition of \"\n           \"ChunkHeader::set_chunk_type_and_num_records(): \"\n           \"number of records out of range\";\n    WriteLittleEndian<uint64_t>(\n        static_cast<uint64_t>(chunk_type) | (num_records << 8),\n        bytes_ + 3 * sizeof(uint64_t));\n  }\n  void set_decoded_data_size(uint64_t value) {\n    WriteLittleEndian<uint64_t>(value, bytes_ + 4 * sizeof(uint64_t));\n  }\n\n  // Representation (Little Endian):\n  //  - `uint64_t`: `header_hash`\n  //  - `uint64_t`: `data_size`\n  //  - `uint64_t`: `data_hash`\n  //  - `uint64_t`: `chunk_type` (low 8 bits) | `num_records` (high 56 bits)\n  //  - `uint64_t`: `decoded_data_size`\n  char bytes_[5 * sizeof(uint64_t)];\n};\n\nstruct Chunk {\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset() { data = Chain(); }\n  ABSL_ATTRIBUTE_REINITIALIZES void Clear() { data.Clear(); }\n\n  bool WriteTo(Writer& dest) const;\n  bool ReadFrom(Reader& src);\n\n  ChunkHeader header;\n  Chain data;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CHUNK_ENCODING_CHUNK_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/chunk_decoder.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/chunk_encoding/chunk_decoder.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <utility>\n#include <vector>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/types/span.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/bytes/array_backward_writer.h\"\n#include \"riegeli/bytes/chain_backward_writer.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/chunk_encoding/chunk.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/chunk_encoding/field_projection.h\"\n#include \"riegeli/chunk_encoding/simple_decoder.h\"\n#include \"riegeli/chunk_encoding/transpose_decoder.h\"\n#include \"riegeli/messages/parse_message.h\"\n\nnamespace riegeli {\n\nvoid ChunkDecoder::Done() { recoverable_ = false; }\n\nbool ChunkDecoder::Decode(const Chunk& chunk, bool flatten) {\n  Clear();\n  ChainReader<> data_reader(&chunk.data);\n  if (ABSL_PREDICT_FALSE(chunk.header.num_records() > limits_.max_size())) {\n    return Fail(absl::ResourceExhaustedError(\"Too many records\"));\n  }\n  Chain values;\n  if (ABSL_PREDICT_FALSE(!Parse(chunk.header, data_reader, values, flatten))) {\n    limits_.clear();  // Ensure that `index() == num_records()`.\n    return false;\n  }\n  RIEGELI_ASSERT_EQ(limits_.size(), chunk.header.num_records())\n      << \"Wrong number of record end positions\";\n  RIEGELI_ASSERT_EQ(limits_.empty() ? size_t{0} : limits_.back(), values.size())\n      << \"Wrong last record end position\";\n  if (chunk.header.num_records() == 0) {\n    RIEGELI_ASSERT_EQ(values.size(), 0u) << \"Wrong decoded data size\";\n  } else if (field_projection_.includes_all()) {\n    RIEGELI_ASSERT_EQ(values.size(), chunk.header.decoded_data_size())\n        << \"Wrong decoded data size\";\n  } else {\n    RIEGELI_ASSERT_LE(values.size(), chunk.header.decoded_data_size())\n        << \"Wrong decoded data size\";\n  }\n  values_reader_.Reset(std::move(values));\n  return true;\n}\n\ninline bool ChunkDecoder::Parse(const ChunkHeader& header, Reader& src,\n                                Chain& dest, bool flatten) {\n  switch (header.chunk_type()) {\n    case ChunkType::kFileSignature:\n      if (ABSL_PREDICT_FALSE(header.data_size() != 0)) {\n        return Fail(absl::InvalidArgumentError(absl::StrCat(\n            \"Invalid file signature chunk: data size is not zero: \",\n            header.data_size())));\n      }\n      if (ABSL_PREDICT_FALSE(header.num_records() != 0)) {\n        return Fail(absl::InvalidArgumentError(absl::StrCat(\n            \"Invalid file signature chunk: number of records is not zero: \",\n            header.num_records())));\n      }\n      if (ABSL_PREDICT_FALSE(header.decoded_data_size() != 0)) {\n        return Fail(absl::InvalidArgumentError(absl::StrCat(\n            \"Invalid file signature chunk: decoded data size is not zero: \",\n            header.decoded_data_size())));\n      }\n      return true;\n    case ChunkType::kFileMetadata:\n      if (ABSL_PREDICT_FALSE(header.num_records() != 0)) {\n        return Fail(absl::InvalidArgumentError(absl::StrCat(\n            \"Invalid file metadata chunk: number of records is not zero: \",\n            header.num_records())));\n      }\n      return true;\n    case ChunkType::kPadding:\n      if (ABSL_PREDICT_FALSE(header.num_records() != 0)) {\n        return Fail(absl::InvalidArgumentError(absl::StrCat(\n            \"Invalid padding chunk: number of records is not zero: \",\n            header.num_records())));\n      }\n      if (ABSL_PREDICT_FALSE(header.decoded_data_size() != 0)) {\n        return Fail(absl::InvalidArgumentError(absl::StrCat(\n            \"Invalid padding chunk: decoded data size is not zero: \",\n            header.decoded_data_size())));\n      }\n      return true;\n    case ChunkType::kSimple: {\n      SimpleDecoder simple_decoder(\n          SimpleDecoder::Options().set_recycling_pool_options(\n              recycling_pool_options_));\n      if (ABSL_PREDICT_FALSE(!simple_decoder.Decode(&src, header.num_records(),\n                                                    header.decoded_data_size(),\n                                                    limits_))) {\n        return Fail(simple_decoder.status());\n      }\n      bool read_ok;\n      if (flatten &&\n          // If all data are already available in a flat buffer, the chunk is\n          // likely uncompressed in a `ChainReader`, and reading it to a `Chain`\n          // will avoid copying it while preserving flatness.\n          simple_decoder.reader().available() < header.decoded_data_size()) {\n        const absl::Span<char> buffer =\n            dest.AppendFixedBuffer(IntCast<size_t>(header.decoded_data_size()));\n        read_ok = simple_decoder.reader().Read(buffer.size(), buffer.data());\n      } else {\n        read_ok = simple_decoder.reader().Read(\n            IntCast<size_t>(header.decoded_data_size()), dest);\n      }\n      if (ABSL_PREDICT_FALSE(!read_ok)) {\n        return Fail(simple_decoder.reader().StatusOrAnnotate(\n            absl::InvalidArgumentError(\"Reading record values failed\")));\n      }\n      if (ABSL_PREDICT_FALSE(!simple_decoder.VerifyEndAndClose())) {\n        return Fail(simple_decoder.status());\n      }\n      if (ABSL_PREDICT_FALSE(!src.VerifyEndAndClose())) {\n        return Fail(src.status());\n      }\n      return true;\n    }\n    case ChunkType::kTransposed: {\n      TransposeDecoder transpose_decoder(\n          TransposeDecoder::Options().set_recycling_pool_options(\n              recycling_pool_options_));\n      bool decode_ok;\n      if (flatten && field_projection_.includes_all()) {\n        ArrayBackwardWriter<> dest_writer(dest.AppendFixedBuffer(\n            IntCast<size_t>(header.decoded_data_size())));\n        decode_ok = transpose_decoder.Decode(\n            header.num_records(), header.decoded_data_size(), field_projection_,\n            src, dest_writer, limits_);\n        if (ABSL_PREDICT_FALSE(!dest_writer.Close())) {\n          return Fail(dest_writer.status());\n        }\n      } else {\n        ChainBackwardWriter<> dest_writer(&dest);\n        if (field_projection_.includes_all()) {\n          dest_writer.SetWriteSizeHint(header.decoded_data_size());\n        }\n        decode_ok = transpose_decoder.Decode(\n            header.num_records(), header.decoded_data_size(), field_projection_,\n            src, dest_writer, limits_);\n        if (ABSL_PREDICT_FALSE(!dest_writer.Close())) {\n          return Fail(dest_writer.status());\n        }\n        if (flatten) dest.Flatten();\n      }\n      if (ABSL_PREDICT_FALSE(!decode_ok)) {\n        return Fail(transpose_decoder.status());\n      }\n      if (ABSL_PREDICT_FALSE(!src.VerifyEndAndClose())) {\n        return Fail(src.status());\n      }\n      return true;\n    }\n  }\n  if (header.num_records() == 0) {\n    // Ignore chunks with no records, even if the type is unknown.\n    return true;\n  }\n  return Fail(absl::UnimplementedError(absl::StrCat(\n      \"Unknown chunk type: \", static_cast<uint64_t>(header.chunk_type()))));\n}\n\nbool ChunkDecoder::ReadRecord(google::protobuf::MessageLite& record) {\n  if (ABSL_PREDICT_FALSE(!ok() || index() == num_records())) return false;\n  const size_t start = IntCast<size_t>(values_reader_.pos());\n  const size_t limit = limits_[IntCast<size_t>(index_)];\n  RIEGELI_ASSERT_LE(start, limit)\n      << \"Failed invariant of ChunkDecoder: record end positions not sorted\";\n  if (absl::Status status =\n          ParseMessage(ReaderSpan(&values_reader_, limit - start), record);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    RIEGELI_EVAL_ASSERT(values_reader_.Seek(limit)) << values_reader_.status();\n    recoverable_ = true;\n    return Fail(std::move(status));\n  }\n  ++index_;\n  return true;\n}\n\nbool ChunkDecoder::Recover() {\n  if (!recoverable_) return false;\n  RIEGELI_ASSERT(!ok()) << \"Failed invariant of ChunkDecoder: \"\n                           \"recovery applicable but ChunkDecoder OK\";\n  recoverable_ = false;\n  MarkNotFailed();\n  ++index_;\n  return true;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/chunk_encoding/chunk_decoder.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_CHUNK_DECODER_H_\n#define RIEGELI_CHUNK_ENCODING_CHUNK_DECODER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <initializer_list>\n#include <string>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/chunk_encoding/chunk.h\"\n#include \"riegeli/chunk_encoding/field_projection.h\"\n\nnamespace riegeli {\n\nclass ChunkDecoder : public Object {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // Specifies the set of fields to be included in returned records, allowing\n    // to exclude the remaining fields (but does not guarantee exclusion).\n    // Excluding data makes reading faster.\n    //\n    // Default: `FieldProjection::All()`.\n    Options& set_field_projection(\n        Initializer<FieldProjection> field_projection) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      riegeli::Reset(field_projection_, std::move(field_projection));\n      return *this;\n    }\n    Options&& set_field_projection(\n        Initializer<FieldProjection> field_projection) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_field_projection(std::move(field_projection)));\n    }\n    Options& set_field_projection(\n        std::initializer_list<Field> field_projection) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      set_field_projection(Initializer<FieldProjection>(field_projection));\n      return *this;\n    }\n    Options&& set_field_projection(\n        std::initializer_list<Field> field_projection) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_field_projection(std::move(field_projection)));\n    }\n    FieldProjection& field_projection() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return field_projection_;\n    }\n    const FieldProjection& field_projection() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return field_projection_;\n    }\n\n    // Options for a global `RecyclingPool` of decompression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // decompression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    Options& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    Options&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    FieldProjection field_projection_ = FieldProjection::All();\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Creates an empty `ChunkDecoder`.\n  explicit ChunkDecoder(Options options = Options());\n\n  ChunkDecoder(ChunkDecoder&& that) noexcept;\n  ChunkDecoder& operator=(ChunkDecoder&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `ChunkDecoder`. This avoids\n  // constructing a temporary `ChunkDecoder` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Options options = Options());\n\n  // Resets the `ChunkDecoder` to an empty chunk. Keeps options unchanged.\n  void Clear();\n\n  // Resets the `ChunkDecoder` to an empty chunk. Keeps options unchanged,\n  // except that field projection is set to `field_projection`.\n  void ClearAndSetFieldProjection(\n      Initializer<FieldProjection> field_projection) {\n    Clear();\n    riegeli::Reset(field_projection_, std::move(field_projection));\n  }\n  void ClearAndSetFieldProjection(\n      std::initializer_list<Field> field_projection) {\n    ClearAndSetFieldProjection(Initializer<FieldProjection>(field_projection));\n  }\n\n  // Resets the `ChunkDecoder` and parses the chunk. Keeps options unchanged.\n  //\n  // If `flatten`, prefer making records readily available as\n  // `absl::string_view`.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool Decode(const Chunk& chunk, bool flatten = false);\n\n  // Reads the next record.\n  //\n  // `ReadRecord(google::protobuf::MessageLite&)` parses raw bytes to a proto\n  // message after reading. The remaining overloads read raw bytes (they never\n  // generate a new failure). For `ReadRecord(absl::string_view&)` the\n  // `absl::string_view` is valid until the next non-const operation on this\n  // `ChunkDecoder`.\n  //\n  // Return values:\n  //  * `true`                 - success (`record` is set, `ok()`)\n  //  * `false` (when `ok()`)  - chunk ends\n  //  * `false` (when `!ok()`) - failure\n  bool ReadRecord(google::protobuf::MessageLite& record);\n  bool ReadRecord(absl::string_view& record);\n  bool ReadRecord(std::string& record);\n  bool ReadRecord(Chain& record);\n  bool ReadRecord(absl::Cord& record);\n\n  // If `!ok()` and the failure was caused by an unparsable message, then\n  // `Recover()` allows reading again by skipping the unparsable message.\n  //\n  // If `ok()`, or if `!ok()` but the failure was not caused by an unparsable\n  // message, then `Recover()` does nothing and returns `false`.\n  //\n  // Return values:\n  //  * `true`  - success\n  //  * `false` - failure not caused by an unparsable message\n  bool Recover();\n\n  // Returns the current record index. Unchanged by `Close()`.\n  uint64_t index() const { return index_; }\n\n  // Sets the current record index.\n  //\n  // If `index > num_records()`, the current index is set to `num_records()`.\n  //\n  // Precondition: `ok()`\n  void SetIndex(uint64_t index);\n\n  // Returns the number of records. Unchanged by `Close()`.\n  uint64_t num_records() const { return IntCast<uint64_t>(limits_.size()); }\n\n protected:\n  void Done() override;\n\n private:\n  bool Parse(const ChunkHeader& header, Reader& src, Chain& dest, bool flatten);\n\n  FieldProjection field_projection_;\n  RecyclingPoolOptions recycling_pool_options_;\n  // Invariants if `ok()`:\n  //   `limits_` are sorted\n  //   `(limits_.empty() ? 0 : limits_.back())` == size of `values_reader_`\n  //   `(index_ == 0 ? 0 : limits_[index_ - 1]) == values_reader_.pos()`\n  std::vector<size_t> limits_;\n  ChainReader<Chain> values_reader_;\n  // Invariant: if `ok()` then `index_ <= num_records()`\n  uint64_t index_ = 0;\n  // Whether `Recover()` is applicable.\n  //\n  // Invariant: if `recoverable_` then `!ok()`\n  bool recoverable_ = false;\n};\n\n// Implementation details follow.\n\ninline ChunkDecoder::ChunkDecoder(Options options)\n    : field_projection_(std::move(options.field_projection())),\n      recycling_pool_options_(options.recycling_pool_options()),\n      values_reader_(riegeli::Maker()) {}\n\ninline ChunkDecoder::ChunkDecoder(ChunkDecoder&& that) noexcept\n    : Object(static_cast<Object&&>(that)),\n      field_projection_(std::move(that.field_projection_)),\n      recycling_pool_options_(that.recycling_pool_options_),\n      limits_(std::move(that.limits_)),\n      values_reader_(std::move(that.values_reader_)),\n      index_(that.index_),\n      recoverable_(std::exchange(that.recoverable_, false)) {}\n\ninline ChunkDecoder& ChunkDecoder::operator=(ChunkDecoder&& that) noexcept {\n  Object::operator=(static_cast<Object&&>(that));\n  field_projection_ = std::move(that.field_projection_);\n  recycling_pool_options_ = that.recycling_pool_options_;\n  limits_ = std::move(that.limits_);\n  values_reader_ = std::move(that.values_reader_);\n  index_ = that.index_;\n  recoverable_ = std::exchange(that.recoverable_, false);\n  return *this;\n}\n\ninline void ChunkDecoder::Reset(Options options) {\n  field_projection_ = std::move(options.field_projection());\n  recycling_pool_options_ = options.recycling_pool_options();\n  Clear();\n}\n\ninline void ChunkDecoder::Clear() {\n  Object::Reset();\n  limits_.clear();\n  values_reader_.Reset(riegeli::Maker());\n  index_ = 0;\n  recoverable_ = false;\n}\n\ninline bool ChunkDecoder::ReadRecord(absl::string_view& record) {\n  if (ABSL_PREDICT_FALSE(!ok() || index() == num_records())) {\n    record = absl::string_view();\n    return false;\n  }\n  const size_t start = IntCast<size_t>(values_reader_.pos());\n  const size_t limit = limits_[IntCast<size_t>(index_)];\n  RIEGELI_ASSERT_LE(start, limit)\n      << \"Failed invariant of ChunkDecoder: record end positions not sorted\";\n  RIEGELI_EVAL_ASSERT(values_reader_.Read(limit - start, record))\n      << values_reader_.status();\n  ++index_;\n  return true;\n}\n\ninline bool ChunkDecoder::ReadRecord(std::string& record) {\n  if (ABSL_PREDICT_FALSE(!ok() || index() == num_records())) {\n    record.clear();\n    return false;\n  }\n  const size_t start = IntCast<size_t>(values_reader_.pos());\n  const size_t limit = limits_[IntCast<size_t>(index_)];\n  RIEGELI_ASSERT_LE(start, limit)\n      << \"Failed invariant of ChunkDecoder: record end positions not sorted\";\n  RIEGELI_EVAL_ASSERT(values_reader_.Read(limit - start, record))\n      << values_reader_.status();\n  ++index_;\n  return true;\n}\n\ninline bool ChunkDecoder::ReadRecord(Chain& record) {\n  if (ABSL_PREDICT_FALSE(!ok() || index() == num_records())) {\n    record.Clear();\n    return false;\n  }\n  const size_t start = IntCast<size_t>(values_reader_.pos());\n  const size_t limit = limits_[IntCast<size_t>(index_)];\n  RIEGELI_ASSERT_LE(start, limit)\n      << \"Failed invariant of ChunkDecoder: record end positions not sorted\";\n  RIEGELI_EVAL_ASSERT(values_reader_.Read(limit - start, record))\n      << values_reader_.status();\n  ++index_;\n  return true;\n}\n\ninline bool ChunkDecoder::ReadRecord(absl::Cord& record) {\n  if (ABSL_PREDICT_FALSE(!ok() || index() == num_records())) {\n    record.Clear();\n    return false;\n  }\n  const size_t start = IntCast<size_t>(values_reader_.pos());\n  const size_t limit = limits_[IntCast<size_t>(index_)];\n  RIEGELI_ASSERT_LE(start, limit)\n      << \"Failed invariant of ChunkDecoder: record end positions not sorted\";\n  RIEGELI_EVAL_ASSERT(values_reader_.Read(limit - start, record))\n      << values_reader_.status();\n  ++index_;\n  return true;\n}\n\ninline void ChunkDecoder::SetIndex(uint64_t index) {\n  RIEGELI_ASSERT_OK(*this) << \"Failed precondition of ChunkDecoder::SetIndex()\";\n  index_ = UnsignedMin(index, num_records());\n  const size_t start =\n      index_ == 0 ? size_t{0} : limits_[IntCast<size_t>(index_ - 1)];\n  RIEGELI_EVAL_ASSERT(values_reader_.Seek(start)) << values_reader_.status();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CHUNK_ENCODING_CHUNK_DECODER_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/chunk_encoder.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/chunk_encoding/chunk_encoder.h\"\n\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/messages/serialize_message.h\"\n\nnamespace riegeli {\n\nvoid ChunkEncoder::Done() {\n  num_records_ = 0;\n  decoded_data_size_ = 0;\n}\n\nbool ChunkEncoder::AddRecord(const google::protobuf::MessageLite& record,\n                             SerializeMessageOptions serialize_options) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chain serialized;\n  if (absl::Status status =\n          SerializeMessage(record, serialized, serialize_options);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return Fail(std::move(status));\n  }\n  return AddRecord(std::move(serialized));\n}\n\nbool ChunkEncoder::AddRecord(Chain&& record) {\n  // Not `std::move(record)`: forward to `AddRecord(const Chain&)`.\n  return AddRecord(record);\n}\n\nbool ChunkEncoder::AddRecord(absl::Cord&& record) {\n  // Not `std::move(record)`: forward to `AddRecord(const absl::Cord&)`.\n  return AddRecord(record);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/chunk_encoding/chunk_encoder.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_CHUNK_ENCODER_H_\n#define RIEGELI_CHUNK_ENCODING_CHUNK_ENCODER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <type_traits>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/messages/serialize_message.h\"\n\nnamespace riegeli {\n\nclass ChunkEncoder : public Object {\n public:\n  // Creates an empty `ChunkEncoder`.\n  ChunkEncoder() noexcept {}\n\n  ChunkEncoder(const ChunkEncoder&) = delete;\n  ChunkEncoder& operator=(const ChunkEncoder&) = delete;\n\n  // Resets the `ChunkEncoder` back to empty.\n  virtual void Clear();\n\n  // Adds the next record.\n  //\n  // `AddRecord(google::protobuf::MessageLite)` serializes a proto message to\n  // raw bytes beforehand. The remaining overloads accept raw bytes.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool AddRecord(const google::protobuf::MessageLite& record);\n  virtual bool AddRecord(const google::protobuf::MessageLite& record,\n                         SerializeMessageOptions serialize_options);\n  virtual bool AddRecord(BytesRef record) = 0;\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool AddRecord(const char* record) {\n    return AddRecord(absl::string_view(record));\n  }\n  virtual bool AddRecord(ExternalRef record) = 0;\n  template <typename Src,\n            std::enable_if_t<SupportsExternalRefWhole<Src>::value, int> = 0>\n  bool AddRecord(Src&& src);\n  virtual bool AddRecord(const Chain& record) = 0;\n  virtual bool AddRecord(Chain&& record);\n  virtual bool AddRecord(const absl::Cord& record) = 0;\n  virtual bool AddRecord(absl::Cord&& record);\n\n  // Add multiple records, expressed as concatenated record values and sorted\n  // record end positions.\n  //\n  // Preconditions:\n  //   `limits` are sorted\n  //   `(limits.empty() ? 0 : limits.back()) == records.size()`\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  virtual bool AddRecords(Chain records, std::vector<size_t> limits) = 0;\n\n  // Returns the number of records added so far.\n  uint64_t num_records() const { return num_records_; }\n\n  // Returns the sum of record sizes added so far.\n  uint64_t decoded_data_size() const { return decoded_data_size_; }\n\n  // Encodes the chunk to `dest`, setting `chunk_type`, `num_records`, and\n  // `decoded_data_size`. Closes the `ChunkEncoder` on success.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`);\n  //              if `!dest.ok()` then the problem was at `dest`\n  virtual bool EncodeAndClose(Writer& dest, ChunkType& chunk_type,\n                              uint64_t& num_records,\n                              uint64_t& decoded_data_size) = 0;\n\n protected:\n  void Done() override;\n\n  uint64_t num_records_ = 0;\n  uint64_t decoded_data_size_ = 0;\n};\n\n// Implementation details follow.\n\ninline void ChunkEncoder::Clear() {\n  Object::Reset();\n  num_records_ = 0;\n  decoded_data_size_ = 0;\n}\n\ninline bool ChunkEncoder::AddRecord(\n    const google::protobuf::MessageLite& record) {\n  return AddRecord(record, SerializeMessageOptions());\n}\n\ntemplate <typename Src,\n          std::enable_if_t<SupportsExternalRefWhole<Src>::value, int>>\ninline bool ChunkEncoder::AddRecord(Src&& src) {\n  return AddRecord(ExternalRef(std::forward<Src>(src)));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CHUNK_ENCODING_CHUNK_ENCODER_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/compressor.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/chunk_encoding/compressor.h\"\n\n#include <stdint.h>\n\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/chain_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/brotli_encoder_selection.h\"\n#include \"riegeli/chunk_encoding/compressor_options.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/snappy/snappy_writer.h\"\n#include \"riegeli/varint/varint_writing.h\"\n#include \"riegeli/zstd/zstd_writer.h\"\n\nnamespace riegeli::chunk_encoding_internal {\n\nCompressor::Compressor(CompressorOptions compressor_options,\n                       TuningOptions tuning_options)\n    : compressor_options_(std::move(compressor_options)),\n      tuning_options_(std::move(tuning_options)) {\n  Initialize();\n  SetWriteSizeHint();\n}\n\nvoid Compressor::Clear(TuningOptions tuning_options) {\n  tuning_options_ = std::move(tuning_options);\n  Clear();\n}\n\nvoid Compressor::Clear() {\n  Object::Reset();\n  Initialize();\n  SetWriteSizeHint();\n}\n\ninline void Compressor::Initialize() {\n  switch (compressor_options_.compression_type()) {\n    case CompressionType::kNone:\n      writer_ = std::make_unique<ChainWriter<>>(&compressed_);\n      return;\n    case CompressionType::kBrotli:\n      writer_ = NewBrotliWriter(&compressed_, compressor_options_,\n                                tuning_options_.recycling_pool_options());\n      return;\n    case CompressionType::kZstd:\n      writer_ = std::make_unique<ZstdWriter<ChainWriter<>>>(\n          riegeli::Maker(&compressed_),\n          ZstdWriterBase::Options()\n              .set_compression_level(compressor_options_.compression_level())\n              .set_window_log(compressor_options_.zstd_window_log())\n              .set_pledged_size(tuning_options_.pledged_size())\n              .set_recycling_pool_options(\n                  tuning_options_.recycling_pool_options()));\n      return;\n    case CompressionType::kSnappy:\n      writer_ = std::make_unique<SnappyWriter<ChainWriter<>>>(\n          riegeli::Maker(&compressed_),\n          SnappyWriterBase::Options().set_compression_level(\n              compressor_options_.compression_level()));\n      return;\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown compression type: \"\n      << static_cast<unsigned>(compressor_options_.compression_type());\n}\n\ninline void Compressor::SetWriteSizeHint() {\n  writer_->SetWriteSizeHint(tuning_options_.pledged_size() != std::nullopt\n                                ? tuning_options_.pledged_size()\n                                : tuning_options_.size_hint());\n}\n\nbool Compressor::EncodeAndClose(Writer& dest) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const Position uncompressed_size = writer().pos();\n  if (ABSL_PREDICT_FALSE(!writer().Close())) return Fail(writer().status());\n  if (compressor_options_.compression_type() != CompressionType::kNone) {\n    if (ABSL_PREDICT_FALSE(\n            !WriteVarint64(IntCast<uint64_t>(uncompressed_size), dest))) {\n      return Fail(dest.status());\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!dest.Write(std::move(compressed_)))) {\n    return Fail(dest.status());\n  }\n  return Close();\n}\n\nbool Compressor::LengthPrefixedEncodeAndClose(Writer& dest) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const Position uncompressed_size = writer().pos();\n  if (ABSL_PREDICT_FALSE(!writer().Close())) return Fail(writer().status());\n  uint64_t compressed_size = compressed_.size();\n  if (compressor_options_.compression_type() != CompressionType::kNone) {\n    compressed_size += LengthVarint64(IntCast<uint64_t>(uncompressed_size));\n  }\n  if (ABSL_PREDICT_FALSE(!WriteVarint64(compressed_size, dest))) {\n    return Fail(dest.status());\n  }\n  if (compressor_options_.compression_type() != CompressionType::kNone) {\n    if (ABSL_PREDICT_FALSE(\n            !WriteVarint64(IntCast<uint64_t>(uncompressed_size), dest))) {\n      return Fail(dest.status());\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!dest.Write(std::move(compressed_)))) {\n    return Fail(dest.status());\n  }\n  return Close();\n}\n\n}  // namespace riegeli::chunk_encoding_internal\n"
  },
  {
    "path": "riegeli/chunk_encoding/compressor.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_COMPRESSOR_H_\n#define RIEGELI_CHUNK_ENCODING_COMPRESSOR_H_\n\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/compressor_options.h\"\n\nnamespace riegeli::chunk_encoding_internal {\n\nclass Compressor : public Object {\n public:\n  class TuningOptions {\n   public:\n    TuningOptions() noexcept {}\n\n    // Exact uncompressed size, or `std::nullopt` if unknown. This may improve\n    // compression density and performance, and may cause the size to be stored\n    // in the compressed stream header.\n    //\n    // If the pledged size turns out to not match reality, compression may fail.\n    //\n    // Default: `std::nullopt`.\n    TuningOptions& set_pledged_size(std::optional<Position> pledged_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      pledged_size_ = pledged_size;\n      return *this;\n    }\n    TuningOptions&& set_pledged_size(std::optional<Position> pledged_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_pledged_size(pledged_size));\n    }\n    std::optional<Position> pledged_size() const { return pledged_size_; }\n\n    // Expected uncompressed size, or `std::nullopt` if unknown. This may\n    // improve compression density and performance.\n    //\n    // If the size hint turns out to not match reality, nothing breaks.\n    //\n    // `pledged_size()`, if not `std::nullopt`, overrides `size_hint()`.\n    //\n    // Default: `std::nullopt`.\n    TuningOptions& set_size_hint(std::optional<Position> size_hint) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      size_hint_ = size_hint;\n      return *this;\n    }\n    TuningOptions&& set_size_hint(std::optional<Position> size_hint) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_size_hint(size_hint));\n    }\n    std::optional<Position> size_hint() const { return size_hint_; }\n\n    // Options for a global `RecyclingPool` of compression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // compression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    TuningOptions& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    TuningOptions&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    std::optional<Position> pledged_size_;\n    std::optional<Position> size_hint_;\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Creates a closed `Compressor`.\n  explicit Compressor(Closed) noexcept : Object(kClosed) {}\n\n  // Creates an empty `Compressor`.\n  explicit Compressor(CompressorOptions compressor_options,\n                      TuningOptions tuning_options = TuningOptions());\n\n  Compressor(const Compressor&) = delete;\n  Compressor& operator=(const Compressor&) = delete;\n\n  // Resets the `Compressor` back to empty. Keeps compressor options unchanged.\n  // Changes tuning options.\n  void Clear(TuningOptions tuning_options);\n\n  // Resets the `Compressor` back to empty. Keeps compressor options and tuning\n  // options unchanged.\n  void Clear();\n\n  // Returns the `Writer` to which uncompressed data should be written.\n  //\n  // Precondition: `ok()`\n  Writer& writer() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Writes compressed data to `dest`. Closes the `Compressor` on success.\n  //\n  // If `compressor_options.compression_type()` is not `kNone`, writes\n  // uncompressed size as a varint before the data.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool EncodeAndClose(Writer& dest);\n\n  // Like `EncodeAndClose()`, but writes the compressed size as a varint before\n  // anything else. The compressed size includes the length of the uncompressed\n  // size.\n  bool LengthPrefixedEncodeAndClose(Writer& dest);\n\n private:\n  void Initialize();\n  void SetWriteSizeHint();\n\n  CompressorOptions compressor_options_;\n  TuningOptions tuning_options_;\n  Chain compressed_;\n  std::unique_ptr<Writer> writer_;\n};\n\n// Implementation details follow.\n\ninline Writer& Compressor::writer() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT_OK(*this) << \"Failed precondition of Compressor::writer()\";\n  return *writer_;\n}\n\n}  // namespace riegeli::chunk_encoding_internal\n\n#endif  // RIEGELI_CHUNK_ENCODING_COMPRESSOR_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/compressor_options.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/chunk_encoding/compressor_options.h\"\n\n#include <optional>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/options_parser.h\"\n#include \"riegeli/brotli/brotli_writer.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/snappy/snappy_writer.h\"\n#include \"riegeli/zstd/zstd_writer.h\"\n\nnamespace riegeli {\n\nabsl::Status CompressorOptions::FromString(absl::string_view text) {\n  // Set just `compression_type_` first because other parsers depend on\n  // `compression_type_`.\n  {\n    OptionsParser options_parser;\n    options_parser.AddOption(\n        \"uncompressed\",\n        ValueParser::And(ValueParser::FailIfSeen(\"brotli\", \"zstd\", \"snappy\"),\n                         [this](ValueParser& value_parser) {\n                           compression_type_ = CompressionType::kNone;\n                           return true;\n                         }));\n    options_parser.AddOption(\n        \"brotli\", ValueParser::And(\n                      ValueParser::FailIfSeen(\"uncompressed\", \"zstd\", \"snappy\"),\n                      [this](ValueParser& value_parser) {\n                        compression_type_ = CompressionType::kBrotli;\n                        return true;\n                      }));\n    options_parser.AddOption(\n        \"zstd\", ValueParser::And(\n                    ValueParser::FailIfSeen(\"uncompressed\", \"brotli\", \"snappy\"),\n                    [this](ValueParser& value_parser) {\n                      compression_type_ = CompressionType::kZstd;\n                      return true;\n                    }));\n    options_parser.AddOption(\n        \"snappy\", ValueParser::And(\n                      ValueParser::FailIfSeen(\"uncompressed\", \"brotli\", \"zstd\"),\n                      [this](ValueParser& value_parser) {\n                        compression_type_ = CompressionType::kSnappy;\n                        return true;\n                      }));\n    options_parser.AddOption(\"window_log\",\n                             [](ValueParser& value_parser) { return true; });\n    options_parser.AddOption(\"brotli_encoder\",\n                             [](ValueParser& value_parser) { return true; });\n    if (ABSL_PREDICT_FALSE(!options_parser.FromString(text))) {\n      return options_parser.status();\n    }\n  }\n  int window_log;\n  OptionsParser options_parser;\n  options_parser.AddOption(\n      \"uncompressed\",\n      ValueParser::And(ValueParser::FailIfSeen(\"window_log\"),\n                       ValueParser::Empty(0, &compression_level_)));\n  options_parser.AddOption(\n      \"brotli\",\n      ValueParser::Or(\n          ValueParser::Empty(\n              BrotliWriterBase::Options::kDefaultCompressionLevel,\n              &compression_level_),\n          ValueParser::Int(BrotliWriterBase::Options::kMinCompressionLevel,\n                           BrotliWriterBase::Options::kMaxCompressionLevel,\n                           &compression_level_)));\n  options_parser.AddOption(\n      \"zstd\",\n      ValueParser::Or(\n          ValueParser::Empty(ZstdWriterBase::Options::kDefaultCompressionLevel,\n                             &compression_level_),\n          ValueParser::Int(ZstdWriterBase::Options::kMinCompressionLevel,\n                           ZstdWriterBase::Options::kMaxCompressionLevel,\n                           &compression_level_)));\n  options_parser.AddOption(\n      \"snappy\",\n      ValueParser::And(\n          ValueParser::FailIfSeen(\"window_log\"),\n          ValueParser::Or(\n              ValueParser::Empty(\n                  SnappyWriterBase::Options::kDefaultCompressionLevel,\n                  &compression_level_),\n              ValueParser::Int(SnappyWriterBase::Options::kMinCompressionLevel,\n                               SnappyWriterBase::Options::kMaxCompressionLevel,\n                               &compression_level_))));\n  options_parser.AddOption(\"window_log\", [&] {\n    switch (compression_type_) {\n      case CompressionType::kNone:\n        return ValueParser::FailIfSeen(\"uncompressed\");\n      case CompressionType::kBrotli:\n        return ValueParser::Or(\n            ValueParser::Enum({{\"auto\", std::nullopt}}, &window_log_),\n            ValueParser::And(\n                ValueParser::Int(BrotliWriterBase::Options::kMinWindowLog,\n                                 BrotliWriterBase::Options::kMaxWindowLog,\n                                 &window_log),\n                [this, &window_log](ValueParser& value_parser) {\n                  window_log_ = window_log;\n                  return true;\n                }));\n      case CompressionType::kZstd:\n        return ValueParser::Or(\n            ValueParser::Enum({{\"auto\", std::nullopt}}, &window_log_),\n            ValueParser::And(\n                ValueParser::Int(ZstdWriterBase::Options::kMinWindowLog,\n                                 ZstdWriterBase::Options::kMaxWindowLog,\n                                 &window_log),\n                [this, &window_log](ValueParser& value_parser) {\n                  window_log_ = window_log;\n                  return true;\n                }));\n      case CompressionType::kSnappy:\n        return ValueParser::FailIfSeen(\"snappy\");\n    }\n    RIEGELI_ASSUME_UNREACHABLE() << \"Unknown compression type: \"\n                                 << static_cast<unsigned>(compression_type_);\n  }());\n  options_parser.AddOption(\n      \"brotli_encoder\",\n      ValueParser::Enum(\n          {{\"rbrotli_or_cbrotli\", BrotliEncoder::kRBrotliOrCBrotli},\n           {\"cbrotli\", BrotliEncoder::kCBrotli},\n           {\"rbrotli\", BrotliEncoder::kRBrotli}},\n          &brotli_encoder_));\n  if (ABSL_PREDICT_FALSE(!options_parser.FromString(text))) {\n    return options_parser.status();\n  }\n  return absl::OkStatus();\n}\n\nint CompressorOptions::brotli_window_log() const {\n  RIEGELI_ASSERT_EQ(compression_type_, CompressionType::kBrotli)\n      << \"Failed precondition of CompressorOptions::brotli_window_log(): \"\n         \"compression type must be Brotli\";\n  if (window_log_ == std::nullopt) {\n    return BrotliWriterBase::Options::kDefaultWindowLog;\n  } else {\n    RIEGELI_ASSERT_GE(*window_log_, BrotliWriterBase::Options::kMinWindowLog)\n        << \"Failed precondition of CompressorOptions::set_window_log(): \"\n           \"window log out of range for Brotli\";\n    RIEGELI_ASSERT_LE(*window_log_, BrotliWriterBase::Options::kMaxWindowLog)\n        << \"Failed precondition of CompressorOptions::set_window_log(): \"\n           \"window log out of range for Brotli\";\n    return *window_log_;\n  }\n}\n\nstd::optional<int> CompressorOptions::zstd_window_log() const {\n  RIEGELI_ASSERT_EQ(compression_type_, CompressionType::kZstd)\n      << \"Failed precondition of CompressorOptions::zstd_window_log(): \"\n         \"compression type must be Zstd\";\n  if (window_log_ != std::nullopt) {\n    RIEGELI_ASSERT_GE(*window_log_, ZstdWriterBase::Options::kMinWindowLog)\n        << \"Failed precondition of CompressorOptions::set_window_log(): \"\n           \"window log out of range for Zstd\";\n    RIEGELI_ASSERT_LE(*window_log_, ZstdWriterBase::Options::kMaxWindowLog)\n        << \"Failed precondition of CompressorOptions::set_window_log(): \"\n           \"window log out of range for Zstd\";\n  }\n  return window_log_;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/chunk_encoding/compressor_options.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_COMPRESSOR_OPTIONS_H_\n#define RIEGELI_CHUNK_ENCODING_COMPRESSOR_OPTIONS_H_\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/brotli/brotli_writer.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/snappy/snappy_writer.h\"\n#include \"riegeli/zstd/zstd_writer.h\"\n\nnamespace riegeli {\n\n// The implementation of the Brotli encoder to use. Experimental, meant for\n// evaluation. Prefer to keep the default.\n//\n// Rust Brotli is currently not available in open sourced Riegeli.\nenum class BrotliEncoder {\n  kRBrotliOrCBrotli,  // Rust Brotli if available, C Brotli otherwise. Default.\n  kCBrotli,           // C Brotli.\n  kRBrotli,           // Rust Brotli if available, fail otherwise.\n};\n\nclass CompressorOptions {\n public:\n  CompressorOptions() noexcept {}\n\n  // Parses options from text:\n  // ```\n  //   options ::= option? (\",\" option?)*\n  //   option ::=\n  //     \"uncompressed\" |\n  //     \"brotli\" (\":\" brotli_level)? |\n  //     \"zstd\" (\":\" zstd_level)? |\n  //     \"snappy\" (\":\" snappy_level)? |\n  //     \"window_log\" \":\" window_log |\n  //     \"brotli_encoder\" \":\" (\"rbrotli_or_cbrotli\" | \"cbrotli\" | \"rbrotli\")\n  //   brotli_level ::= integer in the range [0..11] (default 6)\n  //   zstd_level ::= integer in the range [-131072..22] (default 3)\n  //   snappy_level ::= integer in the range [1..2] (default 1)\n  //   window_log ::= \"auto\" or integer in the range [10..31]\n  // ```\n  //\n  // Returns status:\n  //  * `status.ok()`  - success\n  //  * `!status.ok()` - failure\n  absl::Status FromString(absl::string_view text);\n\n  // Changes compression algorithm to Uncompressed (turns compression off).\n  CompressorOptions& set_uncompressed() & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    compression_type_ = CompressionType::kNone;\n    compression_level_ = 0;\n    return *this;\n  }\n  CompressorOptions&& set_uncompressed() && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_uncompressed());\n  }\n\n  // Changes compression algorithm to Brotli. Sets compression level which\n  // tunes the tradeoff between compression density and compression speed\n  // (higher = better density but slower).\n  //\n  // `compression_level` must be between `kMinBrotli` (0) and `kMaxBrotli` (11).\n  // Default: `kDefaultBrotli` (6).\n  //\n  // This is the default compression algorithm.\n  static constexpr int kMinBrotli =\n      BrotliWriterBase::Options::kMinCompressionLevel;\n  static constexpr int kMaxBrotli =\n      BrotliWriterBase::Options::kMaxCompressionLevel;\n  static constexpr int kDefaultBrotli =\n      BrotliWriterBase::Options::kDefaultCompressionLevel;\n  CompressorOptions& set_brotli(int compression_level = kDefaultBrotli) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_GE(compression_level, kMinBrotli)\n        << \"Failed precondition of CompressorOptions::set_brotli(): \"\n           \"compression level out of range\";\n    RIEGELI_ASSERT_LE(compression_level, kMaxBrotli)\n        << \"Failed precondition of CompressorOptions::set_brotli(): \"\n           \"compression level out of range\";\n    compression_type_ = CompressionType::kBrotli;\n    compression_level_ = compression_level;\n    return *this;\n  }\n  CompressorOptions&& set_brotli(int compression_level = kDefaultBrotli) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_brotli(compression_level));\n  }\n\n  // Changes compression algorithm to Zstd. Sets compression level which tunes\n  // the tradeoff between compression density and compression speed (higher =\n  // better density but slower).\n  //\n  // `compression_level` must be between `kMinZstd` (-131072) and\n  // `kMaxZstd` (22). Level 0 is currently equivalent to 3.\n  // Default: `kDefaultZstd` (3).\n  static constexpr int kMinZstd = ZstdWriterBase::Options::kMinCompressionLevel;\n  static constexpr int kMaxZstd = ZstdWriterBase::Options::kMaxCompressionLevel;\n  static constexpr int kDefaultZstd =\n      ZstdWriterBase::Options::kDefaultCompressionLevel;\n  CompressorOptions& set_zstd(int compression_level = kDefaultZstd) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_GE(compression_level, kMinZstd)\n        << \"Failed precondition of CompressorOptions::set_zstd(): \"\n           \"compression level out of range\";\n    RIEGELI_ASSERT_LE(compression_level, kMaxZstd)\n        << \"Failed precondition of CompressorOptions::set_zstd(): \"\n           \"compression level out of range\";\n    compression_type_ = CompressionType::kZstd;\n    compression_level_ = compression_level;\n    return *this;\n  }\n  CompressorOptions&& set_zstd(int compression_level = kDefaultZstd) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_zstd(compression_level));\n  }\n\n  // Changes compression algorithm to Snappy.  Sets compression level which\n  // tunes the tradeoff between compression density and compression speed\n  // (higher = better density but slower).\n  static constexpr int kMinSnappy =\n      SnappyWriterBase::Options::kMinCompressionLevel;\n  static constexpr int kMaxSnappy =\n      SnappyWriterBase::Options::kMaxCompressionLevel;\n  static constexpr int kDefaultSnappy =\n      SnappyWriterBase::Options::kDefaultCompressionLevel;\n  CompressorOptions& set_snappy(int compression_level = kDefaultSnappy) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    compression_type_ = CompressionType::kSnappy;\n    compression_level_ = compression_level;\n    return *this;\n  }\n  CompressorOptions&& set_snappy(int compression_level = kDefaultSnappy) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_snappy(compression_level));\n  }\n\n  CompressionType compression_type() const { return compression_type_; }\n\n  int compression_level() const { return compression_level_; }\n\n  // Logarithm of the LZ77 sliding window size. This tunes the tradeoff\n  // between compression density and memory usage (higher = better density but\n  // more memory).\n  //\n  // Special value `std::nullopt` means to keep the default (Brotli: 22,\n  // Zstd: derived from compression level and chunk size).\n  //\n  // For Uncompressed and Snappy, `window_log` must be `std::nullopt`.\n  //\n  // For Brotli, `window_log` must be `std::nullopt` or between\n  // `BrotliWriterBase::Options::kMinWindowLog` (10) and\n  // `BrotliWriterBase::Options::kMaxWindowLog` (30).\n  //\n  // For Zstd, `window_log` must be `std::nullopt` or between\n  // `ZstdWriterBase::Options::kMinWindowLog` (10) and\n  // `ZstdWriterBase::Options::kMaxWindowLog` (30 in 32-bit build,\n  // 31 in 64-bit build).\n  //\n  // Default: `std::nullopt`.\n  static constexpr int kMinWindowLog =\n      SignedMin(BrotliWriterBase::Options::kMinWindowLog,\n                ZstdWriterBase::Options::kMinWindowLog);\n  static constexpr int kMaxWindowLog =\n      SignedMax(BrotliWriterBase::Options::kMaxWindowLog,\n                ZstdWriterBase::Options::kMaxWindowLog);\n  CompressorOptions& set_window_log(std::optional<int> window_log) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    if (window_log != std::nullopt) {\n      RIEGELI_ASSERT_GE(*window_log, kMinWindowLog)\n          << \"Failed precondition of CompressorOptions::set_window_log(): \"\n             \"window log out of range\";\n      RIEGELI_ASSERT_LE(*window_log, kMaxWindowLog)\n          << \"Failed precondition of CompressorOptions::set_window_log(): \"\n             \"window log out of range\";\n    }\n    window_log_ = window_log;\n    return *this;\n  }\n  CompressorOptions&& set_window_log(std::optional<int> window_log) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_window_log(window_log));\n  }\n  std::optional<int> window_log() const { return window_log_; }\n\n  // Returns `window_log()` translated for `BrotliWriter`.\n  //\n  // Precondition: `compression_type() == CompressionType::kBrotli`\n  int brotli_window_log() const;\n\n  // Returns `window_log()` translated for `ZstdWriter`.\n  //\n  // Precondition: `compression_type() == CompressionType::kZstd`\n  std::optional<int> zstd_window_log() const;\n\n  // The implementation of the Brotli encoder to use. Experimental, meant for\n  // evaluation. Prefer to keep the default.\n  //\n  // This is ignored if `compression_type() != CompressionType::kBrotli`.\n  //\n  // If Rust Brotli is used, the interpretation of compression levels is\n  // slightly different (in particular compression levels smaller than 3 are\n  // equivalent to 3, and compression levels larger than 7 are equivalent to 7),\n  // and `window_log()` is ignored.\n  //\n  // Default: `BrotliEncoder::kRBrotliOrCBrotli`.\n  CompressorOptions& set_brotli_encoder(BrotliEncoder brotli_encoder) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    brotli_encoder_ = brotli_encoder;\n    return *this;\n  }\n  CompressorOptions&& set_brotli_encoder(BrotliEncoder brotli_encoder) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_brotli_encoder(brotli_encoder));\n  }\n  BrotliEncoder brotli_encoder() const { return brotli_encoder_; }\n\n private:\n  CompressionType compression_type_ = CompressionType::kBrotli;\n  int compression_level_ = kDefaultBrotli;\n  std::optional<int> window_log_;\n  BrotliEncoder brotli_encoder_ = BrotliEncoder::kRBrotliOrCBrotli;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CHUNK_ENCODING_COMPRESSOR_OPTIONS_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/constants.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_CONSTANTS_H_\n#define RIEGELI_CHUNK_ENCODING_CONSTANTS_H_\n\n#include <stdint.h>\n\n#include <limits>\n\nnamespace riegeli {\n\n// These values are frozen in the file format.\nenum class ChunkType : uint8_t {\n  kFileSignature = 's',\n  kFileMetadata = 'm',\n  kPadding = 'p',\n  kSimple = 'r',\n  kTransposed = 't',\n};\n\n// These values are frozen in the file format.\nenum class CompressionType : uint8_t {\n  kNone = 0,\n  kBrotli = 'b',\n  kZstd = 'z',\n  kSnappy = 's',\n};\n\ninline constexpr uint64_t kMaxNumRecords =\n    std::numeric_limits<uint64_t>::max() >> 8;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CHUNK_ENCODING_CONSTANTS_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/decompressor.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/chunk_encoding/decompressor.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <optional>\n\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/varint/varint_reading.h\"\n\nnamespace riegeli::chunk_encoding_internal {\n\nstd::optional<uint64_t> UncompressedSize(const Chain& compressed_data,\n                                         CompressionType compression_type) {\n  if (compression_type == CompressionType::kNone) return compressed_data.size();\n  ChainReader<> compressed_data_reader(&compressed_data);\n  uint64_t size;\n  if (ABSL_PREDICT_FALSE(!ReadVarint64(compressed_data_reader, size))) {\n    return std::nullopt;\n  }\n  return size;\n}\n\n}  // namespace riegeli::chunk_encoding_internal\n"
  },
  {
    "path": "riegeli/chunk_encoding/decompressor.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_DECOMPRESSOR_H_\n#define RIEGELI_CHUNK_ENCODING_DECOMPRESSOR_H_\n\n#include <stdint.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"riegeli/base/any.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/brotli/brotli_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/snappy/snappy_reader.h\"\n#include \"riegeli/varint/varint_reading.h\"\n#include \"riegeli/zstd/zstd_reader.h\"\n\nnamespace riegeli::chunk_encoding_internal {\n\n// Returns uncompressed size of `compressed_data`.\n//\n// If `compression_type` is `kNone`, uncompressed size is the same as compressed\n// size, otherwise reads uncompressed size as a varint from the beginning of\n// compressed_data.\n//\n// Returns `std::nullopt` on failure.\nstd::optional<uint64_t> UncompressedSize(const Chain& compressed_data,\n                                         CompressionType compression_type);\n\n// Options for a `Decompressor`.\nclass DecompressorOptions {\n public:\n  DecompressorOptions() noexcept {}\n\n  // Options for a global `RecyclingPool` of decompression contexts.\n  //\n  // They tune the amount of memory which is kept to speed up creation of new\n  // decompression sessions, and usage of a background thread to clean it.\n  //\n  // Default: `RecyclingPoolOptions()`.\n  DecompressorOptions& set_recycling_pool_options(\n      const RecyclingPoolOptions& recycling_pool_options) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    recycling_pool_options_ = recycling_pool_options;\n    return *this;\n  }\n  DecompressorOptions&& set_recycling_pool_options(\n      const RecyclingPoolOptions& recycling_pool_options) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_recycling_pool_options(recycling_pool_options));\n  }\n  const RecyclingPoolOptions& recycling_pool_options() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return recycling_pool_options_;\n  }\n\n private:\n  RecyclingPoolOptions recycling_pool_options_;\n};\n\n// Decompresses a compressed stream.\n//\n// If `compression_type` is not `kNone`, reads uncompressed size as a varint\n// from the beginning of compressed data.\ntemplate <typename Src = Reader*>\nclass Decompressor : public Object {\n public:\n  using Options = DecompressorOptions;\n\n  // Creates a closed `Decompressor`.\n  explicit Decompressor(Closed) noexcept : Object(kClosed) {}\n\n  // Will read from the compressed stream provided by `src`.\n  explicit Decompressor(Initializer<Src> src, CompressionType compression_type,\n                        DecompressorOptions options = DecompressorOptions());\n\n  Decompressor(Decompressor&& that) = default;\n  Decompressor& operator=(Decompressor&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `Decompressor`. This avoids\n  // constructing a temporary `Decompressor` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      Initializer<Src> src, CompressionType compression_type,\n      DecompressorOptions options = DecompressorOptions());\n\n  // Returns the `Reader` from which uncompressed data should be read.\n  //\n  // Precondition: `ok()`\n  Reader& reader() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Verifies that the source ends at the current position (i.e. has no more\n  // compressed data and has no data after the compressed stream), failing the\n  // `Decompressor` if not. Closes the `Decompressor`.\n  //\n  // Return values:\n  //  * `true`  - success (the source ends at the former current position)\n  //  * `false` - failure (the source does not end at the former current\n  //                       position or the `Decompressor` was not OK before\n  //                       closing)\n  bool VerifyEndAndClose();\n\n  // Verifies that the source ends at the current position (i.e. has no more\n  // compressed data and has no data after the compressed stream), failing the\n  // `Decompressor` if not.\n  void VerifyEnd();\n\n protected:\n  void Done() override;\n\n private:\n  void Initialize(Initializer<Src> src, CompressionType compression_type,\n                  const RecyclingPoolOptions& recycling_pool_options);\n\n  Any<Reader*>::Inlining<Src, BrotliReader<Src>, ZstdReader<Src>,\n                         SnappyReader<Src>>\n      decompressed_;\n};\n\n// Implementation details follow.\n\ntemplate <typename Src>\ninline Decompressor<Src>::Decompressor(Initializer<Src> src,\n                                       CompressionType compression_type,\n                                       DecompressorOptions options) {\n  Initialize(std::move(src), compression_type,\n             options.recycling_pool_options());\n}\n\ntemplate <typename Src>\ninline void Decompressor<Src>::Reset(Closed) {\n  Object::Reset(kClosed);\n  decompressed_.Reset();\n}\n\ntemplate <typename Src>\ninline void Decompressor<Src>::Reset(Initializer<Src> src,\n                                     CompressionType compression_type,\n                                     DecompressorOptions options) {\n  Object::Reset();\n  Initialize(std::move(src), compression_type,\n             options.recycling_pool_options());\n}\n\ntemplate <typename Src>\ninline void Decompressor<Src>::Initialize(\n    Initializer<Src> src, CompressionType compression_type,\n    const RecyclingPoolOptions& recycling_pool_options) {\n  if (compression_type == CompressionType::kNone) {\n    decompressed_ = std::move(src);\n    return;\n  }\n  Dependency<Reader*, Src> compressed_reader(std::move(src));\n  uint64_t uncompressed_size;\n  if (ABSL_PREDICT_FALSE(\n          !ReadVarint64(*compressed_reader, uncompressed_size))) {\n    Fail(compressed_reader->StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Reading uncompressed size failed\")));\n    return;\n  }\n  switch (compression_type) {\n    case CompressionType::kNone:\n      RIEGELI_ASSUME_UNREACHABLE() << \"kNone handled above\";\n    case CompressionType::kBrotli:\n      decompressed_ = riegeli::Maker<BrotliReader<Src>>(\n          std::move(compressed_reader.manager()));\n      return;\n    case CompressionType::kZstd:\n      decompressed_ = riegeli::Maker<ZstdReader<Src>>(\n          std::move(compressed_reader.manager()),\n          ZstdReaderBase::Options().set_recycling_pool_options(\n              recycling_pool_options));\n      return;\n    case CompressionType::kSnappy:\n      decompressed_ = riegeli::Maker<SnappyReader<Src>>(\n          std::move(compressed_reader.manager()));\n      return;\n  }\n  Fail(absl::UnimplementedError(absl::StrCat(\n      \"Unknown compression type: \", static_cast<unsigned>(compression_type))));\n}\n\ntemplate <typename Src>\ninline Reader& Decompressor<Src>::reader() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT_OK(*this) << \"Failed precondition of Decompressor::reader()\";\n  return *decompressed_;\n}\n\ntemplate <typename Src>\nvoid Decompressor<Src>::Done() {\n  if (ABSL_PREDICT_FALSE(!decompressed_->Close())) {\n    Fail(decompressed_->status());\n  }\n}\n\ntemplate <typename Src>\ninline bool Decompressor<Src>::VerifyEndAndClose() {\n  VerifyEnd();\n  return Close();\n}\n\ntemplate <typename Src>\ninline void Decompressor<Src>::VerifyEnd() {\n  if (ABSL_PREDICT_TRUE(ok())) decompressed_->VerifyEnd();\n}\n\n}  // namespace riegeli::chunk_encoding_internal\n\n#endif  // RIEGELI_CHUNK_ENCODING_DECOMPRESSOR_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/deferred_encoder.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/chunk_encoding/deferred_encoder.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/bytes/chain_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/chunk_encoder.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/messages/serialize_message.h\"\n\nnamespace riegeli {\n\nvoid DeferredEncoder::Clear() {\n  ChunkEncoder::Clear();\n  base_encoder_->Clear();\n  records_writer_.Reset();\n  limits_.clear();\n}\n\nbool DeferredEncoder::AddRecord(const google::protobuf::MessageLite& record,\n                                SerializeMessageOptions serialize_options) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const size_t size = serialize_options.GetByteSize(record);\n  if (ABSL_PREDICT_FALSE(num_records_ ==\n                         UnsignedMin(limits_.max_size(), kMaxNumRecords))) {\n    return Fail(absl::ResourceExhaustedError(\"Too many records\"));\n  }\n  if (ABSL_PREDICT_FALSE(size > std::numeric_limits<uint64_t>::max() -\n                                    decoded_data_size_)) {\n    return Fail(absl::ResourceExhaustedError(\"Decoded data size too large\"));\n  }\n  ++num_records_;\n  decoded_data_size_ += IntCast<uint64_t>(size);\n  if (absl::Status status =\n          SerializeMessage(record, records_writer_, serialize_options);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return Fail(std::move(status));\n  }\n  limits_.push_back(IntCast<size_t>(records_writer_.pos()));\n  return true;\n}\n\nbool DeferredEncoder::AddRecord(BytesRef record) {\n  return AddRecordImpl(record);\n}\n\nbool DeferredEncoder::AddRecord(ExternalRef record) {\n  return AddRecordImpl(std::move(record));\n}\n\nbool DeferredEncoder::AddRecord(const Chain& record) {\n  return AddRecordImpl(record);\n}\n\nbool DeferredEncoder::AddRecord(Chain&& record) {\n  return AddRecordImpl(std::move(record));\n}\n\nbool DeferredEncoder::AddRecord(const absl::Cord& record) {\n  return AddRecordImpl(record);\n}\n\nbool DeferredEncoder::AddRecord(absl::Cord&& record) {\n  return AddRecordImpl(std::move(record));\n}\n\ntemplate <typename Record>\nbool DeferredEncoder::AddRecordImpl(Record&& record) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(num_records_ ==\n                         UnsignedMin(limits_.max_size(), kMaxNumRecords))) {\n    return Fail(absl::ResourceExhaustedError(\"Too many records\"));\n  }\n  if (ABSL_PREDICT_FALSE(record.size() > std::numeric_limits<uint64_t>::max() -\n                                             decoded_data_size_)) {\n    return Fail(absl::ResourceExhaustedError(\"Decoded data size too large\"));\n  }\n  ++num_records_;\n  decoded_data_size_ += IntCast<uint64_t>(record.size());\n  if (ABSL_PREDICT_FALSE(\n          !records_writer_.Write(std::forward<Record>(record)))) {\n    return Fail(records_writer_.status());\n  }\n  limits_.push_back(IntCast<size_t>(records_writer_.pos()));\n  return true;\n}\n\nbool DeferredEncoder::AddRecords(Chain records, std::vector<size_t> limits) {\n  RIEGELI_ASSERT_EQ(limits.empty() ? 0u : limits.back(), records.size())\n      << \"Failed precondition of ChunkEncoder::AddRecords(): \"\n         \"record end positions do not match concatenated record values\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(limits.size() >\n                         UnsignedMin(limits_.max_size(), kMaxNumRecords) -\n                             num_records_)) {\n    return Fail(absl::ResourceExhaustedError(\"Too many records\"));\n  }\n  num_records_ += IntCast<uint64_t>(limits.size());\n  decoded_data_size_ += IntCast<uint64_t>(records.size());\n  if (ABSL_PREDICT_FALSE(!records_writer_.Write(std::move(records)))) {\n    return Fail(records_writer_.status());\n  }\n  if (limits_.empty()) {\n    limits_ = std::move(limits);\n  } else {\n    const size_t base = limits_.back();\n    for (size_t& limit : limits) limit += base;\n    limits_.insert(limits_.cend(), limits.begin(), limits.end());\n  }\n  return true;\n}\n\nbool DeferredEncoder::EncodeAndClose(Writer& dest, ChunkType& chunk_type,\n                                     uint64_t& num_records,\n                                     uint64_t& decoded_data_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!records_writer_.Close())) {\n    return Fail(records_writer_.status());\n  }\n  if (ABSL_PREDICT_FALSE(!base_encoder_->AddRecords(\n          std::move(records_writer_.dest()), std::move(limits_))) ||\n      ABSL_PREDICT_FALSE(!base_encoder_->EncodeAndClose(\n          dest, chunk_type, num_records, decoded_data_size))) {\n    Fail(base_encoder_->status());\n  }\n  return Close();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/chunk_encoding/deferred_encoder.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_DEFERRED_ENCODER_H_\n#define RIEGELI_CHUNK_ENCODING_DEFERRED_ENCODER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <memory>\n#include <utility>\n#include <vector>\n\n#include \"absl/strings/cord.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/bytes/chain_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/chunk_encoder.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/messages/serialize_message.h\"\n\nnamespace riegeli {\n\n// `DeferredEncoder` performs a minimal amount of the encoding work in\n// `AddRecord()`, deferring as much as possible to `EncodeAndClose()`.\n// It does more memory copying than the base encoder though.\nclass DeferredEncoder : public ChunkEncoder {\n public:\n  explicit DeferredEncoder(std::unique_ptr<ChunkEncoder> base_encoder);\n\n  void Clear() override;\n\n  using ChunkEncoder::AddRecord;\n  bool AddRecord(const google::protobuf::MessageLite& record,\n                 SerializeMessageOptions serialize_options) override;\n  bool AddRecord(BytesRef record) override;\n  bool AddRecord(ExternalRef record) override;\n  bool AddRecord(const Chain& record) override;\n  bool AddRecord(Chain&& record) override;\n  bool AddRecord(const absl::Cord& record) override;\n  bool AddRecord(absl::Cord&& record) override;\n\n  bool AddRecords(Chain records, std::vector<size_t> limits) override;\n\n  bool EncodeAndClose(Writer& dest, ChunkType& chunk_type,\n                      uint64_t& num_records,\n                      uint64_t& decoded_data_size) override;\n\n private:\n  // This template is defined and used only in deferred_encoder.cc.\n  template <typename Record>\n  bool AddRecordImpl(Record&& record);\n\n  std::unique_ptr<ChunkEncoder> base_encoder_;\n  // `Writer` of concatenated record values.\n  ChainWriter<Chain> records_writer_;\n  // Sorted record end positions.\n  //\n  // Invariant: `limits_.size() == num_records_`\n  std::vector<size_t> limits_;\n\n  // Invariant:\n  //   `records_writer_.pos() == (limits_.empty() ? 0 : limits_.back())`\n};\n\n// Implementation details follow.\n\ninline DeferredEncoder::DeferredEncoder(\n    std::unique_ptr<ChunkEncoder> base_encoder)\n    : base_encoder_(std::move(base_encoder)) {}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CHUNK_ENCODING_DEFERRED_ENCODER_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/field_projection.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_FIELD_PROJECTION_H_\n#define RIEGELI_CHUNK_ENCODING_FIELD_PROJECTION_H_\n\n#include <initializer_list>\n#include <utility>\n\n#include \"absl/container/inlined_vector.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n\nnamespace riegeli {\n\n// Specifies a proto field path.\nclass Field {\n public:\n  using Path = absl::InlinedVector<int, 1>;\n\n  // A special field number value which can be added to the end of the path.\n  //\n  // It preserves field existence but ignores its value, which is replaced with\n  // a default value for the type (zero, empty string, empty message).\n  //\n  // This is useful to include a required field which is not otherwise needed.\n  // This works similarly to specifying a non-existent child field, but applies\n  // not only to submessages.\n  //\n  // Warning: for a repeated field this preserves the field count only if the\n  // field is not packed.\n  static constexpr int kExistenceOnly = 0;\n\n  // Specifies the path using a sequence of proto field numbers descending from\n  // the root message.\n  //\n  // Field numbers can be obtained from `Type::k*FieldNumber` constants exported\n  // by compiled proto messages, or from `FieldDescriptor::number()`.\n  /*implicit*/ Field(std::initializer_list<int> path);\n  Field& operator=(std::initializer_list<int> path);\n\n  // Starts with the root message. Field path can be built with\n  // `AddFieldNumber()`.\n  Field() = default;\n\n  Field(const Field& that) = default;\n  Field& operator=(const Field& that) = default;\n\n  Field(Field&& that) = default;\n  Field& operator=(Field&& that) = default;\n\n  // Adds a field to the end of the path.\n  Field& AddFieldNumber(int field_number) &;\n  Field&& AddFieldNumber(int field_number) &&;\n\n  // Returns the sequence of proto field numbers descending from the root\n  // message.\n  const Path& path() const { return path_; }\n\n private:\n  static void AssertValid(int field_number);\n\n  Path path_;\n};\n\n// Specifies a set of fields to include.\nclass FieldProjection {\n public:\n  using Fields = absl::InlinedVector<Field, 1>;\n\n  // Includes all fields.\n  static FieldProjection All();\n\n  // Includes only the specified fields.\n  /*implicit*/ FieldProjection(std::initializer_list<Field> fields);\n  FieldProjection& operator=(std::initializer_list<Field> fields);\n\n  // Starts with an empty set to include. Fields can be added by `AddField()`.\n  FieldProjection() = default;\n\n  FieldProjection(const FieldProjection&) = default;\n  FieldProjection& operator=(const FieldProjection&) = default;\n\n  FieldProjection(FieldProjection&&) = default;\n  FieldProjection& operator=(FieldProjection&&) = default;\n\n  // Adds a field to the set to include.\n  FieldProjection& AddField(Initializer<Field> field) &;\n  FieldProjection&& AddField(Initializer<Field> field) &&;\n  FieldProjection& AddField(std::initializer_list<int> path) &;\n  FieldProjection&& AddField(std::initializer_list<int> path) &&;\n\n  // Returns true if all fields are included, i.e. if the root message is\n  // included.\n  bool includes_all() const;\n\n  // Returns the set of fields to include.\n  const Fields& fields() const { return fields_; }\n\n private:\n  Fields fields_;\n};\n\n// Implementation details follow.\n\ninline Field::Field(std::initializer_list<int> path) : path_(path) {\n  for (const int field_number : path_) AssertValid(field_number);\n}\n\ninline Field& Field::operator=(std::initializer_list<int> path) {\n  for (const int field_number : path) AssertValid(field_number);\n  path_ = path;\n  return *this;\n}\n\ninline void Field::AssertValid(int field_number) {\n  static_assert(kExistenceOnly == 0,\n                \"Field::AssertValid() assumes that kExistenceOnly == 0\");\n  RIEGELI_ASSERT_GE(field_number, 0) << \"Field number out of range\";\n  RIEGELI_ASSERT_LE(field_number, (1 << 29) - 1) << \"Field number out of range\";\n}\n\ninline Field& Field::AddFieldNumber(int field_number) & {\n  AssertValid(field_number);\n  path_.push_back(field_number);\n  return *this;\n}\n\ninline Field&& Field::AddFieldNumber(int field_number) && {\n  return std::move(AddFieldNumber(field_number));\n}\n\ninline FieldProjection FieldProjection::All() {\n  FieldProjection field_projection;\n  field_projection.AddField(Field());\n  return field_projection;\n}\n\ninline FieldProjection::FieldProjection(std::initializer_list<Field> fields)\n    : fields_(fields) {}\n\ninline FieldProjection& FieldProjection::operator=(\n    std::initializer_list<Field> fields) {\n  fields_ = fields;\n  return *this;\n}\n\ninline FieldProjection& FieldProjection::AddField(Initializer<Field> field) & {\n  fields_.emplace_back(std::move(field));\n  return *this;\n}\n\ninline FieldProjection&& FieldProjection::AddField(\n    Initializer<Field> field) && {\n  return std::move(AddField(std::move(field)));\n}\n\ninline FieldProjection& FieldProjection::AddField(\n    std::initializer_list<int> path) & {\n  return AddField(riegeli::Maker<Field>(path));\n}\n\ninline FieldProjection&& FieldProjection::AddField(\n    std::initializer_list<int> path) && {\n  return std::move(AddField(path));\n}\n\ninline bool FieldProjection::includes_all() const {\n  for (const Field& field : fields_) {\n    if (field.path().empty()) return true;\n  }\n  return false;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CHUNK_ENCODING_FIELD_PROJECTION_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/hash.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/chunk_encoding/hash.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <optional>\n\n#include \"absl/container/fixed_array.h\"\n#include \"absl/strings/string_view.h\"\n#include \"highwayhash/hh_types.h\"\n#include \"highwayhash/highwayhash_target.h\"\n#include \"highwayhash/instruction_sets.h\"\n#include \"riegeli/base/chain.h\"\n\nnamespace riegeli::chunk_encoding_internal {\n\nnamespace {\n\nalignas(32) const highwayhash::HHKey kHashKey = {\n    0x2f696c6567656952,  // 'Riegeli/'\n    0x0a7364726f636572,  // 'records\\n'\n    0x2f696c6567656952,  // 'Riegeli/'\n    0x0a7364726f636572,  // 'records\\n'\n};\n\n}  // namespace\n\nuint64_t Hash(absl::string_view data) {\n  highwayhash::HHResult64 result;\n  highwayhash::InstructionSets::Run<highwayhash::HighwayHash>(\n      kHashKey, data.data(), data.size(), &result);\n  return result;\n}\n\nuint64_t Hash(const Chain& data) {\n  if (const std::optional<absl::string_view> flat = data.TryFlat();\n      flat != std::nullopt) {\n    return Hash(*flat);\n  }\n  absl::FixedArray<highwayhash::StringView> fragments(data.blocks().size());\n  size_t i = 0;\n  for (const absl::string_view fragment : data.blocks()) {\n    fragments[i++] = highwayhash::StringView{fragment.data(), fragment.size()};\n  }\n  highwayhash::HHResult64 result;\n  highwayhash::InstructionSets::Run<highwayhash::HighwayHashCat>(\n      kHashKey, fragments.data(), fragments.size(), &result);\n  return result;\n}\n\n}  // namespace riegeli::chunk_encoding_internal\n"
  },
  {
    "path": "riegeli/chunk_encoding/hash.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_HASH_H_\n#define RIEGELI_CHUNK_ENCODING_HASH_H_\n\n#include <stdint.h>\n\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/chain.h\"\n\nnamespace riegeli::chunk_encoding_internal {\n\nuint64_t Hash(absl::string_view data);\nuint64_t Hash(const Chain& data);\n\n}  // namespace riegeli::chunk_encoding_internal\n\n#endif  // RIEGELI_CHUNK_ENCODING_HASH_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/simple_decoder.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/chunk_encoding/simple_decoder.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <vector>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/chunk_encoding/decompressor.h\"\n#include \"riegeli/varint/varint_reading.h\"\n\nnamespace riegeli {\n\nvoid SimpleDecoder::Done() {\n  if (ABSL_PREDICT_FALSE(!values_decompressor_.Close())) {\n    Fail(values_decompressor_.status());\n  }\n}\n\nbool SimpleDecoder::Decode(Reader* src, uint64_t num_records,\n                           uint64_t decoded_data_size,\n                           std::vector<size_t>& limits) {\n  Object::Reset();\n  if (ABSL_PREDICT_FALSE(num_records > limits.max_size())) {\n    return Fail(absl::ResourceExhaustedError(\"Too many records\"));\n  }\n  if (ABSL_PREDICT_FALSE(decoded_data_size >\n                         std::numeric_limits<size_t>::max())) {\n    return Fail(absl::ResourceExhaustedError(\"Records too large\"));\n  }\n\n  uint8_t compression_type_byte;\n  if (ABSL_PREDICT_FALSE(!src->ReadByte(compression_type_byte))) {\n    return Fail(src->StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Reading compression type failed\")));\n  }\n  const CompressionType compression_type =\n      static_cast<CompressionType>(compression_type_byte);\n\n  uint64_t sizes_size;\n  if (ABSL_PREDICT_FALSE(!ReadVarint64(*src, sizes_size))) {\n    return Fail(src->StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Reading size of sizes failed\")));\n  }\n\n  chunk_encoding_internal::Decompressor<LimitingReader<>> sizes_decompressor(\n      riegeli::Maker(\n          src, LimitingReaderBase::Options().set_exact_length(sizes_size)),\n      compression_type,\n      chunk_encoding_internal::DecompressorOptions().set_recycling_pool_options(\n          recycling_pool_options_));\n  if (ABSL_PREDICT_FALSE(!sizes_decompressor.ok())) {\n    return Fail(sizes_decompressor.status());\n  }\n  limits.clear();\n  size_t limit = 0;\n  while (limits.size() != num_records) {\n    uint64_t size;\n    if (ABSL_PREDICT_FALSE(!ReadVarint64(sizes_decompressor.reader(), size))) {\n      return Fail(sizes_decompressor.reader().StatusOrAnnotate(\n          absl::InvalidArgumentError(\"Reading record size failed\")));\n    }\n    if (ABSL_PREDICT_FALSE(size > decoded_data_size - limit)) {\n      return Fail(\n          absl::InvalidArgumentError(\"Decoded data size larger than expected\"));\n    }\n    limit += IntCast<size_t>(size);\n    limits.push_back(limit);\n  }\n  if (ABSL_PREDICT_FALSE(!sizes_decompressor.VerifyEndAndClose())) {\n    return Fail(sizes_decompressor.status());\n  }\n  if (ABSL_PREDICT_FALSE(limit != decoded_data_size)) {\n    return Fail(\n        absl::InvalidArgumentError(\"Decoded data size smaller than expected\"));\n  }\n\n  values_decompressor_.Reset(\n      src, compression_type,\n      chunk_encoding_internal::DecompressorOptions().set_recycling_pool_options(\n          recycling_pool_options_));\n  if (ABSL_PREDICT_FALSE(!values_decompressor_.ok())) {\n    return Fail(values_decompressor_.status());\n  }\n  return true;\n}\n\nbool SimpleDecoder::VerifyEndAndClose() {\n  values_decompressor_.VerifyEnd();\n  return Close();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/chunk_encoding/simple_decoder.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_SIMPLE_DECODER_H_\n#define RIEGELI_CHUNK_ENCODING_SIMPLE_DECODER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/chunk_encoding/decompressor.h\"\n\nnamespace riegeli {\n\nclass SimpleDecoder : public Object {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // Options for a global `RecyclingPool` of decompression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // decompression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    Options& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    Options&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Creates a closed `SimpleDecoder`.\n  explicit SimpleDecoder(Options options = Options())\n      : Object(kClosed),\n        recycling_pool_options_(options.recycling_pool_options()),\n        values_decompressor_(kClosed) {}\n\n  SimpleDecoder(const SimpleDecoder&) = delete;\n  SimpleDecoder& operator=(const SimpleDecoder&) = delete;\n\n  // Resets the `SimpleDecoder` and parses the chunk.\n  //\n  // Makes concatenated record values available for reading from `reader()`.\n  // Sets `limits` to sorted record end positions.\n  //\n  // `*src` is not owned by this `SimpleDecoder` and must be kept alive but not\n  // accessed until closing the `SimpleDecoder`.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool Decode(Reader* src, uint64_t num_records, uint64_t decoded_data_size,\n              std::vector<size_t>& limits);\n\n  // Returns the `Reader` from which concatenated record values should be read.\n  //\n  // Precondition: `ok()`\n  Reader& reader() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Verifies that the concatenated record values end at the current position,\n  // failing the `SimpleDecoder` if not. Closes the `SimpleDecoder`.\n  //\n  // Return values:\n  //  * `true`  - success (concatenated messages end at the former current\n  //              position)\n  //  * `false` - failure (concatenated messages do not end at the former\n  //              current position or the `SimpleDecoder` was not OK before\n  //              closing)\n  bool VerifyEndAndClose();\n\n protected:\n  void Done() override;\n\n private:\n  RecyclingPoolOptions recycling_pool_options_;\n  chunk_encoding_internal::Decompressor<> values_decompressor_;\n};\n\n// Implementation details follow.\n\ninline Reader& SimpleDecoder::reader() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT_OK(*this) << \"Failed precondition of SimpleDecoder::reader()\";\n  return values_decompressor_.reader();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CHUNK_ENCODING_SIMPLE_DECODER_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/simple_encoder.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/chunk_encoding/simple_encoder.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/chunk_encoder.h\"\n#include \"riegeli/chunk_encoding/compressor.h\"\n#include \"riegeli/chunk_encoding/compressor_options.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/messages/serialize_message.h\"\n#include \"riegeli/varint/varint_writing.h\"\n\nnamespace riegeli {\n\nSimpleEncoder::SimpleEncoder(CompressorOptions compressor_options,\n                             TuningOptions tuning_options)\n    : compression_type_(compressor_options.compression_type()),\n      sizes_compressor_(compressor_options,\n                        chunk_encoding_internal::Compressor::TuningOptions()\n                            .set_recycling_pool_options(\n                                tuning_options.recycling_pool_options())),\n      values_compressor_(compressor_options,\n                         chunk_encoding_internal::Compressor::TuningOptions()\n                             .set_size_hint(tuning_options.size_hint())\n                             .set_recycling_pool_options(\n                                 tuning_options.recycling_pool_options())) {}\n\nvoid SimpleEncoder::Clear() {\n  ChunkEncoder::Clear();\n  sizes_compressor_.Clear();\n  values_compressor_.Clear();\n}\n\nbool SimpleEncoder::AddRecord(const google::protobuf::MessageLite& record,\n                              SerializeMessageOptions serialize_options) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const size_t size = serialize_options.GetByteSize(record);\n  if (ABSL_PREDICT_FALSE(num_records_ == kMaxNumRecords)) {\n    return Fail(absl::ResourceExhaustedError(\"Too many records\"));\n  }\n  if (ABSL_PREDICT_FALSE(size > std::numeric_limits<uint64_t>::max() -\n                                    decoded_data_size_)) {\n    return Fail(absl::ResourceExhaustedError(\"Decoded data size too large\"));\n  }\n  ++num_records_;\n  decoded_data_size_ += IntCast<uint64_t>(size);\n  if (ABSL_PREDICT_FALSE(!WriteVarint64(IntCast<uint64_t>(size),\n                                        sizes_compressor_.writer()))) {\n    return Fail(sizes_compressor_.writer().status());\n  }\n  if (absl::Status status = SerializeMessage(\n          record, values_compressor_.writer(), serialize_options);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return Fail(std::move(status));\n  }\n  return true;\n}\n\nbool SimpleEncoder::AddRecord(BytesRef record) { return AddRecordImpl(record); }\n\nbool SimpleEncoder::AddRecord(ExternalRef record) {\n  return AddRecordImpl(std::move(record));\n}\n\nbool SimpleEncoder::AddRecord(const Chain& record) {\n  return AddRecordImpl(record);\n}\n\nbool SimpleEncoder::AddRecord(Chain&& record) {\n  return AddRecordImpl(std::move(record));\n}\n\nbool SimpleEncoder::AddRecord(const absl::Cord& record) {\n  return AddRecordImpl(record);\n}\n\nbool SimpleEncoder::AddRecord(absl::Cord&& record) {\n  return AddRecordImpl(std::move(record));\n}\n\ntemplate <typename Record>\nbool SimpleEncoder::AddRecordImpl(Record&& record) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(num_records_ == kMaxNumRecords)) {\n    return Fail(absl::ResourceExhaustedError(\"Too many records\"));\n  }\n  if (ABSL_PREDICT_FALSE(record.size() > std::numeric_limits<uint64_t>::max() -\n                                             decoded_data_size_)) {\n    return Fail(absl::ResourceExhaustedError(\"Decoded data size too large\"));\n  }\n  ++num_records_;\n  decoded_data_size_ += IntCast<uint64_t>(record.size());\n  if (ABSL_PREDICT_FALSE(!WriteVarint64(IntCast<uint64_t>(record.size()),\n                                        sizes_compressor_.writer()))) {\n    return Fail(sizes_compressor_.writer().status());\n  }\n  if (ABSL_PREDICT_FALSE(\n          !values_compressor_.writer().Write(std::forward<Record>(record)))) {\n    return Fail(values_compressor_.writer().status());\n  }\n  return true;\n}\n\nbool SimpleEncoder::AddRecords(Chain records, std::vector<size_t> limits) {\n  RIEGELI_ASSERT_EQ(limits.empty() ? size_t{0} : limits.back(), records.size())\n      << \"Failed precondition of ChunkEncoder::AddRecords(): \"\n         \"record end positions do not match concatenated record values\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(limits.size() > kMaxNumRecords - num_records_)) {\n    return Fail(absl::ResourceExhaustedError(\"Too many records\"));\n  }\n  if (ABSL_PREDICT_FALSE(records.size() > std::numeric_limits<uint64_t>::max() -\n                                              decoded_data_size_)) {\n    return Fail(absl::ResourceExhaustedError(\"Decoded data size too large\"));\n  }\n  num_records_ += IntCast<uint64_t>(limits.size());\n  decoded_data_size_ += IntCast<uint64_t>(records.size());\n  size_t start = 0;\n  for (const size_t limit : limits) {\n    RIEGELI_ASSERT_GE(limit, start)\n        << \"Failed precondition of ChunkEncoder::AddRecords(): \"\n           \"record end positions not sorted\";\n    RIEGELI_ASSERT_LE(limit, records.size())\n        << \"Failed precondition of ChunkEncoder::AddRecords(): \"\n           \"record end positions do not match concatenated record values\";\n    if (ABSL_PREDICT_FALSE(!WriteVarint64(IntCast<uint64_t>(limit - start),\n                                          sizes_compressor_.writer()))) {\n      return Fail(sizes_compressor_.writer().status());\n    }\n    start = limit;\n  }\n  if (ABSL_PREDICT_FALSE(\n          !values_compressor_.writer().Write(std::move(records)))) {\n    return Fail(values_compressor_.writer().status());\n  }\n  return true;\n}\n\nbool SimpleEncoder::EncodeAndClose(Writer& dest, ChunkType& chunk_type,\n                                   uint64_t& num_records,\n                                   uint64_t& decoded_data_size) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  chunk_type = ChunkType::kSimple;\n  num_records = num_records_;\n  decoded_data_size = decoded_data_size_;\n\n  if (ABSL_PREDICT_FALSE(\n          !dest.WriteByte(static_cast<uint8_t>(compression_type_)))) {\n    return Fail(dest.status());\n  }\n\n  if (ABSL_PREDICT_FALSE(\n          !sizes_compressor_.LengthPrefixedEncodeAndClose(dest))) {\n    return Fail(sizes_compressor_.status());\n  }\n\n  if (ABSL_PREDICT_FALSE(!values_compressor_.EncodeAndClose(dest))) {\n    return Fail(values_compressor_.status());\n  }\n  return Close();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/chunk_encoding/simple_encoder.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_SIMPLE_ENCODER_H_\n#define RIEGELI_CHUNK_ENCODING_SIMPLE_ENCODER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <optional>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/chunk_encoder.h\"\n#include \"riegeli/chunk_encoding/compressor.h\"\n#include \"riegeli/chunk_encoding/compressor_options.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/messages/serialize_message.h\"\n\nnamespace riegeli {\n\n// Format:\n//  - Compression type\n//  - Size of record sizes (compressed if applicable)\n//  - Record sizes (possibly compressed):\n//    - Array of `num_records` varints: sizes of records\n//  - Record values (possibly compressed):\n//    - Concatenated record data (bytes)\n//\n// If compression is used, a compressed block is prefixed by its varint-encoded\n// uncompressed size.\nclass SimpleEncoder : public ChunkEncoder {\n public:\n  class TuningOptions {\n   public:\n    TuningOptions() noexcept {}\n\n    // Expected uncompressed size of concatenated values, or `std::nullopt` if\n    // unknown. This may improve compression density and performance.\n    //\n    // If the size hint turns out to not match reality, nothing breaks.\n    //\n    // Default: `std::nullopt`.\n    TuningOptions& set_size_hint(std::optional<Position> size_hint) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      size_hint_ = size_hint;\n      return *this;\n    }\n    TuningOptions&& set_size_hint(std::optional<Position> size_hint) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_size_hint(size_hint));\n    }\n    std::optional<Position> size_hint() const { return size_hint_; }\n\n    // Options for a global `RecyclingPool` of compression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // compression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    TuningOptions& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    TuningOptions&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    std::optional<Position> size_hint_;\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Creates an empty `SimpleEncoder`.\n  explicit SimpleEncoder(CompressorOptions compressor_options,\n                         TuningOptions tuning_options = TuningOptions());\n\n  void Clear() override;\n\n  using ChunkEncoder::AddRecord;\n  bool AddRecord(const google::protobuf::MessageLite& record,\n                 SerializeMessageOptions serialize_options) override;\n  bool AddRecord(BytesRef record) override;\n  bool AddRecord(ExternalRef record) override;\n  bool AddRecord(const Chain& record) override;\n  bool AddRecord(Chain&& record) override;\n  bool AddRecord(const absl::Cord& record) override;\n  bool AddRecord(absl::Cord&& record) override;\n\n  bool AddRecords(Chain records, std::vector<size_t> limits) override;\n\n  bool EncodeAndClose(Writer& dest, ChunkType& chunk_type,\n                      uint64_t& num_records,\n                      uint64_t& decoded_data_size) override;\n\n private:\n  // This template is defined and used only in simple_encoder.cc.\n  template <typename Record>\n  bool AddRecordImpl(Record&& record);\n\n  CompressionType compression_type_;\n  chunk_encoding_internal::Compressor sizes_compressor_;\n  chunk_encoding_internal::Compressor values_compressor_;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CHUNK_ENCODING_SIMPLE_ENCODER_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/transpose_decoder.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/chunk_encoding/transpose_decoder.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <algorithm>\n#include <cstring>\n#include <limits>\n#include <memory>\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/container/fixed_array.h\"\n#include \"absl/container/flat_hash_map.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/global.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/limiting_backward_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/string_reader.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/chunk_encoding/decompressor.h\"\n#include \"riegeli/chunk_encoding/field_projection.h\"\n#include \"riegeli/chunk_encoding/transpose_internal.h\"\n#include \"riegeli/messages/message_wire_format.h\"\n#include \"riegeli/varint/varint_reading.h\"\n#include \"riegeli/varint/varint_writing.h\"\n\nnamespace riegeli {\nnamespace {\n\nReader* kEmptyReader() {\n  Reader* const reader = &Global([] { return riegeli::StringReader<>(); });\n  RIEGELI_ASSERT_OK(*reader) << \"kEmptyReader() has been closed\";\n  return reader;\n}\n\nconstexpr uint32_t kInvalidPos = std::numeric_limits<uint32_t>::max();\n\n// Information about one data bucket used in projection.\nstruct DataBucket {\n  // Raw bucket data, valid if not all buffers are already decompressed,\n  // otherwise empty.\n  Chain compressed_data;\n  // Sizes of data buffers in the bucket, valid if not all buffers are already\n  // decompressed, otherwise empty.\n  std::vector<size_t> buffer_sizes;\n  // Decompressor for the remaining data, valid if some but not all buffers are\n  // already decompressed, otherwise closed.\n  chunk_encoding_internal::Decompressor<ChainReader<>> decompressor{kClosed};\n  // A prefix of decompressed data buffers, lazily extended.\n  std::vector<ChainReader<Chain>> buffers;\n};\n\n// Should the data content of the field be decoded?\nenum class FieldIncluded {\n  kYes,\n  kNo,\n  kExistenceOnly,\n};\n\n// Returns `true` if `tag` is a valid protocol buffer tag.\nbool ValidTag(uint32_t tag) {\n  switch (GetTagWireType(tag)) {\n    case WireType::kVarint:\n    case WireType::kFixed32:\n    case WireType::kFixed64:\n    case WireType::kLengthDelimited:\n    case WireType::kStartGroup:\n    case WireType::kEndGroup:\n      return tag >= 8;\n    case WireType::kInvalid6:\n    case WireType::kInvalid7:\n      return false;\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Impossible wire type: \" << static_cast<int>(GetTagWireType(tag));\n}\n\n// The types of callbacks in state machine states.\nenum class CallbackType : uint8_t {\n  kNoOp,\n  kMessageStart,\n  kSubmessageStart,\n  kSubmessageEnd,\n  kSelectCallback,\n  kSkippedSubmessageStart,\n  kSkippedSubmessageEnd,\n  kNonProto,\n  kFailure,\n\n// `kCopyTag_*` has to be the first `CallbackType` in `TYPES_FOR_TAG_LEN` for\n// `GetCopyTagCallbackType()` to work.\n#define TYPES_FOR_TAG_LEN(tag_length)                                         \\\n  kCopyTag_##tag_length, kVarint_1_##tag_length, kVarint_2_##tag_length,      \\\n      kVarint_3_##tag_length, kVarint_4_##tag_length, kVarint_5_##tag_length, \\\n      kVarint_6_##tag_length, kVarint_7_##tag_length, kVarint_8_##tag_length, \\\n      kVarint_9_##tag_length, kVarint_10_##tag_length, kFixed32_##tag_length, \\\n      kFixed64_##tag_length, kFixed32Existence_##tag_length,                  \\\n      kFixed64Existence_##tag_length, kString_##tag_length,                   \\\n      kStartProjectionGroup_##tag_length, kEndProjectionGroup_##tag_length\n\n  TYPES_FOR_TAG_LEN(1),\n  TYPES_FOR_TAG_LEN(2),\n  TYPES_FOR_TAG_LEN(3),\n  TYPES_FOR_TAG_LEN(4),\n  TYPES_FOR_TAG_LEN(5),\n#undef TYPES_FOR_TAG_LEN\n\n  // We need `kCopyTag_*` callback for length 6 as well because of inline\n  // numerics. `kCopyTag_6` has to be the first `CallbackType` after\n  // `TYPES_FOR_TAG_LEN` for `GetCopyTagCallbackType()` to work.\n  kCopyTag_6,\n  kUnknown,\n};\n\nstatic_assert(static_cast<uint8_t>(CallbackType::kUnknown) < (1 << 7),\n              \"CallbackType has too many cases to fit in 7 bits.\");\n\nconstexpr CallbackType operator+(CallbackType a, uint8_t b) {\n  return static_cast<CallbackType>(static_cast<uint8_t>(a) + b);\n}\n\nconstexpr uint8_t operator-(CallbackType a, CallbackType b) {\n  return static_cast<uint8_t>(a) - static_cast<uint8_t>(b);\n}\n\n// Node template that can be used to resolve the `CallbackType` of the node in\n// decoding phase.\nstruct StateMachineNodeTemplate {\n  // `bucket_index` and `buffer_within_bucket_index` identify the decoder\n  // to read data from.\n  uint32_t bucket_index;\n  uint32_t buffer_within_bucket_index;\n  // Proto tag of the node.\n  uint32_t tag;\n  // Tag subtype.\n  chunk_encoding_internal::Subtype subtype;\n  // Length of the varint encoded tag.\n  uint8_t tag_length;\n};\n\n// Returns copy tag callback type for `tag_length`.\ninline CallbackType GetCopyTagCallbackType(size_t tag_length) {\n  RIEGELI_ASSERT_GT(tag_length, 0u) << \"Zero tag length\";\n  RIEGELI_ASSERT_LE(tag_length, kMaxLengthVarint32 + 1)\n      << \"Tag length too large\";\n  return CallbackType::kCopyTag_1 +\n         (tag_length - 1) *\n             (CallbackType::kCopyTag_2 - CallbackType::kCopyTag_1);\n}\n\n// Returns varint callback type for `subtype` and `tag_length`.\ninline CallbackType GetVarintCallbackType(\n    chunk_encoding_internal::Subtype subtype, size_t tag_length) {\n  RIEGELI_ASSERT_GT(tag_length, 0u) << \"Zero tag length\";\n  RIEGELI_ASSERT_LE(tag_length, kMaxLengthVarint32) << \"Tag length too large\";\n  if (subtype > chunk_encoding_internal::Subtype::kVarintInlineMax)\n    return CallbackType::kUnknown;\n  if (subtype >= chunk_encoding_internal::Subtype::kVarintInline0) {\n    return GetCopyTagCallbackType(tag_length + 1);\n  }\n  return CallbackType::kVarint_1_1 +\n         (subtype - chunk_encoding_internal::Subtype::kVarint1) *\n             (CallbackType::kVarint_2_1 - CallbackType::kVarint_1_1) +\n         (tag_length - 1) *\n             (CallbackType::kVarint_1_2 - CallbackType::kVarint_1_1);\n}\n\n// Returns fixed32 callback type for `tag_length`.\ninline CallbackType GetFixed32CallbackType(size_t tag_length) {\n  RIEGELI_ASSERT_GT(tag_length, 0u) << \"Zero tag length\";\n  RIEGELI_ASSERT_LE(tag_length, kMaxLengthVarint32) << \"Tag length too large\";\n  return CallbackType::kFixed32_1 +\n         (tag_length - 1) *\n             (CallbackType::kFixed32_2 - CallbackType::kFixed32_1);\n}\n\n// Returns fixed64 callback type for `tag_length`.\ninline CallbackType GetFixed64CallbackType(size_t tag_length) {\n  RIEGELI_ASSERT_GT(tag_length, 0u) << \"Zero tag length\";\n  RIEGELI_ASSERT_LE(tag_length, kMaxLengthVarint32) << \"Tag length too large\";\n  return CallbackType::kFixed64_1 +\n         (tag_length - 1) *\n             (CallbackType::kFixed64_2 - CallbackType::kFixed64_1);\n}\n\n// Returns fixed32 existence callback type for `tag_length`.\ninline CallbackType GetFixed32ExistenceCallbackType(size_t tag_length) {\n  RIEGELI_ASSERT_GT(tag_length, 0u) << \"Zero tag length\";\n  RIEGELI_ASSERT_LE(tag_length, kMaxLengthVarint32) << \"Tag length too large\";\n  return CallbackType::kFixed32Existence_1 +\n         (tag_length - 1) * (CallbackType::kFixed32Existence_2 -\n                             CallbackType::kFixed32Existence_1);\n}\n\n// Returns fixed64 existence callback type for `tag_length`.\ninline CallbackType GetFixed64ExistenceCallbackType(size_t tag_length) {\n  RIEGELI_ASSERT_GT(tag_length, 0u) << \"Zero tag length\";\n  RIEGELI_ASSERT_LE(tag_length, kMaxLengthVarint32) << \"Tag length too large\";\n  return CallbackType::kFixed64Existence_1 +\n         (tag_length - 1) * (CallbackType::kFixed64Existence_2 -\n                             CallbackType::kFixed64Existence_1);\n}\n\n// Returns string callback type for `subtype` and `tag_length`.\ninline CallbackType GetStringCallbackType(\n    chunk_encoding_internal::Subtype subtype, size_t tag_length) {\n  RIEGELI_ASSERT_GT(tag_length, 0u) << \"Zero tag length\";\n  RIEGELI_ASSERT_LE(tag_length, kMaxLengthVarint32) << \"Tag length too large\";\n  switch (subtype) {\n    case chunk_encoding_internal::Subtype::kLengthDelimitedString:\n      return CallbackType::kString_1 +\n             (tag_length - 1) *\n                 (CallbackType::kString_2 - CallbackType::kString_1);\n    case chunk_encoding_internal::Subtype::kLengthDelimitedEndOfSubmessage:\n      return CallbackType::kSubmessageEnd;\n    default:\n      // Note: Nodes with `kLengthDelimitedStartOfSubmessage` are not created.\n      // Start of submessage is indicated with `MessageId::kStartOfSubmessage`\n      // and uses `CallbackType::kSubmessageStart`.\n      return CallbackType::kUnknown;\n  }\n}\n\n// Returns string callback type for `subtype` and `tag_length` to exclude field.\ninline CallbackType GetStringExcludeCallbackType(\n    chunk_encoding_internal::Subtype subtype, size_t tag_length) {\n  RIEGELI_ASSERT_GT(tag_length, 0u) << \"Zero tag length\";\n  RIEGELI_ASSERT_LE(tag_length, kMaxLengthVarint32) << \"Tag length too large\";\n  switch (subtype) {\n    case chunk_encoding_internal::Subtype::kLengthDelimitedString:\n      return CallbackType::kNoOp;\n    case chunk_encoding_internal::Subtype::kLengthDelimitedEndOfSubmessage:\n      return CallbackType::kSkippedSubmessageEnd;\n    default:\n      return CallbackType::kUnknown;\n  }\n}\n\n// Returns string existence callback type for `subtype` and `tag_length`.\ninline CallbackType GetStringExistenceCallbackType(\n    chunk_encoding_internal::Subtype subtype, size_t tag_length) {\n  RIEGELI_ASSERT_GT(tag_length, 0u) << \"Zero tag length\";\n  RIEGELI_ASSERT_LE(tag_length, kMaxLengthVarint32) << \"Tag length too large\";\n  switch (subtype) {\n    case chunk_encoding_internal::Subtype::kLengthDelimitedString:\n      // We use the fact that there is a zero stored in `tag_data`. This decodes\n      // as an empty string in proto decoder.\n      return GetCopyTagCallbackType(tag_length + 1);\n    case chunk_encoding_internal::Subtype::kLengthDelimitedEndOfSubmessage:\n      return CallbackType::kSubmessageEnd;\n    default:\n      return CallbackType::kUnknown;\n  }\n}\n\ninline CallbackType GetStartProjectionGroupCallbackType(size_t tag_length) {\n  RIEGELI_ASSERT_GT(tag_length, 0u) << \"Zero tag length\";\n  RIEGELI_ASSERT_LE(tag_length, kMaxLengthVarint32) << \"Tag length too large\";\n  return CallbackType::kStartProjectionGroup_1 +\n         (tag_length - 1) * (CallbackType::kStartProjectionGroup_2 -\n                             CallbackType::kStartProjectionGroup_1);\n}\n\ninline CallbackType GetEndProjectionGroupCallbackType(size_t tag_length) {\n  RIEGELI_ASSERT_GT(tag_length, 0u) << \"Zero tag length\";\n  RIEGELI_ASSERT_LE(tag_length, kMaxLengthVarint32) << \"Tag length too large\";\n  return CallbackType::kEndProjectionGroup_1 +\n         (tag_length - 1) * (CallbackType::kEndProjectionGroup_2 -\n                             CallbackType::kEndProjectionGroup_1);\n}\n\n// Get callback for node.\ninline CallbackType GetCallbackType(FieldIncluded field_included, uint32_t tag,\n                                    chunk_encoding_internal::Subtype subtype,\n                                    size_t tag_length,\n                                    bool projection_enabled) {\n  RIEGELI_ASSERT_GT(tag_length, 0u) << \"Zero tag length\";\n  RIEGELI_ASSERT_LE(tag_length, kMaxLengthVarint32) << \"Tag length too large\";\n  switch (field_included) {\n    case FieldIncluded::kYes:\n      switch (GetTagWireType(tag)) {\n        case WireType::kVarint:\n          return GetVarintCallbackType(subtype, tag_length);\n        case WireType::kFixed32:\n          return GetFixed32CallbackType(tag_length);\n        case WireType::kFixed64:\n          return GetFixed64CallbackType(tag_length);\n        case WireType::kLengthDelimited:\n          return GetStringCallbackType(subtype, tag_length);\n        case WireType::kStartGroup:\n          return projection_enabled\n                     ? GetStartProjectionGroupCallbackType(tag_length)\n                     : GetCopyTagCallbackType(tag_length);\n        case WireType::kEndGroup:\n          return projection_enabled\n                     ? GetEndProjectionGroupCallbackType(tag_length)\n                     : GetCopyTagCallbackType(tag_length);\n        default:\n          return CallbackType::kUnknown;\n      }\n    case FieldIncluded::kNo:\n      switch (GetTagWireType(tag)) {\n        case WireType::kVarint:\n        case WireType::kFixed32:\n        case WireType::kFixed64:\n          return CallbackType::kNoOp;\n        case WireType::kLengthDelimited:\n          return GetStringExcludeCallbackType(subtype, tag_length);\n        case WireType::kStartGroup:\n          return CallbackType::kSkippedSubmessageStart;\n        case WireType::kEndGroup:\n          return CallbackType::kSkippedSubmessageEnd;\n        default:\n          return CallbackType::kUnknown;\n      }\n    case FieldIncluded::kExistenceOnly:\n      switch (GetTagWireType(tag)) {\n        case WireType::kVarint:\n          return GetCopyTagCallbackType(tag_length + 1);\n        case WireType::kFixed32:\n          return GetFixed32ExistenceCallbackType(tag_length);\n        case WireType::kFixed64:\n          return GetFixed64ExistenceCallbackType(tag_length);\n        case WireType::kLengthDelimited:\n          return GetStringExistenceCallbackType(subtype, tag_length);\n        case WireType::kStartGroup:\n          return GetStartProjectionGroupCallbackType(tag_length);\n        case WireType::kEndGroup:\n          return GetEndProjectionGroupCallbackType(tag_length);\n        default:\n          return CallbackType::kUnknown;\n      }\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown FieldIncluded: \" << static_cast<int>(field_included);\n}\n\n}  // namespace\n\n// Node of the state machine read from input.\nstruct TransposeDecoder::StateMachineNode {\n  // Tag for the field decoded by this node, may benefit from being aligned.\n  //\n  // `tag_data` contains varint encoded tag (1 to 5 bytes) followed by inline\n  // numeric (if any) or zero otherwise.\n  char tag_data[kMaxLengthVarint32 + 1];\n  // Size of the tag data. Must be in the range [1..5].\n  uint8_t tag_data_size : 7;\n  // Whether the callback is implicit.\n  bool is_implicit : 1;\n  CallbackType callback_type;\n  union {\n    // Buffer to read data from.\n    Reader* buffer;\n    // In projection mode, the node is updated in decoding phase based on the\n    // current submessage stack and this template.\n    StateMachineNodeTemplate* node_template;\n  };\n  // Node to move to after finishing the callback for this node.\n  StateMachineNode* next_node;\n};\n\nstruct TransposeDecoder::Context {\n  static_assert(sizeof(StateMachineNode) == 8 + 2 * sizeof(void*),\n                \"Unexpected padding in StateMachineNode.\");\n  // Compression type of the input.\n  CompressionType compression_type = CompressionType::kNone;\n  // Buffer containing all the data.\n  // Note: Used only when projection is disabled.\n  std::vector<ChainReader<Chain>> buffers;\n  // Buffer for lengths of nonproto messages.\n  Reader* nonproto_lengths = nullptr;\n  // State machine read from the input.\n  std::vector<StateMachineNode> state_machine_nodes;\n  // Node to start decoding from.\n  uint32_t first_node = 0;\n  // State machine transitions. One byte = one transition.\n  chunk_encoding_internal::Decompressor<> transitions{kClosed};\n\n  enum class IncludeType : uint8_t {\n    // Field is included.\n    kIncludeFully,\n    // Some child fields are included.\n    kIncludeChild,\n    // Field is existence only.\n    kExistenceOnly,\n  };\n\n  // --- Fields used in projection. ---\n  // Holds information about included field.\n  struct IncludedField {\n    // IDs are sequentially assigned to fields from FieldProjection.\n    uint32_t field_id;\n    IncludeType include_type;\n  };\n  // Fields form a tree structure stored in `include_fields` map. If `p` is\n  // the ID of parent submessage then `include_fields[std::make_pair(p, f)]`\n  // holds the include information of the child with field number `f`. The root\n  // ID is assumed to be `kInvalidPos` and the root `IncludeType` is assumed to\n  // be `kIncludeChild`.\n  absl::flat_hash_map<std::pair<uint32_t, int>, IncludedField> include_fields;\n  // Data buckets.\n  std::vector<DataBucket> buckets;\n  // Template that can later be used later to finalize `StateMachineNode`.\n  std::vector<StateMachineNodeTemplate> node_templates;\n};\n\nbool TransposeDecoder::Decode(uint64_t num_records, uint64_t decoded_data_size,\n                              const FieldProjection& field_projection,\n                              Reader& src, BackwardWriter& dest,\n                              std::vector<size_t>& limits) {\n  RIEGELI_ASSERT_EQ(dest.pos(), 0u)\n      << \"Failed precondition of TransposeDecoder::Reset(): \"\n         \"non-zero destination position\";\n  Object::Reset();\n  if (ABSL_PREDICT_FALSE(num_records > limits.max_size())) {\n    return Fail(absl::ResourceExhaustedError(\"Too many records\"));\n  }\n  if (ABSL_PREDICT_FALSE(decoded_data_size >\n                         std::numeric_limits<size_t>::max())) {\n    return Fail(absl::ResourceExhaustedError(\"Records too large\"));\n  }\n\n  Context context;\n  if (ABSL_PREDICT_FALSE(!Parse(context, src, field_projection))) return false;\n  LimitingBackwardWriter<> limiting_dest(\n      &dest, LimitingBackwardWriterBase::Options()\n                 .set_max_length(decoded_data_size)\n                 .set_exact(field_projection.includes_all()));\n  if (ABSL_PREDICT_FALSE(\n          !Decode(context, num_records, limiting_dest, limits))) {\n    limiting_dest.Close();\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(!limiting_dest.Close())) {\n    return Fail(limiting_dest.status());\n  }\n  return true;\n}\n\ninline bool TransposeDecoder::Parse(Context& context, Reader& src,\n                                    const FieldProjection& field_projection) {\n  bool projection_enabled = true;\n  for (const Field& include_field : field_projection.fields()) {\n    if (include_field.path().empty()) {\n      projection_enabled = false;\n      break;\n    }\n    size_t path_len = include_field.path().size();\n    bool existence_only =\n        include_field.path()[path_len - 1] == Field::kExistenceOnly;\n    if (existence_only) {\n      --path_len;\n      if (path_len == 0) continue;\n    }\n    uint32_t current_id = kInvalidPos;\n    for (size_t i = 0; i < path_len; ++i) {\n      const int field_number = include_field.path()[i];\n      if (field_number == Field::kExistenceOnly) return false;\n      uint32_t next_id = context.include_fields.size();\n      Context::IncludeType include_type = Context::IncludeType::kIncludeChild;\n      if (i + 1 == path_len) {\n        include_type = existence_only ? Context::IncludeType::kExistenceOnly\n                                      : Context::IncludeType::kIncludeFully;\n      }\n      Context::IncludedField& val =\n          context.include_fields\n              .emplace(std::make_pair(current_id, field_number),\n                       Context::IncludedField{next_id, include_type})\n              .first->second;\n      current_id = val.field_id;\n      static_assert(Context::IncludeType::kExistenceOnly >\n                            Context::IncludeType::kIncludeChild &&\n                        Context::IncludeType::kIncludeChild >\n                            Context::IncludeType::kIncludeFully,\n                    \"Statement below assumes this ordering\");\n      val.include_type = std::min(val.include_type, include_type);\n    }\n  }\n\n  uint8_t compression_type_byte;\n  if (ABSL_PREDICT_FALSE(!src.ReadByte(compression_type_byte))) {\n    return Fail(src.StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Reading compression type failed\")));\n  }\n  context.compression_type =\n      static_cast<CompressionType>(compression_type_byte);\n\n  uint64_t header_size;\n  Chain header;\n  if (ABSL_PREDICT_FALSE(!ReadVarint64(src, header_size) ||\n                         !src.Read(header_size, header))) {\n    return Fail(src.StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Reading header failed\")));\n  }\n  chunk_encoding_internal::Decompressor<ChainReader<>> header_decompressor(\n      riegeli::Maker(&header), context.compression_type,\n      chunk_encoding_internal::DecompressorOptions().set_recycling_pool_options(\n          recycling_pool_options_));\n  if (ABSL_PREDICT_FALSE(!header_decompressor.ok())) {\n    return Fail(header_decompressor.status());\n  }\n\n  uint32_t num_buffers;\n  std::optional<absl::FixedArray<uint32_t>> first_buffer_indices;\n  std::optional<absl::FixedArray<uint32_t>> bucket_indices;\n  if (projection_enabled) {\n    if (ABSL_PREDICT_FALSE(!ParseBuffersForFiltering(\n            context, header_decompressor.reader(), src, first_buffer_indices,\n            bucket_indices))) {\n      return false;\n    }\n    num_buffers = IntCast<uint32_t>(bucket_indices->size());\n  } else {\n    if (ABSL_PREDICT_FALSE(\n            !ParseBuffers(context, header_decompressor.reader(), src))) {\n      return false;\n    }\n    num_buffers = IntCast<uint32_t>(context.buffers.size());\n  }\n\n  uint32_t state_machine_size;\n  if (ABSL_PREDICT_FALSE(\n          !ReadVarint32(header_decompressor.reader(), state_machine_size))) {\n    return Fail(header_decompressor.reader().StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Reading state machine size failed\")));\n  }\n  // Additional `0xff` nodes to correctly handle invalid/malicious inputs.\n  context.state_machine_nodes.resize(size_t{state_machine_size} + 0xff);\n  if (projection_enabled) {\n    context.node_templates.resize(size_t{state_machine_size});\n  }\n  std::vector<StateMachineNode>& state_machine_nodes =\n      context.state_machine_nodes;\n  bool has_nonproto_op = false;\n  size_t num_subtypes = 0;\n  absl::FixedArray<uint32_t> tags(size_t{state_machine_size});\n  for (size_t i = 0; i < state_machine_size; ++i) {\n    uint32_t tag;\n    if (ABSL_PREDICT_FALSE(!ReadVarint32(header_decompressor.reader(), tag))) {\n      return Fail(header_decompressor.reader().StatusOrAnnotate(\n          absl::InvalidArgumentError(\"Reading field tag failed\")));\n    }\n    tags[i] = tag;\n    if (ValidTag(tag) && chunk_encoding_internal::HasSubtype(tag)) {\n      ++num_subtypes;\n    }\n  }\n  absl::FixedArray<uint32_t> next_node_indices(size_t{state_machine_size});\n  for (size_t i = 0; i < state_machine_size; ++i) {\n    uint32_t next_node;\n    if (ABSL_PREDICT_FALSE(\n            !ReadVarint32(header_decompressor.reader(), next_node))) {\n      return Fail(header_decompressor.reader().StatusOrAnnotate(\n          absl::InvalidArgumentError(\"Reading next node index failed\")));\n    }\n    next_node_indices[i] = next_node;\n  }\n  std::string subtypes;\n  if (ABSL_PREDICT_FALSE(\n          !header_decompressor.reader().Read(num_subtypes, subtypes))) {\n    return Fail(header_decompressor.reader().StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Reading subtypes failed\")));\n  }\n  size_t subtype_index = 0;\n  for (size_t i = 0; i < state_machine_size; ++i) {\n    uint32_t tag = tags[i];\n    StateMachineNode& state_machine_node = state_machine_nodes[i];\n    state_machine_node.buffer = nullptr;\n    switch (static_cast<chunk_encoding_internal::MessageId>(tag)) {\n      case chunk_encoding_internal::MessageId::kNoOp:\n        state_machine_node.callback_type = CallbackType::kNoOp;\n        break;\n      case chunk_encoding_internal::MessageId::kNonProto: {\n        state_machine_node.callback_type = CallbackType::kNonProto;\n        uint32_t buffer_index;\n        if (ABSL_PREDICT_FALSE(\n                !ReadVarint32(header_decompressor.reader(), buffer_index))) {\n          return Fail(header_decompressor.reader().StatusOrAnnotate(\n              absl::InvalidArgumentError(\"Reading buffer index failed\")));\n        }\n        if (ABSL_PREDICT_FALSE(buffer_index >= num_buffers)) {\n          return Fail(absl::InvalidArgumentError(\"Buffer index too large\"));\n        }\n        if (projection_enabled) {\n          const uint32_t bucket = (*bucket_indices)[buffer_index];\n          state_machine_node.buffer = GetBuffer(\n              context, bucket, buffer_index - (*first_buffer_indices)[bucket]);\n          if (ABSL_PREDICT_FALSE(state_machine_node.buffer == nullptr)) {\n            return false;\n          }\n        } else {\n          state_machine_node.buffer = &context.buffers[buffer_index];\n        }\n        has_nonproto_op = true;\n      } break;\n      case chunk_encoding_internal::MessageId::kStartOfMessage:\n        state_machine_node.callback_type = CallbackType::kMessageStart;\n        break;\n      case chunk_encoding_internal::MessageId::kStartOfSubmessage:\n        if (projection_enabled) {\n          context.node_templates[i].tag = static_cast<uint32_t>(\n              chunk_encoding_internal::MessageId::kStartOfSubmessage);\n          state_machine_node.node_template = &context.node_templates[i];\n          state_machine_node.callback_type = CallbackType::kSelectCallback;\n        } else {\n          state_machine_node.callback_type = CallbackType::kSubmessageStart;\n        }\n        break;\n      default: {\n        chunk_encoding_internal::Subtype subtype =\n            chunk_encoding_internal::Subtype::kTrivial;\n        static_assert(\n            chunk_encoding_internal::Subtype::kLengthDelimitedString ==\n                chunk_encoding_internal::Subtype::kTrivial,\n            \"chunk_encoding_internal::Subtypes kLengthDelimitedString and \"\n            \"kTrivial must be equal\");\n        // End of submessage is encoded as `kSubmessageWireType`.\n        if (GetTagWireType(tag) ==\n            chunk_encoding_internal::kSubmessageWireType) {\n          tag -= static_cast<uint32_t>(\n                     chunk_encoding_internal::kSubmessageWireType) -\n                 static_cast<uint32_t>(WireType::kLengthDelimited);\n          subtype =\n              chunk_encoding_internal::Subtype::kLengthDelimitedEndOfSubmessage;\n        }\n        if (ABSL_PREDICT_FALSE(!ValidTag(tag))) {\n          return Fail(absl::InvalidArgumentError(\"Invalid tag\"));\n        }\n        char* const tag_end = WriteVarint32(tag, state_machine_node.tag_data);\n        const size_t tag_length =\n            PtrDistance(state_machine_node.tag_data, tag_end);\n        if (chunk_encoding_internal::HasSubtype(tag)) {\n          subtype = static_cast<chunk_encoding_internal::Subtype>(\n              subtypes[subtype_index++]);\n        }\n        if (projection_enabled) {\n          if (chunk_encoding_internal::HasDataBuffer(tag, subtype)) {\n            uint32_t buffer_index;\n            if (ABSL_PREDICT_FALSE(!ReadVarint32(header_decompressor.reader(),\n                                                 buffer_index))) {\n              return Fail(header_decompressor.reader().StatusOrAnnotate(\n                  absl::InvalidArgumentError(\"Reading buffer index failed\")));\n            }\n            if (ABSL_PREDICT_FALSE(buffer_index >= num_buffers)) {\n              return Fail(absl::InvalidArgumentError(\"Buffer index too large\"));\n            }\n            const uint32_t bucket = (*bucket_indices)[buffer_index];\n            context.node_templates[i].bucket_index = bucket;\n            context.node_templates[i].buffer_within_bucket_index =\n                buffer_index - (*first_buffer_indices)[bucket];\n          } else {\n            context.node_templates[i].bucket_index = kInvalidPos;\n          }\n          context.node_templates[i].tag = tag;\n          context.node_templates[i].subtype = subtype;\n          context.node_templates[i].tag_length = IntCast<uint8_t>(tag_length);\n          state_machine_node.node_template = &context.node_templates[i];\n          state_machine_node.callback_type = CallbackType::kSelectCallback;\n        } else {\n          if (chunk_encoding_internal::HasDataBuffer(tag, subtype)) {\n            uint32_t buffer_index;\n            if (ABSL_PREDICT_FALSE(!ReadVarint32(header_decompressor.reader(),\n                                                 buffer_index))) {\n              return Fail(header_decompressor.reader().StatusOrAnnotate(\n                  absl::InvalidArgumentError(\"Reading buffer index failed\")));\n            }\n            if (ABSL_PREDICT_FALSE(buffer_index >= num_buffers)) {\n              return Fail(absl::InvalidArgumentError(\"Buffer index too large\"));\n            }\n            state_machine_node.buffer = &context.buffers[buffer_index];\n          }\n          state_machine_node.callback_type =\n              GetCallbackType(FieldIncluded::kYes, tag, subtype, tag_length,\n                              projection_enabled);\n          if (ABSL_PREDICT_FALSE(state_machine_node.callback_type ==\n                                 CallbackType::kUnknown)) {\n            return Fail(absl::InvalidArgumentError(\"Invalid node\"));\n          }\n        }\n        // Store subtype right past tag in case this is inline numeric.\n        if (GetTagWireType(tag) == WireType::kVarint &&\n            subtype >= chunk_encoding_internal::Subtype::kVarintInline0) {\n          state_machine_node.tag_data[tag_length] =\n              subtype - chunk_encoding_internal::Subtype::kVarintInline0;\n        } else {\n          state_machine_node.tag_data[tag_length] = 0;\n        }\n        state_machine_node.tag_data_size = IntCast<uint8_t>(tag_length);\n      }\n    }\n    uint32_t next_node_id = next_node_indices[i];\n    if (next_node_id >= state_machine_size) {\n      // Callback is implicit.\n      next_node_id -= state_machine_size;\n      state_machine_node.is_implicit = true;\n    } else {\n      state_machine_node.is_implicit = false;\n    }\n    if (ABSL_PREDICT_FALSE(next_node_id >= state_machine_size)) {\n      return Fail(absl::InvalidArgumentError(\"Node index too large\"));\n    }\n\n    state_machine_node.next_node = &state_machine_nodes[next_node_id];\n  }\n\n  if (has_nonproto_op) {\n    // If non-proto state exists then the last buffer is the\n    // `nonproto_lengths` buffer.\n    if (ABSL_PREDICT_FALSE(num_buffers == 0)) {\n      return Fail(\n          absl::InvalidArgumentError(\"Missing buffer for non-proto records\"));\n    }\n    if (projection_enabled) {\n      const uint32_t bucket = (*bucket_indices)[num_buffers - 1];\n      context.nonproto_lengths = GetBuffer(\n          context, bucket, num_buffers - 1 - (*first_buffer_indices)[bucket]);\n      if (ABSL_PREDICT_FALSE(context.nonproto_lengths == nullptr)) return false;\n    } else {\n      context.nonproto_lengths = &context.buffers.back();\n    }\n  }\n\n  if (ABSL_PREDICT_FALSE(\n          !ReadVarint32(header_decompressor.reader(), context.first_node))) {\n    return Fail(header_decompressor.reader().StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Reading first node index failed\")));\n  }\n  if (ABSL_PREDICT_FALSE(context.first_node >= state_machine_size)) {\n    return Fail(absl::InvalidArgumentError(\"First node index too large\"));\n  }\n\n  // Add `0xff` failure nodes so we never overflow this array.\n  for (size_t i = size_t{state_machine_size};\n       i < size_t{state_machine_size} + 0xff; ++i) {\n    state_machine_nodes[i].callback_type = CallbackType::kFailure;\n  }\n\n  if (ABSL_PREDICT_FALSE(ContainsImplicitLoop(&state_machine_nodes))) {\n    return Fail(absl::InvalidArgumentError(\"Nodes contain an implicit loop\"));\n  }\n\n  if (ABSL_PREDICT_FALSE(!header_decompressor.VerifyEndAndClose())) {\n    return Fail(header_decompressor.status());\n  }\n  context.transitions.Reset(\n      &src, context.compression_type,\n      chunk_encoding_internal::DecompressorOptions().set_recycling_pool_options(\n          recycling_pool_options_));\n  if (ABSL_PREDICT_FALSE(!context.transitions.ok())) {\n    return Fail(context.transitions.status());\n  }\n  return true;\n}\n\ninline bool TransposeDecoder::ParseBuffers(Context& context,\n                                           Reader& header_reader, Reader& src) {\n  uint32_t num_buckets;\n  if (ABSL_PREDICT_FALSE(!ReadVarint32(header_reader, num_buckets))) {\n    return Fail(header_reader.StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Reading number of buckets failed\")));\n  }\n  uint32_t num_buffers;\n  if (ABSL_PREDICT_FALSE(!ReadVarint32(header_reader, num_buffers))) {\n    return Fail(header_reader.StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Reading number of buffers failed\")));\n  }\n  if (ABSL_PREDICT_FALSE(num_buffers > context.buffers.max_size())) {\n    return Fail(absl::InvalidArgumentError(\"Too many buffers\"));\n  }\n  if (num_buckets == 0) {\n    if (ABSL_PREDICT_FALSE(num_buffers != 0)) {\n      return Fail(absl::InvalidArgumentError(\"Too few buckets\"));\n    }\n    return true;\n  }\n  context.buffers.reserve(num_buffers);\n  std::vector<chunk_encoding_internal::Decompressor<ChainReader<Chain>>>\n      bucket_decompressors;\n  // Explicitly convert `num_buckets` to `size_t` to avoid a warning\n  // `[-Wtautological-constant-out-of-range-compare]` if `max_size()` is\n  // constexpr.\n  if (ABSL_PREDICT_FALSE(size_t{num_buckets} >\n                         bucket_decompressors.max_size())) {\n    return Fail(absl::ResourceExhaustedError(\"Too many buckets\"));\n  }\n  bucket_decompressors.reserve(num_buckets);\n  for (uint32_t bucket_index = 0; bucket_index < num_buckets; ++bucket_index) {\n    uint64_t bucket_length;\n    if (ABSL_PREDICT_FALSE(!ReadVarint64(header_reader, bucket_length))) {\n      return Fail(header_reader.StatusOrAnnotate(\n          absl::InvalidArgumentError(\"Reading bucket length failed\")));\n    }\n    if (ABSL_PREDICT_FALSE(bucket_length >\n                           std::numeric_limits<size_t>::max())) {\n      return Fail(absl::ResourceExhaustedError(\"Bucket too large\"));\n    }\n    Chain bucket;\n    if (ABSL_PREDICT_FALSE(!src.Read(IntCast<size_t>(bucket_length), bucket))) {\n      return Fail(src.StatusOrAnnotate(\n          absl::InvalidArgumentError(\"Reading bucket failed\")));\n    }\n    bucket_decompressors.emplace_back(\n        riegeli::Maker(std::move(bucket)), context.compression_type,\n        chunk_encoding_internal::DecompressorOptions()\n            .set_recycling_pool_options(recycling_pool_options_));\n    if (ABSL_PREDICT_FALSE(!bucket_decompressors.back().ok())) {\n      return Fail(bucket_decompressors.back().status());\n    }\n  }\n\n  uint32_t bucket_index = 0;\n  for (size_t buffer_index = 0; buffer_index < num_buffers; ++buffer_index) {\n    uint64_t buffer_length;\n    if (ABSL_PREDICT_FALSE(!ReadVarint64(header_reader, buffer_length))) {\n      return Fail(header_reader.StatusOrAnnotate(\n          absl::InvalidArgumentError(\"Reading buffer length failed\")));\n    }\n    if (ABSL_PREDICT_FALSE(buffer_length >\n                           std::numeric_limits<size_t>::max())) {\n      return Fail(absl::ResourceExhaustedError(\"Buffer too large\"));\n    }\n    Chain buffer;\n    if (ABSL_PREDICT_FALSE(!bucket_decompressors[bucket_index].reader().Read(\n            IntCast<size_t>(buffer_length), buffer))) {\n      return Fail(bucket_decompressors[bucket_index].reader().StatusOrAnnotate(\n          absl::InvalidArgumentError(\"Reading buffer failed\")));\n    }\n    context.buffers.emplace_back(std::move(buffer));\n    while (!bucket_decompressors[bucket_index].reader().Pull() &&\n           bucket_index + 1 < num_buckets) {\n      if (ABSL_PREDICT_FALSE(\n              !bucket_decompressors[bucket_index].VerifyEndAndClose())) {\n        return Fail(bucket_decompressors[bucket_index].status());\n      }\n      ++bucket_index;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(bucket_index + 1 < num_buckets)) {\n    return Fail(absl::InvalidArgumentError(\"Too few buckets\"));\n  }\n  if (ABSL_PREDICT_FALSE(\n          !bucket_decompressors[bucket_index].VerifyEndAndClose())) {\n    return Fail(bucket_decompressors[bucket_index].status());\n  }\n  return true;\n}\n\ninline bool TransposeDecoder::ParseBuffersForFiltering(\n    Context& context, Reader& header_reader, Reader& src,\n    std::optional<absl::FixedArray<uint32_t>>& first_buffer_indices,\n    std::optional<absl::FixedArray<uint32_t>>& bucket_indices) {\n  uint32_t num_buckets;\n  if (ABSL_PREDICT_FALSE(!ReadVarint32(header_reader, num_buckets))) {\n    return Fail(header_reader.StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Reading number of buckets failed\")));\n  }\n  if (ABSL_PREDICT_FALSE(num_buckets > context.buckets.max_size())) {\n    return Fail(absl::ResourceExhaustedError(\"Too many buckets\"));\n  }\n  uint32_t num_buffers;\n  if (ABSL_PREDICT_FALSE(!ReadVarint32(header_reader, num_buffers))) {\n    return Fail(header_reader.StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Reading number of buffers failed\")));\n  }\n  if (ABSL_PREDICT_FALSE(num_buffers >\n                         absl::FixedArray<uint32_t>(0).max_size())) {\n    return Fail(absl::ResourceExhaustedError(\"Too many buffers\"));\n  }\n  first_buffer_indices.emplace(num_buckets);\n  bucket_indices.emplace(num_buffers);\n  if (num_buckets == 0) {\n    if (ABSL_PREDICT_FALSE(num_buffers != 0)) {\n      return Fail(absl::InvalidArgumentError(\"Too few buckets\"));\n    }\n    return true;\n  }\n  context.buckets.reserve(num_buckets);\n  for (uint32_t bucket_index = 0; bucket_index < num_buckets; ++bucket_index) {\n    uint64_t bucket_length;\n    if (ABSL_PREDICT_FALSE(!ReadVarint64(header_reader, bucket_length))) {\n      return Fail(header_reader.StatusOrAnnotate(\n          absl::InvalidArgumentError(\"Reading bucket length failed\")));\n    }\n    if (ABSL_PREDICT_FALSE(bucket_length >\n                           std::numeric_limits<size_t>::max())) {\n      return Fail(absl::ResourceExhaustedError(\"Bucket too large\"));\n    }\n    context.buckets.emplace_back();\n    if (ABSL_PREDICT_FALSE(!src.Read(IntCast<size_t>(bucket_length),\n                                     context.buckets.back().compressed_data))) {\n      return Fail(src.StatusOrAnnotate(\n          absl::InvalidArgumentError(\"Reading bucket failed\")));\n    }\n  }\n\n  uint32_t bucket_index = 0;\n  (*first_buffer_indices)[0] = 0;\n  std::optional<uint64_t> remaining_bucket_size =\n      chunk_encoding_internal::UncompressedSize(\n          context.buckets[0].compressed_data, context.compression_type);\n  if (ABSL_PREDICT_FALSE(remaining_bucket_size == std::nullopt)) {\n    return Fail(absl::InvalidArgumentError(\"Reading uncompressed size failed\"));\n  }\n  for (uint32_t buffer_index = 0; buffer_index < num_buffers; ++buffer_index) {\n    uint64_t buffer_length;\n    if (ABSL_PREDICT_FALSE(!ReadVarint64(header_reader, buffer_length))) {\n      return Fail(header_reader.StatusOrAnnotate(\n          absl::InvalidArgumentError(\"Reading buffer length failed\")));\n    }\n    if (ABSL_PREDICT_FALSE(buffer_length >\n                           std::numeric_limits<size_t>::max())) {\n      return Fail(absl::ResourceExhaustedError(\"Buffer too large\"));\n    }\n    context.buckets[bucket_index].buffer_sizes.push_back(\n        IntCast<size_t>(buffer_length));\n    if (ABSL_PREDICT_FALSE(buffer_length > *remaining_bucket_size)) {\n      return Fail(absl::InvalidArgumentError(\"Buffer does not fit in bucket\"));\n    }\n    *remaining_bucket_size -= buffer_length;\n    (*bucket_indices)[buffer_index] = bucket_index;\n    while (*remaining_bucket_size == 0 && bucket_index + 1 < num_buckets) {\n      ++bucket_index;\n      (*first_buffer_indices)[bucket_index] = buffer_index + 1;\n      remaining_bucket_size = chunk_encoding_internal::UncompressedSize(\n          context.buckets[bucket_index].compressed_data,\n          context.compression_type);\n      if (ABSL_PREDICT_FALSE(remaining_bucket_size == std::nullopt)) {\n        return Fail(\n            absl::InvalidArgumentError(\"Reading uncompressed size failed\"));\n      }\n    }\n  }\n  if (ABSL_PREDICT_FALSE(bucket_index + 1 < num_buckets)) {\n    return Fail(absl::InvalidArgumentError(\"Too few buckets\"));\n  }\n  if (ABSL_PREDICT_FALSE(*remaining_bucket_size > 0)) {\n    return Fail(absl::InvalidArgumentError(\"End of data expected\"));\n  }\n  return true;\n}\n\ninline Reader* TransposeDecoder::GetBuffer(Context& context,\n                                           uint32_t bucket_index,\n                                           uint32_t index_within_bucket) {\n  RIEGELI_ASSERT_LT(bucket_index, context.buckets.size())\n      << \"Bucket index out of range\";\n  DataBucket& bucket = context.buckets[bucket_index];\n  RIEGELI_ASSERT_LT(index_within_bucket, !bucket.buffer_sizes.empty()\n                                             ? bucket.buffer_sizes.size()\n                                             : bucket.buffers.size())\n      << \"Index within bucket out of range\";\n  while (index_within_bucket >= bucket.buffers.size()) {\n    if (bucket.buffers.empty()) {\n      // This is the first buffer to be decompressed from this bucket.\n      bucket.decompressor.Reset(\n          riegeli::Maker(&bucket.compressed_data), context.compression_type,\n          chunk_encoding_internal::DecompressorOptions()\n              .set_recycling_pool_options(recycling_pool_options_));\n      if (ABSL_PREDICT_FALSE(!bucket.decompressor.ok())) {\n        Fail(bucket.decompressor.status());\n        return nullptr;\n      }\n      // Important to prevent invalidating pointers by `emplace_back()`.\n      bucket.buffers.reserve(bucket.buffer_sizes.size());\n    }\n    Chain buffer;\n    if (ABSL_PREDICT_FALSE(!bucket.decompressor.reader().Read(\n            bucket.buffer_sizes[bucket.buffers.size()], buffer))) {\n      Fail(bucket.decompressor.reader().StatusOrAnnotate(\n          absl::InvalidArgumentError(\"Reading buffer failed\")));\n      return nullptr;\n    }\n    bucket.buffers.emplace_back(std::move(buffer));\n    if (bucket.buffers.size() == bucket.buffer_sizes.size()) {\n      // This was the last decompressed buffer from this bucket.\n      if (ABSL_PREDICT_FALSE(!bucket.decompressor.VerifyEndAndClose())) {\n        Fail(bucket.decompressor.status());\n        return nullptr;\n      }\n      // Free memory of fields which are no longer needed.\n      bucket.compressed_data = Chain();\n      bucket.buffer_sizes = std::vector<size_t>();\n    }\n  }\n  return &bucket.buffers[index_within_bucket];\n}\n\ninline bool TransposeDecoder::ContainsImplicitLoop(\n    std::vector<StateMachineNode>* state_machine_nodes) {\n  std::vector<size_t> implicit_loop_ids(state_machine_nodes->size(), 0);\n  size_t next_loop_id = 1;\n  for (size_t i = 0; i < state_machine_nodes->size(); ++i) {\n    if (implicit_loop_ids[i] != 0) continue;\n    StateMachineNode* node = &(*state_machine_nodes)[i];\n    implicit_loop_ids[i] = next_loop_id;\n    while (node->is_implicit) {\n      node = node->next_node;\n      const size_t j = node - state_machine_nodes->data();\n      if (implicit_loop_ids[j] == next_loop_id) return true;\n      if (implicit_loop_ids[j] != 0) break;\n      implicit_loop_ids[j] = next_loop_id;\n    }\n    ++next_loop_id;\n  }\n  return false;\n}\n\n// InvalidArgumentError that is not inlined. This reduces register pressure for\n// the Decode loop.\nABSL_ATTRIBUTE_NOINLINE absl::Status InvalidArgumentError(\n    absl::string_view msg) {\n  return absl::InvalidArgumentError(msg);\n}\n\nstruct TransposeDecoder::DecodingState {\n  // `SubmessageStack` is used to keep information about started nested\n  // submessages. Decoding works in non-recursive loop and this class keeps the\n  // information needed to finalize one submessages.\n  //\n  // A manual structure is used instead of `std::vector` to avoid unnecessary\n  // zeroing or object construction in the submessage hot path.\n  class SubmessageStack {\n   public:\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void Push(size_t position,\n                                           StateMachineNode* node) {\n      EnsureSpaceForOne();\n      positions_[size_] = position;\n      nodes_[size_] = node;\n      ++size_;\n    }\n\n    struct SubmessageStackElement {\n      size_t end_of_submessage;\n      StateMachineNode* submessage_node;\n    };\n\n    ABSL_ATTRIBUTE_ALWAYS_INLINE SubmessageStackElement Pop() {\n      RIEGELI_ASSERT(!Empty());\n      --size_;\n      return SubmessageStackElement{positions_[size_], nodes_[size_]};\n    }\n\n    ABSL_ATTRIBUTE_ALWAYS_INLINE bool Empty() const { return size_ == 0; }\n\n    ABSL_ATTRIBUTE_ALWAYS_INLINE absl::Span<const StateMachineNode* const>\n    NodeSpan() const {\n      return absl::MakeConstSpan(nodes_.get(), size_);\n    }\n\n   private:\n    ABSL_ATTRIBUTE_ALWAYS_INLINE void EnsureSpaceForOne() {\n      if (ABSL_PREDICT_FALSE(size_ == capacity_)) {\n        if (ABSL_PREDICT_TRUE(capacity_ == 0)) {\n          capacity_ = 16;\n          positions_ = std::unique_ptr<size_t[]>(new size_t[capacity_]);\n          nodes_ = std::unique_ptr<StateMachineNode*[]>(\n              new StateMachineNode*[capacity_]);\n        } else {\n          capacity_ *= 2;\n          std::unique_ptr<size_t[]> new_positions(new size_t[capacity_]);\n          std::unique_ptr<StateMachineNode*[]> new_nodes(\n              new StateMachineNode*[capacity_]);\n          std::memcpy(new_positions.get(), positions_.get(),\n                      size_ * sizeof(size_t));\n          std::memcpy(new_nodes.get(), nodes_.get(),\n                      size_ * sizeof(StateMachineNode*));\n          positions_ = std::move(new_positions);\n          nodes_ = std::move(new_nodes);\n        }\n      }\n    }\n\n    std::unique_ptr<size_t[]> positions_;\n    std::unique_ptr<StateMachineNode*[]> nodes_;\n    size_t size_ = 0;\n    size_t capacity_ = 0;\n  };\n\n  // All pointers are non-null and owned elsewhere.\n  explicit DecodingState(TransposeDecoder* decoder, Context* context,\n                         uint64_t num_records, BackwardWriter* dest,\n                         std::vector<size_t>* limits)\n      : decoder(decoder),\n        context(context),\n        num_records(num_records),\n        dest(dest),\n        limits(limits),\n        transitions_reader(&context->transitions.reader()),\n        node(&context->state_machine_nodes[context->first_node]),\n        num_iters(node->is_implicit ? 1 : 0) {\n    // For now positions reported by `dest` are pushed to `limits` directly.\n    // Later `limits` will be reversed and complemented.\n    limits->clear();\n    limits->reserve(num_records);\n  }\n\n  DecodingState(const DecodingState&) = delete;\n  DecodingState& operator=(const DecodingState&) = delete;\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool SetCallbackType() {\n    return decoder->SetCallbackType(*context, skipped_submessage_level,\n                                    submessage_stack.NodeSpan(), *node);\n  }\n\n  // Copy tag from `*node` to `dest`.\n  template <size_t tag_length>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool CopyTagCallback() {\n    if (ABSL_PREDICT_FALSE(\n            !dest->Write(absl::string_view(node->tag_data, tag_length)))) {\n      return decoder->Fail(dest->status());\n    }\n    return true;\n  }\n\n  template <uint8_t data_length, uint8_t low, uint8_t high>\n  using EnableIfBetween =\n      std::enable_if_t<(data_length >= low && data_length <= high), int>;\n\n  template <uint8_t data_length>\n  using Uint8Constant = std::integral_constant<uint8_t, data_length>;\n\n  // Masks the first `data_length - 1` elements in `src` with 0x80, and copies\n  // to `dest`.\n  //\n  // TODO: Replace with `if constexpr` in C++17.\n  template <uint8_t data_length, EnableIfBetween<data_length, 9, 10> = 0>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE void MaskBuffer(\n      const char* src, char* dest,\n      std::integral_constant<uint8_t, data_length>) {\n    uint64_t val;\n    std::memcpy(&val, src, 8);\n    val |= uint64_t{0x8080808080808080};\n    std::memcpy(dest, &val, 8);\n    MaskBuffer(src + 8, dest + 8, Uint8Constant<data_length - 8>());\n  }\n\n  template <uint8_t data_length, EnableIfBetween<data_length, 5, 8> = 0>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE void MaskBuffer(\n      const char* src, char* dest,\n      std::integral_constant<uint8_t, data_length>) {\n    uint32_t val;\n    std::memcpy(&val, src, 4);\n    val |= uint32_t{0x80808080};\n    std::memcpy(dest, &val, 4);\n    MaskBuffer(src + 4, dest + 4, Uint8Constant<data_length - 4>());\n  }\n\n  template <uint8_t data_length, EnableIfBetween<data_length, 3, 4> = 0>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE void MaskBuffer(\n      const char* src, char* dest,\n      std::integral_constant<uint8_t, data_length>) {\n    uint16_t val;\n    std::memcpy(&val, src, 2);\n    val |= uint16_t{0x8080};\n    std::memcpy(dest, &val, 2);\n    MaskBuffer(src + 2, dest + 2, Uint8Constant<data_length - 2>());\n  }\n\n  template <uint8_t data_length, EnableIfBetween<data_length, 2, 2> = 0>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE void MaskBuffer(\n      const char* src, char* dest,\n      std::integral_constant<uint8_t, data_length>) {\n    *dest = static_cast<char>(static_cast<uint8_t>(*src) | uint8_t{0x80});\n    MaskBuffer(src + 1, dest + 1, Uint8Constant<data_length - 1>());\n  }\n\n  template <uint8_t data_length, EnableIfBetween<data_length, 1, 1> = 0>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE void MaskBuffer(\n      const char* src, char* dest,\n      std::integral_constant<uint8_t, data_length>) {\n    *dest = *src;\n  }\n\n  // Decode varint value from `*node` to `dest`.\n  template <size_t tag_length, uint8_t data_length>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool VarintCallback() {\n    if (ABSL_PREDICT_FALSE(!dest->Push(tag_length + data_length))) {\n      return decoder->Fail(dest->status());\n    }\n    dest->move_cursor(tag_length + data_length);\n    // Use a temporary buffer to allow it to be in a register for processing.\n    char unmasked[data_length];\n    char* const buffer = dest->cursor();\n    if (ABSL_PREDICT_FALSE(!node->buffer->Read(data_length, unmasked))) {\n      return decoder->Fail(node->buffer->StatusOrAnnotate(\n          InvalidArgumentError(\"Reading varint field failed\")));\n    }\n    MaskBuffer(unmasked, buffer + tag_length, Uint8Constant<data_length>());\n    std::memcpy(buffer, node->tag_data, tag_length);\n    return true;\n  }\n\n  // Decode fixed32 or fixed64 value from `*node` to `dest`.\n  template <size_t tag_length, size_t data_length>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool FixedCallback() {\n    if (ABSL_PREDICT_FALSE(!dest->Push(tag_length + data_length))) {\n      return decoder->Fail(dest->status());\n    }\n    dest->move_cursor(tag_length + data_length);\n    char* const buffer = dest->cursor();\n    if (ABSL_PREDICT_FALSE(\n            !node->buffer->Read(data_length, buffer + tag_length))) {\n      return decoder->Fail(node->buffer->StatusOrAnnotate(\n          InvalidArgumentError(\"Reading fixed field failed\")));\n    }\n    std::memcpy(buffer, node->tag_data, tag_length);\n    return true;\n  }\n\n  // Create zero fixed32 or fixed64 value in `dest`.\n  template <size_t tag_length, size_t data_length>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool FixedExistenceCallback() {\n    if (ABSL_PREDICT_FALSE(!dest->Push(tag_length + data_length))) {\n      return decoder->Fail(dest->status());\n    }\n    dest->move_cursor(tag_length + data_length);\n    char* const buffer = dest->cursor();\n    std::memset(buffer + tag_length, '\\0', data_length);\n    std::memcpy(buffer, node->tag_data, tag_length);\n    return true;\n  }\n\n  // Decode string value from `*node` to `dest`.\n  template <size_t tag_length>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool StringCallback() {\n    node->buffer->Pull(kMaxLengthVarint32);\n    uint32_t length;\n    const size_t length_length =\n        ReadVarint32(node->buffer->cursor(), node->buffer->available(), length);\n    if (ABSL_PREDICT_FALSE(length_length == 0)) {\n      return decoder->Fail(node->buffer->StatusOrAnnotate(\n          InvalidArgumentError(\"Reading string length failed\")));\n    }\n    if (ABSL_PREDICT_FALSE(length > std::numeric_limits<uint32_t>::max() -\n                                        length_length)) {\n      return decoder->Fail(InvalidArgumentError(\"String length overflow\"));\n    }\n    if (ABSL_PREDICT_FALSE(\n            !node->buffer->Copy(length_length + length, *dest))) {\n      return decoder->Fail(\n          !dest->ok() ? dest->status()\n                      : node->buffer->StatusOrAnnotate(InvalidArgumentError(\n                            \"Reading string field failed\")));\n    }\n    if (ABSL_PREDICT_FALSE(\n            !dest->Write(absl::string_view(node->tag_data, tag_length)))) {\n      return decoder->Fail(dest->status());\n    }\n    return true;\n  }\n\n  template <size_t tag_length>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool StartProjectionGroupCallback() {\n    if (ABSL_PREDICT_FALSE(submessage_stack.Empty())) {\n      return decoder->Fail(InvalidArgumentError(\"Submessage stack underflow\"));\n    }\n    submessage_stack.Pop();\n    return CopyTagCallback<tag_length>();\n  }\n\n  template <size_t tag_length>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool EndProjectionGroupCallback() {\n    submessage_stack.Push(IntCast<size_t>(dest->pos()), node);\n    return CopyTagCallback<tag_length>();\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool SubmessageStartCallback() {\n    if (ABSL_PREDICT_FALSE(submessage_stack.Empty())) {\n      return decoder->Fail(InvalidArgumentError(\"Submessage stack underflow\"));\n    }\n    auto elem = submessage_stack.Pop();\n    RIEGELI_ASSERT_GE(dest->pos(), elem.end_of_submessage)\n        << \"Destination position decreased\";\n    const size_t length = IntCast<size_t>(dest->pos()) - elem.end_of_submessage;\n    if (ABSL_PREDICT_FALSE(length > std::numeric_limits<uint32_t>::max())) {\n      return decoder->Fail(InvalidArgumentError(\"Message too large\"));\n    }\n    const size_t varint_length = LengthVarint32(IntCast<uint32_t>(length));\n    uint8_t tag_data_size = elem.submessage_node->tag_data_size;\n    size_t header_length = varint_length + tag_data_size;\n    if (ABSL_PREDICT_FALSE(!dest->Push(header_length))) {\n      return decoder->Fail(dest->status());\n    }\n    dest->move_cursor(header_length);\n    char* cursor = dest->cursor();\n    std::memcpy(cursor, elem.submessage_node->tag_data, tag_data_size);\n    WriteVarint32(IntCast<uint32_t>(length), cursor + tag_data_size);\n    return true;\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool MessageStartCallback() {\n    if (ABSL_PREDICT_FALSE(!submessage_stack.Empty())) {\n      return decoder->Fail(InvalidArgumentError(\"Submessages still open\"));\n    }\n    if (ABSL_PREDICT_FALSE(limits->size() == num_records)) {\n      return decoder->Fail(InvalidArgumentError(\"Too many records\"));\n    }\n    limits->push_back(IntCast<size_t>(dest->pos()));\n    return true;\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool NonprotoCallback() {\n    uint32_t length;\n    if (ABSL_PREDICT_FALSE(!ReadVarint32(*context->nonproto_lengths, length))) {\n      return decoder->Fail(context->nonproto_lengths->StatusOrAnnotate(\n          InvalidArgumentError(\"Reading non-proto record length failed\")));\n    }\n    if (ABSL_PREDICT_FALSE(!node->buffer->Copy(length, *dest))) {\n      return decoder->Fail(\n          !dest->ok() ? dest->status()\n                      : node->buffer->StatusOrAnnotate(InvalidArgumentError(\n                            \"Reading non-proto record failed\")));\n    }\n    return MessageStartCallback();\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool HandleNode() {\n    // Use a loop instead of recursion to avoid unbounded stack growth.\n    // for loop is only executed more than once in the case of kSelectCallback.\n    for (;;) {\n      switch (node->callback_type) {\n        case CallbackType::kSelectCallback:\n          if (ABSL_PREDICT_FALSE(!SetCallbackType())) return false;\n          continue;\n\n        case CallbackType::kSkippedSubmessageEnd:\n          ++skipped_submessage_level;\n          return true;\n\n        case CallbackType::kSkippedSubmessageStart:\n          if (ABSL_PREDICT_FALSE(skipped_submessage_level == 0)) {\n            return decoder->Fail(\n                InvalidArgumentError(\"Skipped submessage stack underflow\"));\n          }\n          --skipped_submessage_level;\n          return true;\n\n        case CallbackType::kSubmessageEnd:\n          submessage_stack.Push(IntCast<size_t>(dest->pos()), node);\n          return true;\n\n        case CallbackType::kSubmessageStart:\n          return SubmessageStartCallback();\n\n#define ACTIONS_FOR_TAG_LEN(tag_length)                  \\\n  case CallbackType::kCopyTag_##tag_length:              \\\n    return CopyTagCallback<tag_length>();                \\\n  case CallbackType::kVarint_1_##tag_length:             \\\n    return VarintCallback<tag_length, 1>();              \\\n  case CallbackType::kVarint_2_##tag_length:             \\\n    return VarintCallback<tag_length, 2>();              \\\n  case CallbackType::kVarint_3_##tag_length:             \\\n    return VarintCallback<tag_length, 3>();              \\\n  case CallbackType::kVarint_4_##tag_length:             \\\n    return VarintCallback<tag_length, 4>();              \\\n  case CallbackType::kVarint_5_##tag_length:             \\\n    return VarintCallback<tag_length, 5>();              \\\n  case CallbackType::kVarint_6_##tag_length:             \\\n    return VarintCallback<tag_length, 6>();              \\\n  case CallbackType::kVarint_7_##tag_length:             \\\n    return VarintCallback<tag_length, 7>();              \\\n  case CallbackType::kVarint_8_##tag_length:             \\\n    return VarintCallback<tag_length, 8>();              \\\n  case CallbackType::kVarint_9_##tag_length:             \\\n    return VarintCallback<tag_length, 9>();              \\\n  case CallbackType::kVarint_10_##tag_length:            \\\n    return VarintCallback<tag_length, 10>();             \\\n  case CallbackType::kFixed32_##tag_length:              \\\n    return FixedCallback<tag_length, 4>();               \\\n  case CallbackType::kFixed64_##tag_length:              \\\n    return FixedCallback<tag_length, 8>();               \\\n  case CallbackType::kFixed32Existence_##tag_length:     \\\n    return FixedExistenceCallback<tag_length, 4>();      \\\n  case CallbackType::kFixed64Existence_##tag_length:     \\\n    return FixedExistenceCallback<tag_length, 8>();      \\\n  case CallbackType::kString_##tag_length:               \\\n    return StringCallback<tag_length>();                 \\\n  case CallbackType::kStartProjectionGroup_##tag_length: \\\n    return StartProjectionGroupCallback<tag_length>();   \\\n  case CallbackType::kEndProjectionGroup_##tag_length:   \\\n    return EndProjectionGroupCallback<tag_length>()\n\n          ACTIONS_FOR_TAG_LEN(1);\n          ACTIONS_FOR_TAG_LEN(2);\n          ACTIONS_FOR_TAG_LEN(3);\n          ACTIONS_FOR_TAG_LEN(4);\n          ACTIONS_FOR_TAG_LEN(5);\n#undef ACTIONS_FOR_TAG_LEN\n\n        case CallbackType::kCopyTag_6:\n          return CopyTagCallback<6>();\n\n        case CallbackType::kUnknown:\n        case CallbackType::kFailure:\n          return decoder->Fail(InvalidArgumentError(\"Invalid node index\"));\n\n        case CallbackType::kNonProto:\n          return NonprotoCallback();\n\n        case CallbackType::kMessageStart:\n          return MessageStartCallback();\n\n        case CallbackType::kNoOp:\n          return true;\n      }\n      RIEGELI_ASSUME_UNREACHABLE()\n          << \"Unknown callback type: \" << static_cast<int>(node->callback_type);\n    }\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool TransitionNode() {\n    node = node->next_node;\n    if (num_iters == 0) {\n      uint8_t transition_byte;\n      if (ABSL_PREDICT_FALSE(!transitions_reader->ReadByte(transition_byte))) {\n        return false;\n      }\n      node += (transition_byte >> 2);\n      num_iters = (transition_byte & 3) + (node->is_implicit ? 1 : 0);\n    } else {\n      num_iters -= node->is_implicit ? 0 : 1;\n    }\n    return true;\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool Finish() {\n    if (ABSL_PREDICT_FALSE(!context->transitions.VerifyEndAndClose())) {\n      return decoder->Fail(context->transitions.status());\n    }\n    if (ABSL_PREDICT_FALSE(!submessage_stack.Empty())) {\n      return decoder->Fail(InvalidArgumentError(\"Submessages still open\"));\n    }\n    if (ABSL_PREDICT_FALSE(skipped_submessage_level != 0)) {\n      return decoder->Fail(\n          InvalidArgumentError(\"Skipped submessages still open\"));\n    }\n    if (ABSL_PREDICT_FALSE(limits->size() != num_records)) {\n      return decoder->Fail(InvalidArgumentError(\"Too few records\"));\n    }\n    const size_t size = limits->empty() ? size_t{0} : limits->back();\n    if (ABSL_PREDICT_FALSE(size != dest->pos())) {\n      return decoder->Fail(InvalidArgumentError(\"Unfinished message\"));\n    }\n    // Reverse `limits` and complement them, but keep the last limit unchanged\n    // (because both old and new limits exclude 0 at the beginning and include\n    // size at the end), e.g. for records of sizes {10, 20, 30, 40}:\n    // {40, 70, 90, 100} -> {10, 30, 60, 100}.\n    std::vector<size_t>::iterator first = limits->begin();\n    std::vector<size_t>::iterator last = limits->end();\n    if (first != last) {\n      --last;\n      while (first < last) {\n        --last;\n        const size_t tmp = size - *first;\n        *first = size - *last;\n        *last = tmp;\n        ++first;\n      }\n    }\n    return true;\n  }\n\n  // Initial state\n  TransposeDecoder* decoder;\n  Context* context;\n  uint64_t num_records;\n  BackwardWriter* dest;\n  std::vector<size_t>* limits;\n  Reader* transitions_reader;\n\n  // Variable state\n\n  // The current node.\n  StateMachineNode* node;\n  // The depth of the current field relative to the parent submessage that\n  // was excluded in projection.\n  int skipped_submessage_level = 0;\n  // Stack of all open sub-messages.\n  SubmessageStack submessage_stack;\n  // Number of following iteration that go directly to `node->next_node`\n  // without reading transition byte. Maximum value is 4.\n  int8_t num_iters;\n};\n\ninline bool TransposeDecoder::Decode(Context& context, uint64_t num_records,\n                                     BackwardWriter& dest,\n                                     std::vector<size_t>& limits) {\n  DecodingState state(this, &context, num_records, &dest, &limits);\n  do {\n    if (ABSL_PREDICT_FALSE(!state.HandleNode())) return false;\n  } while (ABSL_PREDICT_TRUE(state.TransitionNode()));\n  return state.Finish();\n}\n\n// Do not inline this function. This helps Clang to generate better code for\n// the main loop in `Decode()`.\nABSL_ATTRIBUTE_NOINLINE inline bool TransposeDecoder::SetCallbackType(\n    Context& context, int skipped_submessage_level,\n    absl::Span<const StateMachineNode* const> submessage_stack,\n    StateMachineNode& node) {\n  StateMachineNodeTemplate* node_template = node.node_template;\n  if (node_template->tag ==\n      static_cast<uint32_t>(\n          chunk_encoding_internal::MessageId::kStartOfSubmessage)) {\n    if (skipped_submessage_level > 0) {\n      node.callback_type = CallbackType::kSkippedSubmessageStart;\n    } else {\n      node.callback_type = CallbackType::kSubmessageStart;\n    }\n  } else {\n    FieldIncluded field_included = FieldIncluded::kNo;\n    uint32_t field_id = kInvalidPos;\n    if (skipped_submessage_level == 0) {\n      field_included = FieldIncluded::kExistenceOnly;\n      for (const StateMachineNode* elem : submessage_stack) {\n        uint32_t tag;\n        RIEGELI_EVAL_ASSERT(\n            ReadVarint32(elem->tag_data, kMaxLengthVarint32, tag) > 0)\n            << \"Invalid tag\";\n        const absl::flat_hash_map<std::pair<uint32_t, int>,\n                                  Context::IncludedField>::const_iterator iter =\n            context.include_fields.find(\n                std::make_pair(field_id, GetTagFieldNumber(tag)));\n        if (iter == context.include_fields.end()) {\n          field_included = FieldIncluded::kNo;\n          break;\n        }\n        if (iter->second.include_type == Context::IncludeType::kIncludeFully) {\n          field_included = FieldIncluded::kYes;\n          break;\n        }\n        field_id = iter->second.field_id;\n      }\n    }\n    // If tag is a `kStartGroup`, there are two options:\n    // 1. Either related `kEndGroup` was skipped and\n    //    `skipped_submessage_level > 0`.\n    //    In this case `field_included` is already set to `kNo`.\n    // 2. If `kEndGroup` was not skipped, then its tag is on the top of the\n    //    `submessage_stack` and in that case we already checked its tag in\n    //    `include_fields` in the loop above.\n    const bool start_group_tag =\n        GetTagWireType(node_template->tag) == WireType::kStartGroup;\n    if (!start_group_tag && field_included == FieldIncluded::kExistenceOnly) {\n      uint32_t tag;\n      RIEGELI_EVAL_ASSERT(ReadVarint32(node.tag_data, kMaxLengthVarint32, tag) >\n                          0)\n          << \"Invalid tag\";\n      const absl::flat_hash_map<std::pair<uint32_t, int>,\n                                Context::IncludedField>::const_iterator iter =\n          context.include_fields.find(\n              std::make_pair(field_id, GetTagFieldNumber(tag)));\n      if (iter == context.include_fields.end()) {\n        field_included = FieldIncluded::kNo;\n      } else {\n        if (iter->second.include_type == Context::IncludeType::kIncludeFully ||\n            iter->second.include_type == Context::IncludeType::kIncludeChild) {\n          field_included = FieldIncluded::kYes;\n        }\n      }\n    }\n    if (node_template->bucket_index != kInvalidPos) {\n      switch (field_included) {\n        case FieldIncluded::kYes:\n          node.buffer = GetBuffer(context, node_template->bucket_index,\n                                  node_template->buffer_within_bucket_index);\n          if (ABSL_PREDICT_FALSE(node.buffer == nullptr)) return false;\n          break;\n        case FieldIncluded::kNo:\n        case FieldIncluded::kExistenceOnly:\n          node.buffer = kEmptyReader();\n          break;\n      }\n    } else {\n      node.buffer = kEmptyReader();\n    }\n    node.callback_type = GetCallbackType(field_included, node_template->tag,\n                                         node_template->subtype,\n                                         node_template->tag_length, true);\n    if (field_included == FieldIncluded::kExistenceOnly &&\n        GetTagWireType(node_template->tag) == WireType::kVarint) {\n      // The tag in `tag_data` was followed by a subtype but must be followed by\n      // zero now.\n      node.tag_data[node_template->tag_length] = 0;\n    }\n  }\n  return true;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/chunk_encoding/transpose_decoder.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_TRANSPOSE_DECODER_H_\n#define RIEGELI_CHUNK_ENCODING_TRANSPOSE_DECODER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <optional>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/container/fixed_array.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/chunk_encoding/field_projection.h\"\n\nnamespace riegeli {\n\nclass TransposeDecoder : public Object {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // Options for a global `RecyclingPool` of decompression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // decompression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    Options& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    Options&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Creates a closed `TransposeDecoder`.\n  explicit TransposeDecoder(Options options = Options())\n      : Object(kClosed),\n        recycling_pool_options_(options.recycling_pool_options()) {}\n\n  TransposeDecoder(const TransposeDecoder&) = delete;\n  TransposeDecoder& operator=(const TransposeDecoder&) = delete;\n\n  // Resets the `TransposeDecoder` and parses the chunk.\n  //\n  // Writes concatenated record values to `dest`. Sets `limits` to sorted\n  // record end positions.\n  //\n  // Precondition: `dest.pos() == 0`\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`);\n  //              if `!dest.ok()` then the problem was at `dest`\n  bool Decode(uint64_t num_records, uint64_t decoded_data_size,\n              const FieldProjection& field_projection, Reader& src,\n              BackwardWriter& dest, std::vector<size_t>& limits);\n\n private:\n  struct StateMachineNode;\n  struct Context;\n  struct DecodingState;\n\n  bool Parse(Context& context, Reader& src,\n             const FieldProjection& field_projection);\n\n  // Parse data buffers in `header_reader` and `src` into `context.buffers`.\n  // This method is used when projection is disabled and all buffers are\n  // initially decompressed.\n  bool ParseBuffers(Context& context, Reader& header_reader, Reader& src);\n\n  // Parse data buffers in `header_reader` and `src` into `context.buckets`.\n  // When projection is enabled, buckets are decompressed on demand.\n  // `bucket_indices` contains bucket index for each buffer.\n  // `first_buffer_indices` contains the index of first buffer for each bucket.\n  bool ParseBuffersForFiltering(\n      Context& context, Reader& header_reader, Reader& src,\n      std::optional<absl::FixedArray<uint32_t>>& first_buffer_indices,\n      std::optional<absl::FixedArray<uint32_t>>& bucket_indices);\n\n  // Precondition: `projection_enabled`.\n  Reader* GetBuffer(Context& context, uint32_t bucket_index,\n                    uint32_t index_within_bucket);\n\n  static bool ContainsImplicitLoop(\n      std::vector<StateMachineNode>* state_machine_nodes);\n\n  bool Decode(Context& context, uint64_t num_records, BackwardWriter& dest,\n              std::vector<size_t>& limits);\n\n  // Set `callback_type` in `node` based on `skipped_submessage_level`,\n  // `submessage_stack`, and `node.node_template`.\n  bool SetCallbackType(\n      Context& context, int skipped_submessage_level,\n      absl::Span<const StateMachineNode* const> submessage_stack,\n      StateMachineNode& node);\n\n  RecyclingPoolOptions recycling_pool_options_;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CHUNK_ENCODING_TRANSPOSE_DECODER_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/transpose_encoder.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/chunk_encoding/transpose_encoder.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <algorithm>\n#include <limits>\n#include <memory>\n#include <optional>\n#include <queue>\n#include <string>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/container/flat_hash_map.h\"\n#include \"absl/container/inlined_vector.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/chain_backward_writer.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/chain_writer.h\"\n#include \"riegeli/bytes/cord_reader.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/string_reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/chunk_encoder.h\"\n#include \"riegeli/chunk_encoding/compressor.h\"\n#include \"riegeli/chunk_encoding/compressor_options.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/chunk_encoding/transpose_internal.h\"\n#include \"riegeli/messages/message_wire_format.h\"\n#include \"riegeli/varint/varint_reading.h\"\n#include \"riegeli/varint/varint_writing.h\"\n\nnamespace riegeli {\n\nnamespace {\n\nconstexpr uint32_t kInvalidPos = std::numeric_limits<uint32_t>::max();\n// Maximum varint value to encode as varint subtype instead of using the buffer.\nconstexpr uint8_t kMaxVarintInline = 3;\n\nstatic_assert(kMaxVarintInline < 0x80,\n              \"Only one byte is used to store inline varint and its value must \"\n              \"concide with its varint encoding\");\n\n// Maximum depth of the nested message we break into columns. Submessages with\n// deeper nesting are encoded as strings.\nconstexpr int kMaxRecursionDepth = 100;\n\n// Returns `true` if `record` is a valid protocol buffer message in the\n// canonical encoding. The purpose of this method is to distinguish a string\n// from a submessage in the proto wire format and to perform validity checks\n// that are asserted later (such as that double proto field is followed by at\n// least 8 bytes of data).\n//\n// Note: Protocol buffer with suboptimal varint encoded tags and values (such as\n// `0x87, 0x00` instead of `0x07`) would parse successfully with the default\n// proto parser. This can happen for binary strings in proto. However, we need\n// to produce exactly the same bytes in the output so we reject message encoded\n// in non-canonical way.\nbool IsProtoMessage(Reader& record) {\n  // We validate that all started proto groups are closed with endgroup tag.\n  std::vector<int> started_groups;\n  while (record.Pull()) {\n    uint32_t tag;\n    if (!ReadCanonicalVarint32(record, tag)) return false;\n    const int field_number = GetTagFieldNumber(tag);\n    if (field_number == 0) return false;\n    switch (GetTagWireType(tag)) {\n      case WireType::kVarint:\n        if (!SkipCanonicalVarint64(record)) return false;\n        continue;\n      case WireType::kFixed32:\n        if (!record.Skip(sizeof(uint32_t))) return false;\n        continue;\n      case WireType::kFixed64:\n        if (!record.Skip(sizeof(uint64_t))) return false;\n        continue;\n      case WireType::kLengthDelimited: {\n        uint32_t length;\n        if (!ReadCanonicalVarint32(record, length) || !record.Skip(length)) {\n          return false;\n        }\n        continue;\n      }\n      case WireType::kStartGroup:\n        started_groups.push_back(field_number);\n        continue;\n      case WireType::kEndGroup:\n        if (started_groups.empty() || started_groups.back() != field_number) {\n          return false;\n        }\n        started_groups.pop_back();\n        continue;\n      case WireType::kInvalid6:\n      case WireType::kInvalid7:\n        return false;\n    }\n    RIEGELI_ASSUME_UNREACHABLE()\n        << \"Impossible wire type: \" << static_cast<int>(GetTagWireType(tag));\n  }\n  RIEGELI_ASSERT_OK(record);\n  return started_groups.empty();\n}\n\n// `PriorityQueueEntry` is used in `tag_priority` to order destinations by the\n// number of transitions into them.\nstruct PriorityQueueEntry : WithCompare<PriorityQueueEntry> {\n  PriorityQueueEntry() = default;\n\n  explicit PriorityQueueEntry(uint32_t dest_index, size_t num_transitions)\n      : dest_index(dest_index), num_transitions(num_transitions) {}\n\n  friend bool operator==(const PriorityQueueEntry& a,\n                         const PriorityQueueEntry& b) {\n    return a.dest_index == b.dest_index &&\n           a.num_transitions == b.num_transitions;\n  }\n  friend StrongOrdering RIEGELI_COMPARE(const PriorityQueueEntry& a,\n                                        const PriorityQueueEntry& b) {\n    // Sort by `num_transitions`. Largest first.\n    if (const StrongOrdering ordering =\n            riegeli::Compare(b.num_transitions, a.num_transitions);\n        ordering != 0) {\n      return ordering;\n    }\n    // Break ties for reproducible ordering.\n    return riegeli::Compare(a.dest_index, b.dest_index);\n  }\n\n  // Index of the destination in `tags_list_`.\n  uint32_t dest_index = 0;\n  // Number of transitions into destination.\n  size_t num_transitions = 0;\n};\n\n}  // namespace\n\ninline TransposeEncoder::MessageNode::MessageNode(\n    chunk_encoding_internal::MessageId message_id)\n    : message_id(message_id) {}\n\ninline TransposeEncoder::NodeId::NodeId(\n    chunk_encoding_internal::MessageId parent_message_id, uint32_t tag)\n    : parent_message_id(parent_message_id), tag(tag) {}\n\ninline TransposeEncoder::StateInfo::StateInfo()\n    : etag_index(kInvalidPos),\n      base(kInvalidPos),\n      canonical_source(kInvalidPos) {}\n\ninline TransposeEncoder::StateInfo::StateInfo(uint32_t etag_index,\n                                              uint32_t base)\n    : etag_index(etag_index), base(base), canonical_source(kInvalidPos) {}\n\ninline TransposeEncoder::DestInfo::DestInfo() : pos(kInvalidPos) {}\n\ninline TransposeEncoder::EncodedTagInfo::EncodedTagInfo(\n    NodeId node_id, chunk_encoding_internal::Subtype subtype)\n    : node_id(node_id),\n      subtype(subtype),\n      state_machine_pos(kInvalidPos),\n      public_list_noop_pos(kInvalidPos),\n      base(kInvalidPos) {}\n\ninline TransposeEncoder::BufferWithMetadata::BufferWithMetadata(NodeId node_id)\n    : buffer(std::make_unique<Chain>()), node_id(node_id) {}\n\nTransposeEncoder::TransposeEncoder(CompressorOptions compressor_options,\n                                   TuningOptions tuning_options)\n    : compressor_options_(std::move(compressor_options)),\n      bucket_size_(compressor_options_.compression_type() ==\n                           CompressionType::kNone\n                       ? std::numeric_limits<uint64_t>::max()\n                       : tuning_options.bucket_size()),\n      recycling_pool_options_(tuning_options.recycling_pool_options()) {}\n\nTransposeEncoder::~TransposeEncoder() {}\n\nvoid TransposeEncoder::Clear() {\n  ChunkEncoder::Clear();\n  tags_list_.clear();\n  encoded_tags_.clear();\n  for (std::vector<BufferWithMetadata>& buffers : data_) buffers.clear();\n  message_stack_.clear();\n  group_stack_.clear();\n  message_nodes_.clear();\n  nonproto_lengths_writer_.Reset();\n  next_message_id_ = chunk_encoding_internal::MessageId::kRoot + 1;\n}\n\nbool TransposeEncoder::AddRecord(BytesRef record) {\n  StringReader<> reader(record);\n  return AddRecordInternal(reader);\n}\n\nbool TransposeEncoder::AddRecord(ExternalRef record) {\n  if (record.size() <= kMaxBytesToCopy) {\n    StringReader<> reader(record);\n    return AddRecordInternal(reader);\n  } else {\n    ChainReader<Chain> reader(riegeli::Maker<Chain>(std::move(record)));\n    return AddRecordInternal(reader);\n  }\n}\n\nbool TransposeEncoder::AddRecord(const Chain& record) {\n  ChainReader<> reader(&record);\n  return AddRecordInternal(reader);\n}\n\nbool TransposeEncoder::AddRecord(const absl::Cord& record) {\n  CordReader<> reader(&record);\n  return AddRecordInternal(reader);\n}\n\nbool TransposeEncoder::AddRecords(Chain records, std::vector<size_t> limits) {\n  RIEGELI_ASSERT_EQ(limits.empty() ? 0u : limits.back(), records.size())\n      << \"Failed precondition of ChunkEncoder::AddRecords(): \"\n         \"record end positions do not match concatenated record values\";\n  LimitingReader<ChainReader<>> record_reader(\n      riegeli::Maker(&records), LimitingReaderBase::Options().set_exact_pos(0));\n  for (const size_t limit : limits) {\n    RIEGELI_ASSERT_GE(limit, record_reader.pos())\n        << \"Failed precondition of ChunkEncoder::AddRecords(): \"\n           \"record end positions not sorted\";\n    record_reader.set_max_pos(limit);\n    if (ABSL_PREDICT_FALSE(!AddRecordInternal(record_reader))) return false;\n    RIEGELI_ASSERT_EQ(record_reader.pos(), limit)\n        << \"Record was not read up to its end\";\n  }\n  RIEGELI_EVAL_ASSERT(record_reader.Close()) << record_reader.status();\n  return true;\n}\n\ninline bool TransposeEncoder::AddRecordInternal(Reader& record) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  RIEGELI_ASSERT_OK(record)\n      << \"Failed precondition of TransposeEncoder::AddRecordInternal()\";\n  const Position pos_before = record.pos();\n  std::optional<Position> size = record.Size();\n  RIEGELI_ASSERT(size != std::nullopt) << record.status();\n  RIEGELI_ASSERT_LE(pos_before, *size)\n      << \"Current position after the end of record\";\n  *size -= pos_before;\n  if (ABSL_PREDICT_FALSE(num_records_ == kMaxNumRecords)) {\n    return Fail(absl::ResourceExhaustedError(\"Too many records\"));\n  }\n  if (ABSL_PREDICT_FALSE(*size > std::numeric_limits<uint64_t>::max() -\n                                     decoded_data_size_)) {\n    return Fail(absl::ResourceExhaustedError(\"Decoded data size too large\"));\n  }\n  ++num_records_;\n  decoded_data_size_ += IntCast<uint64_t>(*size);\n  const bool is_proto = IsProtoMessage(record);\n  RIEGELI_EVAL_ASSERT(record.Seek(pos_before)) << record.status();\n  if (is_proto) {\n    encoded_tags_.push_back(GetPosInTagsList(\n        GetNode(NodeId(chunk_encoding_internal::MessageId::kStartOfMessage, 0)),\n        chunk_encoding_internal::Subtype::kTrivial));\n    return AddMessage(record);\n  } else {\n    Node* node =\n        GetNode(NodeId(chunk_encoding_internal::MessageId::kNonProto, 0));\n    encoded_tags_.push_back(\n        GetPosInTagsList(node, chunk_encoding_internal::Subtype::kTrivial));\n    BackwardWriter* const buffer = GetBuffer(node, BufferType::kNonProto);\n    if (ABSL_PREDICT_FALSE(!record.Copy(IntCast<size_t>(*size), *buffer))) {\n      return Fail(buffer->status());\n    }\n    if (ABSL_PREDICT_FALSE(!WriteVarint64(IntCast<uint64_t>(*size),\n                                          nonproto_lengths_writer_))) {\n      return Fail(nonproto_lengths_writer_.status());\n    }\n    return true;\n  }\n}\n\ninline BackwardWriter* TransposeEncoder::GetBuffer(Node* node,\n                                                   BufferType type) {\n  if (!node->second.writer) {\n    std::vector<BufferWithMetadata>& buffers =\n        data_[static_cast<uint32_t>(type)];\n    buffers.emplace_back(node->first);\n    node->second.writer =\n        std::make_unique<ChainBackwardWriter<>>(buffers.back().buffer.get());\n  }\n  return node->second.writer.get();\n}\n\ninline uint32_t TransposeEncoder::GetPosInTagsList(\n    Node* node, chunk_encoding_internal::Subtype subtype) {\n  size_t pos = static_cast<size_t>(subtype);\n  if (node->second.encoded_tag_pos.size() <= pos) {\n    node->second.encoded_tag_pos.resize(pos + 1,\n                                        std::numeric_limits<uint32_t>::max());\n  }\n  uint32_t* ret = &node->second.encoded_tag_pos[pos];\n  if (*ret == std::numeric_limits<uint32_t>::max()) {\n    *ret = tags_list_.size();\n    tags_list_.emplace_back(node->first, subtype);\n  }\n  return *ret;\n}\n\ninline TransposeEncoder::Node* TransposeEncoder::GetNode(NodeId node_id) {\n  const std::pair<absl::flat_hash_map<NodeId, MessageNode>::iterator, bool>\n      insert_result = message_nodes_.try_emplace(node_id, next_message_id_);\n  if (insert_result.second) ++next_message_id_;\n  return &*insert_result.first;\n}\n\n// Precondition: `IsProtoMessage` returns `true` for this record.\n// Note: Encoded tags are appended into `encoded_tags_` but data is prepended\n// into respective buffers. `encoded_tags_` will be later traversed backwards.\ninline bool TransposeEncoder::AddMessage(Reader& record) {\n  chunk_encoding_internal::MessageId parent_message_id =\n      chunk_encoding_internal::MessageId::kRoot;\n  LimitingReader<> limited_record(&record);\n  for (;;) {\n    while (limited_record.Pull()) {\n      uint32_t tag;\n      RIEGELI_EVAL_ASSERT(ReadVarint32(limited_record, tag))\n          << \"Invalid tag: \" << limited_record.status();\n      Node* node = GetNode(NodeId(parent_message_id, tag));\n      switch (GetTagWireType(tag)) {\n        case WireType::kVarint: {\n          // Storing value as `uint64_t[2]` instead of `uint8_t[10]` lets Clang\n          // and GCC generate better code for clearing high bit of each byte.\n          uint64_t value[2];\n          static_assert(sizeof(value) >= kMaxLengthVarint64,\n                        \"value too small to hold a varint64\");\n          const size_t value_length =\n              CopyVarint64(limited_record, reinterpret_cast<char*>(value));\n          RIEGELI_ASSERT(value_length > 0)\n              << \"Invalid varint: \" << limited_record.status();\n          if (reinterpret_cast<const unsigned char*>(value)[0] <=\n              kMaxVarintInline) {\n            encoded_tags_.push_back(GetPosInTagsList(\n                node, chunk_encoding_internal::Subtype::kVarintInline0 +\n                          reinterpret_cast<const unsigned char*>(value)[0]));\n          } else {\n            encoded_tags_.push_back(GetPosInTagsList(\n                node, chunk_encoding_internal::Subtype::kVarint1 +\n                          IntCast<uint8_t>(value_length - 1)));\n            // Clear high bit of each byte.\n            for (uint64_t& word : value) word &= ~uint64_t{0x8080808080808080};\n            BackwardWriter* const buffer = GetBuffer(node, BufferType::kVarint);\n            if (ABSL_PREDICT_FALSE(!buffer->Write(absl::string_view(\n                    reinterpret_cast<const char*>(value), value_length)))) {\n              return Fail(buffer->status());\n            }\n          }\n        } break;\n        case WireType::kFixed32: {\n          encoded_tags_.push_back(GetPosInTagsList(\n              node, chunk_encoding_internal::Subtype::kTrivial));\n          BackwardWriter* const buffer = GetBuffer(node, BufferType::kFixed32);\n          if (ABSL_PREDICT_FALSE(\n                  !limited_record.Copy(sizeof(uint32_t), *buffer))) {\n            return Fail(buffer->status());\n          }\n        } break;\n        case WireType::kFixed64: {\n          encoded_tags_.push_back(GetPosInTagsList(\n              node, chunk_encoding_internal::Subtype::kTrivial));\n          BackwardWriter* const buffer = GetBuffer(node, BufferType::kFixed64);\n          if (ABSL_PREDICT_FALSE(\n                  !limited_record.Copy(sizeof(uint64_t), *buffer))) {\n            return Fail(buffer->status());\n          }\n        } break;\n        case WireType::kLengthDelimited: {\n          const Position length_pos = limited_record.pos();\n          uint32_t length;\n          RIEGELI_EVAL_ASSERT(ReadVarint32(limited_record, length))\n              << \"Invalid length: \" << limited_record.status();\n          const Position value_pos = limited_record.pos();\n          const Position parent_max_record_pos = limited_record.max_pos();\n          limited_record.set_max_length(length);\n          // Non-toplevel empty strings are treated as strings, not messages.\n          // They have a simpler encoding this way (one node instead of two).\n          if (length > 0 &&\n              ABSL_PREDICT_TRUE(message_stack_.size() + group_stack_.size() <\n                                kMaxRecursionDepth) &&\n              IsProtoMessage(limited_record)) {\n            encoded_tags_.push_back(\n                GetPosInTagsList(node, chunk_encoding_internal::Subtype::\n                                           kLengthDelimitedStartOfSubmessage));\n            RIEGELI_EVAL_ASSERT(limited_record.Seek(value_pos))\n                << limited_record.status();\n            const uint32_t end_of_submessage_pos =\n                GetPosInTagsList(node, chunk_encoding_internal::Subtype::\n                                           kLengthDelimitedEndOfSubmessage);\n#if __cpp_aggregate_paren_init\n            message_stack_.emplace_back(end_of_submessage_pos,\n                                        parent_message_id,\n                                        IntCast<size_t>(parent_max_record_pos));\n#else\n            message_stack_.push_back(\n                MessageFrame{end_of_submessage_pos, parent_message_id,\n                             IntCast<size_t>(parent_max_record_pos)});\n#endif\n            parent_message_id = node->second.message_id;\n            continue;\n          }\n          encoded_tags_.push_back(GetPosInTagsList(\n              node, chunk_encoding_internal::Subtype::kLengthDelimitedString));\n          RIEGELI_EVAL_ASSERT(limited_record.Seek(length_pos))\n              << limited_record.status();\n          BackwardWriter* const buffer = GetBuffer(node, BufferType::kString);\n          if (ABSL_PREDICT_FALSE(!limited_record.Copy(\n                  IntCast<size_t>(value_pos - length_pos) + length, *buffer))) {\n            return Fail(buffer->status());\n          }\n          limited_record.set_max_pos(parent_max_record_pos);\n        } break;\n        case WireType::kStartGroup: {\n          encoded_tags_.push_back(GetPosInTagsList(\n              node, chunk_encoding_internal::Subtype::kTrivial));\n          group_stack_.push_back(parent_message_id);\n          parent_message_id = node->second.message_id;\n        } break;\n        case WireType::kEndGroup:\n          parent_message_id = group_stack_.back();\n          group_stack_.pop_back();\n          // Note that `parent_message_id` was updated above so the `node` does\n          // not belong to `(parent_message_id, tag)` as in all the other cases.\n          // But we don't reload `node` because this still works. All we need is\n          // some unique consistent node.\n          encoded_tags_.push_back(GetPosInTagsList(\n              node, chunk_encoding_internal::Subtype::kTrivial));\n          break;\n        default:\n          RIEGELI_ASSUME_UNREACHABLE()\n              << \"Invalid wire type: \"\n              << static_cast<uint32_t>(GetTagWireType(tag));\n      }\n    }\n    RIEGELI_ASSERT_OK(limited_record);\n    if (message_stack_.empty()) return true;\n    const MessageFrame& message_frame = message_stack_.back();\n    encoded_tags_.push_back(message_frame.end_of_submessage_pos);\n    parent_message_id = message_frame.parent_message_id;\n    limited_record.set_max_pos(message_frame.parent_max_record_pos);\n    message_stack_.pop_back();\n  }\n}\n\ninline bool TransposeEncoder::AddBuffer(\n    std::optional<size_t> new_uncompressed_bucket_size, const Chain& buffer,\n    chunk_encoding_internal::Compressor& bucket_compressor, Writer& data_writer,\n    std::vector<size_t>& compressed_bucket_sizes,\n    std::vector<size_t>& buffer_sizes) {\n  buffer_sizes.push_back(buffer.size());\n  if (new_uncompressed_bucket_size != std::nullopt) {\n    if (bucket_compressor.writer().pos() > 0) {\n      const Position pos_before = data_writer.pos();\n      if (ABSL_PREDICT_FALSE(!bucket_compressor.EncodeAndClose(data_writer))) {\n        return Fail(bucket_compressor.status());\n      }\n      RIEGELI_ASSERT_GE(data_writer.pos(), pos_before)\n          << \"Data writer position decreased\";\n      compressed_bucket_sizes.push_back(\n          IntCast<size_t>(data_writer.pos() - pos_before));\n    }\n    bucket_compressor.Clear(\n        chunk_encoding_internal::Compressor::TuningOptions()\n            .set_pledged_size(*new_uncompressed_bucket_size)\n            .set_recycling_pool_options(recycling_pool_options_));\n  }\n  if (ABSL_PREDICT_FALSE(!bucket_compressor.writer().Write(buffer))) {\n    return Fail(bucket_compressor.writer().status());\n  }\n  return true;\n}\n\ninline bool TransposeEncoder::WriteBuffers(\n    Writer& header_writer, Writer& data_writer,\n    absl::flat_hash_map<NodeId, uint32_t>* buffer_pos) {\n  size_t num_buffers = 0;\n  for (std::vector<BufferWithMetadata>& buffers : data_) {\n    // Sort buffers by length, smallest to largest.\n    std::sort(\n        buffers.begin(), buffers.end(),\n        [](const BufferWithMetadata& a, const BufferWithMetadata& b) {\n          if (a.buffer->size() != b.buffer->size()) {\n            return a.buffer->size() < b.buffer->size();\n          }\n          if (a.node_id.parent_message_id != b.node_id.parent_message_id) {\n            return a.node_id.parent_message_id < b.node_id.parent_message_id;\n          }\n          return a.node_id.tag < b.node_id.tag;\n        });\n    num_buffers += buffers.size();\n  }\n  const Chain& nonproto_lengths = nonproto_lengths_writer_.dest();\n  if (!nonproto_lengths.empty()) ++num_buffers;\n\n  std::vector<size_t> compressed_bucket_sizes;\n  std::vector<size_t> buffer_sizes;\n  buffer_sizes.reserve(num_buffers);\n\n  chunk_encoding_internal::Compressor bucket_compressor(\n      compressor_options_,\n      chunk_encoding_internal::Compressor::TuningOptions()\n          .set_recycling_pool_options(recycling_pool_options_));\n  for (absl::Span<const BufferWithMetadata> buffers : data_) {\n    // Split data into buckets.\n    size_t remaining_buffers_size = 0;\n    for (const BufferWithMetadata& buffer : buffers) {\n      remaining_buffers_size += buffer.buffer->size();\n    }\n\n    std::vector<size_t> uncompressed_bucket_sizes;\n    size_t current_bucket_size = 0;\n    for (absl::Span<const BufferWithMetadata>::const_reverse_iterator iter =\n             buffers.crbegin();\n         iter != buffers.crend(); ++iter) {\n      const size_t current_buffer_size = iter->buffer->size();\n      if (current_bucket_size > 0 &&\n          current_bucket_size + current_buffer_size / 2 >= bucket_size_) {\n        uncompressed_bucket_sizes.push_back(current_bucket_size);\n        current_bucket_size = 0;\n      }\n      current_bucket_size += current_buffer_size;\n      remaining_buffers_size -= current_buffer_size;\n      if (remaining_buffers_size <= bucket_size_ / 2) {\n        current_bucket_size += remaining_buffers_size;\n        break;\n      }\n    }\n    if (current_bucket_size > 0) {\n      uncompressed_bucket_sizes.push_back(current_bucket_size);\n    }\n\n    current_bucket_size = 0;\n    for (const BufferWithMetadata& buffer : buffers) {\n      std::optional<size_t> new_uncompressed_bucket_size;\n      if (current_bucket_size == 0) {\n        RIEGELI_ASSERT(!uncompressed_bucket_sizes.empty())\n            << \"Bucket sizes and buffer sizes do not match\";\n        current_bucket_size = uncompressed_bucket_sizes.back();\n        uncompressed_bucket_sizes.pop_back();\n        new_uncompressed_bucket_size = current_bucket_size;\n      }\n      RIEGELI_ASSERT_GE(current_bucket_size, buffer.buffer->size())\n          << \"Bucket sizes and buffer sizes do not match\";\n      current_bucket_size -= buffer.buffer->size();\n      if (ABSL_PREDICT_FALSE(!AddBuffer(\n              new_uncompressed_bucket_size, *buffer.buffer, bucket_compressor,\n              data_writer, compressed_bucket_sizes, buffer_sizes))) {\n        return false;\n      }\n      const std::pair<absl::flat_hash_map<NodeId, uint32_t>::iterator, bool>\n          insert_result = buffer_pos->emplace(\n              buffer.node_id, IntCast<uint32_t>(buffer_pos->size()));\n      RIEGELI_ASSERT(insert_result.second)\n          << \"Field already has buffer assigned: \"\n          << static_cast<uint32_t>(buffer.node_id.parent_message_id) << \"/\"\n          << buffer.node_id.tag;\n    }\n    RIEGELI_ASSERT(uncompressed_bucket_sizes.empty())\n        << \"Bucket sizes and buffer sizes do not match\";\n    RIEGELI_ASSERT_EQ(current_bucket_size, 0u)\n        << \"Bucket sizes and buffer sizes do not match\";\n  }\n  if (!nonproto_lengths.empty()) {\n    // `nonproto_lengths` is the last buffer if non-empty.\n    if (ABSL_PREDICT_FALSE(!AddBuffer(nonproto_lengths.size(), nonproto_lengths,\n                                      bucket_compressor, data_writer,\n                                      compressed_bucket_sizes, buffer_sizes))) {\n      return false;\n    }\n    // Note: `nonproto_lengths` needs no `buffer_pos`.\n  }\n\n  if (bucket_compressor.writer().pos() > 0) {\n    // Last bucket.\n    const Position pos_before = data_writer.pos();\n    if (ABSL_PREDICT_FALSE(!bucket_compressor.EncodeAndClose(data_writer))) {\n      return Fail(bucket_compressor.status());\n    }\n    RIEGELI_ASSERT_GE(data_writer.pos(), pos_before)\n        << \"Data writer position decreased\";\n    compressed_bucket_sizes.push_back(\n        IntCast<size_t>(data_writer.pos() - pos_before));\n  }\n\n  if (ABSL_PREDICT_FALSE(!WriteVarint32(\n          IntCast<uint32_t>(compressed_bucket_sizes.size()), header_writer)) ||\n      ABSL_PREDICT_FALSE(!WriteVarint32(IntCast<uint32_t>(buffer_sizes.size()),\n                                        header_writer))) {\n    return Fail(header_writer.status());\n  }\n  for (const size_t length : compressed_bucket_sizes) {\n    if (ABSL_PREDICT_FALSE(\n            !WriteVarint64(IntCast<uint64_t>(length), header_writer))) {\n      return Fail(header_writer.status());\n    }\n  }\n  for (const size_t length : buffer_sizes) {\n    if (ABSL_PREDICT_FALSE(\n            !WriteVarint64(IntCast<uint64_t>(length), header_writer))) {\n      return Fail(header_writer.status());\n    }\n  }\n  return true;\n}\n\ninline bool TransposeEncoder::WriteStatesAndData(\n    uint32_t max_transition, absl::Span<const StateInfo> state_machine,\n    Writer& header_writer, Writer& data_writer) {\n  if (!encoded_tags_.empty() &&\n      tags_list_[encoded_tags_[0]].dest_info.size() == 1) {\n    // There should be no implicit transition from the last state. If there was\n    // one, then it would not be obvious whether to stop or continue decoding.\n    // Only if transition is explicit we check whether there is more transition\n    // bytes.\n    absl::flat_hash_map<uint32_t, DestInfo>& dest_info =\n        tags_list_[encoded_tags_[0]].dest_info;\n    const uint32_t first_key = dest_info.begin()->first;\n    dest_info[first_key + 1];\n    RIEGELI_ASSERT_NE(tags_list_[encoded_tags_[0]].dest_info.size(), 1u)\n        << \"Number of transitions from the last state did not increase\";\n  }\n  absl::flat_hash_map<NodeId, uint32_t> buffer_pos;\n  if (ABSL_PREDICT_FALSE(\n          !WriteBuffers(header_writer, data_writer, &buffer_pos))) {\n    return false;\n  }\n\n  std::string subtype_to_write;\n  std::vector<uint32_t> buffer_index_to_write;\n  std::vector<uint32_t> base_to_write;\n\n  base_to_write.reserve(state_machine.size());\n\n  if (ABSL_PREDICT_FALSE(!WriteVarint32(IntCast<uint32_t>(state_machine.size()),\n                                        header_writer))) {\n    return Fail(header_writer.status());\n  }\n  for (const StateInfo state_info : state_machine) {\n    if (state_info.etag_index == kInvalidPos) {\n      // `kNoOp` state.\n      if (ABSL_PREDICT_FALSE(!WriteVarint32(\n              static_cast<uint32_t>(chunk_encoding_internal::MessageId::kNoOp),\n              header_writer))) {\n        return Fail(header_writer.status());\n      }\n      base_to_write.push_back(state_info.base);\n      continue;\n    }\n    const EncodedTagInfo& etag_info = tags_list_[state_info.etag_index];\n    NodeId node_id = etag_info.node_id;\n    chunk_encoding_internal::Subtype subtype = etag_info.subtype;\n    if (node_id.tag != 0) {\n      const bool is_string =\n          GetTagWireType(node_id.tag) == WireType::kLengthDelimited;\n      if (is_string && subtype == chunk_encoding_internal::Subtype::\n                                      kLengthDelimitedStartOfSubmessage) {\n        if (ABSL_PREDICT_FALSE(!WriteVarint32(\n                static_cast<uint32_t>(\n                    chunk_encoding_internal::MessageId::kStartOfSubmessage),\n                header_writer))) {\n          return Fail(header_writer.status());\n        }\n      } else if (is_string && subtype == chunk_encoding_internal::Subtype::\n                                             kLengthDelimitedEndOfSubmessage) {\n        // End of submessage is encoded as `kSubmessageWireType` instead of\n        // `WireType::kLengthDelimited`.\n        if (ABSL_PREDICT_FALSE(!WriteVarint32(\n                node_id.tag +\n                    (static_cast<uint32_t>(\n                         chunk_encoding_internal::kSubmessageWireType) -\n                     static_cast<uint32_t>(WireType::kLengthDelimited)),\n                header_writer))) {\n          return Fail(header_writer.status());\n        }\n      } else {\n        if (ABSL_PREDICT_FALSE(!WriteVarint32(node_id.tag, header_writer))) {\n          return Fail(header_writer.status());\n        }\n        if (chunk_encoding_internal::HasSubtype(node_id.tag)) {\n          subtype_to_write.push_back(static_cast<char>(subtype));\n        }\n        if (chunk_encoding_internal::HasDataBuffer(node_id.tag, subtype)) {\n          const absl::flat_hash_map<NodeId, uint32_t>::const_iterator iter =\n              buffer_pos.find(NodeId(node_id.parent_message_id, node_id.tag));\n          RIEGELI_ASSERT(iter != buffer_pos.end())\n              << \"Buffer not found: \"\n              << static_cast<uint32_t>(node_id.parent_message_id) << \"/\"\n              << node_id.tag;\n          buffer_index_to_write.push_back(iter->second);\n        }\n      }\n    } else {\n      // `kNonProto` and `kStartOfMessage` special IDs.\n      if (ABSL_PREDICT_FALSE(\n              !WriteVarint32(static_cast<uint32_t>(node_id.parent_message_id),\n                             header_writer))) {\n        return Fail(header_writer.status());\n      }\n      if (node_id.parent_message_id ==\n          chunk_encoding_internal::MessageId::kNonProto) {\n        // `kNonProto` has data buffer.\n        const absl::flat_hash_map<NodeId, uint32_t>::const_iterator iter =\n            buffer_pos.find(\n                NodeId(chunk_encoding_internal::MessageId::kNonProto, 0));\n        RIEGELI_ASSERT(iter != buffer_pos.end())\n            << \"Buffer of non-proto records not found\";\n        buffer_index_to_write.push_back(iter->second);\n      } else {\n        RIEGELI_ASSERT_EQ(\n            static_cast<uint32_t>(node_id.parent_message_id),\n            static_cast<uint32_t>(\n                chunk_encoding_internal::MessageId::kStartOfMessage))\n            << \"Unexpected message ID with no tag\";\n      }\n    }\n    if (tags_list_[state_info.etag_index].base != kInvalidPos) {\n      // Signal implicit transition by adding `state_machine.size()`.\n      base_to_write.push_back(\n          tags_list_[state_info.etag_index].base +\n          (tags_list_[state_info.etag_index].dest_info.size() == 1\n               ? IntCast<uint32_t>(state_machine.size())\n               : uint32_t{0}));\n    } else {\n      // If there is no outgoing transition from this state, just output zero.\n      base_to_write.push_back(0);\n    }\n  }\n  for (const uint32_t value : base_to_write) {\n    if (ABSL_PREDICT_FALSE(!WriteVarint32(value, header_writer))) {\n      return Fail(header_writer.status());\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!header_writer.Write(std::move(subtype_to_write)))) {\n    return Fail(header_writer.status());\n  }\n  for (const uint32_t value : buffer_index_to_write) {\n    if (ABSL_PREDICT_FALSE(!WriteVarint32(value, header_writer))) {\n      return Fail(header_writer.status());\n    }\n  }\n\n  // Find the smallest index that has first tag.\n  // Note: `encoded_tags_` is stored in reverse order so we look for the last\n  // element of `encoded_tags_`.\n  uint32_t first_tag_pos = 0;\n  if (!encoded_tags_.empty()) {\n    while (state_machine[first_tag_pos].etag_index != encoded_tags_.back()) {\n      ++first_tag_pos;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!WriteVarint32(first_tag_pos, header_writer))) {\n    return Fail(header_writer.status());\n  }\n\n  chunk_encoding_internal::Compressor transitions_compressor(\n      compressor_options_,\n      chunk_encoding_internal::Compressor::TuningOptions()\n          .set_recycling_pool_options(recycling_pool_options_));\n  if (ABSL_PREDICT_FALSE(!WriteTransitions(max_transition, state_machine,\n                                           transitions_compressor.writer()))) {\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(!transitions_compressor.EncodeAndClose(data_writer))) {\n    return Fail(transitions_compressor.status());\n  }\n  return true;\n}\n\ninline bool TransposeEncoder::WriteTransitions(\n    uint32_t max_transition, absl::Span<const StateInfo> state_machine,\n    Writer& transitions_writer) {\n  if (encoded_tags_.empty()) return true;\n  uint32_t prev_etag = encoded_tags_.back();\n  uint32_t current_base = tags_list_[prev_etag].base;\n  // Assuming an approximately balanced tree of `kNoOp` nodes covering\n  // transitions from the given node in the state machine, the maximum number of\n  // bytes needed to encode one transition should be the depth of the tree, i.e.\n  // `O(log_max_transition(state_machine_size))`. We allocate buffer of size\n  // `kWriteBufSize` to store the entire encoded transition.\n  // For experiments with low `max_transition` we use much larger buffer then\n  // needed for optimal `max_transition == 63`.\n  constexpr size_t kWriteBufSize = 32;\n  uint8_t write[kWriteBufSize];\n  std::optional<uint8_t> last_transition;\n  // Go through all transitions and encode them.\n  for (uint32_t i = IntCast<uint32_t>(encoded_tags_.size() - 1); i > 0; --i) {\n    // There are multiple options how transition may be encoded:\n    // 1. Transition is common and it's in the private list for the previous\n    //    node.\n    // 2. Transition is common and is served from public list. This can have two\n    //    forms:\n    //      a) Previous node has no private list so we simply serve the\n    //         transition using the public node list.\n    //      b) Node has private list so we first make a `kNoOp` transition to\n    //         the public list and then continue as above.\n    uint32_t tag = encoded_tags_[i - 1];\n    // Check whether this is implicit transition.\n    if (tags_list_[prev_etag].dest_info.size() != 1) {\n      // Position in the private list.\n      uint32_t pos = tags_list_[prev_etag].dest_info[tag].pos;\n      if (pos == kInvalidPos) {\n        // `pos` is not in the private list, go to `public_list_noop_pos` if\n        // available.\n        // Otherwise base is already in the public list (option 2a).\n        pos = tags_list_[prev_etag].public_list_noop_pos;\n        if (pos != kInvalidPos) {\n          // Option 2b.\n          const uint32_t orig_pos = pos;\n          size_t write_start = kWriteBufSize;\n          // Encode transition from `current_base` to `public_list_noop_pos`\n          // which is a `kNoOp` that would lead us to the public list.\n          while (current_base > pos || pos - current_base > max_transition) {\n            // While desired pos is not reachable using one transition, move to\n            // `canonical_source`.\n            const uint32_t cs = state_machine[pos].canonical_source;\n            RIEGELI_ASSERT_LT(cs, state_machine.size())\n                << \"Canonical source out of range: \" << pos;\n            RIEGELI_ASSERT_LE(state_machine[cs].base, pos)\n                << \"Position unreachable from its base: \" << pos;\n            RIEGELI_ASSERT_LE(pos - state_machine[cs].base, max_transition)\n                << \"Position unreachable from its base: \" << pos;\n            RIEGELI_ASSERT_NE(write_start, 0u) << \"Write buffer overflow\";\n            write[--write_start] =\n                IntCast<uint8_t>(pos - state_machine[cs].base);\n            pos = cs;\n          }\n          RIEGELI_ASSERT_NE(write_start, 0u) << \"Write buffer overflow\";\n          write[--write_start] = IntCast<uint8_t>(pos - current_base);\n\n          for (size_t j = write_start; j < kWriteBufSize; ++j) {\n            if (write[j] == 0 && last_transition != std::nullopt &&\n                (*last_transition & 3) < 3) {\n              ++*last_transition;\n            } else {\n              if (last_transition != std::nullopt) {\n                if (ABSL_PREDICT_FALSE(\n                        !transitions_writer.WriteByte(*last_transition))) {\n                  return Fail(transitions_writer.status());\n                }\n              }\n              last_transition = IntCast<uint8_t>(write[j] << 2);\n            }\n          }\n          // `current_base` is the base of the `kNoOp` that we reached using the\n          // transitions so far.\n          current_base = state_machine[orig_pos].base;\n        }\n        // `pos` becomes the position of the state in the public list.\n        pos = tags_list_[tag].state_machine_pos;\n      }\n      RIEGELI_ASSERT_NE(current_base, kInvalidPos)\n          << \"No outgoing transition from current base\";\n      RIEGELI_ASSERT_LT(pos, state_machine.size()) << \"Position out of range\";\n      size_t write_start = kWriteBufSize;\n      // Encode transition from `current_base` to `pos`.\n      while (current_base > pos || pos - current_base > max_transition) {\n        // While desired pos is not reachable using one transition, move to\n        // `canonical_source`.\n        const uint32_t cs = state_machine[pos].canonical_source;\n        RIEGELI_ASSERT_LT(cs, state_machine.size())\n            << \"Canonical source out of range: \" << pos;\n        RIEGELI_ASSERT_LE(state_machine[cs].base, pos)\n            << \"Position unreachable from its base: \" << pos;\n        RIEGELI_ASSERT_LE(pos - state_machine[cs].base, max_transition)\n            << \"Position unreachable from its base: \" << pos;\n        RIEGELI_ASSERT_NE(write_start, 0u) << \"Write buffer overflow\";\n        write[--write_start] = IntCast<uint8_t>(pos - state_machine[cs].base);\n        pos = cs;\n      }\n      RIEGELI_ASSERT_NE(write_start, 0u) << \"Write buffer overflow\";\n      write[--write_start] = IntCast<uint8_t>(pos - current_base);\n      for (size_t j = write_start; j < kWriteBufSize; ++j) {\n        if (write[j] == 0 && last_transition != std::nullopt &&\n            (*last_transition & 3) < 3) {\n          ++*last_transition;\n        } else {\n          if (last_transition != std::nullopt) {\n            if (ABSL_PREDICT_FALSE(\n                    !transitions_writer.WriteByte(*last_transition))) {\n              return Fail(transitions_writer.status());\n            }\n          }\n          last_transition = IntCast<uint8_t>(write[j] << 2);\n        }\n      }\n    } else {\n      RIEGELI_ASSERT_EQ(state_machine[tags_list_[prev_etag].base].etag_index,\n                        tag)\n          << \"Implicit transition goes to a wrong tag\";\n    }\n    prev_etag = tag;\n    current_base = tags_list_[prev_etag].base;\n  }\n  if (last_transition != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(!transitions_writer.WriteByte(*last_transition))) {\n      return Fail(transitions_writer.status());\n    }\n  }\n  return true;\n}\n\ninline void TransposeEncoder::CollectTransitionStatistics() {\n  // Go through all the transitions from back to front and collect transition\n  // distribution statistics.\n  uint32_t prev_pos = encoded_tags_.back();\n  for (size_t i = encoded_tags_.size() - 1; i > 0; --i) {\n    const uint32_t pos = encoded_tags_[i - 1];\n    ++tags_list_[prev_pos].dest_info[pos].num_transitions;\n    ++tags_list_[pos].num_incoming_transitions;\n    prev_pos = pos;\n  }\n\n  if (tags_list_[encoded_tags_.back()].num_incoming_transitions == 0) {\n    // This guarantees that the initial state is created even if it has no other\n    // incoming transition.\n    tags_list_[encoded_tags_.back()].num_incoming_transitions = 1;\n  }\n}\n\ninline void TransposeEncoder::ComputeBaseIndices(\n    uint32_t max_transition, uint32_t public_list_base,\n    absl::Span<const std::pair<uint32_t, uint32_t>> public_list_noops,\n    std::vector<StateInfo>& state_machine) {\n  // The related transitions reach a state in the public list so the valid\n  // approach would be to simply set all of these to `public_list_base`.\n  // However, we observe that most of the tags only target few destinations so\n  // we can do better if we find the base that is closer to reachable states.\n  //\n  // We do this by computing `base` of the block that can reach all required\n  // destination and `min_pos` of the state that is used in any such transition.\n\n  // Compute `base` indices for `kNoOp` states.\n  for (const std::pair<uint32_t, uint32_t>& tag_index_and_state_index :\n       public_list_noops) {\n    // Start of block that can reach all required destinations.\n    uint32_t base = kInvalidPos;\n    // Smallest position of node used in transition.\n    uint32_t min_pos = kInvalidPos;\n    for (const std::pair<const uint32_t, DestInfo>& dest_info :\n         tags_list_[tag_index_and_state_index.first].dest_info) {\n      uint32_t pos = dest_info.second.pos;\n      if (pos != kInvalidPos) {\n        // This tag has a node in the private list.\n        continue;\n      }\n      // Position of the state that we need to reach.\n      pos = tags_list_[dest_info.first].state_machine_pos;\n      RIEGELI_ASSERT_NE(pos, kInvalidPos) << \"Invalid position\";\n      // Assuming we processed some states already and `base` is already set to\n      // non-`kInvalidPos` we find the base of the block that is the common\n      // ancestor for both `pos` and current `base`.\n      // If `base <= pos && pos - base <= max_transition` then `pos` can\n      // be encoded from `base` using one byte and `base` starts the block we\n      // are looking for. If this is not the case then either:\n      //  - `base > pos` and `pos` is reachable from one of the common ancestor\n      //    blocks of `base` and `pos`. In that case we move `base` to the\n      //    parent block of `base`.\n      //  - `pos - base > max_transition` and to reach `pos` we need more than\n      //    one transition. In that case we ensure the reachability of `pos` by\n      //    ensuring reachability of its `canonical_source` which belongs to the\n      //    parent block of `pos`.\n      // Note: We assume that transitions in the public list always go from\n      // lower to higher indices. This is ensured by the public list generation\n      // code.\n      // Note: If `base` is `kInvalidPos`, the condition `base > pos` is true\n      // and we handle the first state in there.\n      while (base > pos || pos - base > max_transition) {\n        if (base > pos) {\n          // `cs` is the `canonical_source` that leads to the block we are\n          // looking for.\n          uint32_t cs;\n          if (base == kInvalidPos) {\n            // `base` not initialized yet. We'll use canonical_source of `pos`.\n            cs = state_machine[pos].canonical_source;\n          } else {\n            // Set `cs` to the `kNoOp` that leads to `base`.\n            cs = state_machine[base].canonical_source;\n            // If `cs` is `kInvalidPos` then `base` was already in the first\n            // block. But then `base > pos` can't be true.\n            RIEGELI_ASSERT_NE(cs, kInvalidPos) << \"Unreachable base: \" << base;\n            // Transitions to previously added states will use `cs` so we update\n            // `min_pos`.\n            min_pos = UnsignedMin(min_pos, cs);\n            // To find `base` the block that contains `cs` we move one level\n            // above.\n            cs = state_machine[cs].canonical_source;\n          }\n          if (cs == kInvalidPos) {\n            // No `canonical_source` means `base` is in the first block.\n            base = public_list_base;\n          } else {\n            // Otherwise it's the base of the current `cs`.\n            base = state_machine[cs].base;\n          }\n        } else {\n          // Update `pos` to `canonical_source` of `pos`.\n          const uint32_t cs = state_machine[pos].canonical_source;\n          RIEGELI_ASSERT_LT(cs, state_machine.size())\n              << \"Canonical source out of range: \" << pos;\n          RIEGELI_ASSERT_LE(state_machine[cs].base, pos)\n              << \"Position unreachable from its base: \" << pos;\n          RIEGELI_ASSERT_LE(pos - state_machine[cs].base, max_transition)\n              << \"Position unreachable from its base: \" << pos;\n          pos = cs;\n        }\n      }\n      min_pos = UnsignedMin(min_pos, pos);\n    }\n    RIEGELI_ASSERT_NE(min_pos, kInvalidPos)\n        << \"No outgoing transition from a public NoOp\";\n    state_machine[tag_index_and_state_index.second].base = min_pos;\n  }\n\n  // The same as above for tags without private list.\n  for (EncodedTagInfo& tag : tags_list_) {\n    if (tag.base != kInvalidPos) {\n      // Skip tags with private list.\n      continue;\n    }\n    uint32_t base = kInvalidPos;\n    uint32_t min_pos = kInvalidPos;\n    for (const std::pair<const uint32_t, DestInfo>& dest_info : tag.dest_info) {\n      uint32_t pos = dest_info.second.pos;\n      if (pos != kInvalidPos) {\n        // Skip destinations in the private list.\n        continue;\n      }\n      pos = tags_list_[dest_info.first].state_machine_pos;\n      RIEGELI_ASSERT_NE(pos, kInvalidPos) << \"Invalid position\";\n      while (base > pos || pos - base > max_transition) {\n        if (base > pos) {\n          uint32_t cs;\n          if (base == kInvalidPos) {\n            cs = state_machine[pos].canonical_source;\n          } else {\n            cs = state_machine[base].canonical_source;\n            RIEGELI_ASSERT_NE(cs, kInvalidPos) << \"Unreachable base: \" << base;\n            min_pos = UnsignedMin(min_pos, cs);\n            cs = state_machine[cs].canonical_source;\n          }\n          if (cs == kInvalidPos) {\n            base = public_list_base;\n          } else {\n            base = state_machine[cs].base;\n          }\n        } else {\n          const uint32_t cs = state_machine[pos].canonical_source;\n          RIEGELI_ASSERT_LT(cs, state_machine.size())\n              << \"Canonical source out of range: \" << pos;\n          RIEGELI_ASSERT_LE(state_machine[cs].base, pos)\n              << \"Position unreachable from its base: \" << pos;\n          RIEGELI_ASSERT_LE(pos - state_machine[cs].base, max_transition)\n              << \"Position unreachable from its base: \" << pos;\n          pos = cs;\n        }\n      }\n      min_pos = UnsignedMin(min_pos, pos);\n    }\n    if (min_pos != kInvalidPos) tag.base = min_pos;\n  }\n}\n\ninline std::vector<TransposeEncoder::StateInfo>\nTransposeEncoder::CreateStateMachine(uint32_t max_transition,\n                                     uint32_t min_count_for_state) {\n  std::vector<StateInfo> state_machine;\n  if (encoded_tags_.empty()) {\n    state_machine.emplace_back(kInvalidPos, 0);\n    return state_machine;\n  }\n\n  CollectTransitionStatistics();\n\n  // Go through all the tag infos and update transitions that will be included\n  // in the private list for the node.\n  constexpr uint32_t kInListPos = 0;\n  for (EncodedTagInfo& tag_info : tags_list_) {\n    for (std::pair<const uint32_t, DestInfo>& dest_and_count :\n         tag_info.dest_info) {\n      if (dest_and_count.second.num_transitions >= min_count_for_state) {\n        // Subtract transitions so we have the right estimate of the remaining\n        // transitions into each node.\n        tags_list_[dest_and_count.first].num_incoming_transitions -=\n            dest_and_count.second.num_transitions;\n        // Mark transition to be included in list.\n        dest_and_count.second.pos = kInListPos;\n      }\n    }\n  }\n\n  // Priority_queue to order nodes by transition count.\n  std::priority_queue<PriorityQueueEntry> tag_priority;\n  // Pair of `(tag_index, noop_position)` where `noop_position` is the index of\n  // the `kNoOp` state created for this tag that has base index in the public\n  // node list.\n  std::vector<std::pair<uint32_t, uint32_t>> public_list_noops;\n  // Helper vector to track the base index for `kNoOp` nodes added in the loop\n  // below.\n  std::vector<uint32_t> noop_base;\n  // Create private lists of states for all nodes that have one.\n  // After this loop:\n  //  - `state_machine` will contain states of created private lists.\n  //  - `base` in `tags_list_` will be set for tags with private list.\n  //  - `dest_info` in `tags_list_` will have `pos != kInvalidPos` for those\n  //    nodes that already have state.\n  //  - `public_list_noops` will have a record for all `kNoOp` states reaching\n  //    public list.\n  for (uint32_t tag_id = 0; tag_id < tags_list_.size(); ++tag_id) {\n    EncodedTagInfo& tag_info = tags_list_[tag_id];\n    const uint32_t sz = IntCast<uint32_t>(tag_info.dest_info.size());\n    // If we exclude just one state we add it instead of creating the `kNoOp`\n    // state.\n    PriorityQueueEntry excluded_state;\n    // Number of transitions into public list states.\n    uint32_t num_excluded_transitions = 0;\n    for (const std::pair<const uint32_t, DestInfo>& dest_info :\n         tag_info.dest_info) {\n      // If destination was marked as `kInListPos` or all transitions into it go\n      // from this node.\n      if (dest_info.second.pos == kInListPos ||\n          dest_info.second.num_transitions ==\n              tags_list_[dest_info.first].num_incoming_transitions) {\n        if (dest_info.second.pos != kInListPos) {\n          // Not yet subtracted.\n          tags_list_[dest_info.first].num_incoming_transitions -=\n              dest_info.second.num_transitions;\n        }\n        // Add to the priority queue.\n        tag_priority.emplace(dest_info.first, dest_info.second.num_transitions);\n      } else {\n        num_excluded_transitions += dest_info.second.num_transitions;\n        excluded_state = PriorityQueueEntry(dest_info.first,\n                                            dest_info.second.num_transitions);\n      }\n    }\n    uint32_t num_states = IntCast<uint32_t>(tag_priority.size());\n    if (num_states == 0) {\n      // No private list for this tag.\n      continue;\n    }\n    if (num_states + 1 == sz) {\n      // If only one state would go to the public list, just add it.\n      ++num_states;\n      tag_priority.push(excluded_state);\n      tags_list_[excluded_state.dest_index].num_incoming_transitions -=\n          excluded_state.num_transitions;\n    }\n    if (num_states != sz) {\n      // If not all nodes are in the private list, we'll need `kNoOp` into the\n      // public list.\n      tag_priority.emplace(kInvalidPos, num_excluded_transitions);\n      ++num_states;\n    }\n    // Update `base` for this tag.\n    tag_info.base = IntCast<uint32_t>(state_machine.size());\n    // Number of `kNoOp` nodes for transitions that can't be encoded using one\n    // byte.\n    const uint32_t noop_nodes = num_states <= max_transition + 1\n                                    ? uint32_t{0}\n                                    : (num_states - 2) / max_transition;\n    num_states += noop_nodes;\n    // We create states back to front. After loop below there will be\n    // `state_machine.size() + num_states` states.\n    uint32_t prev_state = IntCast<uint32_t>(state_machine.size()) + num_states;\n    state_machine.resize(prev_state);\n    // States are created in blocks. All blocks except the last one have\n    // `max_transition + 1` states. `block_size` is initialized to the size of\n    // the last block.\n    uint32_t block_size = (num_states - 1) % (max_transition + 1) + 1;\n    noop_base.clear();\n    for (;;) {\n      // Sum of all `num_transitions` into this block. It will be used as the\n      // weight of the `kNoOp` created for this block.\n      uint32_t total_block_nodes_weight = 0;\n      for (uint32_t i = 0; i < block_size; ++i) {\n        RIEGELI_ASSERT(!tag_priority.empty()) << \"No remaining nodes\";\n        total_block_nodes_weight += tag_priority.top().num_transitions;\n        const uint32_t node_index = tag_priority.top().dest_index;\n        if (node_index == kInvalidPos) {\n          // `kNoOp` that goes to the public list.\n          state_machine[--prev_state] = StateInfo(kInvalidPos, kInvalidPos);\n          tag_info.public_list_noop_pos = prev_state;\n          public_list_noops.emplace_back(tag_id, prev_state);\n        } else if (node_index >= tags_list_.size()) {\n          // `kNoOp` that goes to private list.\n          const uint32_t base = noop_base[node_index - tags_list_.size()];\n          state_machine[--prev_state] = StateInfo(kInvalidPos, base);\n          // Update canonical source for block that this node serves.\n          for (uint32_t j = 0; j <= max_transition; ++j) {\n            if (j + base >= state_machine.size()) break;\n            state_machine[j + base].canonical_source = prev_state;\n          }\n        } else {\n          // Regular state.\n          state_machine[--prev_state] = StateInfo(node_index, kInvalidPos);\n          tag_info.dest_info[node_index].pos = prev_state;\n        }\n        tag_priority.pop();\n      }\n      if (tag_priority.empty()) break;\n      // Add new `kNoOp` node into `tag_priority` to serve the block that was\n      // just created. Use position greater than `tags_list_.size()` to\n      // distinguish it from both regular state and `public_list_noop`.\n      tag_priority.emplace(tags_list_.size() + noop_base.size(),\n                           total_block_nodes_weight);\n      // Set the base to the start of the block.\n      noop_base.push_back(prev_state);\n      // All remaining blocks are `max_transition + 1` states long.\n      block_size = max_transition + 1;\n    }\n  }\n\n  // Base index of the public state list.\n  const uint32_t public_list_base = IntCast<uint32_t>(state_machine.size());\n\n  // Add all tags with non-zero incoming transition count to the priority queue.\n  for (uint32_t i = 0; i < tags_list_.size(); ++i) {\n    if (tags_list_[i].num_incoming_transitions != 0) {\n      tag_priority.emplace(i, tags_list_[i].num_incoming_transitions);\n    }\n  }\n\n  // Create a public list of states. The loop is similar to the public list\n  // creation above.\n  // After this loop:\n  //  - All states in the state machine are created.\n  //  - All tags that have an state in the public list have `state_machine_pos`\n  //    set.\n  uint32_t num_states = IntCast<uint32_t>(tag_priority.size());\n  if (num_states > 0) {\n    const uint32_t noop_nodes = num_states <= max_transition + 1\n                                    ? uint32_t{0}\n                                    : (num_states - 2) / max_transition;\n    num_states += noop_nodes;\n    // Note: The code that assigns `base` indices to states assumes that all\n    // `kNoOp` transitions to the child block increase the state index. This is\n    // ensured by creating the blocks in reverse order.\n    uint32_t prev_node = IntCast<uint32_t>(state_machine.size()) + num_states;\n    state_machine.resize(prev_node);\n    uint32_t block_size = (num_states - 1) % (max_transition + 1) + 1;\n    noop_base.clear();\n    for (;;) {\n      uint32_t total_block_nodes_weight = 0;\n      for (uint32_t i = 0; i < block_size; ++i) {\n        RIEGELI_ASSERT(!tag_priority.empty()) << \"No remaining nodes\";\n        total_block_nodes_weight += tag_priority.top().num_transitions;\n        const uint32_t node_index = tag_priority.top().dest_index;\n        if (node_index >= tags_list_.size()) {\n          // `kNoOp` state.\n          const uint32_t base = noop_base[node_index - tags_list_.size()];\n          state_machine[--prev_node] = StateInfo(kInvalidPos, base);\n          for (uint32_t j = 0; j <= max_transition; ++j) {\n            if (j + base >= state_machine.size()) break;\n            state_machine[j + base].canonical_source = prev_node;\n          }\n        } else {\n          // Regular state.\n          state_machine[--prev_node] = StateInfo(node_index, kInvalidPos);\n          tags_list_[node_index].state_machine_pos = prev_node;\n        }\n        tag_priority.pop();\n      }\n      if (tag_priority.empty()) break;\n      tag_priority.emplace(tags_list_.size() + noop_base.size(),\n                           total_block_nodes_weight);\n      noop_base.push_back(prev_node);\n      block_size = max_transition + 1;\n    }\n  }\n\n  // At this point, the only thing missing is the `base` index for tags without\n  // a private list and for `kNoOp` nodes that go to public list.\n  ComputeBaseIndices(max_transition, public_list_base, public_list_noops,\n                     state_machine);\n\n  return state_machine;\n}\n\n// Maximum transition number. Transitions are encoded as values in the range\n// [0..`kMaxTransition`].\nconstexpr uint32_t kMaxTransition = 63;\n// Minimum number of transitions between nodes A and B for state for node B to\n// appear in the private state list for node A.\nconstexpr uint32_t kMinCountForState = 10;\n\nbool TransposeEncoder::EncodeAndClose(Writer& dest, ChunkType& chunk_type,\n                                      uint64_t& num_records,\n                                      uint64_t& decoded_data_size) {\n  chunk_type = ChunkType::kTransposed;\n  return EncodeAndCloseInternal(kMaxTransition, kMinCountForState, dest,\n                                num_records, decoded_data_size);\n}\n\nbool TransposeEncoder::EncodeAndCloseInternal(uint32_t max_transition,\n                                              uint32_t min_count_for_state,\n                                              Writer& dest,\n                                              uint64_t& num_records,\n                                              uint64_t& decoded_data_size) {\n  RIEGELI_ASSERT_LE(max_transition, 63u)\n      << \"Failed precondition of TransposeEncoder::EncodeAndCloseInternal(): \"\n         \"maximum transition too large to encode\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  num_records = num_records_;\n  decoded_data_size = decoded_data_size_;\n  for (const std::pair<const NodeId, MessageNode>& entry : message_nodes_) {\n    if (entry.second.writer != nullptr) {\n      if (ABSL_PREDICT_FALSE(!entry.second.writer->Close())) {\n        return Fail(entry.second.writer->status());\n      }\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!nonproto_lengths_writer_.Close())) {\n    return Fail(nonproto_lengths_writer_.status());\n  }\n\n  if (ABSL_PREDICT_FALSE(!dest.WriteByte(\n          static_cast<uint8_t>(compressor_options_.compression_type())))) {\n    return Fail(dest.status());\n  }\n\n  const std::vector<StateInfo> state_machine =\n      CreateStateMachine(max_transition, min_count_for_state);\n\n  ChainWriter<Chain> header_writer;\n  ChainWriter<Chain> data_writer;\n  if (ABSL_PREDICT_FALSE(!WriteStatesAndData(max_transition, state_machine,\n                                             header_writer, data_writer))) {\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(!header_writer.Close())) {\n    return Fail(header_writer.status());\n  }\n  if (ABSL_PREDICT_FALSE(!data_writer.Close())) {\n    return Fail(data_writer.status());\n  }\n\n  chunk_encoding_internal::Compressor header_compressor(\n      compressor_options_,\n      chunk_encoding_internal::Compressor::TuningOptions()\n          .set_pledged_size(header_writer.dest().size())\n          .set_recycling_pool_options(recycling_pool_options_));\n  if (ABSL_PREDICT_FALSE(\n          !header_compressor.writer().Write(std::move(header_writer.dest())))) {\n    return Fail(header_compressor.writer().status());\n  }\n  if (ABSL_PREDICT_FALSE(\n          !header_compressor.LengthPrefixedEncodeAndClose(dest))) {\n    return Fail(header_compressor.status());\n  }\n  if (ABSL_PREDICT_FALSE(!dest.Write(std::move(data_writer.dest())))) {\n    return Fail(dest.status());\n  }\n  return Close();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/chunk_encoding/transpose_encoder.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_TRANSPOSE_ENCODER_H_\n#define RIEGELI_CHUNK_ENCODING_TRANSPOSE_ENCODER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/container/flat_hash_map.h\"\n#include \"absl/container/inlined_vector.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/chain_backward_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/chunk_encoder.h\"\n#include \"riegeli/chunk_encoding/compressor.h\"\n#include \"riegeli/chunk_encoding/compressor_options.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/chunk_encoding/transpose_internal.h\"\n\nnamespace riegeli {\n\n// Format (values are varint encoded unless indicated otherwise):\n//  - Compression type\n//  - Header length (compressed length if applicable)\n//  - Header (possibly compressed):\n//    - Number of separately compressed buckets that data buffers are split into\n//      [`num_buckets`]\n//    - Number of data buffers [`num_buffers`]\n//    - Array of `num_buckets` varints: sizes of buckets (compressed size\n//      if applicable)\n//    - Array of `num_buffers` varints: lengths of buffers (uncompressed)\n//    - Number of state machine states [`num_state`]\n//    - States encoded in 4 blocks:\n//      - Array of `num_state` Tags/ReservedIDs\n//      - Array of `num_state` next node indices\n//      - Array of subtypes (for all tags where applicable)\n//      - Array of data buffer indices (for all tags/subtypes where applicable)\n//    - Initial state index\n//  - `num_buckets` buckets:\n//    - Bucket data (possibly compressed):\n//      - Concatenated data buffers in this bucket (bytes)\n//  - Transitions (possibly compressed):\n//    - State machine transitions (bytes)\nclass TransposeEncoder : public ChunkEncoder {\n public:\n  class TuningOptions {\n   public:\n    TuningOptions() noexcept {}\n\n    // The default approximate bucket size, used if compression is enabled.\n    // Finer bucket granularity (i.e. smaller size) worsens compression density\n    // but makes field projection more effective.\n    //\n    // Default: `std::numeric_limits<uint64_t>::max()`.\n    TuningOptions& set_bucket_size(uint64_t bucket_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      bucket_size_ = bucket_size;\n      return *this;\n    }\n    TuningOptions&& set_bucket_size(uint64_t bucket_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_bucket_size(bucket_size));\n    }\n    uint64_t bucket_size() const { return bucket_size_; }\n\n    // Options for a global `RecyclingPool` of compression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // compression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    TuningOptions& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    TuningOptions&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    uint64_t bucket_size_ = std::numeric_limits<uint64_t>::max();\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Creates an empty `TransposeEncoder`.\n  explicit TransposeEncoder(CompressorOptions options,\n                            TuningOptions tuning_options = TuningOptions());\n\n  ~TransposeEncoder();\n\n  void Clear() override;\n\n  // `record` should be a protocol message in binary format. Transpose works\n  // just fine even if `record` is a corrupted protocol message or an arbitrary\n  // string. Such records are internally stored separately -- these are not\n  // broken down into columns.\n  using ChunkEncoder::AddRecord;\n  bool AddRecord(BytesRef record) override;\n  bool AddRecord(ExternalRef record) override;\n  bool AddRecord(const Chain& record) override;\n  bool AddRecord(const absl::Cord& record) override;\n\n  bool AddRecords(Chain records, std::vector<size_t> limits) override;\n\n  bool EncodeAndClose(Writer& dest, ChunkType& chunk_type,\n                      uint64_t& num_records,\n                      uint64_t& decoded_data_size) override;\n\n private:\n  bool AddRecordInternal(Reader& record);\n\n  // Encode messages added with `AddRecord()` calls and write the result to\n  // `dest`.\n  bool EncodeAndCloseInternal(uint32_t max_transition,\n                              uint32_t min_count_for_state, Writer& dest,\n                              uint64_t& num_records,\n                              uint64_t& decoded_data_size);\n\n  // Types of data buffers protocol buffer fields are split into.\n  // The buffer type information is not used in any way except to group similar\n  // buffers together hoping that this helps with compression context modeling.\n  enum class BufferType : int {\n    // Varint-encoded numbers, with the highest bit (signifying end of number)\n    // stripped.\n    kVarint,\n    // Fixed width 32bit integer or floating point numbers.\n    kFixed32,\n    // Fixed width 64bit integer or floating point numbers.\n    kFixed64,\n    // String length + data.\n    kString,\n    // All non-proto messages.\n    kNonProto,\n    kNumBufferTypes,\n  };\n\n  static constexpr size_t kNumBufferTypes =\n      static_cast<size_t>(BufferType::kNumBufferTypes);\n\n  // Information about a field with unique proto path.\n  struct MessageNode {\n    explicit MessageNode(chunk_encoding_internal::MessageId message_id);\n    // Some nodes (such as `kStartGroup`) contain no data. Buffer is assigned in\n    // the first `GetBuffer()` call when we have data to write.\n    std::unique_ptr<BackwardWriter> writer;\n    // Unique ID for every instance of this class within `TransposeEncoder`.\n    chunk_encoding_internal::MessageId message_id;\n    // Position of encoded tag in `tags_list_` per subtype.\n    // Size 14 works well with `kMaxVarintInline == 3`.\n    absl::InlinedVector<uint32_t, 14> encoded_tag_pos;\n  };\n\n  // We build a tree structure of protocol buffer tags. `NodeId` uniquely\n  // identifies a node in this tree.\n  struct NodeId : WithEqual<NodeId> {\n    explicit NodeId(chunk_encoding_internal::MessageId parent_message_id,\n                    uint32_t tag);\n\n    friend bool operator==(NodeId a, NodeId b) {\n      return a.parent_message_id == b.parent_message_id && a.tag == b.tag;\n    }\n    template <typename HashState>\n    friend HashState AbslHashValue(HashState hash_state, NodeId self) {\n      return HashState::combine(std::move(hash_state), self.parent_message_id,\n                                self.tag);\n    }\n\n    chunk_encoding_internal::MessageId parent_message_id;\n    uint32_t tag;\n  };\n\n  // Add message recursively to the internal data structures.\n  // Precondition: `message` is a valid proto message, i.e. `IsProtoMessage()`\n  // on this message returns `true`.\n  bool AddMessage(Reader& record);\n\n  // Write all buffer lengths to `header_writer` and data buffers in `data_` to\n  // `data_writer` (compressed using `compressor_`). Fill map with the\n  // sequential position of each buffer written.\n  bool WriteBuffers(Writer& header_writer, Writer& data_writer,\n                    absl::flat_hash_map<NodeId, uint32_t>* buffer_pos);\n\n  // One state of the state machine created in encoder.\n  struct StateInfo {\n    StateInfo();\n    explicit StateInfo(uint32_t etag_index, uint32_t base);\n    // Index of the `encoded_tag` in `tags_list_` represented by this state.\n    // `kInvalidPos` for `kNoOp` states.\n    uint32_t etag_index;\n    // Base index of this state. Transitions from this state can target only\n    // states in the range [`base`..`base + kMaxTransition`].\n    // `kInvalidPos` if no outgoing transition.\n    uint32_t base;\n    // Usual source of transition into this state. Set if there is `kNoOp`\n    // generated to reach a group of states including this one.\n    // `kInvalidPos` if no such state.\n    uint32_t canonical_source;\n  };\n\n  // Add `buffer` to `bucket_compressor.writer()`.\n  // If `new_uncompressed_bucket_size` is not `std::nullopt`, flush the current\n  // bucket to `data_writer` first and create a new bucket of that size.\n  bool AddBuffer(std::optional<size_t> new_uncompressed_bucket_size,\n                 const Chain& buffer,\n                 chunk_encoding_internal::Compressor& bucket_compressor,\n                 Writer& data_writer,\n                 std::vector<size_t>& compressed_bucket_sizes,\n                 std::vector<size_t>& buffer_sizes);\n\n  // Compute base indices for states in `state_machine` that don't have one yet.\n  // `public_list_base` is the index of the start of the public list.\n  // `public_list_noops` is the list of `kNoOp` states that don't have a base\n  // set yet. It contains pairs of (`tag_index`, `state_index`).\n  void ComputeBaseIndices(\n      uint32_t max_transition, uint32_t public_list_base,\n      absl::Span<const std::pair<uint32_t, uint32_t>> public_list_noops,\n      std::vector<StateInfo>& state_machine);\n\n  // Traverse `encoded_tags_` and populate `num_incoming_transitions` and\n  // `dest_info` in `tags_list_` based on transition distribution.\n  void CollectTransitionStatistics();\n\n  // Create a state machine for `encoded_tags_`.\n  std::vector<StateInfo> CreateStateMachine(uint32_t max_transition,\n                                            uint32_t min_count_for_state);\n\n  // Write state machine states into `header_writer` and all data buffers and\n  // transitions into `data_writer` (compressed using `compressor_`).\n  bool WriteStatesAndData(uint32_t max_transition,\n                          absl::Span<const StateInfo> state_machine,\n                          Writer& header_writer, Writer& data_writer);\n\n  // Write all state machine transitions from `encoded_tags_` into\n  // `compressor_.writer()`.\n  bool WriteTransitions(uint32_t max_transition,\n                        absl::Span<const StateInfo> state_machine,\n                        Writer& transitions_writer);\n\n  // Value type of node in Nodes map.\n  using Node = absl::flat_hash_map<NodeId, MessageNode>::value_type;\n\n  // Returns node pointer from `node_id`.\n  Node* GetNode(NodeId node_id);\n\n  // Get possition of the (`node`, `subtype`) pair in `tags_list_`, adding it\n  // if not in the list yet.\n  uint32_t GetPosInTagsList(Node* node,\n                            chunk_encoding_internal::Subtype subtype);\n\n  // Get `BackwardWriter` for node. `type` is used to select the right category\n  // for the buffer if not created yet.\n  BackwardWriter* GetBuffer(Node* node, BufferType type);\n\n  // Information about the state machine transition destination.\n  struct DestInfo {\n    DestInfo();\n    // Position of the destination in destination list created for this state.\n    // `kInvalidPos` if transition destination is not in the list. In that case\n    // transition is encoded using the public list of states.\n    uint32_t pos;\n    // Number of transition to this destination.\n    size_t num_transitions = 0;\n  };\n\n  // Information about encoded tag.\n  struct EncodedTagInfo {\n    explicit EncodedTagInfo(NodeId node_id,\n                            chunk_encoding_internal::Subtype subtype);\n    NodeId node_id;\n    chunk_encoding_internal::Subtype subtype;\n    // Maps all destinations reachable from this encoded tag to `DestInfo`.\n    absl::flat_hash_map<uint32_t, DestInfo> dest_info;\n    // Number of incoming tranitions into this state.\n    size_t num_incoming_transitions = 0;\n    // Index of this state in the state machine.\n    uint32_t state_machine_pos;\n    // Position of `kNoOp` node in the private list that has base in public\n    // list. If outgoing transitions are split into frequent and infrequent, a\n    // list of frequent destinations is created and `kNoOp` node is added that\n    // serves infrequent transitions.\n    uint32_t public_list_noop_pos;\n    // Base index of this encoded tag. Transitions from this tag can target only\n    // states in the range [`base`..`base + kMaxTransition`].\n    // `kInvalidPos` if no outgoing transition.\n    uint32_t base;\n  };\n\n  // Information about the data buffer.\n  struct BufferWithMetadata {\n    explicit BufferWithMetadata(NodeId node_id);\n    // Buffer itself, wrapped in `std::unique_ptr` so that its address remains\n    // constant when additional buffers are added.\n    std::unique_ptr<Chain> buffer;\n    // `NodeId` this buffer belongs to.\n    NodeId node_id;\n  };\n\n  // Information about a submessage.\n  struct MessageFrame {\n    uint32_t end_of_submessage_pos;\n    chunk_encoding_internal::MessageId parent_message_id;\n    size_t parent_max_record_pos;\n  };\n\n  CompressorOptions compressor_options_;\n  // The default approximate bucket size, used if compression is enabled.\n  // Finer bucket granularity (i.e. smaller size) worsens compression density\n  // but makes field projection more effective.\n  uint64_t bucket_size_;\n  RecyclingPoolOptions recycling_pool_options_;\n\n  // List of all distinct Encoded tags.\n  std::vector<EncodedTagInfo> tags_list_;\n  // Sequence of tags on input as indices into `tags_list_`.\n  std::vector<uint32_t> encoded_tags_;\n  // Data buffers in separate vectors per buffer type.\n  std::vector<BufferWithMetadata> data_[kNumBufferTypes];\n  // Stack of open submessages.\n  std::vector<MessageFrame> message_stack_;\n  // Every group creates a new message ID. We keep track of open groups in this\n  // vector.\n  std::vector<chunk_encoding_internal::MessageId> group_stack_;\n  // Tree of message nodes.\n  absl::flat_hash_map<NodeId, MessageNode> message_nodes_;\n  ChainBackwardWriter<Chain> nonproto_lengths_writer_;\n  // Counter used to assign unique IDs to the message nodes.\n  chunk_encoding_internal::MessageId next_message_id_ =\n      chunk_encoding_internal::MessageId::kRoot + 1;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CHUNK_ENCODING_TRANSPOSE_ENCODER_H_\n"
  },
  {
    "path": "riegeli/chunk_encoding/transpose_internal.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CHUNK_ENCODING_TRANSPOSE_INTERNAL_H_\n#define RIEGELI_CHUNK_ENCODING_TRANSPOSE_INTERNAL_H_\n\n#include <stdint.h>\n\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/messages/message_wire_format.h\"\n#include \"riegeli/varint/varint_writing.h\"\n\nnamespace riegeli::chunk_encoding_internal {\n\nenum class MessageId : uint32_t {\n  kNoOp,\n  kNonProto,\n  kStartOfSubmessage,\n  kStartOfMessage,\n  // `kRoot` marks the root node in memory. It is never encoded.\n  kRoot,\n  // Remaining message ids are assigned sequentially one per `NodeId`.\n};\n\ninline MessageId operator+(MessageId a, uint32_t b) {\n  return static_cast<MessageId>(static_cast<uint32_t>(a) + b);\n}\n\ninline MessageId& operator++(MessageId& a) { return a = a + 1; }\n\nstatic_assert(static_cast<uint32_t>(MessageId::kRoot) <= 8,\n              \"Reserved ids must not overlap valid proto tags\");\n\n// `kSubmessageWireType` does marks the end of a submessage, distinguishing it\n// from the end of a string or bytes field, which is encoded using\n// `WireType::kLengthDelimited`.\ninline constexpr WireType kSubmessageWireType = static_cast<WireType>(6);\n\nenum class Subtype : uint8_t {\n  kTrivial = 0,\n\n  // Subtypes of `WireType::kVarint`:\n  // Varint of the given length, in the buffer.\n  kVarint1 = 0,\n  kVarintMax = static_cast<uint8_t>(kVarint1) + kMaxLengthVarint64 - 1,\n  // Varint of the given value, inline.\n  kVarintInline0 = static_cast<uint8_t>(kVarintMax) + 1,\n  kVarintInlineMax = static_cast<uint8_t>(kVarintInline0) + 0x7f,\n\n  // Subtypes of `WireType::kLengthDelimited`:\n  kLengthDelimitedString = 0,\n  kLengthDelimitedStartOfSubmessage = 1,\n  kLengthDelimitedEndOfSubmessage = 2,\n};\n\ninline Subtype operator+(Subtype a, uint8_t b) {\n  return static_cast<Subtype>(static_cast<uint8_t>(a) + b);\n}\n\ninline uint8_t operator-(Subtype a, Subtype b) {\n  return static_cast<uint8_t>(a) - static_cast<uint8_t>(b);\n}\n\n// Returns whether `tag`/`subtype` pair has a data buffer.\n// Precondition: `tag` is a valid proto tag.\ninline bool HasDataBuffer(uint32_t tag, Subtype subtype) {\n  switch (GetTagWireType(tag)) {\n    case WireType::kVarint:\n      // Protocol buffer has buffer if value is not inlined.\n      return subtype < Subtype::kVarintInline0;\n    case WireType::kFixed32:\n    case WireType::kFixed64:\n      return true;\n    case WireType::kLengthDelimited:\n      // If subtype is `kLengthDelimitedStartOfSubmessage` or\n      // `kLengthDelimitedEndOfSubmessage`, we have no buffer.\n      return subtype == Subtype::kLengthDelimitedString;\n    case WireType::kStartGroup:\n    case WireType::kEndGroup:\n      return false;\n    default:\n      RIEGELI_ASSUME_UNREACHABLE() << \"Unknown wire type in \" << tag;\n  }\n}\n\n// Returns `true` if this tag is followed by subtype.\n// Precondition: `tag` is a valid proto tag.\ninline bool HasSubtype(uint32_t tag) {\n  switch (GetTagWireType(tag)) {\n    case WireType::kVarint:\n      return true;\n      // A `kLengthDelimited` tag is not followed by subtype, even though\n      // `kLengthDelimited` nodes have subtypes, because submessage start is\n      // encoded as `MessageId::kStartOfSubmessage`, and submessage end is\n      // encoded with `kSubmessageWireType` that is taken into account before\n      // calling this method.\n    case WireType::kFixed32:\n    case WireType::kFixed64:\n    case WireType::kLengthDelimited:\n    case WireType::kStartGroup:\n    case WireType::kEndGroup:\n      return false;\n    default:\n      RIEGELI_ASSUME_UNREACHABLE() << \"Unknown wire type in \" << tag;\n  }\n}\n\n}  // namespace riegeli::chunk_encoding_internal\n\n#endif  // RIEGELI_CHUNK_ENCODING_TRANSPOSE_INTERNAL_H_\n"
  },
  {
    "path": "riegeli/containers/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"linear_sorted_string_set\",\n    srcs = [\"linear_sorted_string_set.cc\"],\n    hdrs = [\"linear_sorted_string_set.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:compact_string\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:debug\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:iterable\",\n        \"//riegeli/base:optional_compact_string\",\n        \"//riegeli/base:stream_utils\",\n        \"//riegeli/bytes:compact_string_writer\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/endian:endian_reading\",\n        \"//riegeli/varint:varint_reading\",\n        \"//riegeli/varint:varint_writing\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:bits\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/status:statusor\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"chunked_sorted_string_set\",\n    srcs = [\"chunked_sorted_string_set.cc\"],\n    hdrs = [\"chunked_sorted_string_set.h\"],\n    deps = [\n        \":linear_sorted_string_set\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:binary_search\",\n        \"//riegeli/base:compact_string\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:debug\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:iterable\",\n        \"//riegeli/base:memory_estimator\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/varint:varint_reading\",\n        \"//riegeli/varint:varint_writing\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/container:inlined_vector\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/status:statusor\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/containers/chunked_sorted_string_set.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/containers/chunked_sorted_string_set.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <algorithm>\n#include <initializer_list>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/status/statusor.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/binary_search.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/debug.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/containers/linear_sorted_string_set.h\"\n#include \"riegeli/varint/varint_reading.h\"\n#include \"riegeli/varint/varint_writing.h\"\n\nnamespace riegeli {\n\nChunkedSortedStringSet ChunkedSortedStringSet::FromSorted(\n    std::initializer_list<absl::string_view> src, Options options) {\n  return FromSorted<>(src, std::move(options));\n}\n\nChunkedSortedStringSet ChunkedSortedStringSet::FromUnsorted(\n    std::initializer_list<absl::string_view> src, Options options) {\n  return FromUnsorted<>(src, std::move(options));\n}\n\ninline ChunkedSortedStringSet::ChunkedSortedStringSet(Chunks&& chunks)\n    : chunks_(std::move(chunks)) {}\n\nbool ChunkedSortedStringSet::ContainsImpl(absl::string_view element) const {\n  if (chunks_.empty()) return false;\n  SearchResult<Chunks::const_iterator> chunk =\n      BinarySearch(chunks_.cbegin() + 1, chunks_.cend(), CompareFirst{element});\n  --chunk.found;\n\n  if (chunk.ordering == 0) return true;\n  if (chunk.found == chunks_.cbegin()) {\n    return chunk.found->set.contains(element);\n  }\n  return chunk.found->set.contains_skip_first(element);\n}\n\nbool ChunkedSortedStringSet::ContainsWithIndexImpl(absl::string_view element,\n                                                   size_t& index) const {\n  if (chunks_.empty()) {\n    index = 0;\n    return false;\n  }\n  SearchResult<Chunks::const_iterator> chunk =\n      BinarySearch(chunks_.cbegin() + 1, chunks_.cend(), CompareFirst{element});\n  --chunk.found;\n\n  if (chunk.ordering == 0) {\n    index = chunk.found->cumulative_end_index;\n    return true;\n  }\n  if (chunk.found == chunks_.cbegin()) {\n    return chunk.found->set.contains(element, &index);\n  }\n  const bool result = chunk.found->set.contains_skip_first(element, &index);\n  index += chunk.found[-1].cumulative_end_index;\n  return result;\n}\n\nbool ChunkedSortedStringSet::Equal(const ChunkedSortedStringSet& a,\n                                   const ChunkedSortedStringSet& b) {\n  return a.size() == b.size() &&\n         std::equal(a.split_elements().cbegin(), SplitElementIterator(),\n                    b.split_elements().cbegin(), SplitElementIterator());\n}\n\nStrongOrdering ChunkedSortedStringSet::Compare(\n    const ChunkedSortedStringSet& a, const ChunkedSortedStringSet& b) {\n  SplitElementIterator a_iter = a.split_elements().cbegin();\n  SplitElementIterator b_iter = b.split_elements().cbegin();\n  while (a_iter != SplitElementIterator()) {\n    if (b_iter == SplitElementIterator()) return StrongOrdering::greater;\n    if (const StrongOrdering ordering = riegeli::Compare(*a_iter, *b_iter);\n        ordering != 0) {\n      return ordering;\n    }\n    ++a_iter;\n    ++b_iter;\n  }\n  return b_iter == SplitElementIterator() ? StrongOrdering::equal\n                                          : StrongOrdering::less;\n}\n\nsize_t ChunkedSortedStringSet::EncodedSize() const {\n  size_t encoded_size = LengthVarint64(chunks_.size());\n  for (const Chunk& chunk : chunks_) encoded_size += chunk.set.EncodedSize();\n  return encoded_size;\n}\n\nabsl::Status ChunkedSortedStringSet::EncodeImpl(Writer& dest) const {\n  if (ABSL_PREDICT_FALSE(!WriteVarint64(chunks_.size(), dest))) {\n    return dest.status();\n  }\n  for (const Chunk& chunk : chunks_) {\n    if (absl::Status status = chunk.set.Encode(dest);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n  }\n  return absl::OkStatus();\n}\n\nabsl::Status ChunkedSortedStringSet::DecodeImpl(Reader& src,\n                                                DecodeOptions options) {\n  uint64_t num_chunks;\n  if (ABSL_PREDICT_FALSE(!ReadVarint64(src, num_chunks))) {\n    return src.StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Malformed ChunkedSortedStringSet encoding \"\n                                   \"(num_chunks)\"));\n  }\n  if (ABSL_PREDICT_FALSE(num_chunks > options.max_num_chunks())) {\n    return src.AnnotateStatus(absl::ResourceExhaustedError(absl::StrCat(\n        \"Maximum ChunkedSortedStringSet number of chunks exceeded: \",\n        num_chunks, \" > \", options.max_num_chunks())));\n  }\n\n  LinearSortedStringSet::DecodeState decode_state;\n  const LinearSortedStringSet::DecodeOptions linear_options =\n      LinearSortedStringSet::DecodeOptions()\n          .set_validate(options.validate())\n          .set_max_encoded_size(options.max_encoded_chunk_size())\n          .set_decode_state(&decode_state);\n  Chunks chunks(IntCast<size_t>(num_chunks));\n  for (Chunk& chunk : chunks) {\n    if (absl::Status status = chunk.set.Decode(src, linear_options);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n    if (ABSL_PREDICT_FALSE(chunk.set.empty())) {\n      return src.AnnotateStatus(absl::InvalidArgumentError(\n          \"Malformed ChunkedSortedStringSet encoding \"\n          \"(empty chunk)\"));\n    }\n    chunk.cumulative_end_index = decode_state.cumulative_size;\n  }\n  chunks_ = std::move(chunks);\n  return absl::OkStatus();\n}\n\ntemplate <typename LinearIterator>\nChunkedSortedStringSet::IteratorImpl<LinearIterator>&\nChunkedSortedStringSet::IteratorImpl<LinearIterator>::operator++() {\n  RIEGELI_ASSERT(current_iterator_ != LinearIterator())\n      << \"Failed precondition of \"\n         \"ChunkedSortedStringSet::IteratorImpl::operator++: \"\n         \"iterator is end()\";\n  ++current_iterator_;\n  if (ABSL_PREDICT_TRUE(current_iterator_ != LinearIterator())) {\n    // Staying in the same chunk.\n    return *this;\n  }\n  RIEGELI_ASSERT(current_chunk_iterator_ != set_->chunks_.cend())\n      << \"Failed invariant of ChunkedSortedStringSet::Iterator: \"\n         \"current_chunk_iterator_ is end() but current_iterator_ was not\";\n  ++current_chunk_iterator_;\n  if (ABSL_PREDICT_FALSE(current_chunk_iterator_ == set_->chunks_.cend())) {\n    // Reached the end.\n    return *this;\n  }\n  // Moving to the next chunk.\n  current_iterator_ =\n      GetLinearIterator<LinearIterator>(current_chunk_iterator_->set);\n  return *this;\n}\n\ntemplate ChunkedSortedStringSet::IteratorImpl<LinearSortedStringSet::Iterator>&\nChunkedSortedStringSet::IteratorImpl<\n    LinearSortedStringSet::Iterator>::operator++();\ntemplate ChunkedSortedStringSet::IteratorImpl<\n    LinearSortedStringSet::SplitElementIterator>&\nChunkedSortedStringSet::IteratorImpl<\n    LinearSortedStringSet::SplitElementIterator>::operator++();\n\nChunkedSortedStringSet::Builder::Builder(Options options)\n    : chunk_size_(options.chunk_size()) {\n  ApplySizeHint(options.size_hint());\n}\n\nChunkedSortedStringSet::Builder::Builder(Builder&& that) noexcept\n    : chunk_size_(that.chunk_size_),\n      chunks_(std::exchange(that.chunks_, Chunks())),\n      current_builder_(std::move(that.current_builder_)) {}\n\nChunkedSortedStringSet::Builder& ChunkedSortedStringSet::Builder::operator=(\n    Builder&& that) noexcept {\n  chunk_size_ = that.chunk_size_;\n  chunks_ = std::exchange(that.chunks_, Chunks());\n  current_builder_ = std::move(that.current_builder_);\n  return *this;\n}\n\nChunkedSortedStringSet::Builder::~Builder() = default;\n\nvoid ChunkedSortedStringSet::Builder::Reset(Options options) {\n  chunk_size_ = options.chunk_size();\n  chunks_.clear();\n  current_builder_.Reset();\n  ApplySizeHint(options.size_hint());\n}\n\ninline void ChunkedSortedStringSet::Builder::ApplySizeHint(size_t size_hint) {\n  if (size_hint > 0) chunks_.reserve((size_hint - 1) / chunk_size_ + 1);\n}\n\nbool ChunkedSortedStringSet::Builder::InsertNext(absl::string_view element) {\n  const absl::StatusOr<bool> inserted = TryInsertNext(element);\n  RIEGELI_CHECK_OK(inserted)\n      << \"Failed precondition of ChunkedSortedStringSet::Builder::InsertNext()\";\n  return *inserted;\n}\n\ntemplate <typename Element,\n          std::enable_if_t<std::is_same_v<Element, std::string>, int>>\nbool ChunkedSortedStringSet::Builder::InsertNext(Element&& element) {\n  // `std::move(element)` is correct and `std::forward<Element>(element)` is not\n  // necessary: `Element` is always `std::string`, never an lvalue reference.\n  const absl::StatusOr<bool> inserted = TryInsertNext(std::move(element));\n  RIEGELI_CHECK_OK(inserted)\n      << \"Failed precondition of ChunkedSortedStringSet::Builder::InsertNext()\";\n  return *inserted;\n}\n\ntemplate bool ChunkedSortedStringSet::Builder::InsertNext(\n    std::string&& element);\n\nabsl::StatusOr<bool> ChunkedSortedStringSet::Builder::TryInsertNext(\n    absl::string_view element) {\n  return InsertNextImpl(element);\n}\n\ntemplate <typename Element,\n          std::enable_if_t<std::is_same_v<Element, std::string>, int>>\nabsl::StatusOr<bool> ChunkedSortedStringSet::Builder::TryInsertNext(\n    Element&& element) {\n  // `std::move(element)` is correct and `std::forward<Element>(element)` is not\n  // necessary: `Element` is always `std::string`, never an lvalue reference.\n  return InsertNextImpl(std::move(element));\n}\n\ntemplate absl::StatusOr<bool> ChunkedSortedStringSet::Builder::TryInsertNext(\n    std::string&& element);\n\ntemplate <typename Element>\nabsl::StatusOr<bool> ChunkedSortedStringSet::Builder::InsertNextImpl(\n    Element&& element) {\n  if (ABSL_PREDICT_FALSE(current_builder_.size() == chunk_size_)) {\n    if (ABSL_PREDICT_FALSE(element <= current_builder_.last())) {\n      if (ABSL_PREDICT_TRUE(element == current_builder_.last())) return false;\n      return absl::FailedPreconditionError(\n          absl::StrCat(\"Elements are not sorted: new \", riegeli::Debug(element),\n                       \" < last \", riegeli::Debug(last())));\n    }\n    AddChunk();\n  }\n  if (absl::StatusOr<bool> inserted =\n          current_builder_.TryInsertNext(std::forward<Element>(element));\n      ABSL_PREDICT_FALSE(!inserted.ok() || !*inserted)) {\n    return inserted;\n  }\n  return true;\n}\n\nChunkedSortedStringSet ChunkedSortedStringSet::Builder::Build() {\n  if (!current_builder_.empty()) AddChunk();\n  ChunkedSortedStringSet set(std::exchange(chunks_, Chunks()));\n  RIEGELI_ASSERT(empty())\n      << \"Failed postcondition of ChunkedSortedStringSet::Builder::Build(): \"\n         \"builder should be empty\";\n  return set;\n}\n\ninline void ChunkedSortedStringSet::Builder::AddChunk() {\n  const size_t cumulative_end_index =\n      (chunks_.empty() ? 0 : chunks_.back().cumulative_end_index) +\n      current_builder_.size();\n#if __cpp_aggregate_paren_init\n  chunks_.emplace_back(current_builder_.Build(), cumulative_end_index);\n#else\n  chunks_.push_back(Chunk{current_builder_.Build(), cumulative_end_index});\n#endif\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/containers/chunked_sorted_string_set.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CONTAINERS_CHUNKED_SORTED_STRING_SET_H_\n#define RIEGELI_CONTAINERS_CHUNKED_SORTED_STRING_SET_H_\n\n#include <stddef.h>\n\n#include <algorithm>\n#include <initializer_list>\n#include <iterator>\n#include <string>\n#include <type_traits>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/macros.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/container/inlined_vector.h\"\n#include \"absl/status/status.h\"\n#include \"absl/status/statusor.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compact_string.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/iterable.h\"\n#include \"riegeli/base/memory_estimator.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/containers/linear_sorted_string_set.h\"\n\nnamespace riegeli {\n\n// A sorted set of strings, split into chunks, compressed by recognizing shared\n// prefixes within each chunk.\n//\n// `ChunkedSortedStringSet` is optimized for memory usage.\nclass ChunkedSortedStringSet : public WithCompare<ChunkedSortedStringSet> {\n private:\n  template <typename LinearIterator>\n  class IteratorImpl;\n\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // Tunes the number of elements encoded together. A larger `chunk_size`\n    // reduces memory usage, but the time complexity of lookups is roughly\n    // proportional to `chunk_size`.\n    //\n    // Default: `kDefaultChunkSize` (16).\n    static constexpr size_t kDefaultChunkSize = 16;\n    Options& set_chunk_size(size_t chunk_size) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      chunk_size_ = chunk_size;\n      return *this;\n    }\n    Options&& set_chunk_size(size_t chunk_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_chunk_size(chunk_size));\n    }\n    size_t chunk_size() const { return chunk_size_; }\n\n    // Expected final size, or 0 if unknown. This may improve performance and\n    // memory usage.\n    //\n    // If the size hint turns out to not match reality, nothing breaks.\n    //\n    // Default: 0.\n    Options& set_size_hint(size_t size_hint) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      size_hint_ = size_hint;\n      return *this;\n    }\n    Options&& set_size_hint(size_t size_hint) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_size_hint(size_hint));\n    }\n    size_t size_hint() const { return size_hint_; }\n\n   private:\n    size_t chunk_size_ = kDefaultChunkSize;\n    size_t size_hint_ = 0;\n  };\n\n  using SplitElement = LinearSortedStringSet::SplitElement;\n  using SplitElementIterator =\n      IteratorImpl<LinearSortedStringSet::SplitElementIterator>;\n  class SplitElements;\n  class Builder;\n  class NextInsertIterator;\n\n  // Options for `Decode()`.\n  class DecodeOptions {\n   public:\n    DecodeOptions() noexcept {}\n\n    // If `false`, performs partial validation of the structure of data, which\n    // is sufficient to prevent undefined behavior when the set is used. The\n    // only aspect not validated is that elements are sorted and unique. This is\n    // faster. If elements are not sorted and unique, then iteration yields\n    // elements in the stored order, and `contains()` may fail to find an\n    // element which can be seen during iteration.\n    //\n    // If `true`, performs full validation of encoded data, including checking\n    // that elements are sorted and unique. This is slower. This can be used for\n    // parsing untrusted data.\n    //\n    // Default: `false`.\n    DecodeOptions& set_validate(bool validate) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      validate_ = validate;\n      return *this;\n    }\n    DecodeOptions&& set_validate(bool validate) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_validate(validate));\n    }\n    bool validate() const { return validate_; }\n\n    // `Decode()` fails if more than `set_max_num_chunks()` chunks would need to\n    // be created. This can be used for parsing untrusted data.\n    //\n    // Default: `Chunks().max_size()` with `Chunks` being the internal type\n    // for representing chunks.\n    DecodeOptions& set_max_num_chunks(size_t max_num_chunks) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      max_num_chunks_ = max_num_chunks;\n      return *this;\n    }\n    DecodeOptions&& set_max_num_chunks(size_t max_num_chunks) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_num_chunks(max_num_chunks));\n    }\n    size_t max_num_chunks() const { return max_num_chunks_; }\n\n    // `Decode()` fails if more than `max_encoded_chunk_size()` bytes would need\n    // to be allocated for any chunk. This can be used for parsing untrusted\n    // data.\n    //\n    // Default: `CompactString::max_size()`.\n    DecodeOptions& set_max_encoded_chunk_size(size_t max_encoded_chunk_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      max_encoded_chunk_size_ = max_encoded_chunk_size;\n      return *this;\n    }\n    DecodeOptions&& set_max_encoded_chunk_size(size_t max_encoded_chunk_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_encoded_chunk_size(max_encoded_chunk_size));\n    }\n    size_t max_encoded_chunk_size() const { return max_encoded_chunk_size_; }\n\n   private:\n    bool validate_ = false;\n    size_t max_num_chunks_ = Chunks().max_size();\n    size_t max_encoded_chunk_size_ = CompactString::max_size();\n  };\n\n  using value_type = absl::string_view;\n  using reference = value_type;\n  using const_reference = reference;\n  using iterator = IteratorImpl<LinearSortedStringSet::iterator>;\n  using const_iterator = iterator;\n  using size_type = size_t;\n  using difference_type = ptrdiff_t;\n\n  // Creates a set consisting of the given elements. They must be sorted.\n  // Consecutive duplicates are inserted only once.\n  //\n  // The type of `src` must support iteration yielding `absl::string_view`:\n  // `for (const absl::string_view element : src)`,\n  // e.g. `std::vector<std::string>`.\n  //\n  // If `Src` supports random access iteration,\n  // `std::distance(begin(src), end(src))` is automatically used as\n  // `Options::size_hint()`.\n  template <\n      typename Src,\n      std::enable_if_t<IsIterableOf<Src, absl::string_view>::value, int> = 0>\n  static ChunkedSortedStringSet FromSorted(Src&& src,\n                                           Options options = Options());\n  static ChunkedSortedStringSet FromSorted(\n      std::initializer_list<absl::string_view> src,\n      Options options = Options());\n\n  // Creates a set consisting of the given elements. They do not need to be\n  // sorted. Duplicates are inserted only once.\n  //\n  // The type of `src` must support iteration yielding `absl::string_view`:\n  // `for (const absl::string_view element : src)`,\n  // e.g. `std::vector<std::string>`.\n  //\n  // If duplicates are expected, `Options::size_hint()` should apply before\n  // removing duplicates.\n  //\n  // If `Src` supports random access iteration,\n  // `std::distance(begin(src), end(src))` is automatically used as\n  // `Options::size_hint()`.\n  template <\n      typename Src,\n      std::enable_if_t<IsIterableOf<Src, absl::string_view>::value, int> = 0>\n  static ChunkedSortedStringSet FromUnsorted(Src&& src,\n                                             Options options = Options());\n  static ChunkedSortedStringSet FromUnsorted(\n      std::initializer_list<absl::string_view> src,\n      Options options = Options());\n\n  // An empty set.\n  ChunkedSortedStringSet() = default;\n\n  ChunkedSortedStringSet(const ChunkedSortedStringSet& that) = default;\n  ChunkedSortedStringSet& operator=(const ChunkedSortedStringSet& that) =\n      default;\n\n  ChunkedSortedStringSet(ChunkedSortedStringSet&& that) noexcept = default;\n  ChunkedSortedStringSet& operator=(ChunkedSortedStringSet&& that) noexcept =\n      default;\n\n  // Iteration over the set.\n  iterator begin() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  iterator cbegin() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  iterator end() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  iterator cend() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns a proxy for `LinearSortedStringSet` where each element is\n  // represented as `SplitElement` rather than `absl::string_view`. This is\n  // more efficient but less convenient.\n  //\n  // The `SplitElements` object is valid while the `LinearSortedStringSet` is\n  // valid.\n  SplitElements split_elements() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns `true` if the set is empty.\n  bool empty() const { return chunks_.empty(); }\n\n  // Returns the number of elements.\n  size_t size() const {\n    return chunks_.empty() ? 0 : chunks_.back().cumulative_end_index;\n  }\n\n  // Returns `true` if `element` is present in the set.\n  //\n  // If `index != nullptr`, sets `*index` to the index of `element` in the set,\n  // or to the index where it would be inserted.\n  //\n  // Time complexity: `O(log(size / chunk_size) + chunk_size)`.\n  bool contains(absl::string_view element, size_t* index = nullptr) const;\n\n  friend bool operator==(const ChunkedSortedStringSet& a,\n                         const ChunkedSortedStringSet& b) {\n    return Equal(a, b);\n  }\n  friend StrongOrdering RIEGELI_COMPARE(const ChunkedSortedStringSet& a,\n                                        const ChunkedSortedStringSet& b) {\n    return Compare(a, b);\n  }\n\n  template <typename HashState>\n  friend HashState AbslHashValue(HashState hash_state,\n                                 const ChunkedSortedStringSet& self) {\n    return self.HashValue(std::move(hash_state));\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const ChunkedSortedStringSet* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->chunks_);\n  }\n\n  // Returns the size of data that would be written by `Encode()`.\n  size_t EncodedSize() const;\n\n  // Encodes the set to a sequence of bytes.\n  //\n  // As for now the encoding is not guaranteed to not change in future.\n  // Please ask qrczak@google.com if you need stability.\n  template <typename Dest,\n            std::enable_if_t<TargetRefSupportsDependency<Writer*, Dest>::value,\n                             int> = 0>\n  absl::Status Encode(Dest&& dest) const;\n\n  // Decodes the set from the encoded form.\n  template <typename Src,\n            std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value,\n                             int> = 0>\n  absl::Status Decode(Src&& src, DecodeOptions options = DecodeOptions());\n\n private:\n  struct Chunk {\n    LinearSortedStringSet set;\n    size_t cumulative_end_index;\n  };\n\n  using Chunks = absl::InlinedVector<Chunk, 1>;\n\n  struct CompareFirst {\n    StrongOrdering operator()(Chunks::const_iterator current) const {\n      return riegeli::Compare(current->set.first(), element);\n    }\n    absl::string_view element;\n  };\n\n  template <typename LinearIterator>\n  static LinearIterator GetLinearIterator(const LinearSortedStringSet& set);\n\n  explicit ChunkedSortedStringSet(Chunks&& chunks);\n\n  bool ContainsImpl(absl::string_view element) const;\n  bool ContainsWithIndexImpl(absl::string_view element, size_t& index) const;\n  static bool Equal(const ChunkedSortedStringSet& a,\n                    const ChunkedSortedStringSet& b);\n  static StrongOrdering Compare(const ChunkedSortedStringSet& a,\n                                const ChunkedSortedStringSet& b);\n  template <typename HashState>\n  HashState HashValue(HashState hash_state) const;\n\n  absl::Status EncodeImpl(Writer& dest) const;\n  absl::Status DecodeImpl(Reader& src, DecodeOptions options);\n\n  // Invariant: no `chunks_` are empty.\n  Chunks chunks_;\n};\n\n// Iterates over a `LinearSortedStringSet` in the sorted order.\ntemplate <typename LinearIterator>\nclass ChunkedSortedStringSet::IteratorImpl\n    : public WithEqual<IteratorImpl<LinearIterator>> {\n public:\n  // `iterator_concept` is only `std::input_iterator_tag` because the\n  // `std::forward_iterator` requirement and above require references to remain\n  // valid while the range exists.\n  using iterator_concept = std::input_iterator_tag;\n  // `iterator_category` is only `std::input_iterator_tag` also because the\n  // `LegacyForwardIterator` requirement and above require `reference` to be\n  // a true reference type.\n  using iterator_category = std::input_iterator_tag;\n  using value_type = typename LinearIterator::value_type;\n  using reference = value_type;\n  using pointer = ArrowProxy<reference>;\n  using difference_type = ptrdiff_t;\n\n  // A sentinel value, equal to `end()`.\n  IteratorImpl() = default;\n\n  IteratorImpl(const IteratorImpl& that) = default;\n  IteratorImpl& operator=(const IteratorImpl& that) = default;\n\n  IteratorImpl(IteratorImpl&& that) noexcept = default;\n  IteratorImpl& operator=(IteratorImpl&& that) noexcept = default;\n\n  // Returns the current element.\n  //\n  // The `value_type` is valid until the next non-const operation on this\n  // `IteratorImpl` because data behind the `value_type` are conditionally owned\n  // by the `IteratorImpl`.\n  reference operator*() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT(current_iterator_ != LinearIterator())\n        << \"Failed precondition of \"\n           \"ChunkedSortedStringSet::IteratorImpl::operator*: \"\n           \"iterator is end()\";\n    return *current_iterator_;\n  }\n\n  pointer operator->() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT(current_iterator_ != LinearIterator())\n        << \"Failed precondition of \"\n           \"ChunkedSortedStringSet::IteratorImpl::operator->: \"\n           \"iterator is end()\";\n    return pointer(**this);\n  }\n\n  IteratorImpl& operator++();\n  IteratorImpl operator++(int) {\n    const IteratorImpl tmp = *this;\n    ++*this;\n    return tmp;\n  }\n\n  // Iterators can be compared even if they are associated with different\n  // `ChunkedSortedStringSet` objects. All `end()` values are equal, while all\n  // other values are not equal.\n  friend bool operator==(const IteratorImpl& a, const IteratorImpl& b) {\n    return a.current_iterator_ == b.current_iterator_;\n  }\n\n private:\n  friend class ChunkedSortedStringSet;  // For `IteratorImpl()`.\n\n  using Chunks = ChunkedSortedStringSet::Chunks;\n\n  explicit IteratorImpl(const ChunkedSortedStringSet* set)\n      : current_iterator_(\n            set->chunks_.empty()\n                ? LinearIterator()\n                : GetLinearIterator<LinearIterator>(set->chunks_.front().set)),\n        current_chunk_iterator_(set->chunks_.cbegin()),\n        set_(set) {}\n\n  // Invariant:\n  //    if `current_chunk_iterator_ == set_->chunks_.cend()` then\n  //        `current_iterator_ == LinearIterator()`\n  LinearIterator current_iterator_;\n  Chunks::const_iterator current_chunk_iterator_ = Chunks::const_iterator();\n  const ChunkedSortedStringSet* set_ = nullptr;\n};\n\n// A proxy for `ChunkedSortedStringSet` where each element is represented as\n// `SplitElement` rather than `absl::string_view`. This is more efficient but\n// less convenient.\nclass ChunkedSortedStringSet::SplitElements {\n public:\n  using value_type = SplitElement;\n  using reference = value_type;\n  using const_reference = reference;\n  using iterator = SplitElementIterator;\n  using const_iterator = iterator;\n  using size_type = size_t;\n  using difference_type = ptrdiff_t;\n\n  SplitElements(const SplitElements& that) = default;\n  SplitElements& operator=(const SplitElements& that) = default;\n\n  // Iteration over the set.\n  //\n  // The `SplitElementIterator` is valid while the `ChunkedSortedStringSet` is\n  // valid. The `SplitElements` object does not need to be kept valid.\n  SplitElementIterator begin() const { return SplitElementIterator(set_); }\n  SplitElementIterator cbegin() const { return begin(); }\n  SplitElementIterator end() const { return SplitElementIterator(); }\n  SplitElementIterator cend() const { return end(); }\n\n private:\n  friend class ChunkedSortedStringSet;  // For `SplitElements()`.\n\n  explicit SplitElements(const ChunkedSortedStringSet* set) : set_(set) {}\n\n  const ChunkedSortedStringSet* set_;\n};\n\n// Builds a `ChunkedSortedStringSet` from a sorted sequence of strings.\nclass ChunkedSortedStringSet::Builder {\n public:\n  // Begins with an empty set.\n  explicit Builder(Options options = Options());\n\n  Builder(Builder&& that) noexcept;\n  Builder& operator=(Builder&& that) noexcept;\n\n  ~Builder();\n\n  // Makes `*this` equivalent to a newly constructed `Builder`.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Options options = Options());\n\n  // Returns an output iterator which inserts elements to this `Builder`.\n  // Consecutive duplicates are inserted only once.\n  //\n  // Each inserted element must be greater than or equal to the last inserted\n  // element.\n  //\n  // Inserting with a `NextInsertIterator` is equivalent to calling\n  // `InsertNext()`. In particular if multiple iterators and explicit\n  // `InsertNext()` calls are used together, then their combined element\n  // sequence must be ordered.\n  NextInsertIterator NextInserter();\n\n  // Inserts an element. Consecutive duplicates are inserted only once.\n  //\n  // Precondition: `element` is greater than or equal to the last inserted\n  // element.\n  //\n  // Returns `true` if `element` was inserted, or `false` if it is equal to the\n  // last inserted element.\n  //\n  // If `std::string&&` is passed, it is moved only if the result is `true`.\n  //\n  // `std::string&&` is accepted with a template to avoid implicit conversions\n  // to `std::string` which can be ambiguous against `absl::string_view`\n  // (e.g. `const char*`).\n  bool InsertNext(absl::string_view element);\n  template <typename Element,\n            std::enable_if_t<std::is_same_v<Element, std::string>, int> = 0>\n  bool InsertNext(Element&& element);\n\n  // Inserts an element. Elements out of order are skipped.\n  //\n  // Returns `true` if `element` was inserted, `false` if it is equal to the\n  // last inserted element, or `absl::FailedPreconditionError()` if it is less\n  // than the last inserted element.\n  //\n  // If `std::string&&` is passed, it is moved only if the result is `true`.\n  //\n  // `std::string&&` is accepted with a template to avoid implicit conversions\n  // to `std::string` which can be ambiguous against `absl::string_view`\n  // (e.g. `const char*`).\n  absl::StatusOr<bool> TryInsertNext(absl::string_view element);\n  template <typename Element,\n            std::enable_if_t<std::is_same_v<Element, std::string>, int> = 0>\n  absl::StatusOr<bool> TryInsertNext(Element&& element);\n\n  // Returns `true` if the set is empty.\n  bool empty() const { return chunks_.empty() && current_builder_.empty(); }\n\n  // Returns the number of elements.\n  size_t size() const {\n    return (chunks_.empty() ? 0 : chunks_.back().cumulative_end_index) +\n           current_builder_.size();\n  }\n\n  // Returns the last inserted element. The set must not be empty.\n  absl::string_view last() const {\n    RIEGELI_ASSERT(!empty())\n        << \"Failed precondition of ChunkedSortedStringSet::Builder::last(): \"\n           \"empty set\";\n    return current_builder_.last();\n  }\n\n  // Builds the `ChunkedSortedStringSet` and resets the `Builder` to empty\n  // state.\n  ChunkedSortedStringSet Build();\n\n private:\n  void ApplySizeHint(size_t size_hint);\n  void AddChunk();\n\n  // This template is defined and used only in chunked_sorted_string_set.cc.\n  template <typename Element>\n  absl::StatusOr<bool> InsertNextImpl(Element&& element);\n\n  size_t chunk_size_;\n  Chunks chunks_;\n  LinearSortedStringSet::Builder current_builder_;\n};\n\n// Inserts elements to a `ChunkedSortedStringSet::Builder`. Consecutive\n// duplicates are inserted only once.\n//\n// Each inserted element must be greater than or equal to the last inserted\n// element.\nclass ChunkedSortedStringSet::NextInsertIterator {\n public:\n  using iterator_concept = std::output_iterator_tag;\n  using iterator_category = std::output_iterator_tag;\n  using value_type = absl::string_view;\n  using pointer = void;\n  using difference_type = ptrdiff_t;\n\n  class reference {\n   public:\n    // Inserts the next element.\n    //\n    // `std::string&&` is accepted with a template to avoid implicit conversions\n    // to `std::string` which can be ambiguous against `absl::string_view`\n    // (e.g. `const char*`).\n    const reference& operator=(absl::string_view element) const {\n      builder_->InsertNext(element);\n      return *this;\n    }\n    template <typename Element,\n              std::enable_if_t<std::is_same_v<Element, std::string>, int> = 0>\n    const reference& operator=(Element&& element) const {\n      // `std::move(element)` is correct and `std::forward<Element>(element)` is\n      // not necessary: `Element` is always `std::string`, never an lvalue\n      // reference.\n      builder_->InsertNext(std::move(element));\n      return *this;\n    }\n\n   private:\n    friend class NextInsertIterator;\n    explicit reference(Builder* builder) : builder_(builder) {}\n    Builder* builder_;\n  };\n\n  // A sentinel value.\n  NextInsertIterator() = default;\n\n  NextInsertIterator(const NextInsertIterator& that) = default;\n  NextInsertIterator& operator=(const NextInsertIterator& that) = default;\n\n  reference operator*() const {\n    RIEGELI_ASSERT_NE(builder_, nullptr)\n        << \"Failed precondition of NextInsertIterator::operator*: \"\n           \"iterator is sentinel\";\n    return reference(builder_);\n  }\n\n  NextInsertIterator& operator++() { return *this; }\n  NextInsertIterator operator++(int) { return ++*this; }\n\n  Builder* builder() const { return builder_; }\n\n private:\n  friend class Builder;  // For `NextInsertIterator()`.\n\n  explicit NextInsertIterator(Builder* builder) : builder_(builder) {}\n\n  Builder* builder_ = nullptr;\n};\n\n// Implementation details follow.\n\ntemplate <typename Src,\n          std::enable_if_t<IsIterableOf<Src, absl::string_view>::value, int>>\nChunkedSortedStringSet ChunkedSortedStringSet::FromSorted(Src&& src,\n                                                          Options options) {\n  using std::begin;\n  auto iter = begin(src);\n  using std::end;\n  auto end_iter = end(src);\n  if (IsRandomAccessIterable<Src>::value) {\n    options.set_size_hint(std::distance(iter, end_iter));\n  }\n  ChunkedSortedStringSet::Builder builder(std::move(options));\n  for (; iter != end_iter; ++iter) {\n    builder.InsertNext(*MaybeMakeMoveIterator<Src>(iter));\n  }\n  return builder.Build();\n}\n\ntemplate <typename Src,\n          std::enable_if_t<IsIterableOf<Src, absl::string_view>::value, int>>\ninline ChunkedSortedStringSet ChunkedSortedStringSet::FromUnsorted(\n    Src&& src, Options options) {\n  using std::begin;\n  auto iter = begin(src);\n  using std::end;\n  auto end_iter = end(src);\n  if (IsRandomAccessIterable<Src>::value) {\n    options.set_size_hint(std::distance(iter, end_iter));\n  }\n  using SrcIterator = decltype(iter);\n  std::vector<SrcIterator> iterators;\n  iterators.reserve(options.size_hint());\n  for (; iter != end_iter; ++iter) iterators.push_back(iter);\n  std::sort(iterators.begin(), iterators.end(),\n            [](const SrcIterator& a, const SrcIterator& b) {\n              return absl::string_view(*a) < absl::string_view(*b);\n            });\n\n  options.set_size_hint(iterators.size());\n  ChunkedSortedStringSet::Builder builder(std::move(options));\n  for (const SrcIterator& iter : iterators) {\n    builder.InsertNext(*MaybeMakeMoveIterator<Src>(iter));\n  }\n  return builder.Build();\n}\n\ninline ChunkedSortedStringSet::iterator ChunkedSortedStringSet::begin() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return iterator(this);\n}\n\ninline ChunkedSortedStringSet::iterator ChunkedSortedStringSet::cbegin() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return begin();\n}\n\ninline ChunkedSortedStringSet::iterator ChunkedSortedStringSet::end() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return iterator();\n}\n\ninline ChunkedSortedStringSet::iterator ChunkedSortedStringSet::cend() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return end();\n}\n\ninline bool ChunkedSortedStringSet::contains(absl::string_view element,\n                                             size_t* index) const {\n  if (index == nullptr) {\n    return ContainsImpl(element);\n  } else {\n    return ContainsWithIndexImpl(element, *index);\n  }\n}\n\ninline ChunkedSortedStringSet::SplitElements\nChunkedSortedStringSet::split_elements() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return SplitElements(this);\n}\n\ntemplate <>\ninline LinearSortedStringSet::Iterator\nChunkedSortedStringSet::GetLinearIterator<LinearSortedStringSet::Iterator>(\n    const LinearSortedStringSet& set) {\n  return set.cbegin();\n}\n\ntemplate <>\ninline LinearSortedStringSet::SplitElementIterator ChunkedSortedStringSet::\n    GetLinearIterator<LinearSortedStringSet::SplitElementIterator>(\n        const LinearSortedStringSet& set) {\n  return set.split_elements().cbegin();\n}\n\ntemplate <typename HashState>\nHashState ChunkedSortedStringSet::HashValue(HashState hash_state) const {\n  for (const absl::string_view element : *this) {\n    hash_state = HashState::combine(std::move(hash_state), element);\n  }\n  return HashState::combine(std::move(hash_state), size());\n}\n\ntemplate <\n    typename Dest,\n    std::enable_if_t<TargetRefSupportsDependency<Writer*, Dest>::value, int>>\ninline absl::Status ChunkedSortedStringSet::Encode(Dest&& dest) const {\n  DependencyRef<Writer*, Dest> dest_dep(std::forward<Dest>(dest));\n  if (dest_dep.IsOwning()) dest_dep->SetWriteSizeHint(EncodedSize());\n  absl::Status status = EncodeImpl(*dest_dep);\n  if (dest_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_dep->Close())) {\n      status.Update(dest_dep->status());\n    }\n  }\n  return status;\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ChunkedSortedStringSet::Decode(Src&& src,\n                                                   DecodeOptions options) {\n  DependencyRef<Reader*, Src> src_dep(std::forward<Src>(src));\n  if (src_dep.IsOwning()) src_dep->SetReadAllHint(true);\n  absl::Status status = DecodeImpl(*src_dep, options);\n  if (src_dep.IsOwning()) {\n    if (ABSL_PREDICT_TRUE(status.ok())) src_dep->VerifyEnd();\n    if (ABSL_PREDICT_FALSE(!src_dep->Close())) status.Update(src_dep->status());\n  }\n  return status;\n}\n\nextern template ChunkedSortedStringSet::IteratorImpl<\n    LinearSortedStringSet::Iterator>&\nChunkedSortedStringSet::IteratorImpl<\n    LinearSortedStringSet::Iterator>::operator++();\nextern template ChunkedSortedStringSet::IteratorImpl<\n    LinearSortedStringSet::SplitElementIterator>&\nChunkedSortedStringSet::IteratorImpl<\n    LinearSortedStringSet::SplitElementIterator>::operator++();\n\ninline ChunkedSortedStringSet::NextInsertIterator\nChunkedSortedStringSet::Builder::NextInserter() {\n  return NextInsertIterator(this);\n}\n\nextern template bool ChunkedSortedStringSet::Builder::InsertNext(\n    std::string&& element);\n\nextern template absl::StatusOr<bool>\nChunkedSortedStringSet::Builder::TryInsertNext(std::string&& element);\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CONTAINERS_CHUNKED_SORTED_STRING_SET_H_\n"
  },
  {
    "path": "riegeli/containers/linear_sorted_string_set.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/containers/linear_sorted_string_set.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <algorithm>\n#include <cstring>\n#include <initializer_list>\n#include <ios>\n#include <optional>\n#include <ostream>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/bits.h\"\n#include \"absl/status/status.h\"\n#include \"absl/status/statusor.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compact_string.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/debug.h\"\n#include \"riegeli/base/stream_utils.h\"\n#include \"riegeli/bytes/compact_string_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/endian/endian_reading.h\"\n#include \"riegeli/varint/varint_reading.h\"\n#include \"riegeli/varint/varint_writing.h\"\n\nnamespace riegeli {\n\nnamespace {\n\ninline size_t CommonPrefix(absl::string_view a, absl::string_view b) {\n  const size_t min_length = UnsignedMin(a.size(), b.size());\n  size_t length = 0;\n  if (min_length < sizeof(uint64_t)) {\n    // Compare byte by byte.\n    while (length < min_length) {\n      if (a[length] != b[length]) return length;\n      ++length;\n    }\n    return length;\n  }\n\n  // Compare whole blocks, except for the last pair.\n  const size_t limit = min_length - sizeof(uint64_t);\n  while (length < limit) {\n    const uint64_t xor_result = ReadLittleEndian<uint64_t>(a.data() + length) ^\n                                ReadLittleEndian<uint64_t>(b.data() + length);\n    if (xor_result != 0) {\n      return length + IntCast<size_t>(absl::countr_zero(xor_result)) / 8;\n    }\n    length += sizeof(uint64_t);\n  }\n  // Compare the last, possible incomplete blocks as whole blocks shifted\n  // backwards.\n  const uint64_t xor_result = ReadLittleEndian<uint64_t>(a.data() + limit) ^\n                              ReadLittleEndian<uint64_t>(b.data() + limit);\n  if (xor_result != 0) {\n    return limit + IntCast<size_t>(absl::countr_zero(xor_result)) / 8;\n  }\n  return limit + sizeof(uint64_t);\n}\n\n}  // namespace\n\nLinearSortedStringSet LinearSortedStringSet::FromSorted(\n    std::initializer_list<absl::string_view> src) {\n  return FromSorted<>(src);\n}\n\nLinearSortedStringSet LinearSortedStringSet::FromUnsorted(\n    std::initializer_list<absl::string_view> src) {\n  return FromUnsorted<>(src);\n}\n\ninline LinearSortedStringSet::LinearSortedStringSet(CompactString&& encoded)\n    : encoded_(std::move(encoded)) {}\n\nsize_t LinearSortedStringSet::size() const {\n  size_t size = 0;\n  size_t current_length = 0;\n  const absl::string_view encoded_view = encoded_;\n  const char* ptr = encoded_view.data();\n  const char* const limit = ptr + encoded_view.size();\n  while (ptr != limit) {\n    uint64_t tagged_length;\n    const size_t tagged_length_length =\n        ReadVarint64(ptr, PtrDistance(ptr, limit), tagged_length);\n    RIEGELI_ASSUME_GT(tagged_length_length, 0u)\n        << \"Malformed LinearSortedStringSet encoding (tagged_length)\";\n    ptr += tagged_length_length;\n    const uint64_t unshared_length = tagged_length >> 1;\n    uint64_t shared_length;\n    if ((tagged_length & 1) == 0) {\n      // `shared_length == 0` and is not stored.\n      shared_length = 0;\n    } else {\n      // `shared_length > 0` and is stored.\n      const size_t shared_length_length =\n          ReadVarint64(ptr, PtrDistance(ptr, limit), shared_length);\n      RIEGELI_ASSUME_GT(shared_length_length, 0u)\n          << \"Malformed LinearSortedStringSet encoding (shared_length)\";\n      ptr += shared_length_length;\n      // Compare `<` instead of `<=`, before `++shared_length`.\n      RIEGELI_ASSERT_LT(shared_length, current_length)\n          << \"Malformed LinearSortedStringSet encoding \"\n             \"(shared_length larger than previous element)\";\n      ++shared_length;\n    }\n    RIEGELI_ASSERT_LE(unshared_length, PtrDistance(ptr, limit))\n        << \"Malformed LinearSortedStringSet encoding (unshared)\";\n    current_length = IntCast<size_t>(shared_length + unshared_length);\n    ptr += IntCast<size_t>(unshared_length);\n    ++size;\n  }\n  return size;\n}\n\nabsl::string_view LinearSortedStringSet::first() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  RIEGELI_ASSERT(!empty())\n      << \"Failed precondition of LinearSortedStringSet::first(): \"\n         \"empty set\";\n  const absl::string_view encoded_view = encoded_;\n  uint64_t tagged_length;\n  const size_t tagged_length_length =\n      ReadVarint64(encoded_view.data(), encoded_view.size(), tagged_length);\n  RIEGELI_ASSUME_GT(tagged_length_length, 0u)\n      << \"Malformed LinearSortedStringSet encoding (tagged_length)\";\n  RIEGELI_ASSERT_EQ(tagged_length & 1, 0u)\n      << \"Malformed LinearSortedStringSet encoding \"\n         \"(first element has shared_length > 0)\";\n  const uint64_t length = tagged_length >> 1;\n  RIEGELI_ASSERT_LE(length, encoded_view.size() - tagged_length_length)\n      << \"Malformed LinearSortedStringSet encoding (unshared)\";\n  return absl::string_view(encoded_view.data() + tagged_length_length,\n                           IntCast<size_t>(length));\n}\n\nbool LinearSortedStringSet::ContainsImpl(absl::string_view element,\n                                         SplitElementIterator iterator,\n                                         size_t& cumulative_index) {\n  // Length of the prefix shared between `element` and `*iterator`.\n  size_t common_length = 0;\n  for (; iterator != SplitElementIterator(); ++iterator, ++cumulative_index) {\n    // It would be incorrect to assume that if\n    // `found.prefix().size() < common_length` then `*iterator > element`\n    // because `found.prefix().size()` is not guaranteed to be maximal.\n    const SplitElement found = *iterator;\n    common_length = UnsignedMin(common_length, found.prefix().size());\n    RIEGELI_ASSUME_LE(common_length, element.size())\n        << \"The invariant common_length <= element.size() should hold\";\n    if (common_length < found.prefix().size()) {\n      common_length += CommonPrefix(found.prefix().substr(common_length),\n                                    element.substr(common_length));\n      if (common_length < found.prefix().size()) {\n        RIEGELI_ASSUME_LE(common_length, element.size())\n            << \"The invariant common_length <= element.size() should hold\";\n        // The first difference, if any, is at\n        // `found.prefix().data() + common_length`.\n        RIEGELI_ASSERT_EQ(found.prefix().substr(0, common_length),\n                          element.substr(0, common_length))\n            << \"common_length should cover an equal prefix\";\n        const absl::string_view found_middle =\n            found.prefix().substr(common_length);\n        const absl::string_view element_suffix = element.substr(common_length);\n        RIEGELI_ASSERT(!found_middle.empty())\n            << \"Implied by common_length < found_prefix().size()\";\n        RIEGELI_ASSERT(element_suffix.empty() ||\n                       found_middle.front() != element_suffix.front())\n            << \"common_length should cover the maximal common prefix\";\n        if (element_suffix.empty() ||\n            static_cast<unsigned char>(found_middle.front()) >\n                static_cast<unsigned char>(element_suffix.front())) {\n          return false;\n        }\n        continue;\n      }\n    }\n\n    RIEGELI_ASSERT_GE(common_length, found.prefix().size())\n        << \"common_length < found.prefix().size() was handled above\";\n    size_t common_length_in_suffix = common_length - found.prefix().size();\n    RIEGELI_ASSUME_LE(common_length_in_suffix, found.suffix().size())\n        << \"The invariant common_length <= found.size() should hold\";\n    RIEGELI_ASSUME_LE(common_length, element.size())\n        << \"The invariant common_length <= element.size() should hold\";\n    common_length +=\n        CommonPrefix(found.suffix().substr(common_length_in_suffix),\n                     element.substr(common_length));\n    common_length_in_suffix = common_length - found.prefix().size();\n    RIEGELI_ASSUME_LE(common_length_in_suffix, found.suffix().size())\n        << \"The invariant common_length <= found.size() should hold\";\n    RIEGELI_ASSUME_LE(common_length, element.size())\n        << \"The invariant common_length <= element.size() should hold\";\n    // The first difference, if any, is at\n    // `found.suffix().data() + (common_length - found_prefix().size())`.\n    RIEGELI_ASSERT_EQ(\n        SplitElement(found.prefix(),\n                     found.suffix().substr(0, common_length_in_suffix)),\n        element.substr(0, common_length))\n        << \"common_length should cover an equal prefix\";\n    const absl::string_view found_suffix =\n        found.suffix().substr(common_length_in_suffix);\n    const absl::string_view element_suffix = element.substr(common_length);\n    RIEGELI_ASSERT(found_suffix.empty() || element_suffix.empty() ||\n                   found_suffix.front() != element_suffix.front())\n        << \"common_length should cover the maximal common prefix\";\n    if (found_suffix.empty()) {\n      if (element_suffix.empty()) return true;\n    } else {\n      if (element_suffix.empty() ||\n          static_cast<unsigned char>(found_suffix.front()) >\n              static_cast<unsigned char>(element_suffix.front())) {\n        return false;\n      }\n    }\n  }\n  return false;  // Not found.\n}\n\nbool LinearSortedStringSet::Equal(const LinearSortedStringSet& a,\n                                  const LinearSortedStringSet& b) {\n  return std::equal(a.split_elements().cbegin(), SplitElementIterator(),\n                    b.split_elements().cbegin(), SplitElementIterator());\n}\n\nStrongOrdering LinearSortedStringSet::Compare(const LinearSortedStringSet& a,\n                                              const LinearSortedStringSet& b) {\n  SplitElementIterator a_iter = a.split_elements().cbegin();\n  SplitElementIterator b_iter = b.split_elements().cbegin();\n  while (a_iter != SplitElementIterator()) {\n    if (b_iter == SplitElementIterator()) return StrongOrdering::greater;\n    if (const StrongOrdering ordering = riegeli::Compare(*a_iter, *b_iter);\n        ordering != 0) {\n      return ordering;\n    }\n    ++a_iter;\n    ++b_iter;\n  }\n  return b_iter == SplitElementIterator() ? StrongOrdering::equal\n                                          : StrongOrdering::less;\n}\n\nabsl::Status LinearSortedStringSet::EncodeImpl(Writer& dest) const {\n  if (ABSL_PREDICT_FALSE(!WriteVarint64(uint64_t{encoded_.size()}, dest))) {\n    return dest.status();\n  }\n  if (ABSL_PREDICT_FALSE(!dest.Write(encoded_))) return dest.status();\n  return absl::OkStatus();\n}\n\nabsl::Status LinearSortedStringSet::DecodeImpl(Reader& src,\n                                               DecodeOptions options) {\n  uint64_t encoded_size;\n  if (ABSL_PREDICT_FALSE(!ReadVarint64(src, encoded_size))) {\n    return src.StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Malformed LinearSortedStringSet encoding \"\n                                   \"(encoded_size)\"));\n  }\n  if (ABSL_PREDICT_FALSE(encoded_size > options.max_encoded_size())) {\n    return src.AnnotateStatus(absl::ResourceExhaustedError(absl::StrCat(\n        \"Maximum LinearSortedStringSet encoded length exceeded: \", encoded_size,\n        \" > \", options.max_encoded_size())));\n  }\n  CompactString encoded(IntCast<size_t>(encoded_size));\n  if (ABSL_PREDICT_FALSE(\n          !src.Read(IntCast<size_t>(encoded_size), encoded.data()))) {\n    return src.StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Malformed LinearSortedStringSet encoding \"\n                                   \"(encoded)\"));\n  }\n\n  // Validate `encoded` and update `*options.decode_state()`.\n  size_t size = 0;\n  size_t current_length = 0;\n  CompactString current_if_validated_and_shared;\n  std::optional<absl::string_view> current_if_validated;\n  if (options.decode_state() != nullptr &&\n      options.decode_state()->last != nullptr) {\n    current_if_validated = *options.decode_state()->last;\n  }\n  const absl::string_view encoded_view = encoded;\n  const char* ptr = encoded_view.data();\n  const char* const limit = ptr + encoded_view.size();\n  while (ptr != limit) {\n    uint64_t tagged_length;\n    const size_t tagged_length_length =\n        ReadVarint64(ptr, PtrDistance(ptr, limit), tagged_length);\n    if (ABSL_PREDICT_FALSE(tagged_length_length == 0)) {\n      return src.AnnotateStatus(absl::InvalidArgumentError(\n          \"Malformed LinearSortedStringSet encoding (tagged_length)\"));\n    }\n    ptr += tagged_length_length;\n    const uint64_t unshared_length = tagged_length >> 1;\n    if ((tagged_length & 1) == 0) {\n      // `shared_length == 0` and is not stored.\n      if (ABSL_PREDICT_FALSE(unshared_length > PtrDistance(ptr, limit))) {\n        return src.AnnotateStatus(absl::InvalidArgumentError(\n            \"Malformed LinearSortedStringSet encoding (unshared)\"));\n      }\n      current_length = IntCast<size_t>(unshared_length);\n      if (options.validate()) {\n        if (ABSL_PREDICT_TRUE(current_if_validated != std::nullopt) &&\n            ABSL_PREDICT_FALSE(absl::string_view(ptr, current_length) <=\n                               *current_if_validated)) {\n          return src.AnnotateStatus(absl::InvalidArgumentError(absl::StrCat(\n              \"Elements are not sorted and unique: new \",\n              riegeli::Debug(absl::string_view(ptr, current_length)),\n              \" <= last \", riegeli::Debug(*current_if_validated))));\n        }\n        current_if_validated_and_shared.clear();\n        current_if_validated = absl::string_view(ptr, current_length);\n      }\n    } else {\n      // `shared_length > 0` and is stored.\n      uint64_t shared_length;\n      const size_t shared_length_length =\n          ReadVarint64(ptr, PtrDistance(ptr, limit), shared_length);\n      if (ABSL_PREDICT_FALSE(shared_length_length == 0)) {\n        return src.AnnotateStatus(absl::InvalidArgumentError(\n            \"Malformed LinearSortedStringSet encoding (shared_length)\"));\n      }\n      ptr += shared_length_length;\n      // Compare `>=` instead of `>`, before `++shared_length`.\n      if (ABSL_PREDICT_FALSE(shared_length >= current_length)) {\n        return src.AnnotateStatus(absl::InvalidArgumentError(\n            \"Malformed LinearSortedStringSet encoding \"\n            \"(shared_length larger than previous element)\"));\n      }\n      ++shared_length;\n      if (ABSL_PREDICT_FALSE(unshared_length > PtrDistance(ptr, limit))) {\n        return src.AnnotateStatus(absl::InvalidArgumentError(\n            \"Malformed LinearSortedStringSet encoding (unshared)\"));\n      }\n      current_length = IntCast<size_t>(shared_length + unshared_length);\n      if (options.validate()) {\n        if (ABSL_PREDICT_TRUE(current_if_validated != std::nullopt) &&\n            ABSL_PREDICT_FALSE(absl::string_view(ptr, unshared_length) <=\n                               current_if_validated->substr(shared_length))) {\n          return src.AnnotateStatus(absl::InvalidArgumentError(absl::StrCat(\n              \"Elements are not sorted and unique: new \",\n              riegeli::Debug(\n                  SplitElement(current_if_validated->substr(0, shared_length),\n                               absl::string_view(ptr, unshared_length))),\n              \" <= last \", riegeli::Debug(*current_if_validated))));\n        }\n        // The unshared part of the next element will be written here.\n        char* current_unshared;\n        if (current_if_validated_and_shared.empty()) {\n          RIEGELI_ASSERT(current_if_validated != std::nullopt)\n              << \"shared_length > 0 implies that this is not the first element\";\n          char* const current_data =\n              current_if_validated_and_shared.resize(current_length, 0);\n          std::memcpy(current_data, current_if_validated->data(),\n                      IntCast<size_t>(shared_length));\n          current_unshared = current_data + IntCast<size_t>(shared_length);\n        } else {\n          current_unshared = current_if_validated_and_shared.resize(\n              current_length, IntCast<size_t>(shared_length));\n        }\n        std::memcpy(current_unshared, ptr, IntCast<size_t>(unshared_length));\n        current_if_validated = current_if_validated_and_shared;\n      }\n    }\n    ptr += IntCast<size_t>(unshared_length);\n    ++size;\n  }\n  if (options.decode_state() != nullptr && size > 0) {\n    options.decode_state()->cumulative_size += size;\n    if (options.validate()) {\n      if (current_if_validated_and_shared.empty()) {\n        options.decode_state()->last = *current_if_validated;\n      } else {\n        options.decode_state()->last =\n            std::move(current_if_validated_and_shared);\n      }\n    }\n  }\n  encoded_ = std::move(encoded);\n  return absl::OkStatus();\n}\n\nLinearSortedStringSet::Iterator& LinearSortedStringSet::Iterator::operator++() {\n  RIEGELI_ASSERT_NE(cursor_, nullptr)\n      << \"Failed precondition of \"\n         \"LinearSortedStringSet::Iterator::operator++: \"\n         \"iterator is end()\";\n  if (cursor_ == limit_) {\n    // `end()` was reached.\n    cursor_ = nullptr;  // Mark `end()`.\n    length_if_unshared_ = 0;\n    current_if_shared_ = CompactString();  // Free memory.\n    return *this;\n  }\n  const char* ptr = cursor_;\n  uint64_t tagged_length;\n  const size_t tagged_length_length =\n      ReadVarint64(ptr, PtrDistance(ptr, limit_), tagged_length);\n  RIEGELI_ASSUME_GT(tagged_length_length, 0u)\n      << \"Malformed LinearSortedStringSet encoding (tagged_length)\";\n  ptr += tagged_length_length;\n  const uint64_t unshared_length = tagged_length >> 1;\n  if ((tagged_length & 1) == 0) {\n    // `shared_length == 0` and is not stored.\n    RIEGELI_ASSERT_LE(unshared_length, PtrDistance(ptr, limit_))\n        << \"Malformed LinearSortedStringSet encoding (unshared)\";\n    current_if_shared_.clear();\n    length_if_unshared_ = IntCast<size_t>(unshared_length);\n    ptr += IntCast<size_t>(unshared_length);\n    cursor_ = ptr;\n    return *this;\n  }\n  // `shared_length > 0` and is stored.\n  uint64_t shared_length;\n  const size_t shared_length_length =\n      ReadVarint64(ptr, PtrDistance(ptr, limit_), shared_length);\n  RIEGELI_ASSUME_GT(shared_length_length, 0u)\n      << \"Malformed LinearSortedStringSet encoding (shared_length)\";\n  ptr += shared_length_length;\n  // Compare `<` instead of `<=`, before `++shared_length`.\n  RIEGELI_ASSERT_LT(shared_length, length_if_unshared_ > 0\n                                       ? length_if_unshared_\n                                       : current_if_shared_.size())\n      << \"Malformed LinearSortedStringSet encoding \"\n         \"(shared_length larger than previous element)\";\n  ++shared_length;\n  RIEGELI_ASSERT_LE(unshared_length, PtrDistance(ptr, limit_))\n      << \"Malformed LinearSortedStringSet encoding (unshared)\";\n  const size_t new_size = IntCast<size_t>(shared_length + unshared_length);\n  // The unshared part of the next element will be written here.\n  char* current_unshared;\n  if (length_if_unshared_ > 0) {\n    char* const current_data = current_if_shared_.resize(new_size, 0);\n    std::memcpy(current_data, cursor_ - length_if_unshared_,\n                IntCast<size_t>(shared_length));\n    current_unshared = current_data + IntCast<size_t>(shared_length);\n  } else {\n    current_unshared =\n        current_if_shared_.resize(new_size, IntCast<size_t>(shared_length));\n  }\n  std::memcpy(current_unshared, ptr, IntCast<size_t>(unshared_length));\n  length_if_unshared_ = 0;\n  ptr += IntCast<size_t>(unshared_length);\n  cursor_ = ptr;\n  return *this;\n}\n\nLinearSortedStringSet::SplitElement::operator std::string() const {\n  return absl::StrCat(prefix(), suffix());\n}\n\nvoid LinearSortedStringSet::SplitElement::Output(std::ostream& dest) const {\n  WriteWithPadding(dest, size(), [&] {\n    dest.write(prefix().data(), IntCast<std::streamsize>(prefix().size()));\n    dest.write(suffix().data(), IntCast<std::streamsize>(suffix().size()));\n  });\n}\n\nLinearSortedStringSet::SplitElementIterator&\nLinearSortedStringSet::SplitElementIterator::operator++() {\n  RIEGELI_ASSERT_NE(cursor_, nullptr)\n      << \"Failed precondition of \"\n         \"LinearSortedStringSet::SplitElementIterator::operator++: \"\n         \"iterator is end()\";\n  if (cursor_ == limit_) {\n    // `end()` was reached.\n    cursor_ = nullptr;                    // Mark `end()`.\n    prefix_if_stored_ = CompactString();  // Free memory.\n    prefix_ = absl::string_view();\n    suffix_length_ = 0;\n    return *this;\n  }\n  const char* ptr = cursor_;\n  uint64_t tagged_length;\n  const size_t tagged_length_length =\n      ReadVarint64(ptr, PtrDistance(ptr, limit_), tagged_length);\n  RIEGELI_ASSUME_GT(tagged_length_length, 0u)\n      << \"Malformed LinearSortedStringSet encoding (tagged_length)\";\n  ptr += tagged_length_length;\n  const uint64_t unshared_length = tagged_length >> 1;\n  if ((tagged_length & 1) == 0) {\n    // `shared_length == 0` and is not stored.\n    RIEGELI_ASSERT_LE(unshared_length, PtrDistance(ptr, limit_))\n        << \"Malformed LinearSortedStringSet encoding (unshared)\";\n    prefix_ = absl::string_view();\n    suffix_length_ = IntCast<size_t>(unshared_length);\n    ptr += IntCast<size_t>(unshared_length);\n    cursor_ = ptr;\n    return *this;\n  }\n  // `shared_length > 0` and is stored.\n  uint64_t shared_length;\n  const size_t shared_length_length =\n      ReadVarint64(ptr, PtrDistance(ptr, limit_), shared_length);\n  RIEGELI_ASSUME_GT(shared_length_length, 0u)\n      << \"Malformed LinearSortedStringSet encoding (shared_length)\";\n  ptr += shared_length_length;\n  // Compare `<` instead of `<=`, before `++shared_length`.\n  RIEGELI_ASSERT_LT(shared_length, prefix_.size() + suffix_length_)\n      << \"Malformed LinearSortedStringSet encoding \"\n         \"(shared_length larger than previous element)\";\n  ++shared_length;\n  RIEGELI_ASSERT_LE(unshared_length, PtrDistance(ptr, limit_))\n      << \"Malformed LinearSortedStringSet encoding (unshared)\";\n  if (shared_length <= prefix_.size()) {\n    prefix_ = prefix_.substr(0, IntCast<size_t>(shared_length));\n  } else if (prefix_.empty()) {\n    prefix_ = absl::string_view(cursor_ - suffix_length_,\n                                IntCast<size_t>(shared_length));\n  } else {\n    // Append\n    // `absl::string_view(cursor_ - suffix_length_,\n    //                    IntCast<size_t>(shared_length) - prefix_.size())`\n    // to `prefix_`, using `prefix_if_stored_` for storage.\n\n    // The new prefix.\n    char* prefix_data;\n    // The suffix of the new prefix which is not shared with the previous\n    // element will be written here.\n    char* prefix_unshared;\n    if (prefix_if_stored_.data() == prefix_.data()) {\n      RIEGELI_ASSERT_GE(prefix_if_stored_.size(), prefix_.size())\n          << \"Failed invariant of LinearSortedStringSet::SplitElementIterator: \"\n             \"prefix_ overflows prefix_if_stored_\";\n      // `prefix_if_stored_` already begins with `prefix_`.\n      prefix_unshared = prefix_if_stored_.resize(IntCast<size_t>(shared_length),\n                                                 prefix_.size());\n      prefix_data = prefix_unshared - prefix_.size();\n    } else {\n      // Copy `prefix_` to the beginning of `prefix_if_stored_` first.\n      prefix_data = prefix_if_stored_.resize(IntCast<size_t>(shared_length), 0);\n      std::memcpy(prefix_data, prefix_.data(), prefix_.size());\n      prefix_unshared = prefix_data + prefix_.size();\n    }\n    std::memcpy(prefix_unshared, cursor_ - suffix_length_,\n                IntCast<size_t>(shared_length) - prefix_.size());\n    prefix_ = absl::string_view(prefix_data, IntCast<size_t>(shared_length));\n  }\n  ptr += IntCast<size_t>(unshared_length);\n  cursor_ = ptr;\n  suffix_length_ = IntCast<size_t>(unshared_length);\n  return *this;\n}\n\nbool LinearSortedStringSet::SplitElement::Equal(const SplitElement& a,\n                                                const SplitElement& b) {\n  if (a.size() != b.size()) return false;\n  if (a.prefix().size() < b.prefix().size()) {\n    const size_t split_point_distance = b.prefix().size() - a.prefix().size();\n    RIEGELI_ASSUME_LE(split_point_distance, a.suffix().size())\n        << \"implied by a.size() == b.size()\";\n    RIEGELI_ASSUME_EQ(a.suffix().size() - split_point_distance,\n                      b.suffix().size())\n        << \"implied by a.size() == b.size()\";\n    return a.prefix() != b.prefix().substr(0, a.prefix().size()) &&\n           a.suffix().substr(0, split_point_distance) !=\n               b.prefix().substr(a.prefix().size()) &&\n           a.suffix().substr(split_point_distance) == b.suffix();\n  } else if (a.prefix().size() == b.prefix().size()) {\n    RIEGELI_ASSUME_EQ(a.suffix().size(), b.suffix().size())\n        << \"implied by a.size() == b.size() \"\n           \"and a.prefix().size() == b.prefix().size()\";\n    return a.prefix() == b.prefix() && a.suffix() == b.suffix();\n  } else {\n    const size_t split_point_distance = a.prefix().size() - b.prefix().size();\n    RIEGELI_ASSUME_LE(split_point_distance, b.suffix().size())\n        << \"implied by a.size() == b.size()\";\n    RIEGELI_ASSUME_EQ(b.suffix().size() - split_point_distance,\n                      a.suffix().size())\n        << \"implied by a.size() == b.size()\";\n    return a.prefix().substr(0, b.prefix().size()) == b.prefix() &&\n           a.prefix().substr(b.prefix().size()) ==\n               b.suffix().substr(0, split_point_distance) &&\n           a.suffix() == b.suffix().substr(split_point_distance);\n  }\n}\n\nStrongOrdering LinearSortedStringSet::SplitElement::Compare(\n    const SplitElement& a, const SplitElement& b) {\n  if (a.prefix().size() < b.prefix().size()) {\n    if (const StrongOrdering ordering = riegeli::Compare(\n            a.prefix(), b.prefix().substr(0, a.prefix().size()));\n        ordering != 0) {\n      return ordering;\n    }\n    return riegeli::Compare(\n        a.suffix(),\n        SplitElement(b.prefix().substr(a.prefix().size()), b.suffix()));\n  } else if (a.prefix().size() == b.prefix().size()) {\n    if (const StrongOrdering ordering =\n            riegeli::Compare(a.prefix(), b.prefix());\n        ordering != 0) {\n      return ordering;\n    }\n    return riegeli::Compare(a.suffix(), b.suffix());\n  } else {\n    if (const StrongOrdering ordering = riegeli::Compare(\n            a.prefix().substr(0, b.prefix().size()), b.prefix());\n        ordering != 0) {\n      return ordering;\n    }\n    return riegeli::Compare(\n        SplitElement(a.prefix().substr(b.prefix().size()), a.suffix()),\n        b.suffix());\n  }\n}\n\nbool LinearSortedStringSet::SplitElement::Equal(const SplitElement& a,\n                                                absl::string_view b) {\n  if (a.size() != b.size()) return false;\n  RIEGELI_ASSUME_LE(a.prefix().size(), b.size())\n      << \"implied by a.size() == b.size()\";\n  RIEGELI_ASSUME_EQ(a.suffix().size(), b.size() - a.prefix().size())\n      << \"implied by a.size() == b.size()\";\n  return a.prefix() == b.substr(0, a.prefix().size()) &&\n         a.suffix() == b.substr(a.prefix().size());\n}\n\nStrongOrdering LinearSortedStringSet::SplitElement::Compare(\n    const SplitElement& a, absl::string_view b) {\n  if (a.prefix().size() <= b.size()) {\n    if (const StrongOrdering ordering =\n            riegeli::Compare(a.prefix(), b.substr(0, a.prefix().size()));\n        ordering != 0) {\n      return ordering;\n    }\n    return riegeli::Compare(a.suffix(), b.substr(a.prefix().size()));\n  } else {\n    if (const StrongOrdering ordering =\n            riegeli::Compare(a.prefix().substr(0, b.size()), b);\n        ordering != 0) {\n      return ordering;\n    }\n    return StrongOrdering::greater;\n  }\n}\n\nLinearSortedStringSet::Builder::Builder() = default;\n\nLinearSortedStringSet::Builder::Builder(Builder&& that) noexcept\n    : writer_(\n          std::exchange(that.writer_, CompactStringWriter<CompactString>())),\n      size_(std::exchange(that.size_, 0)),\n      last_(std::exchange(that.last_, std::string())) {}\n\nLinearSortedStringSet::Builder& LinearSortedStringSet::Builder::operator=(\n    Builder&& that) noexcept {\n  writer_ = std::exchange(that.writer_, CompactStringWriter<CompactString>());\n  size_ = std::exchange(that.size_, 0);\n  last_ = std::exchange(that.last_, std::string());\n  return *this;\n}\n\nLinearSortedStringSet::Builder::~Builder() = default;\n\nvoid LinearSortedStringSet::Builder::Reset() {\n  writer_.Reset();\n  size_ = 0;\n  last_.clear();\n}\n\nbool LinearSortedStringSet::Builder::InsertNext(absl::string_view element) {\n  const absl::StatusOr<bool> inserted = TryInsertNext(element);\n  RIEGELI_CHECK_OK(inserted)\n      << \"Failed precondition of LinearSortedStringSet::Builder::InsertNext()\";\n  return *inserted;\n}\n\ntemplate <typename Element,\n          std::enable_if_t<std::is_same_v<Element, std::string>, int>>\nbool LinearSortedStringSet::Builder::InsertNext(Element&& element) {\n  // `std::move(element)` is correct and `std::forward<Element>(element)` is not\n  // necessary: `Element` is always `std::string`, never an lvalue reference.\n  const absl::StatusOr<bool> inserted = TryInsertNext(std::move(element));\n  RIEGELI_CHECK_OK(inserted)\n      << \"Failed precondition of LinearSortedStringSet::Builder::InsertNext()\";\n  return *inserted;\n}\n\ntemplate bool LinearSortedStringSet::Builder::InsertNext(std::string&& element);\n\nabsl::StatusOr<bool> LinearSortedStringSet::Builder::TryInsertNext(\n    absl::string_view element) {\n  return InsertNextImpl(\n      element, [this](absl::string_view element, size_t shared_length) {\n        last_.erase(shared_length);\n        const absl::string_view unshared(element.data() + shared_length,\n                                         element.size() - shared_length);\n        last_.append(unshared);\n        RIEGELI_ASSERT_EQ(last_, element) << \"last_ incorrectly reconstructed\";\n        return unshared;\n      });\n}\n\ntemplate <typename Element,\n          std::enable_if_t<std::is_same_v<Element, std::string>, int>>\nabsl::StatusOr<bool> LinearSortedStringSet::Builder::TryInsertNext(\n    Element&& element) {\n  // `std::move(element)` is correct and `std::forward<Element>(element)` is not\n  // necessary: `Element` is always `std::string`, never an lvalue reference.\n  return InsertNextImpl(std::move(element),\n                        [this](std::string&& element, size_t shared_length) {\n                          last_ = std::move(element);\n                          return absl::string_view(last_).substr(shared_length);\n                        });\n}\n\ntemplate absl::StatusOr<bool> LinearSortedStringSet::Builder::TryInsertNext(\n    std::string&& element);\n\ntemplate <typename Element, typename UpdateLast>\nabsl::StatusOr<bool> LinearSortedStringSet::Builder::InsertNextImpl(\n    Element&& element, UpdateLast update_last) {\n  RIEGELI_ASSERT(writer_.is_open())\n      << \"Failed precondition of \"\n         \"LinearSortedStringSet::Builder::TryInsertNext(): \"\n         \"set already built or moved from\";\n  size_t shared_length = CommonPrefix(last_, element);\n  const absl::string_view unshared_element(element.data() + shared_length,\n                                           element.size() - shared_length);\n  const absl::string_view unshared_last(last_.data() + shared_length,\n                                        last_.size() - shared_length);\n  if (ABSL_PREDICT_FALSE(unshared_element <= unshared_last) && !empty()) {\n    if (ABSL_PREDICT_TRUE(unshared_element == unshared_last)) return false;\n    return absl::FailedPreconditionError(\n        absl::StrCat(\"Elements are not sorted: new \", riegeli::Debug(element),\n                     \" < last \", riegeli::Debug(last())));\n  }\n  if (shared_length == 1) {\n    // If only the first byte is shared, write the element fully unshared.\n    // The encoded length is the same, and this allows `Iterator` to avoid\n    // allocating the string.\n    shared_length = 0;\n  }\n  const absl::string_view unshared =\n      update_last(std::forward<Element>(element), shared_length);\n  const size_t unshared_length = unshared.size();\n  // `shared_length` is stored if `shared_length > 0`.\n  const uint64_t tagged_length =\n      (uint64_t{unshared_length} << 1) |\n      (shared_length > 0 ? uint64_t{1} : uint64_t{0});\n  WriteVarint64(tagged_length, writer_);\n  if (shared_length > 0) WriteVarint64(uint64_t{shared_length - 1}, writer_);\n  writer_.Write(unshared);\n  ++size_;\n  return true;\n}\n\nLinearSortedStringSet LinearSortedStringSet::Builder::Build() {\n  RIEGELI_ASSERT(writer_.is_open())\n      << \"Failed precondition of LinearSortedStringSet::Builder::Build(): \"\n         \"set already built or moved from\";\n  RIEGELI_EVAL_ASSERT(writer_.Close())\n      << \"CompactStringWriter has no reason to fail: \" << writer_.status();\n  writer_.dest().shrink_to_fit();\n  LinearSortedStringSet set(std::move(writer_.dest()));\n  writer_.Reset();\n  size_ = 0;\n  last_.clear();\n  RIEGELI_ASSERT(empty())\n      << \"Failed postcondition of LinearSortedStringSet::Builder::Build(): \"\n         \"builder should be empty\";\n  return set;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/containers/linear_sorted_string_set.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CONTAINERS_LINEAR_SORTED_STRING_SET_H_\n#define RIEGELI_CONTAINERS_LINEAR_SORTED_STRING_SET_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <algorithm>\n#include <initializer_list>\n#include <iosfwd>\n#include <iterator>\n#include <string>\n#include <type_traits>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/status/statusor.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compact_string.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/iterable.h\"\n#include \"riegeli/base/optional_compact_string.h\"\n#include \"riegeli/bytes/compact_string_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/varint/varint_writing.h\"\n\nnamespace riegeli {\n\n// A sorted set of strings, compressed by recognizing shared prefixes.\n//\n// `LinearSortedStringSet` is optimized for memory usage. It should be used\n// only with very small sets (up to tens of elements), otherwise consider\n// `ChunkedSortedStringSet`.\nclass LinearSortedStringSet : public WithCompare<LinearSortedStringSet> {\n public:\n  class Iterator;\n  class SplitElement;\n  class SplitElementIterator;\n  class SplitElements;\n  class Builder;\n  class NextInsertIterator;\n\n  // When calling `Decode()` for a sequence of sets whose elements should be\n  // ordered, a `DecodeState` is passed between calls. This is primarily used by\n  // `ChunkedSortedStringSet::Decode()`.\n  struct DecodeState {\n    // Total number of elements decoded so far. The size is calculated as a side\n    // effect of structural validation; calling `size()` later would be slower.\n    size_t cumulative_size = 0;\n    // If not `nullptr`, the last element in the last decoded set. Meaningful\n    // only if `DecodeOptions::validate()`.\n    OptionalCompactString last;\n  };\n\n  // Options for `Decode()`.\n  class DecodeOptions {\n   public:\n    DecodeOptions() noexcept {}\n\n    // If `false`, performs partial validation of the structure of data, which\n    // is sufficient to prevent undefined behavior when the set is used. The\n    // only aspect not validated is that elements are sorted and unique. This is\n    // faster. If elements are not sorted and unique, then iteration yields\n    // elements in the stored order, and `contains()` may fail to find an\n    // element which can be seen during iteration.\n    //\n    // If `true`, performs full validation of encoded data, including checking\n    // that elements are sorted and unique. This is slower. This can be used for\n    // parsing untrusted data.\n    //\n    // Default: `false`.\n    DecodeOptions& set_validate(bool validate) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      validate_ = validate;\n      return *this;\n    }\n    DecodeOptions&& set_validate(bool validate) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_validate(validate));\n    }\n    bool validate() const { return validate_; }\n\n    // `Decode()` fails if more than `max_encoded_size()` bytes would need to be\n    // allocated. This can be used for parsing untrusted data.\n    //\n    // Default: `CompactString::max_size()`.\n    DecodeOptions& set_max_encoded_size(size_t max_encoded_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      max_encoded_size_ = max_encoded_size;\n      return *this;\n    }\n    DecodeOptions&& set_max_encoded_size(size_t max_encoded_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_encoded_size(max_encoded_size));\n    }\n    size_t max_encoded_size() const { return max_encoded_size_; }\n\n    // When calling `Decode()` for a sequence of sets whose elements should be\n    // ordered, a `DecodeState` is passed between calls. This is primarily used\n    // by `ChunkedSortedStringSet::Decode()`.\n    //\n    // Default: `nullptr`.\n    DecodeOptions& set_decode_state(DecodeState* decode_state) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      decode_state_ = decode_state;\n      return *this;\n    }\n    DecodeOptions&& set_decode_state(DecodeState* decode_state) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_decode_state(decode_state));\n    }\n    DecodeState* decode_state() const { return decode_state_; }\n\n   private:\n    bool validate_ = false;\n    size_t max_encoded_size_ = CompactString::max_size();\n    DecodeState* decode_state_ = nullptr;\n  };\n\n  using value_type = absl::string_view;\n  using reference = value_type;\n  using const_reference = reference;\n  using iterator = Iterator;\n  using const_iterator = iterator;\n  using size_type = size_t;\n  using difference_type = ptrdiff_t;\n\n  // Creates a set consisting of the given elements. They must be sorted.\n  // Consecutive duplicates are inserted only once.\n  //\n  // The type of `src` must support iteration yielding `absl::string_view`:\n  // `for (const absl::string_view element : src)`,\n  // e.g. `std::vector<std::string>`.\n  template <\n      typename Src,\n      std::enable_if_t<IsIterableOf<Src, absl::string_view>::value, int> = 0>\n  static LinearSortedStringSet FromSorted(Src&& src);\n  static LinearSortedStringSet FromSorted(\n      std::initializer_list<absl::string_view> src);\n\n  // Creates a set consisting of the given elements. They do not need to be\n  // sorted. Duplicates are inserted only once.\n  //\n  // The type of `src` must support iteration yielding `absl::string_view`:\n  // `for (const absl::string_view element : src)`,\n  // e.g. `std::vector<std::string>`.\n  template <\n      typename Src,\n      std::enable_if_t<IsIterableOf<Src, absl::string_view>::value, int> = 0>\n  static LinearSortedStringSet FromUnsorted(Src&& src);\n  static LinearSortedStringSet FromUnsorted(\n      std::initializer_list<absl::string_view> src);\n\n  // An empty set.\n  LinearSortedStringSet() = default;\n\n  LinearSortedStringSet(const LinearSortedStringSet& that) = default;\n  LinearSortedStringSet& operator=(const LinearSortedStringSet& that) = default;\n\n  LinearSortedStringSet(LinearSortedStringSet&& that) noexcept = default;\n  LinearSortedStringSet& operator=(LinearSortedStringSet&& that) noexcept =\n      default;\n\n  // Iteration over the set.\n  Iterator begin() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  Iterator cbegin() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  Iterator end() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  Iterator cend() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns a proxy for `LinearSortedStringSet` where each element is\n  // represented as `SplitElement` rather than `absl::string_view`. This is\n  // more efficient but less convenient.\n  //\n  // The `SplitElements` object is valid while the `LinearSortedStringSet` is\n  // valid.\n  SplitElements split_elements() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns `true` if the set is empty.\n  bool empty() const { return encoded_.empty(); }\n\n  // Returns the number of elements.\n  //\n  // Time complexity: `O(size)`.\n  size_t size() const;\n\n  // Returns the first element. The set must not be empty.\n  absl::string_view first() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns `true` if `element` is present in the set.\n  //\n  // If `index != nullptr`, sets `*index` to the index of `element` in the set,\n  // or to the index where it would be inserted.\n  //\n  // Time complexity: `O(size)`.\n  bool contains(absl::string_view element, size_t* index = nullptr) const;\n\n  // Like `contains()`, but skips comparing `element` against `first()`.\n  bool contains_skip_first(absl::string_view element,\n                           size_t* index = nullptr) const;\n\n  friend bool operator==(const LinearSortedStringSet& a,\n                         const LinearSortedStringSet& b) {\n    return Equal(a, b);\n  }\n  friend StrongOrdering RIEGELI_COMPARE(const LinearSortedStringSet& a,\n                                        const LinearSortedStringSet& b) {\n    return Compare(a, b);\n  }\n\n  template <typename HashState>\n  friend HashState AbslHashValue(HashState hash_state,\n                                 const LinearSortedStringSet& self) {\n    return self.HashValue(std::move(hash_state));\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const LinearSortedStringSet* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->encoded_);\n  }\n\n  // Returns the size of data that would be written by `Encode()`.\n  size_t EncodedSize() const;\n\n  // Encodes the set to a sequence of bytes.\n  //\n  // As for now the encoding is not guaranteed to not change in future.\n  // Please ask qrczak@google.com if you need stability.\n  template <typename Dest,\n            std::enable_if_t<TargetRefSupportsDependency<Writer*, Dest>::value,\n                             int> = 0>\n  absl::Status Encode(Dest&& dest) const;\n\n  // Decodes the set from the encoded form.\n  template <typename Src,\n            std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value,\n                             int> = 0>\n  absl::Status Decode(Src&& src, DecodeOptions options = DecodeOptions());\n\n private:\n  explicit LinearSortedStringSet(CompactString&& encoded);\n\n  // Increments `cumulative_index` by the number of elements skipped.\n  static bool ContainsImpl(absl::string_view element,\n                           SplitElementIterator iterator,\n                           size_t& cumulative_index);\n  static bool Equal(const LinearSortedStringSet& a,\n                    const LinearSortedStringSet& b);\n  static StrongOrdering Compare(const LinearSortedStringSet& a,\n                                const LinearSortedStringSet& b);\n  template <typename HashState>\n  HashState HashValue(HashState hash_state) const;\n\n  absl::Status EncodeImpl(Writer& dest) const;\n  absl::Status DecodeImpl(Reader& src, DecodeOptions options);\n\n  // Representation of each other element, which consists of the prefix of the\n  // previous element with length shared_length, concatenated with unshared,\n  // where tagged_length = (unshared_length << 1) | (shared_length > 0 ? 1 : 0):\n  //\n  //  * tagged_length     : varint64\n  //  * shared_length - 1 : varint64, if shared_length > 0\n  //  * unshared          : char[unshared_length]\n  CompactString encoded_;\n};\n\n// Iterates over a `LinearSortedStringSet` in the sorted order.\nclass LinearSortedStringSet::Iterator : public WithEqual<Iterator> {\n public:\n  // `iterator_concept` is only `std::input_iterator_tag` because the\n  // `std::forward_iterator` requirement and above require references to remain\n  // valid while the range exists.\n  using iterator_concept = std::input_iterator_tag;\n  // `iterator_category` is only `std::input_iterator_tag` also because the\n  // `LegacyForwardIterator` requirement and above require `reference` to be\n  // a true reference type.\n  using iterator_category = std::input_iterator_tag;\n  using value_type = absl::string_view;\n  using reference = value_type;\n  using pointer = ArrowProxy<reference>;\n  using difference_type = ptrdiff_t;\n\n  // A sentinel value, equal to `end()`.\n  Iterator() = default;\n\n  Iterator(const Iterator& that) = default;\n  Iterator& operator=(const Iterator& that) = default;\n\n  Iterator(Iterator&& that) noexcept = default;\n  Iterator& operator=(Iterator&& that) noexcept = default;\n\n  // Returns the current element.\n  //\n  // The `absl::string_view` is valid until the next non-const operation on this\n  // `Iterator` because data behind the `absl::string_view` are conditionally\n  // owned by the `Iterator`.\n  reference operator*() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_NE(cursor_, nullptr)\n        << \"Failed precondition of \"\n           \"LinearSortedStringSet::Iterator::operator*: \"\n           \"iterator is end()\";\n    if (length_if_unshared_ > 0) {\n      return absl::string_view(cursor_ - length_if_unshared_,\n                               length_if_unshared_);\n    } else {\n      return current_if_shared_;\n    }\n  }\n\n  pointer operator->() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_NE(cursor_, nullptr)\n        << \"Failed precondition of \"\n           \"LinearSortedStringSet::Iterator::operator->: \"\n           \"iterator is end()\";\n    return pointer(**this);\n  }\n\n  Iterator& operator++();\n  Iterator operator++(int) {\n    const Iterator tmp = *this;\n    ++*this;\n    return tmp;\n  }\n\n  // Iterators can be compared even if they are associated with different\n  // `LinearSortedStringSet` objects. All `end()` values are equal, while all\n  // other values are not equal.\n  friend bool operator==(const Iterator& a, const Iterator& b) {\n    return a.cursor_ == b.cursor_;\n  }\n\n private:\n  friend class LinearSortedStringSet;  // For `Iterator()`.\n\n  explicit Iterator(absl::string_view encoded)\n      : cursor_(encoded.data()), limit_(encoded.data() + encoded.size()) {\n    RIEGELI_ASSERT(cursor_ != nullptr)\n        << \"Failed precondition of \"\n           \"LinearSortedStringSet::Iterator::Iterator(): \"\n           \"encoded.data() is nullptr\";\n    ++*this;\n  }\n\n  // `cursor_` points after the encoded current element in\n  // `LinearSortedStringSet::encoded_`, or is `nullptr` for `end()` (this is\n  // unambiguous because `CompactString::data()` is never `nullptr`).\n  const char* cursor_ = nullptr;\n  const char* limit_ = nullptr;\n  // If `length_if_unshared_ > 0`, the current element is\n  // `absl::string_view(cursor_ - length_if_unshared_, length_if_unshared_)`,\n  // and `current_if_shared_` is unused and empty.\n  //\n  // If `length_if_unshared_ == 0`, the decoded current element is\n  // `current_if_shared_`.\n  size_t length_if_unshared_ = 0;\n  // If `*this` is `end()`, or if `length_if_unshared_ > 0`, unused and empty.\n  // Otherwise stores the decoded current element.\n  CompactString current_if_shared_;\n};\n\n// Represents an element as the concatenation of two `absl::string_view` values:\n// prefix and suffix. This is more efficient than a single `absl::string_view`\n// but less convenient.\n//\n// The prefix is known to be shared with the previous element. It is not\n// guaranteed to be the longest shared prefix though.\nclass LinearSortedStringSet::SplitElement : public WithCompare<SplitElement> {\n public:\n  explicit SplitElement(absl::string_view prefix, absl::string_view suffix)\n      : prefix_(prefix), suffix_(suffix) {}\n\n  SplitElement(const SplitElement& that) = default;\n  SplitElement& operator=(const SplitElement& that) = default;\n\n  absl::string_view prefix() const { return prefix_; }\n  absl::string_view suffix() const { return suffix_; }\n\n  explicit operator std::string() const;\n\n  bool empty() const { return prefix().empty() && suffix().empty(); }\n  size_t size() const { return prefix().size() + suffix().size(); }\n\n  const char& operator[](size_t index) const;\n  const char& at(size_t index) const;\n  const char& front() const;\n  const char& back() const;\n\n  void remove_prefix(size_t length);\n  void remove_suffix(size_t length);\n\n  friend bool operator==(const SplitElement& a, const SplitElement& b) {\n    return Equal(a, b);\n  }\n  friend StrongOrdering RIEGELI_COMPARE(const SplitElement& a,\n                                        const SplitElement& b) {\n    return Compare(a, b);\n  }\n\n  friend bool operator==(const SplitElement& a, absl::string_view b) {\n    return Equal(a, b);\n  }\n  friend StrongOrdering RIEGELI_COMPARE(const SplitElement& a,\n                                        absl::string_view b) {\n    return Compare(a, b);\n  }\n\n  // Default stringification by `absl::StrCat()` etc.\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const SplitElement& src) {\n    dest.Append(src.prefix());\n    dest.Append(src.suffix());\n  }\n\n  friend std::ostream& operator<<(std::ostream& dest, const SplitElement& src) {\n    src.Output(dest);\n    return dest;\n  }\n\n  // Supports `riegeli::Debug()`.\n  template <typename DebugStream>\n  friend void RiegeliDebug(const SplitElement& src, DebugStream& dest) {\n    dest.DebugStringQuote();\n    dest.DebugStringFragment(src.prefix());\n    dest.DebugStringFragment(src.suffix());\n    dest.DebugStringQuote();\n  }\n\n private:\n  static bool Equal(const SplitElement& a, const SplitElement& b);\n  static StrongOrdering Compare(const SplitElement& a, const SplitElement& b);\n  static bool Equal(const SplitElement& a, absl::string_view b);\n  static StrongOrdering Compare(const SplitElement& a, absl::string_view b);\n  void Output(std::ostream& dest) const;\n\n  absl::string_view prefix_;\n  absl::string_view suffix_;\n};\n\n// Iterates over a `LinearSortedStringSet` in the sorted order.\n//\n// Each element is represented as `SplitElement` rather than\n// `absl::string_view`, which is more efficient but less convenient.\nclass LinearSortedStringSet::SplitElementIterator\n    : public WithEqual<SplitElementIterator> {\n public:\n  // `iterator_concept` is only `std::input_iterator_tag` because the\n  // `std::forward_iterator` requirement and above require references to remain\n  // valid while the range exists.\n  using iterator_concept = std::input_iterator_tag;\n  // `iterator_category` is only `std::input_iterator_tag` also because the\n  // `LegacyForwardIterator` requirement and above require `reference` to be\n  // a true reference type.\n  using iterator_category = std::input_iterator_tag;\n  using value_type = SplitElement;\n  using reference = value_type;\n  using pointer = ArrowProxy<reference>;\n  using difference_type = ptrdiff_t;\n\n  // A sentinel value, equal to `end()`.\n  SplitElementIterator() = default;\n\n  SplitElementIterator(const SplitElementIterator& that) = default;\n  SplitElementIterator& operator=(const SplitElementIterator& that) = default;\n\n  SplitElementIterator(SplitElementIterator&& that) noexcept = default;\n  SplitElementIterator& operator=(SplitElementIterator&& that) noexcept =\n      default;\n\n  // Returns the current element.\n  //\n  // The `SplitElement` is valid until the next non-const operation on this\n  // `SplitElementIterator` because data behind the `SplitElement` are\n  // conditionally owned by the `SplitElementIterator`.\n  reference operator*() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_NE(cursor_, nullptr)\n        << \"Failed precondition of \"\n           \"LinearSortedStringSet::SplitElementIterator::operator*: \"\n           \"iterator is end()\";\n    return SplitElement(\n        prefix_, absl::string_view(cursor_ - suffix_length_, suffix_length_));\n  }\n\n  pointer operator->() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_NE(cursor_, nullptr)\n        << \"Failed precondition of \"\n           \"LinearSortedStringSet::SplitElementIterator::operator->: \"\n           \"iterator is end()\";\n    return pointer(**this);\n  }\n\n  SplitElementIterator& operator++();\n  SplitElementIterator operator++(int) {\n    const SplitElementIterator tmp = *this;\n    ++*this;\n    return tmp;\n  }\n\n  // Iterators can be compared even if they are associated with different\n  // `LinearSortedStringSet` objects. All `end()` values are equal, while all\n  // other values are not equal.\n  friend bool operator==(const SplitElementIterator& a,\n                         const SplitElementIterator& b) {\n    return a.cursor_ == b.cursor_;\n  }\n\n private:\n  friend class SplitElements;  // For `SplitElementIterator()`.\n\n  explicit SplitElementIterator(absl::string_view encoded)\n      : cursor_(encoded.data()), limit_(encoded.data() + encoded.size()) {\n    ++*this;\n  }\n\n  // `cursor_` points after the encoded current element in\n  // `LinearSortedStringSet::encoded_`, or is `nullptr` for `end()` (this is\n  // unambiguous because `CompactString::data()` is never `nullptr`).\n  const char* cursor_ = nullptr;\n  const char* limit_ = nullptr;\n  // `prefix_if_stored_` is unused or provides storage for `prefix_` (might be\n  // longer than `prefix_`).\n  CompactString prefix_if_stored_;\n  // If `*this` is `end()`, `prefix_.empty()` and `suffix_length_ == 0`.\n  // Otherwise the current element is the concatenation of `prefix_` and\n  // `absl::string_view(cursor_ - suffix_length_, suffix_length_)`.\n  // `prefix_` points to a prefix of `prefix_if_stored_` or to a substring of\n  // encoded data.\n  absl::string_view prefix_;\n  size_t suffix_length_ = 0;\n};\n\n// A proxy for `LinearSortedStringSet` where each element is represented as\n// `SplitElement` rather than `absl::string_view`. This is more efficient but\n// less convenient.\nclass LinearSortedStringSet::SplitElements {\n public:\n  using value_type = SplitElement;\n  using reference = value_type;\n  using const_reference = reference;\n  using iterator = SplitElementIterator;\n  using const_iterator = iterator;\n  using size_type = size_t;\n  using difference_type = ptrdiff_t;\n\n  SplitElements(const SplitElements& that) = default;\n  SplitElements& operator=(const SplitElements& that) = default;\n\n  // Iteration over the set.\n  //\n  // The `SplitElementIterator` is valid while the `LinearSortedStringSet` is\n  // valid. The `SplitElements` object does not need to be kept valid.\n  SplitElementIterator begin() const { return SplitElementIterator(encoded_); }\n  SplitElementIterator cbegin() const { return begin(); }\n  SplitElementIterator end() const { return SplitElementIterator(); }\n  SplitElementIterator cend() const { return end(); }\n\n private:\n  friend class LinearSortedStringSet;  // For `SplitElements()`.\n\n  explicit SplitElements(const LinearSortedStringSet* set)\n      : encoded_(set->encoded_) {}\n\n  // Invariant: `encoded_.data() != nullptr`\n  absl::string_view encoded_;\n};\n\n// Builds a `LinearSortedStringSet` from a sorted sequence of strings.\nclass LinearSortedStringSet::Builder {\n public:\n  // Begins with an empty set.\n  Builder();\n\n  Builder(Builder&& that) noexcept;\n  Builder& operator=(Builder&& that) noexcept;\n\n  ~Builder();\n\n  // Makes `*this` equivalent to a newly constructed `Builder`.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n\n  // Returns an output iterator which inserts elements to this `Builder`.\n  // Consecutive duplicates are inserted only once.\n  //\n  // Each inserted element must be greater than or equal to the last inserted\n  // element.\n  //\n  // Inserting with a `NextInsertIterator` is equivalent to calling\n  // `InsertNext()`. In particular if multiple iterators and explicit\n  // `InsertNext()` calls are used together, then their combined element\n  // sequence must be ordered.\n  NextInsertIterator NextInserter();\n\n  // Inserts an element. Consecutive duplicates are inserted only once.\n  //\n  // Precondition: `element` is greater than or equal to the last inserted\n  // element.\n  //\n  // Returns `true` if `element` was inserted, or `false` if it is equal to the\n  // last inserted element.\n  //\n  // If `std::string&&` is passed, it is moved only if the result is `true`.\n  //\n  // `std::string&&` is accepted with a template to avoid implicit conversions\n  // to `std::string` which can be ambiguous against `absl::string_view`\n  // (e.g. `const char*`).\n  bool InsertNext(absl::string_view element);\n  template <typename Element,\n            std::enable_if_t<std::is_same_v<Element, std::string>, int> = 0>\n  bool InsertNext(Element&& element);\n\n  // Inserts an element. Elements out of order are skipped.\n  //\n  // Returns `true` if `element` was inserted, `false` if it is equal to the\n  // last inserted element, or `absl::FailedPreconditionError()` if it is less\n  // than the last inserted element.\n  //\n  // If `std::string&&` is passed, it is moved only if the result is `true`.\n  //\n  // `std::string&&` is accepted with a template to avoid implicit conversions\n  // to `std::string` which can be ambiguous against `absl::string_view`\n  // (e.g. `const char*`).\n  absl::StatusOr<bool> TryInsertNext(absl::string_view element);\n  template <typename Element,\n            std::enable_if_t<std::is_same_v<Element, std::string>, int> = 0>\n  absl::StatusOr<bool> TryInsertNext(Element&& element);\n\n  // Returns `true` if the set is empty.\n  bool empty() const { return size_ == 0; }\n\n  // Returns the number of elements.\n  size_t size() const { return size_; }\n\n  // Returns the last inserted element. The set must not be empty.\n  absl::string_view last() const {\n    RIEGELI_ASSERT(!empty())\n        << \"Failed precondition of LinearSortedStringSet::Builder::last(): \"\n           \"empty set\";\n    return last_;\n  }\n\n  // Builds the `LinearSortedStringSet` and resets the `Builder` to empty state.\n  LinearSortedStringSet Build();\n\n private:\n  // This template is defined and used only in linear_sorted_string_set.cc.\n  template <typename Element, typename UpdateLast>\n  absl::StatusOr<bool> InsertNextImpl(Element&& element,\n                                      UpdateLast update_last);\n\n  CompactStringWriter<CompactString> writer_;\n  size_t size_ = 0;\n  std::string last_;\n};\n\n// Inserts elements to a `LinearSortedStringSet::Builder`. Consecutive\n// duplicates are inserted only once.\n//\n// Each inserted element must be greater than or equal to the last inserted\n// element.\nclass LinearSortedStringSet::NextInsertIterator {\n public:\n  using iterator_concept = std::output_iterator_tag;\n  using iterator_category = std::output_iterator_tag;\n  using value_type = absl::string_view;\n  using difference_type = ptrdiff_t;\n  using pointer = void;\n\n  class reference {\n   public:\n    // Inserts the next element.\n    //\n    // `std::string&&` is accepted with a template to avoid implicit conversions\n    // to `std::string` which can be ambiguous against `absl::string_view`\n    // (e.g. `const char*`).\n    const reference& operator=(absl::string_view element) const {\n      builder_->InsertNext(element);\n      return *this;\n    }\n    template <typename Element,\n              std::enable_if_t<std::is_same_v<Element, std::string>, int> = 0>\n    const reference& operator=(Element&& element) const {\n      // `std::move(element)` is correct and `std::forward<Element>(element)` is\n      // not necessary: `Element` is always `std::string`, never an lvalue\n      // reference.\n      builder_->InsertNext(std::move(element));\n      return *this;\n    }\n\n   private:\n    friend class NextInsertIterator;\n    explicit reference(Builder* builder) : builder_(builder) {}\n    Builder* builder_;\n  };\n\n  // A sentinel value.\n  NextInsertIterator() = default;\n\n  NextInsertIterator(const NextInsertIterator& that) = default;\n  NextInsertIterator& operator=(const NextInsertIterator& that) = default;\n\n  reference operator*() const {\n    RIEGELI_ASSERT_NE(builder_, nullptr)\n        << \"Failed precondition of NextInsertIterator::operator*: \"\n           \"iterator is sentinel\";\n    return reference(builder_);\n  }\n\n  NextInsertIterator& operator++() { return *this; }\n  NextInsertIterator operator++(int) { return ++*this; }\n\n  Builder* builder() const { return builder_; }\n\n private:\n  friend class Builder;  // For `NextInsertIterator()`.\n\n  explicit NextInsertIterator(Builder* builder) : builder_(builder) {}\n\n  Builder* builder_ = nullptr;\n};\n\n// Implementation details follow.\n\ntemplate <typename Src,\n          std::enable_if_t<IsIterableOf<Src, absl::string_view>::value, int>>\nLinearSortedStringSet LinearSortedStringSet::FromSorted(Src&& src) {\n  using std::begin;\n  auto iter = begin(src);\n  using std::end;\n  auto end_iter = end(src);\n  LinearSortedStringSet::Builder builder;\n  for (; iter != end_iter; ++iter) {\n    builder.InsertNext(*MaybeMakeMoveIterator<Src>(iter));\n  }\n  return builder.Build();\n}\n\ntemplate <typename Src,\n          std::enable_if_t<IsIterableOf<Src, absl::string_view>::value, int>>\ninline LinearSortedStringSet LinearSortedStringSet::FromUnsorted(Src&& src) {\n  using std::begin;\n  auto iter = begin(src);\n  using std::end;\n  auto end_iter = end(src);\n  using SrcIterator = decltype(iter);\n  std::vector<SrcIterator> iterators;\n  if (IsRandomAccessIterable<Src>::value) {\n    iterators.reserve(std::distance(iter, end_iter));\n  }\n  for (; iter != end_iter; ++iter) iterators.push_back(iter);\n  std::sort(iterators.begin(), iterators.end(),\n            [](const SrcIterator& a, const SrcIterator& b) {\n              return absl::string_view(*a) < absl::string_view(*b);\n            });\n\n  LinearSortedStringSet::Builder builder;\n  for (const SrcIterator& iter : iterators) {\n    builder.InsertNext(*MaybeMakeMoveIterator<Src>(iter));\n  }\n  return builder.Build();\n}\n\ninline LinearSortedStringSet::Iterator LinearSortedStringSet::begin() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return Iterator(encoded_);\n}\n\ninline LinearSortedStringSet::Iterator LinearSortedStringSet::cbegin() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return begin();\n}\n\ninline LinearSortedStringSet::Iterator LinearSortedStringSet::end() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return Iterator();\n}\n\ninline LinearSortedStringSet::Iterator LinearSortedStringSet::cend() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return end();\n}\n\ninline bool LinearSortedStringSet::contains(absl::string_view element,\n                                            size_t* index) const {\n  if (index == nullptr) {\n    size_t counter = 0;\n    return ContainsImpl(element, split_elements().cbegin(), counter);\n  } else {\n    *index = 0;\n    return ContainsImpl(element, split_elements().cbegin(), *index);\n  }\n}\n\ninline bool LinearSortedStringSet::contains_skip_first(\n    absl::string_view element, size_t* index) const {\n  SplitElementIterator iterator = split_elements().cbegin();\n  if (iterator == SplitElementIterator()) return false;\n  ++iterator;\n  if (index == nullptr) {\n    size_t counter = 1;\n    return ContainsImpl(element, std::move(iterator), counter);\n  } else {\n    *index = 1;\n    return ContainsImpl(element, std::move(iterator), *index);\n  }\n}\n\ninline LinearSortedStringSet::SplitElements\nLinearSortedStringSet::split_elements() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return SplitElements(this);\n}\n\ntemplate <typename HashState>\nHashState LinearSortedStringSet::HashValue(HashState hash_state) const {\n  size_t size = 0;\n  for (const absl::string_view element : *this) {\n    hash_state = HashState::combine(std::move(hash_state), element);\n    ++size;\n  }\n  return HashState::combine(std::move(hash_state), size);\n}\n\ninline size_t LinearSortedStringSet::EncodedSize() const {\n  return LengthVarint64(uint64_t{encoded_.size()}) + encoded_.size();\n}\n\ntemplate <\n    typename Dest,\n    std::enable_if_t<TargetRefSupportsDependency<Writer*, Dest>::value, int>>\ninline absl::Status LinearSortedStringSet::Encode(Dest&& dest) const {\n  DependencyRef<Writer*, Dest> dest_dep(std::forward<Dest>(dest));\n  if (dest_dep.IsOwning()) dest_dep->SetWriteSizeHint(EncodedSize());\n  absl::Status status = EncodeImpl(*dest_dep);\n  if (dest_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_dep->Close())) {\n      status.Update(dest_dep->status());\n    }\n  }\n  return status;\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status LinearSortedStringSet::Decode(Src&& src,\n                                                  DecodeOptions options) {\n  DependencyRef<Reader*, Src> src_dep(std::forward<Src>(src));\n  if (src_dep.IsOwning()) src_dep->SetReadAllHint(true);\n  absl::Status status = DecodeImpl(*src_dep, options);\n  if (src_dep.IsOwning()) {\n    if (ABSL_PREDICT_TRUE(status.ok())) src_dep->VerifyEnd();\n    if (ABSL_PREDICT_FALSE(!src_dep->Close())) status.Update(src_dep->status());\n  }\n  return status;\n}\n\ninline const char& LinearSortedStringSet::SplitElement::operator[](\n    size_t index) const {\n  RIEGELI_ASSERT_LT(index, size())\n      << \"Failed precondition of \"\n         \"LinearSortedStringSet::SplitElement::operator[]: \"\n         \"index out of range\";\n  return index < prefix().size() ? prefix()[index]\n                                 : suffix()[index - prefix().size()];\n}\n\ninline const char& LinearSortedStringSet::SplitElement::at(size_t index) const {\n  RIEGELI_CHECK_LT(index, size())\n      << \"Failed precondition of LinearSortedStringSet::SplitElement::at(): \"\n         \"index out of range\";\n  return index < prefix().size() ? prefix()[index]\n                                 : suffix()[index - prefix().size()];\n}\n\ninline const char& LinearSortedStringSet::SplitElement::front() const {\n  RIEGELI_ASSERT(!empty()) << \"Failed precondition of \"\n                              \"LinearSortedStringSet::SplitElement::front(): \"\n                              \"empty string\";\n  return prefix().empty() ? suffix().front() : prefix().front();\n}\n\ninline const char& LinearSortedStringSet::SplitElement::back() const {\n  RIEGELI_ASSERT(!empty()) << \"Failed precondition of \"\n                              \"LinearSortedStringSet::SplitElement::back(): \"\n                              \"empty string\";\n  return suffix().empty() ? prefix().back() : suffix().back();\n}\n\ninline void LinearSortedStringSet::SplitElement::remove_prefix(size_t length) {\n  RIEGELI_ASSERT_LE(length, size())\n      << \"Failed precondition of \"\n         \"LinearSortedStringSet::SplitElement::remove_prefix(): \"\n         \"length out of range\";\n  if (length <= prefix().size()) {\n    prefix_.remove_prefix(length);\n  } else {\n    suffix_.remove_prefix(length - prefix().size());\n    prefix_ = absl::string_view();\n  }\n}\n\ninline void LinearSortedStringSet::SplitElement::remove_suffix(size_t length) {\n  RIEGELI_ASSERT_LE(length, size())\n      << \"Failed precondition of \"\n         \"LinearSortedStringSet::SplitElement::remove_suffix(): \"\n         \"length out of range\";\n  if (length <= suffix().size()) {\n    suffix_.remove_suffix(length);\n  } else {\n    prefix_.remove_suffix(length - suffix().size());\n    suffix_ = absl::string_view();\n  }\n}\n\ninline LinearSortedStringSet::NextInsertIterator\nLinearSortedStringSet::Builder::NextInserter() {\n  return NextInsertIterator(this);\n}\n\nextern template bool LinearSortedStringSet::Builder::InsertNext(\n    std::string&& element);\n\nextern template absl::StatusOr<bool>\nLinearSortedStringSet::Builder::TryInsertNext(std::string&& element);\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CONTAINERS_LINEAR_SORTED_STRING_SET_H_\n"
  },
  {
    "path": "riegeli/csv/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"csv_reader\",\n    srcs = [\"csv_reader.cc\"],\n    hdrs = [\"csv_reader.h\"],\n    deps = [\n        \":csv_record\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:debug\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:status\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:string_reader\",\n        \"//riegeli/bytes:string_writer\",\n        \"//riegeli/lines:line_reading\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"csv_writer\",\n    srcs = [\"csv_writer.cc\"],\n    hdrs = [\"csv_writer.h\"],\n    deps = [\n        \":csv_record\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:debug\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:iterable\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:status\",\n        \"//riegeli/bytes:string_writer\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/lines:line_writing\",\n        \"//riegeli/lines:newline\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"csv_record\",\n    srcs = [\"csv_record.cc\"],\n    hdrs = [\"csv_record.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:global\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:iterable\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:shared_ptr\",\n        \"//riegeli/base:string_ref\",\n        \"//riegeli/base:type_traits\",\n        \"//riegeli/bytes:ostream_writer\",\n        \"//riegeli/bytes:string_writer\",\n        \"//riegeli/bytes:stringify_writer\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/container:flat_hash_map\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/synchronization\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/csv/csv_reader.cc",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/csv/csv_reader.h\"\n\n#include <stddef.h>\n\n#include <array>\n#include <functional>\n#include <optional>\n#include <string>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/debug.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/string_reader.h\"\n#include \"riegeli/bytes/string_writer.h\"\n#include \"riegeli/csv/csv_record.h\"\n#include \"riegeli/lines/line_reading.h\"\n\nnamespace riegeli {\n\nvoid CsvReaderBase::Initialize(Reader* src, Options&& options) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of CsvReader: null Reader pointer\";\n  // Set `has_header_` before early returns because `ReadRecord(CsvRecord&)`\n  // uses this as a precondition.\n  if (options.required_header() != std::nullopt ||\n      options.assumed_header() != std::nullopt) {\n    RIEGELI_ASSERT(options.required_header() == std::nullopt ||\n                   options.assumed_header() == std::nullopt)\n        << \"Failed precondition of CsvReader: \"\n           \"required_header() and assumed_header() both set\";\n    has_header_ = true;\n  }\n\n  if (options.comment() != std::nullopt &&\n      ABSL_PREDICT_FALSE(*options.comment() == '\\n' ||\n                         *options.comment() == '\\r')) {\n    Fail(absl::InvalidArgumentError(\n        absl::StrCat(\"Comment character conflicts with record separator: \",\n                     riegeli::Debug(*options.comment()))));\n    return;\n  }\n  if (ABSL_PREDICT_FALSE(options.field_separator() == '\\n' ||\n                         options.field_separator() == '\\r')) {\n    Fail(absl::InvalidArgumentError(\n        absl::StrCat(\"Field separator conflicts with record separator: \",\n                     riegeli::Debug(options.field_separator()))));\n    return;\n  }\n  if (ABSL_PREDICT_FALSE(options.field_separator() == options.comment())) {\n    Fail(absl::InvalidArgumentError(\n        absl::StrCat(\"Field separator conflicts with comment character: \",\n                     riegeli::Debug(options.field_separator()))));\n    return;\n  }\n  if (options.quote() != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(*options.quote() == '\\n' ||\n                           *options.quote() == '\\r')) {\n      Fail(absl::InvalidArgumentError(\n          absl::StrCat(\"Quote character conflicts with record separator: \",\n                       riegeli::Debug(*options.quote()))));\n      return;\n    }\n    if (ABSL_PREDICT_FALSE(*options.quote() == options.comment())) {\n      Fail(absl::InvalidArgumentError(\n          absl::StrCat(\"Quote character conflicts with comment character: \",\n                       riegeli::Debug(*options.quote()))));\n      return;\n    }\n    if (ABSL_PREDICT_FALSE(*options.quote() == options.field_separator())) {\n      Fail(absl::InvalidArgumentError(\n          absl::StrCat(\"Quote character conflicts with field separator: \",\n                       riegeli::Debug(*options.quote()))));\n      return;\n    }\n  }\n  if (options.escape() != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(*options.escape() == '\\n' ||\n                           *options.escape() == '\\r')) {\n      Fail(absl::InvalidArgumentError(\n          absl::StrCat(\"Escape character conflicts with record separator: \",\n                       riegeli::Debug(*options.escape()))));\n      return;\n    }\n    if (ABSL_PREDICT_FALSE(*options.escape() == options.comment())) {\n      Fail(absl::InvalidArgumentError(\n          absl::StrCat(\"Escape character conflicts with comment character: \",\n                       riegeli::Debug(*options.escape()))));\n      return;\n    }\n    if (ABSL_PREDICT_FALSE(*options.escape() == options.field_separator())) {\n      Fail(absl::InvalidArgumentError(\n          absl::StrCat(\"Escape character conflicts with field separator: \",\n                       riegeli::Debug(*options.escape()))));\n      return;\n    }\n    if (ABSL_PREDICT_FALSE(*options.escape() == options.quote())) {\n      Fail(absl::InvalidArgumentError(\n          absl::StrCat(\"Escape character conflicts with quote character: \",\n                       riegeli::Debug(*options.escape()))));\n      return;\n    }\n  }\n\n  char_classes_['\\n'] = CharClass::kLf;\n  char_classes_['\\r'] = CharClass::kCr;\n  if (options.comment() != std::nullopt) {\n    char_classes_[static_cast<unsigned char>(*options.comment())] =\n        CharClass::kComment;\n  }\n  char_classes_[static_cast<unsigned char>(options.field_separator())] =\n      CharClass::kFieldSeparator;\n  if (options.quote() != std::nullopt) {\n    char_classes_[static_cast<unsigned char>(*options.quote())] =\n        CharClass::kQuote;\n  }\n  if (options.escape() != std::nullopt) {\n    char_classes_[static_cast<unsigned char>(*options.escape())] =\n        CharClass::kEscape;\n  }\n  skip_empty_lines_ = options.skip_empty_lines();\n  quote_ = options.quote().value_or('\\0');\n  max_num_fields_ = options.max_num_fields();\n  max_field_length_ = options.max_field_length();\n\n  if (ABSL_PREDICT_FALSE(!src->ok())) {\n    FailWithoutAnnotation(AnnotateOverSrc(src->status()));\n    return;\n  }\n  if (!options.preserve_utf8_bom()) SkipUtf8Bom(*src);\n\n  // Recovery is not applicable to reading the header. Hence `recovery_` is set\n  // after reading the header.\n  if (options.required_header() != std::nullopt) {\n    std::vector<std::string> header;\n    if (ABSL_PREDICT_FALSE(!ReadRecord(header))) {\n      Fail(absl::InvalidArgumentError(\"Empty CSV file\"));\n    } else {\n      --record_index_;\n      if (header == options.required_header()->names()) {\n        header_ = *std::move(options.required_header());\n      } else if (const absl::Status status =\n                     header_.TryReset(options.required_header()->normalizer(),\n                                      std::move(header));\n                 ABSL_PREDICT_FALSE(!status.ok())) {\n        FailAtPreviousRecord(absl::InvalidArgumentError(status.message()));\n      } else {\n        std::vector<absl::string_view> missing_names;\n        for (const absl::string_view field :\n             options.required_header()->names()) {\n          if (ABSL_PREDICT_FALSE(!header_.contains(field))) {\n            missing_names.push_back(field);\n          }\n        }\n        if (ABSL_PREDICT_FALSE(!missing_names.empty())) {\n          StringWriter<std::string> message;\n          message.Write(\"Missing field names: \");\n          for (std::vector<absl::string_view>::const_iterator iter =\n                   missing_names.cbegin();\n               iter != missing_names.cend(); ++iter) {\n            if (iter != missing_names.cbegin()) message.Write(',');\n            csv_internal::WriteDebugQuotedIfNeeded(*iter, message);\n          }\n          message.Write(\"; existing field names: \");\n          for (CsvHeader::const_iterator iter = header_.cbegin();\n               iter != header_.cend(); ++iter) {\n            if (iter != header_.cbegin()) message.Write(',');\n            csv_internal::WriteDebugQuotedIfNeeded(*iter, message);\n          }\n          message.Close();\n          FailAtPreviousRecord(absl::InvalidArgumentError(message.dest()));\n        }\n      }\n    }\n  } else if (options.assumed_header() != std::nullopt) {\n    header_ = *std::move(options.assumed_header());\n  }\n\n  recovery_ = std::move(options.recovery());\n}\n\nvoid CsvReaderBase::FailAtPreviousRecord(absl::Status status) {\n  RIEGELI_ASSERT(!status.ok())\n      << \"Failed precondition of CsvReaderBase::FailAtPreviousRecord(): \"\n         \"status not failed\";\n  RIEGELI_ASSERT(!standalone_record_)\n      << \"Failed precondition of CsvReaderBase::FailAtPreviousRecord(): \"\n         \"should never happen in ReadCsvRecordFromString()\";\n  if (is_open()) {\n    Reader& src = *SrcReader();\n    status = src.AnnotateStatus(std::move(status));\n  }\n  FailWithoutAnnotation(\n      Annotate(status, absl::StrCat(\"at line \", last_line_number())));\n}\n\nabsl::Status CsvReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Reader& src = *SrcReader();\n    status = src.AnnotateStatus(std::move(status));\n  }\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status CsvReaderBase::AnnotateOverSrc(absl::Status status) {\n  if (!standalone_record_) {\n    return Annotate(status, absl::StrCat(\"at line \", line_number()));\n  }\n  return status;\n}\n\nbool CsvReaderBase::FailMaxFieldLengthExceeded() {\n  recoverable_ = true;\n  return Fail(absl::ResourceExhaustedError(\n      absl::StrCat(\"Maximum field length exceeded: \", max_field_length_)));\n}\n\ninline void CsvReaderBase::SkipLine(Reader& src) {\n  const char* ptr = src.cursor();\n  for (;;) {\n    if (ABSL_PREDICT_FALSE(ptr == src.limit())) {\n      src.move_cursor(src.available());\n      if (ABSL_PREDICT_FALSE(!src.Pull())) {\n        // Set `line_number_` as if the last line was terminated by a newline.\n        ++line_number_;\n        return;\n      }\n      ptr = src.cursor();\n    }\n    if (*ptr == '\\n') {\n      ++line_number_;\n      src.set_cursor(ptr + 1);\n      return;\n    }\n    ++ptr;\n  }\n}\n\ninline bool CsvReaderBase::ReadQuoted(Reader& src, std::string& field) {\n  if (ABSL_PREDICT_FALSE(!field.empty())) {\n    recoverable_ = true;\n    return Fail(\n        absl::InvalidArgumentError(\"Unquoted data before opening quote\"));\n  }\n\n  // Data from `src.cursor()` to where `ptr` stops will be appended to `field`.\n  const char* ptr = src.cursor();\n  for (;;) {\n    if (ABSL_PREDICT_FALSE(ptr == src.limit())) {\n      if (ABSL_PREDICT_FALSE(src.available() >\n                             max_field_length_ - field.size())) {\n        return FailMaxFieldLengthExceeded();\n      }\n      field.append(src.cursor(), src.available());\n      src.move_cursor(src.available());\n      if (ABSL_PREDICT_FALSE(!src.Pull())) {\n        if (ABSL_PREDICT_FALSE(!src.ok())) {\n          return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n        }\n        recoverable_ = true;\n        return Fail(absl::InvalidArgumentError(\"Missing closing quote\"));\n      }\n      ptr = src.cursor();\n    }\n    const CharClass char_class =\n        char_classes_[static_cast<unsigned char>(*ptr++)];\n    if (ABSL_PREDICT_TRUE(char_class == CharClass::kOther)) continue;\n    switch (char_class) {\n      case CharClass::kLf:\n        ++line_number_;\n        continue;\n      case CharClass::kCr:\n      case CharClass::kComment:\n      case CharClass::kFieldSeparator:\n        continue;\n      default:\n        break;\n    }\n    const size_t length = PtrDistance(src.cursor(), ptr - 1);\n    if (ABSL_PREDICT_FALSE(length > max_field_length_ - field.size())) {\n      return FailMaxFieldLengthExceeded();\n    }\n    field.append(src.cursor(), length);\n    src.set_cursor(ptr);\n    switch (char_class) {\n      case CharClass::kOther:\n      case CharClass::kLf:\n      case CharClass::kCr:\n      case CharClass::kComment:\n      case CharClass::kFieldSeparator:\n        RIEGELI_ASSUME_UNREACHABLE() << \"Handled before switch\";\n      case CharClass::kQuote:\n        if (ABSL_PREDICT_FALSE(!src.Pull())) {\n          if (ABSL_PREDICT_FALSE(!src.ok())) {\n            return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n          }\n          return true;\n        }\n        if (*src.cursor() == quote_) {\n          // Quote written twice.\n          ptr = src.cursor() + 1;\n          continue;\n        }\n        return true;\n      case CharClass::kEscape:\n        if (ABSL_PREDICT_FALSE(!src.Pull())) {\n          if (ABSL_PREDICT_FALSE(!src.ok())) {\n            return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n          }\n          recoverable_ = true;\n          return Fail(\n              absl::InvalidArgumentError(\"Missing character after escape\"));\n        }\n        ptr = src.cursor() + 1;\n        continue;\n    }\n    RIEGELI_ASSUME_UNREACHABLE()\n        << \"Unknown character class: \" << static_cast<int>(char_class);\n  }\n}\n\ninline bool CsvReaderBase::ReadFields(Reader& src,\n                                      std::vector<std::string>& fields,\n                                      size_t& field_index) {\n  RIEGELI_ASSERT_EQ(field_index, 0u)\n      << \"Failed precondition of CsvReaderBase::ReadFields(): \"\n         \"initial index must be 0\";\nnext_record:\n  last_line_number_ = line_number_;\n  if (standalone_record_) {\n    if (ABSL_PREDICT_FALSE(record_index_ > 0)) return false;\n  } else {\n    if (ABSL_PREDICT_FALSE(!src.Pull())) {\n      // End of file at the beginning of a record.\n      if (ABSL_PREDICT_FALSE(!src.ok())) {\n        return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n      }\n      return false;\n    }\n  }\n\nnext_field:\n  if (ABSL_PREDICT_FALSE(field_index == max_num_fields_)) {\n    recoverable_ = true;\n    return Fail(absl::ResourceExhaustedError(\n        absl::StrCat(\"Maximum number of fields exceeded: \", max_num_fields_)));\n  }\n  if (fields.size() == field_index) {\n    fields.emplace_back();\n  } else {\n    fields[field_index].clear();\n  }\n  std::string& field = fields[field_index];\n\n  // Data from `src.cursor()` to where `ptr` stops will be appended to `field`.\n  const char* ptr = src.cursor();\n  for (;;) {\n    if (ABSL_PREDICT_FALSE(ptr == src.limit())) {\n      if (ABSL_PREDICT_FALSE(src.available() >\n                             max_field_length_ - field.size())) {\n        return FailMaxFieldLengthExceeded();\n      }\n      field.append(src.cursor(), src.available());\n      src.move_cursor(src.available());\n      if (ABSL_PREDICT_FALSE(!src.Pull())) {\n        if (ABSL_PREDICT_FALSE(!src.ok())) {\n          return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n        }\n        // Set `line_number_` as if the last line was terminated by a newline.\n        ++line_number_;\n        return true;\n      }\n      ptr = src.cursor();\n    }\n    const CharClass char_class =\n        char_classes_[static_cast<unsigned char>(*ptr++)];\n    if (ABSL_PREDICT_TRUE(char_class == CharClass::kOther)) continue;\n    switch (char_class) {\n      case CharClass::kComment:\n        if (field_index == 0 && field.empty() && ptr - 1 == src.cursor()) {\n          src.set_cursor(ptr);\n          SkipLine(src);\n          goto next_record;\n        }\n        continue;\n      default:\n        break;\n    }\n    const size_t length = PtrDistance(src.cursor(), ptr - 1);\n    if (ABSL_PREDICT_FALSE(length > max_field_length_ - field.size())) {\n      return FailMaxFieldLengthExceeded();\n    }\n    field.append(src.cursor(), length);\n    src.set_cursor(ptr);\n    switch (char_class) {\n      case CharClass::kOther:\n      case CharClass::kComment:\n        RIEGELI_ASSUME_UNREACHABLE() << \"Handled before switch\";\n      case CharClass::kCr:\n        if (ABSL_PREDICT_FALSE(!src.Pull())) {\n          if (ABSL_PREDICT_FALSE(!src.ok())) {\n            return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n          }\n          recoverable_ = true;\n          return Fail(absl::InvalidArgumentError(\"Missing LF after CR\"));\n        }\n        if (ABSL_PREDICT_FALSE(*src.cursor() != '\\n')) {\n          recoverable_ = true;\n          return Fail(absl::InvalidArgumentError(\"Missing LF after CR\"));\n        }\n        src.move_cursor(1);\n        ABSL_FALLTHROUGH_INTENDED;\n      case CharClass::kLf:\n        ++line_number_;\n        if (skip_empty_lines_ && field_index == 0 && field.empty()) {\n          goto next_record;\n        }\n        if (ABSL_PREDICT_FALSE(standalone_record_)) {\n          return Fail(absl::InvalidArgumentError(\"Unexpected newline\"));\n        }\n        return true;\n      case CharClass::kFieldSeparator:\n        ++field_index;\n        goto next_field;\n      case CharClass::kQuote: {\n        if (ABSL_PREDICT_FALSE(!ReadQuoted(src, field))) return false;\n        if (ABSL_PREDICT_FALSE(!src.Pull())) {\n          if (ABSL_PREDICT_FALSE(!src.ok())) {\n            return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n          }\n          // Set `line_number_` as if the last line was terminated by a newline.\n          ++line_number_;\n          return true;\n        }\n        const CharClass char_class_after_quoted =\n            char_classes_[static_cast<unsigned char>(*src.cursor())];\n        src.move_cursor(1);\n        switch (char_class_after_quoted) {\n          case CharClass::kOther:\n          case CharClass::kComment:\n          case CharClass::kEscape:\n            recoverable_ = true;\n            return Fail(absl::InvalidArgumentError(\n                \"Unquoted data after closing quote\"));\n          case CharClass::kFieldSeparator:\n            ++field_index;\n            goto next_field;\n          case CharClass::kCr:\n            if (ABSL_PREDICT_FALSE(!src.Pull())) {\n              if (ABSL_PREDICT_FALSE(!src.ok())) {\n                return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n              }\n              recoverable_ = true;\n              return Fail(absl::InvalidArgumentError(\"Missing LF after CR\"));\n            }\n            if (ABSL_PREDICT_FALSE(*src.cursor() != '\\n')) {\n              recoverable_ = true;\n              return Fail(absl::InvalidArgumentError(\"Missing LF after CR\"));\n            }\n            src.move_cursor(1);\n            ABSL_FALLTHROUGH_INTENDED;\n          case CharClass::kLf:\n            ++line_number_;\n            if (ABSL_PREDICT_FALSE(standalone_record_)) {\n              return Fail(absl::InvalidArgumentError(\"Unexpected newline\"));\n            }\n            return true;\n          case CharClass::kQuote:\n            RIEGELI_ASSUME_UNREACHABLE() << \"Handled by ReadQuoted()\";\n        }\n        RIEGELI_ASSUME_UNREACHABLE()\n            << \"Unknown character class: \"\n            << static_cast<int>(char_class_after_quoted);\n      }\n      case CharClass::kEscape:\n        if (ABSL_PREDICT_FALSE(!src.Pull())) {\n          if (ABSL_PREDICT_FALSE(!src.ok())) {\n            return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n          }\n          recoverable_ = true;\n          return Fail(\n              absl::InvalidArgumentError(\"Missing character after escape\"));\n        }\n        ptr = src.cursor() + 1;\n        continue;\n    }\n    RIEGELI_ASSUME_UNREACHABLE()\n        << \"Unknown character class: \" << static_cast<int>(char_class);\n  }\n}\n\nbool CsvReaderBase::ReadRecord(CsvRecord& record) {\n  RIEGELI_CHECK(has_header())\n      << \"Failed precondition of CsvReaderBase::ReadRecord(CsvRecord&): \"\n         \"CsvReaderBase::Options::required_header() != nullopt or \"\n         \"assumed_header() != nullopt is required\";\n  if (ABSL_PREDICT_FALSE(!ok())) {\n    record.Reset();\n    return false;\n  }\ntry_again:\n  record.Reset(header_);\n  // Reading directly into `record.fields_` must be careful to maintain the\n  // invariant that `record.header_.size() == record.fields_.size()`.\n  if (ABSL_PREDICT_FALSE(!ReadRecord(record.fields_))) {\n    record.Reset();\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(record.fields_.size() != header_.size())) {\n    --record_index_;\n    const size_t record_size = record.fields_.size();\n    record.Reset();\n    FailAtPreviousRecord(absl::InvalidArgumentError(\n        absl::StrCat(\"Mismatched number of CSV fields: header has \",\n                     header_.size(), \", record has \", record_size)));\n    if (recovery_ != nullptr) {\n      absl::Status status = this->status();\n      MarkNotFailed();\n      if (recovery_(std::move(status), *this)) goto try_again;\n    }\n    return false;\n  }\n  return true;\n}\n\nnamespace csv_internal {\n\ninline bool ReadStandaloneRecord(CsvReaderBase& csv_reader,\n                                 std::vector<std::string>& record) {\n  csv_reader.standalone_record_ = true;\n  return csv_reader.ReadRecordInternal(record);\n}\n\n}  // namespace csv_internal\n\nbool CsvReaderBase::ReadRecord(std::vector<std::string>& record) {\n  return ReadRecordInternal(record);\n}\n\ninline bool CsvReaderBase::ReadRecordInternal(\n    std::vector<std::string>& record) {\n  if (ABSL_PREDICT_FALSE(!ok())) {\n    record.clear();\n    return false;\n  }\n  if (standalone_record_) {\n    RIEGELI_ASSERT_EQ(record_index_, 0u)\n        << \"Failed precondition of CsvReaderBase::ReadRecordInternal(): \"\n           \"called more than once by ReadCsvRecordFromString()\";\n  }\n  Reader& src = *SrcReader();\ntry_again:\n  size_t field_index = 0;\n  // Assign to existing elements of `record` when possible and then `erase()`\n  // excess elements, instead of calling `record.clear()` upfront, to avoid\n  // losing existing `std::string` allocations.\n  if (ABSL_PREDICT_FALSE(!ReadFields(src, record, field_index))) {\n    if (recovery_ != nullptr && recoverable_) {\n      recoverable_ = false;\n      absl::Status status = this->status();\n      MarkNotFailed();\n      SkipLine(src);\n      if (recovery_(std::move(status), *this)) goto try_again;\n      if (standalone_record_) {\n        // Recovery was cancelled. Return the same result as for an empty input:\n        // one empty field.\n        if (record.empty()) {\n          record.emplace_back();\n        } else {\n          record[0].clear();\n        }\n        record.erase(record.begin() + 1, record.end());\n        ++record_index_;\n        return true;\n      }\n    }\n    record.clear();\n    return false;\n  }\n  record.erase(record.begin() + field_index + 1, record.end());\n  ++record_index_;\n  return true;\n}\n\nbool CsvReaderBase::HasNextRecord() {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  for (;;) {\n    last_line_number_ = line_number_;\n    if (ABSL_PREDICT_FALSE(!src.Pull())) {\n      // End of file at the beginning of a record.\n      if (ABSL_PREDICT_FALSE(!src.ok())) {\n        return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n      }\n      return false;\n    }\n    const CharClass char_class =\n        char_classes_[static_cast<unsigned char>(*src.cursor())];\n    switch (char_class) {\n      case CharClass::kCr:\n        src.move_cursor(1);\n        if (ABSL_PREDICT_FALSE(!src.Pull())) {\n          if (ABSL_PREDICT_FALSE(!src.ok())) {\n            return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n          }\n          return false;\n        }\n        if (ABSL_PREDICT_FALSE(*src.cursor() != '\\n')) return false;\n        ABSL_FALLTHROUGH_INTENDED;\n      case CharClass::kLf:\n        if (skip_empty_lines_) {\n          ++line_number_;\n          src.move_cursor(1);\n          continue;\n        }\n        return true;\n      case CharClass::kComment:\n        SkipLine(src);\n        continue;\n      default:\n        return true;\n    }\n  }\n}\n\nabsl::Status ReadCsvRecordFromString(absl::string_view src,\n                                     std::vector<std::string>& record,\n                                     CsvReaderBase::Options options) {\n  RIEGELI_ASSERT(options.required_header() == std::nullopt)\n      << \"Failed precondition of ReadCsvRecordFromString(): \"\n         \"CsvReaderBase::Options::required_header() != nullopt not applicable\";\n  CsvReader csv_reader(riegeli::Maker<StringReader>(src), std::move(options));\n  if (ABSL_PREDICT_FALSE(\n          !csv_internal::ReadStandaloneRecord(csv_reader, record))) {\n    RIEGELI_ASSERT(!csv_reader.ok())\n        << \"ReadStandaloneRecord() returned false but ok() is true\";\n    return csv_reader.status();\n  }\n  return absl::OkStatus();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/csv/csv_reader.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CSV_CSV_READER_H_\n#define RIEGELI_CSV_CSV_READER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <array>\n#include <functional>\n#include <initializer_list>\n#include <limits>\n#include <optional>\n#include <string>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/csv/csv_record.h\"\n\nnamespace riegeli {\n\nclass CsvReaderBase;\n\nnamespace csv_internal {\n\nbool ReadStandaloneRecord(CsvReaderBase& csv_reader,\n                          std::vector<std::string>& record);\n\n}  // namespace csv_internal\n\n// Template parameter independent part of `CsvReader`.\nclass CsvReaderBase : public Object {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // If not `std::nullopt`, automatically reads field names from the first\n    // record, specifies how field names are normalized, and verifies that all\n    // required fields are present (in any order).\n    //\n    // In this case `ReadRecord(CsvRecord&)` is supported. Otherwise no\n    // particular header is assumed, and only `ReadRecord()` to a vector of\n    // fields is supported.\n    //\n    // `set_required_header({})` specifies an empty set of required fields and\n    // thus accepts any field names.\n    //\n    // If the file is empty, actual field names have duplicates, or some\n    // required fields are not present, reading the header fails.\n    //\n    // `required_header()` and `assumed_header()` must not be both set.\n    //\n    // Default: `std::nullopt`.\n    Options& set_required_header(Initializer<std::optional<CsvHeader>> header) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      riegeli::Reset(required_header_, std::move(header));\n      return *this;\n    }\n    Options&& set_required_header(\n        Initializer<std::optional<CsvHeader>> header) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_required_header(std::move(header)));\n    }\n    Options& set_required_header(\n        std::initializer_list<absl::string_view> names) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_required_header(Initializer<std::optional<CsvHeader>>(names));\n    }\n    Options&& set_required_header(\n        std::initializer_list<absl::string_view> names) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_required_header(names));\n    }\n    std::optional<CsvHeader>& required_header() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return required_header_;\n    }\n    const std::optional<CsvHeader>& required_header() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return required_header_;\n    }\n\n    // If not `std::nullopt`, a header is not read from the file, but\n    // `ReadRecord(CsvRecord&)` is supported as if this header was present as\n    // the first record.\n    //\n    // `required_header()` and `assumed_header()` must not be both set.\n    //\n    // Default: `std::nullopt`.\n    Options& set_assumed_header(Initializer<std::optional<CsvHeader>> header) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      riegeli::Reset(assumed_header_, std::move(header));\n      return *this;\n    }\n    Options&& set_assumed_header(\n        Initializer<std::optional<CsvHeader>> header) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_header(std::move(header)));\n    }\n    Options& set_assumed_header(\n        std::initializer_list<absl::string_view> names) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_assumed_header(Initializer<std::optional<CsvHeader>>(names));\n    }\n    Options&& set_assumed_header(\n        std::initializer_list<absl::string_view> names) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_header(names));\n    }\n    std::optional<CsvHeader>& assumed_header() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return assumed_header_;\n    }\n    const std::optional<CsvHeader>& assumed_header() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return assumed_header_;\n    }\n\n    // If `false`, an initial UTF-8 BOM is skipped if present.\n    //\n    // If `true`, an initial UTF-8 BOM if present is treated as a part of the\n    // first field in the first record. This is unlikely to be the intent, but\n    // this conforms to RFC4180.\n    //\n    // Default: `false`.\n    Options& set_preserve_utf8_bom(bool preserve_utf8_bom) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      preserve_utf8_bom_ = preserve_utf8_bom;\n      return *this;\n    }\n    Options&& set_preserve_utf8_bom(bool preserve_utf8_bom) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_preserve_utf8_bom(preserve_utf8_bom));\n    }\n    bool preserve_utf8_bom() const { return preserve_utf8_bom_; }\n\n    // If `false`, an empty line is interpreted as a record with one empty\n    // field. This conforms to RFC4180.\n    //\n    // If `true`, empty lines are skipped.\n    //\n    // Default: `false`.\n    Options& set_skip_empty_lines(bool skip_empty_lines) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      skip_empty_lines_ = skip_empty_lines;\n      return *this;\n    }\n    Options&& set_skip_empty_lines(bool skip_empty_lines) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_skip_empty_lines(skip_empty_lines));\n    }\n    bool skip_empty_lines() const { return skip_empty_lines_; }\n\n    // Comment character.\n    //\n    // If not `std::nullopt`, a line beginning with this character is skipped.\n    // This is not covered by RFC4180.\n    //\n    // Often used: '#'.\n    //\n    // Default: `std::nullopt`.\n    Options& set_comment(std::optional<char> comment) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      comment_ = comment;\n      return *this;\n    }\n    Options&& set_comment(std::optional<char> comment) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_comment(comment));\n    }\n    std::optional<char> comment() const { return comment_; }\n\n    // Field separator.\n    //\n    // Default: ','.\n    Options& set_field_separator(char field_separator) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      field_separator_ = field_separator;\n      return *this;\n    }\n    Options&& set_field_separator(char field_separator) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_field_separator(field_separator));\n    }\n    char field_separator() const { return field_separator_; }\n\n    // Quote character.\n    //\n    // Quotes around a field allow expressing special characters inside the\n    // field: LF, CR, comment character, field separator, or quote character\n    // itself.\n    //\n    // To express a quote itself inside a field, it must be written twice when\n    // the field is quoted, or preceded by an escape character.\n    //\n    // If `quote()` and `escape()` are both `std::nullopt`, special characters\n    // inside fields are not expressible.\n    //\n    // Default: '\"'.\n    Options& set_quote(std::optional<char> quote) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      quote_ = quote;\n      return *this;\n    }\n    Options&& set_quote(std::optional<char> quote) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_quote(quote));\n    }\n    std::optional<char> quote() const { return quote_; }\n\n    // Escape character.\n    //\n    // If not `std::nullopt`, a character preceded by escape is treated\n    // literally instead of possibly having a special meaning. This allows\n    // expressing special characters inside a field: LF, CR, comment character,\n    // field separator, or escape character itself. This is not covered by\n    // RFC4180.\n    //\n    // If `quote()` and `escape()` are both `std::nullopt`, special characters\n    // inside fields are not expressible.\n    //\n    // Default: `std::nullopt`.\n    Options& set_escape(std::optional<char> escape) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      escape_ = escape;\n      return *this;\n    }\n    Options&& set_escape(std::optional<char> escape) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_escape(escape));\n    }\n    std::optional<char> escape() const { return escape_; }\n\n    // Expected maximum number of fields.\n    //\n    // If this number is exceeded, reading fails with\n    // `absl::ResourceExhaustedError()`.\n    //\n    // `max_num_fields` must be at least 1.\n    //\n    // Default: `std::numeric_limits<size_t>::max()`.\n    Options& set_max_num_fields(size_t max_num_fields) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(max_num_fields, 1u)\n          << \"Failed precondition of \"\n             \"CsvReaderBase::Options::set_max_num_fields(): \"\n             \"number of fields out of range\";\n      max_num_fields_ = max_num_fields;\n      return *this;\n    }\n    Options&& set_max_num_fields(size_t max_num_fields) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_num_fields(max_num_fields));\n    }\n    size_t max_num_fields() const { return max_num_fields_; }\n\n    // Expected maximum field length.\n    //\n    // If this length is exceeded, reading fails with\n    // `absl::ResourceExhaustedError()`.\n    //\n    // Default: `std::numeric_limits<size_t>::max()`.\n    Options& set_max_field_length(size_t max_field_length) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      max_field_length_ = max_field_length;\n      return *this;\n    }\n    Options&& set_max_field_length(size_t max_field_length) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_max_field_length(max_field_length));\n    }\n    size_t max_field_length() const { return max_field_length_; }\n\n    // Recovery function called after skipping over an invalid line.\n    //\n    // If `nullptr`, then an invalid line causes `CsvReader` to fail.\n    //\n    // If not `nullptr`, then an invalid line causes `CsvReader` to skip over\n    // the invalid line and call the recovery function. If the recovery function\n    // returns `true`, reading continues. If the recovery function returns\n    // `false`, reading ends as if the end of source was encountered.\n    //\n    // Recovery is not applicable to reading the header with\n    // `Options::required_header() != std::nullopt`.\n    //\n    // Calling `ReadRecord()` may cause the recovery function to be called (in\n    // the same thread).\n    //\n    // Default: `nullptr`.\n    Options& set_recovery(\n        Initializer<std::function<bool(absl::Status, CsvReaderBase&)>>\n            recovery) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      riegeli::Reset(recovery_, std::move(recovery));\n      return *this;\n    }\n    Options&& set_recovery(\n        Initializer<std::function<bool(absl::Status, CsvReaderBase&)>>\n            recovery) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recovery(std::move(recovery)));\n    }\n    std::function<bool(absl::Status, CsvReaderBase&)>& recovery()\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recovery_;\n    }\n    const std::function<bool(absl::Status, CsvReaderBase&)>& recovery() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recovery_;\n    }\n\n   private:\n    std::optional<CsvHeader> required_header_;\n    std::optional<CsvHeader> assumed_header_;\n    bool preserve_utf8_bom_ = false;\n    bool skip_empty_lines_ = false;\n    std::optional<char> comment_;\n    char field_separator_ = ',';\n    std::optional<char> quote_ = '\"';\n    std::optional<char> escape_;\n    size_t max_num_fields_ = std::numeric_limits<size_t>::max();\n    size_t max_field_length_ = std::numeric_limits<size_t>::max();\n    std::function<bool(absl::Status, CsvReaderBase&)> recovery_;\n  };\n\n  // Returns the byte `Reader` being read from. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Changes the recovery function to be called after skipping over an invalid\n  // line.\n  //\n  // See `Options::set_recovery()` for details.\n  void set_recovery(\n      Initializer<std::function<bool(absl::Status, CsvReaderBase&)>> recovery) {\n    riegeli::Reset(recovery_, std::move(recovery));\n  }\n\n  // Returns `true` if reading the header was requested or assumed, i.e.\n  // `Options::required_header() != std::nullopt ||\n  //  Options::assumed_header() != std::nullopt`.\n  //\n  // In this case `ReadRecord(CsvRecord&)` is supported. Otherwise no particular\n  // header is assumed, and only `ReadRecord(std::vector<std::string>&)` is\n  // supported.\n  bool has_header() const { return has_header_; }\n\n  // If `has_header()`, returns field names read from the first record. Returns\n  // an empty header if reading the header failed.\n  //\n  // If `!has_header()`, returns an empty header.\n  const CsvHeader& header() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return header_;\n  }\n\n  // Reads the next record expressed as `CsvRecord`, with named fields.\n  //\n  // The old value of `record`, including `record.header()`, is overwritten.\n  //\n  // If the number of fields read is not the same as expected by the header,\n  // `CsvReader` fails.\n  //\n  // If `ReadRecord()` returns `true`, `record` will contain all fields present\n  // in the `header()`, and thus it is safe to access fields whose presence has\n  // been verified in the `header()`.\n  //\n  // Precondition:\n  //   `has_header()`, i.e. `Options::required_header() != std::nullopt ||\n  //                         Options::assumed_hedaer() != std::nullopt`\n  //\n  // Return values:\n  //  * `true`                 - success (`record` is set)\n  //  * `false` (when `ok()`)  - source ends (`record` is empty)\n  //  * `false` (when `!ok()`) - failure (`record` is empty)\n  bool ReadRecord(CsvRecord& record);\n\n  // Reads the next record expressed as a vector of fields.\n  //\n  // By a common convention each record should consist of the same number of\n  // fields, but this is not enforced.\n  //\n  // Return values:\n  //  * `true`                 - success (`record` is set)\n  //  * `false` (when `ok()`)  - source ends (`record` is empty)\n  //  * `false` (when `!ok()`) - failure (`record` is empty)\n  bool ReadRecord(std::vector<std::string>& record);\n\n  // Determines if a record follows without reading it, but skips intervening\n  // comments.\n  //\n  // Return values:\n  //  * `true`  - `ReadRecord()` would read the next record or fail\n  //  * `false` - `ReadRecord()` would report that source ends or fail\n  bool HasNextRecord();\n\n  // The index of the most recently read record, starting from 0.\n  //\n  // The record count does not include any header read with\n  // `Options::required_header() != std::nullopt`.\n  //\n  // `last_record_index()` is unchanged by `Close()`.\n  //\n  // Precondition: some record was successfully read (`record_index() > 0`).\n  uint64_t last_record_index() const;\n\n  // The index of the next record, starting from 0.\n  //\n  // The record count does not include any header read with\n  // `Options::required_header() != std::nullopt`.\n  //\n  // `record_index()` is unchanged by `Close()`.\n  uint64_t record_index() const { return record_index_; }\n\n  // The number of the first line of the most recently read record (or attempted\n  // to be read), starting from 1.\n  //\n  // This is 1 if no record was attempted to be read.\n  //\n  // A line is terminated by LF or CR-LF (\"\\n\" or \"\\r\\n\").\n  //\n  // `last_line_number()` is unchanged by `Close()`.\n  int64_t last_line_number() const { return last_line_number_; }\n\n  // The number of the next line, starting from 1.\n  //\n  // A line is terminated by LF or CR-LF (\"\\n\" or \"\\r\\n\").\n  //\n  // `line_number()` is unchanged by `Close()`.\n  int64_t line_number() const { return line_number_; }\n\n protected:\n  using Object::Object;\n\n  CsvReaderBase(CsvReaderBase&& that) noexcept;\n  CsvReaderBase& operator=(CsvReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(Reader* src, Options&& options);\n  // Fails, attributing this to `last_line_number()` instead of `line_number()`.\n  ABSL_ATTRIBUTE_COLD void FailAtPreviousRecord(absl::Status status);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n\n private:\n  friend bool csv_internal::ReadStandaloneRecord(\n      CsvReaderBase& csv_reader, std::vector<std::string>& record);\n\n  enum class CharClass : uint8_t {\n    kOther,\n    kLf,\n    kCr,\n    kComment,\n    kFieldSeparator,\n    kQuote,\n    kEscape,\n  };\n\n  ABSL_ATTRIBUTE_COLD bool FailMaxFieldLengthExceeded();\n  void SkipLine(Reader& src);\n  bool ReadQuoted(Reader& src, std::string& field);\n  bool ReadFields(Reader& src, std::vector<std::string>& fields,\n                  size_t& field_index);\n  bool ReadRecordInternal(std::vector<std::string>& record);\n\n  bool standalone_record_ = false;\n  bool has_header_ = false;\n  CsvHeader header_;\n  // Lookup table for interpreting source characters.\n  std::array<CharClass, std::numeric_limits<unsigned char>::max() + 1>\n      char_classes_{};\n  bool skip_empty_lines_ = false;\n  // Meaningful if `char_classes_` contains `CharClass::kQuote`.\n  char quote_ = '\\0';\n  size_t max_num_fields_ = 0;\n  size_t max_field_length_ = 0;\n  std::function<bool(absl::Status, CsvReaderBase&)> recovery_;\n  uint64_t record_index_ = 0;\n  int64_t last_line_number_ = 1;\n  int64_t line_number_ = 1;\n  bool recoverable_ = false;\n};\n\n// `CsvReader` reads records of a CSV (comma-separated values) file.\n//\n// A basic variant of CSV is specified in https://tools.ietf.org/html/rfc4180,\n// and some common extensions are described in\n// https://specs.frictionlessdata.io/csv-dialect/.\n//\n// `CsvReader` reads RFC4180-compliant CSV files, and also supports some\n// extensions.\n//\n// By a common convention the first record consists of field names. This is\n// supported by `Options::required_header()` and `ReadRecord(CsvRecord&)`.\n//\n// A record is terminated by a newline: LF or CR-LF (\"\\n\" or \"\\r\\n\").\n// Line terminator after the last record is optional.\n//\n// If skipping empty lines is requested (usually it is not), empty lines are\n// skipped. If a comment character is set (usually it is not), a line beginning\n// with the comment character is skipped.\n//\n// A record consists of a sequence of fields separated by a field separator\n// (usually ',' or '\\t'). Each record contains at least one field.\n//\n// Quotes (usually '\"') around a field allow expressing special characters\n// inside the field: LF, CR, comment character, field separator, or quote\n// character itself.\n//\n// If an escape character is set (usually it is not), a character preceded by\n// escape is treated literally instead of possibly having a special meaning.\n// This is an alternative way of expressing special characters inside a field.\n//\n// To express a quote itself inside a field, it must be written twice when the\n// field is quoted, or preceded by an escape character.\n//\n// Quotes are also useful for unambiguous interpretation of a record consisting\n// of a single empty field or beginning with UTF-8 BOM.\n//\n// If neither a quote character nor an escape character is set, special\n// characters inside fields are not expressible. In this case, reading a record\n// consisting of a single empty field is incompatible with\n// `Options::skip_empty_lines()`, and reading the first record beginning with\n// UTF-8 BOM requires `Options::set_preserve_utf8_bom()`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the byte `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The current position is synchronized with the byte `Reader` between records.\ntemplate <typename Src = Reader*>\nclass CsvReader : public CsvReaderBase {\n public:\n  // Creates a closed `CsvReader`.\n  explicit CsvReader(Closed) noexcept : CsvReaderBase(kClosed) {}\n\n  // Will read from the byte `Reader` provided by `src`.\n  explicit CsvReader(Initializer<Src> src, Options options = Options());\n\n  CsvReader(CsvReader&& that) = default;\n  CsvReader& operator=(CsvReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `CsvReader`. This avoids\n  // constructing a temporary `CsvReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the byte `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n\n private:\n  // The object providing and possibly owning the byte `Reader`.\n  Dependency<Reader*, Src> src_;\n};\n\nexplicit CsvReader(Closed) -> CsvReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit CsvReader(Src&& src,\n                   CsvReaderBase::Options options = CsvReaderBase::Options())\n    -> CsvReader<TargetT<Src>>;\n\n// Reads a single record from a CSV string.\n//\n// A record terminator must not be present in the string.\n//\n// Precondition: `options.required_header() == std::nullopt`\nabsl::Status ReadCsvRecordFromString(\n    absl::string_view src, std::vector<std::string>& record,\n    CsvReaderBase::Options options = CsvReaderBase::Options());\n\n// Implementation details follow.\n\ninline CsvReaderBase::CsvReaderBase(CsvReaderBase&& that) noexcept\n    : Object(static_cast<Object&&>(that)),\n      standalone_record_(that.standalone_record_),\n      has_header_(that.has_header_),\n      header_(std::move(that.header_)),\n      char_classes_(that.char_classes_),\n      skip_empty_lines_(that.skip_empty_lines_),\n      quote_(that.quote_),\n      max_num_fields_(that.max_num_fields_),\n      max_field_length_(that.max_field_length_),\n      recovery_(std::move(that.recovery_)),\n      record_index_(std::exchange(that.record_index_, 0)),\n      last_line_number_(std::exchange(that.last_line_number_, 1)),\n      line_number_(std::exchange(that.line_number_, 1)),\n      recoverable_(std::exchange(that.recoverable_, false)) {}\n\ninline CsvReaderBase& CsvReaderBase::operator=(CsvReaderBase&& that) noexcept {\n  Object::operator=(static_cast<Object&&>(that));\n  standalone_record_ = that.standalone_record_;\n  has_header_ = that.has_header_;\n  header_ = std::move(that.header_);\n  char_classes_ = that.char_classes_;\n  skip_empty_lines_ = that.skip_empty_lines_;\n  quote_ = that.quote_;\n  max_num_fields_ = that.max_num_fields_;\n  max_field_length_ = that.max_field_length_;\n  recovery_ = std::move(that.recovery_);\n  record_index_ = std::exchange(that.record_index_, 0);\n  last_line_number_ = std::exchange(that.last_line_number_, 1);\n  line_number_ = std::exchange(that.line_number_, 1);\n  recoverable_ = std::exchange(that.recoverable_, false);\n  return *this;\n}\n\ninline void CsvReaderBase::Reset(Closed) {\n  Object::Reset(kClosed);\n  standalone_record_ = false;\n  has_header_ = false;\n  header_.Reset();\n  recovery_ = nullptr;\n  record_index_ = 0;\n  last_line_number_ = 1;\n  line_number_ = 1;\n  recoverable_ = false;\n}\n\ninline void CsvReaderBase::Reset() {\n  Object::Reset();\n  standalone_record_ = false;\n  has_header_ = false;\n  header_.Reset();\n  char_classes_ = {};\n  recovery_ = nullptr;\n  record_index_ = 0;\n  last_line_number_ = 1;\n  line_number_ = 1;\n  recoverable_ = false;\n}\n\ninline uint64_t CsvReaderBase::last_record_index() const {\n  RIEGELI_ASSERT_GT(record_index_, 0u)\n      << \"Failed precondition of CsvReaderBase::last_record_index(): \"\n         \"no record was read\";\n  return record_index_ - 1;\n}\n\ntemplate <typename Src>\ninline CsvReader<Src>::CsvReader(Initializer<Src> src, Options options)\n    : src_(std::move(src)) {\n  Initialize(src_.get(), std::move(options));\n}\n\ntemplate <typename Src>\ninline void CsvReader<Src>::Reset(Closed) {\n  CsvReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void CsvReader<Src>::Reset(Initializer<Src> src, Options options) {\n  CsvReaderBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(src_.get(), std::move(options));\n}\n\ntemplate <typename Src>\nvoid CsvReader<Src>::Done() {\n  CsvReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(AnnotateOverSrc(src_->status()));\n    }\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CSV_CSV_READER_H_\n"
  },
  {
    "path": "riegeli/csv/csv_record.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/csv/csv_record.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <functional>\n#include <initializer_list>\n#include <optional>\n#include <ostream>\n#include <string>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/const_init.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/container/flat_hash_map.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/ascii.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/synchronization/mutex.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/shared_ptr.h\"\n#include \"riegeli/base/string_ref.h\"\n#include \"riegeli/bytes/ostream_writer.h\"\n#include \"riegeli/bytes/string_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nnamespace {\n\ninline void WriteDebugQuoted(absl::string_view src, Writer& dest,\n                             size_t already_scanned) {\n  dest.Write('\"');\n  const char* start = src.data();\n  const char* next_to_check = src.data() + already_scanned;\n  const char* const limit = src.data() + src.size();\n  // Write characters in the range [`start`..`limit`), except that if quotes are\n  // found in the range [`next_to_check`..`limit`), write them twice.\n  while (const char* const next_quote = static_cast<const char*>(std::memchr(\n             next_to_check, '\"', PtrDistance(next_to_check, limit)))) {\n    dest.Write(absl::string_view(start, PtrDistance(start, next_quote + 1)));\n    start = next_quote;\n    next_to_check = next_quote + 1;\n  }\n  dest.Write(absl::string_view(start, PtrDistance(start, limit)), '\"');\n}\n\ninline std::string DebugQuotedIfNeeded(absl::string_view src) {\n  std::string dest;\n  StringWriter<> writer(&dest);\n  csv_internal::WriteDebugQuotedIfNeeded(src, writer);\n  writer.Close();\n  return dest;\n}\n\n}  // namespace\n\nnamespace csv_internal {\n\nvoid WriteDebugQuotedIfNeeded(absl::string_view src, Writer& dest) {\n  for (size_t i = 0; i < src.size(); ++i) {\n    switch (src[i]) {\n      // For correct CSV syntax.\n      case '\\n':\n      case '\\r':\n      case ',':\n      case '\"':\n      // For unambiguous `CsvRecord::DebugString()`.\n      case ':':\n      // For unambiguous appending of the rest of an error message.\n      case ';':\n        WriteDebugQuoted(src, dest, i);\n        return;\n    }\n  }\n  dest.Write(src);\n}\n\n}  // namespace csv_internal\n\nstd::string AsciiCaseInsensitive(absl::string_view name) {\n  return absl::AsciiStrToLower(name);\n}\n\ninline CsvHeader::Payload::Payload(const Payload& that)\n    : normalizer(that.normalizer),\n      index_to_name(that.index_to_name),\n      name_to_index(that.name_to_index) {}\n\nABSL_CONST_INIT absl::Mutex CsvHeader::payload_cache_mutex_(absl::kConstInit);\nABSL_CONST_INIT SharedPtr<CsvHeader::Payload> CsvHeader::payload_cache_;\n\nCsvHeader::CsvHeader(std::initializer_list<absl::string_view> names) {\n  const absl::Status status = TryResetInternal(nullptr, names);\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvHeader::CsvHeader()\";\n}\n\nCsvHeader& CsvHeader::operator=(\n    std::initializer_list<absl::string_view> names) {\n  const absl::Status status = TryResetInternal(nullptr, names);\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvHeader::operator=()\";\n  return *this;\n}\n\nCsvHeader::CsvHeader(std::function<std::string(absl::string_view)> normalizer)\n    : payload_(normalizer == nullptr ? nullptr\n                                     : SharedPtr<Payload>(riegeli::Maker(\n                                           std::move(normalizer)))) {}\n\nCsvHeader::CsvHeader(std::function<std::string(absl::string_view)> normalizer,\n                     std::initializer_list<absl::string_view> names) {\n  const absl::Status status = TryResetInternal(std::move(normalizer), names);\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvHeader::CsvHeader()\";\n}\n\nvoid CsvHeader::Reset() { payload_.Reset(); }\n\nvoid CsvHeader::Reset(std::initializer_list<absl::string_view> names) {\n  const absl::Status status = TryResetInternal(nullptr, names);\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvHeader::Reset()\";\n}\n\nvoid CsvHeader::Reset(\n    std::function<std::string(absl::string_view)> normalizer) {\n  if (normalizer == nullptr) {\n    payload_.Reset();\n  } else {\n    payload_.Reset(riegeli::Maker(std::move(normalizer)));\n  }\n}\n\nvoid CsvHeader::Reset(std::function<std::string(absl::string_view)> normalizer,\n                      std::initializer_list<absl::string_view> names) {\n  const absl::Status status = TryResetInternal(std::move(normalizer), names);\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvHeader::Reset()\";\n}\n\nabsl::Status CsvHeader::TryReset(\n    std::initializer_list<absl::string_view> names) {\n  return TryResetInternal(nullptr, names);\n}\n\nabsl::Status CsvHeader::TryReset(\n    std::function<std::string(absl::string_view)> normalizer,\n    std::initializer_list<absl::string_view> names) {\n  return TryResetInternal(std::move(normalizer), names);\n}\n\nabsl::Status CsvHeader::TryResetUncached(\n    std::function<std::string(absl::string_view)>&& normalizer,\n    std::vector<std::string>&& names) {\n  EnsureUnique();\n  payload_->normalizer = std::move(normalizer);\n  payload_->name_to_index.clear();\n  payload_->name_to_index.reserve(names.size());\n  std::vector<absl::string_view> duplicate_names;\n  for (size_t index = 0; index < names.size(); ++index) {\n    const std::pair<absl::flat_hash_map<std::string, size_t>::iterator, bool>\n        insert_result =\n            payload_->normalizer == nullptr\n                ? payload_->name_to_index.emplace(names[index], index)\n                : payload_->name_to_index.emplace(\n                      payload_->normalizer(names[index]), index);\n    if (ABSL_PREDICT_FALSE(!insert_result.second)) {\n      duplicate_names.push_back(names[index]);\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!duplicate_names.empty())) {\n    payload_.Reset();\n    StringWriter<std::string> message;\n    message.Write(\"Duplicate field names: \");\n    for (std::vector<absl::string_view>::const_iterator iter =\n             duplicate_names.cbegin();\n         iter != duplicate_names.cend(); ++iter) {\n      if (iter != duplicate_names.cbegin()) message.Write(',');\n      csv_internal::WriteDebugQuotedIfNeeded(*iter, message);\n    }\n    message.Close();\n    return absl::FailedPreconditionError(message.dest());\n  }\n  payload_->index_to_name = std::move(names);\n  if (payload_->normalizer == nullptr) {\n    SharedPtr<Payload> old_payload_cache;\n    {\n      absl::MutexLock lock(payload_cache_mutex_);\n      old_payload_cache = std::exchange(payload_cache_, payload_);\n    }\n    // Destroy `old_payload_cache` after releasing `payload_cache_mutex_`.\n  }\n  return absl::OkStatus();\n}\n\nvoid CsvHeader::Reserve(size_t size) {\n  EnsureUnique();\n  payload_->index_to_name.reserve(size);\n  payload_->name_to_index.reserve(size);\n}\n\nvoid CsvHeader::Add(StringInitializer name) {\n  const absl::Status status = TryAdd(std::move(name));\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvHeader::Add()\";\n}\n\nabsl::Status CsvHeader::TryAdd(StringInitializer name) {\n  EnsureUnique();\n  std::string name_string = std::move(name);\n  const size_t index = payload_->index_to_name.size();\n  const std::pair<absl::flat_hash_map<std::string, size_t>::iterator, bool>\n      insert_result = payload_->normalizer == nullptr\n                          ? payload_->name_to_index.emplace(name_string, index)\n                          : payload_->name_to_index.emplace(\n                                payload_->normalizer(name_string), index);\n  if (ABSL_PREDICT_FALSE(!insert_result.second)) {\n    RIEGELI_ASSERT(!empty())\n        << \"It should not have been needed to ensure that an empty CsvHeader \"\n           \"has payload_ == nullptr because a duplicate field name is possible \"\n           \"only if some fields were already present\";\n    StringWriter<std::string> message;\n    message.Write(\"Duplicate field name: \");\n    csv_internal::WriteDebugQuotedIfNeeded(name_string, message);\n    message.Close();\n    return absl::FailedPreconditionError(message.dest());\n  }\n  payload_->index_to_name.push_back(std::move(name_string));\n  return absl::OkStatus();\n}\n\nCsvHeader::iterator CsvHeader::find(absl::string_view name) const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  if (ABSL_PREDICT_FALSE(payload_ == nullptr)) return iterator();\n  const absl::flat_hash_map<std::string, size_t>::const_iterator iter =\n      payload_->normalizer == nullptr\n          ? payload_->name_to_index.find(name)\n          : payload_->name_to_index.find(payload_->normalizer(name));\n  if (ABSL_PREDICT_FALSE(iter == payload_->name_to_index.cend())) {\n    return iterator(payload_->index_to_name.cend());\n  }\n  return iterator(payload_->index_to_name.cbegin() + iter->second);\n}\n\nbool CsvHeader::contains(absl::string_view name) const {\n  if (ABSL_PREDICT_FALSE(payload_ == nullptr)) return false;\n  const absl::flat_hash_map<std::string, size_t>::const_iterator iter =\n      payload_->normalizer == nullptr\n          ? payload_->name_to_index.find(name)\n          : payload_->name_to_index.find(payload_->normalizer(name));\n  return iter != payload_->name_to_index.cend();\n}\n\nstd::optional<size_t> CsvHeader::IndexOf(absl::string_view name) const {\n  if (ABSL_PREDICT_FALSE(payload_ == nullptr)) return std::nullopt;\n  const absl::flat_hash_map<std::string, size_t>::const_iterator iter =\n      payload_->normalizer == nullptr\n          ? payload_->name_to_index.find(name)\n          : payload_->name_to_index.find(payload_->normalizer(name));\n  if (ABSL_PREDICT_FALSE(iter == payload_->name_to_index.cend())) {\n    return std::nullopt;\n  }\n  return iter->second;\n}\n\nbool CsvHeader::Equal(const CsvHeader& a, const CsvHeader& b) {\n  if (ABSL_PREDICT_TRUE(a.payload_ == b.payload_)) return true;\n  if (a.payload_ == nullptr || b.payload_ == nullptr) return false;\n  return a.payload_->index_to_name == b.payload_->index_to_name;\n}\n\ninline void CsvHeader::EnsureUnique() {\n  if (payload_ == nullptr) {\n    payload_.Reset(riegeli::Maker());\n  } else if (ABSL_PREDICT_FALSE(!payload_.IsUnique())) {\n    payload_ = SharedPtr<Payload>(*payload_);\n  }\n}\n\nvoid CsvHeader::WriteDebugStringTo(Writer& dest) const {\n  for (iterator iter = cbegin(); iter != cend(); ++iter) {\n    if (iter != cbegin()) dest.Write(',');\n    csv_internal::WriteDebugQuotedIfNeeded(*iter, dest);\n  }\n}\n\nstd::string CsvHeader::DebugString() const {\n  std::string result;\n  StringWriter<> writer(&result);\n  WriteDebugStringTo(writer);\n  writer.Close();\n  return result;\n}\n\nvoid CsvHeader::Output(std::ostream& dest) const {\n  OStreamWriter<> writer(&dest);\n  WriteDebugStringTo(writer);\n  writer.Close();\n}\n\nCsvRecord::CsvRecord(CsvHeader header,\n                     std::initializer_list<absl::string_view> fields) {\n  const absl::Status status = TryResetInternal(std::move(header), fields);\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvRecord::CsvRecord()\";\n}\n\nvoid CsvRecord::Reset() {\n  header_.Reset();\n  fields_.clear();\n}\n\nvoid CsvRecord::Reset(CsvHeader header) {\n  header_ = std::move(header);\n  fields_.resize(header_.size());\n  Clear();\n}\n\nvoid CsvRecord::Reset(CsvHeader header,\n                      std::initializer_list<absl::string_view> fields) {\n  const absl::Status status = TryResetInternal(std::move(header), fields);\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvRecord::Reset()\";\n}\n\nabsl::Status CsvRecord::TryReset(\n    CsvHeader header, std::initializer_list<absl::string_view> fields) {\n  return TryResetInternal(std::move(header), fields);\n}\n\nabsl::Status CsvRecord::TryResetInternal(CsvHeader&& header,\n                                         std::vector<std::string>&& fields) {\n  if (ABSL_PREDICT_FALSE(header.size() != fields.size())) {\n    header_.Reset();\n    fields_.clear();\n    return absl::FailedPreconditionError(\n        absl::StrCat(\"Mismatched number of CSV fields: header has \",\n                     header.size(), \", record has \", fields.size()));\n  }\n  header_ = std::move(header);\n  fields_ = std::move(fields);\n  return absl::OkStatus();\n}\n\nvoid CsvRecord::Clear() {\n  for (std::string& value : fields_) value.clear();\n}\n\nstd::string& CsvRecord::operator[](absl::string_view name)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  const CsvHeader::iterator name_iter = header_.find(name);\n  RIEGELI_CHECK(name_iter != header_.end())\n      << \"Failed precondition of CsvRecord::operator[]: missing field name: \"\n      << DebugQuotedIfNeeded(name) << \"; existing field names: \" << header_;\n  return fields_[name_iter - header_.begin()];\n}\n\nconst std::string& CsvRecord::operator[](absl::string_view name) const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  const CsvHeader::iterator name_iter = header_.find(name);\n  RIEGELI_CHECK(name_iter != header_.end())\n      << \"Failed precondition of CsvRecord::operator[]: missing field name: \"\n      << DebugQuotedIfNeeded(name) << \"; existing field names: \" << header_;\n  return fields_[name_iter - header_.begin()];\n}\n\nCsvRecord::iterator CsvRecord::find(absl::string_view name)\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  const CsvHeader::iterator name_iter = header_.find(name);\n  RIEGELI_ASSERT(name_iter >= header_.begin() && name_iter <= header_.end())\n      << \"Failed precondition of CsvRecord::find(): \"\n         \"field name iterator does not belong to the same header\";\n  return iterator(name_iter, fields_.begin() + (name_iter - header_.begin()));\n}\n\nCsvRecord::const_iterator CsvRecord::find(absl::string_view name) const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  const CsvHeader::iterator name_iter = header_.find(name);\n  RIEGELI_ASSERT(name_iter >= header_.begin() && name_iter <= header_.end())\n      << \"Failed precondition of CsvRecord::find(): \"\n         \"field name iterator does not belong to the same header\";\n  return const_iterator(name_iter,\n                        fields_.cbegin() + (name_iter - header_.begin()));\n}\n\nbool CsvRecord::contains(absl::string_view name) const {\n  return header_.contains(name);\n}\n\nvoid CsvRecord::Merge(\n    std::initializer_list<std::pair<absl::string_view, absl::string_view>>\n        src) {\n  Merge<std::initializer_list<std::pair<absl::string_view, absl::string_view>>>(\n      std::move(src));\n}\n\nabsl::Status CsvRecord::TryMerge(\n    std::initializer_list<std::pair<absl::string_view, absl::string_view>>\n        src) {\n  return TryMerge<\n      std::initializer_list<std::pair<absl::string_view, absl::string_view>>>(\n      std::move(src));\n}\n\nabsl::Status CsvRecord::FailMissingNames(\n    absl::Span<const std::string> missing_names) const {\n  StringWriter<std::string> message;\n  message.Write(\"Missing field names: \");\n  for (absl::Span<const std::string>::const_iterator iter =\n           missing_names.cbegin();\n       iter != missing_names.cend(); ++iter) {\n    if (iter != missing_names.cbegin()) message.Write(',');\n    csv_internal::WriteDebugQuotedIfNeeded(*iter, message);\n  }\n  message.Write(\"; existing field names: \");\n  for (CsvHeader::const_iterator iter = header_.cbegin();\n       iter != header_.cend(); ++iter) {\n    if (iter != header_.cbegin()) message.Write(',');\n    csv_internal::WriteDebugQuotedIfNeeded(*iter, message);\n  }\n  message.Close();\n  return absl::FailedPreconditionError(message.dest());\n}\n\nbool CsvRecord::Equal(const CsvRecord& a, const CsvRecord& b) {\n  return a.header() == b.header() && a.fields() == b.fields();\n}\n\nvoid CsvRecord::WriteDebugStringTo(Writer& dest) const {\n  RIEGELI_ASSERT_EQ(header_.size(), fields_.size())\n      << \"Failed invariant of CsvRecord: \"\n         \"mismatched length of CSV header and fields\";\n  for (const_iterator iter = cbegin(); iter != cend(); ++iter) {\n    if (iter != cbegin()) dest.Write(',');\n    csv_internal::WriteDebugQuotedIfNeeded(iter->first, dest);\n    dest.Write(':');\n    csv_internal::WriteDebugQuotedIfNeeded(iter->second, dest);\n  }\n}\n\nstd::string CsvRecord::DebugString() const {\n  std::string result;\n  StringWriter<> writer(&result);\n  WriteDebugStringTo(writer);\n  writer.Close();\n  return result;\n}\n\nvoid CsvRecord::Output(std::ostream& dest) const {\n  OStreamWriter<> writer(&dest);\n  WriteDebugStringTo(writer);\n  writer.Close();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/csv/csv_record.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CSV_CSV_RECORD_H_\n#define RIEGELI_CSV_CSV_RECORD_H_\n\n#include <stddef.h>\n\n#include <algorithm>\n#include <functional>\n#include <initializer_list>\n#include <iosfwd>\n#include <iterator>\n#include <new>\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/call_once.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/base/thread_annotations.h\"\n#include \"absl/container/flat_hash_map.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/synchronization/mutex.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/global.h\"\n#include \"riegeli/base/iterable.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/shared_ptr.h\"\n#include \"riegeli/base/string_ref.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/bytes/stringify_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli::csv_internal {\n\n// `ToStringVector()` converts an iterable of elements convertible to\n// `absl::string_view` to a `std::vector<std::string>`.\n\ntemplate <\n    typename Values,\n    std::enable_if_t<IsIterableOf<Values, absl::string_view>::value, int> = 0>\ninline std::vector<std::string> ToStringVector(Values&& values) {\n  using std::begin;\n  using std::end;\n  return std::vector<std::string>(MaybeMakeMoveIterator<Values>(begin(values)),\n                                  MaybeMakeMoveIterator<Values>(end(values)));\n}\n\n}  // namespace riegeli::csv_internal\n\nnamespace riegeli {\n\n// A normalizer for `CsvHeader` and `CsvReaderBase::Options::set_normalizer()`,\n// providing case insensitive matching.\n//\n// You may pass a pointer to this function, without wrapping it in a lambda\n// (it will not be overloaded).\nstd::string AsciiCaseInsensitive(absl::string_view name);\n\n// A set of field names. This is commonly specified in a CSV file header.\n//\n// This is conceptually a set of strings, which remembers the order in which\n// they have been added.\n//\n// Copying a `CsvHeader` object is cheap, sharing the actual set.\nclass CsvHeader : public WithEqual<CsvHeader> {\n public:\n  class iterator : public WithCompare<iterator> {\n   public:\n    using iterator_category = std::random_access_iterator_tag;\n    using value_type = std::string;\n    using reference = const std::string&;\n    using pointer = const std::string*;\n    using difference_type = ptrdiff_t;\n\n    iterator() = default;\n\n    iterator(const iterator& that) = default;\n    iterator& operator=(const iterator& that) = default;\n\n    reference operator*() const;\n    pointer operator->() const;\n    iterator& operator++();\n    iterator operator++(int);\n    iterator& operator--();\n    iterator operator--(int);\n    iterator& operator+=(difference_type n);\n    iterator operator+(difference_type n) const;\n    iterator& operator-=(difference_type n);\n    iterator operator-(difference_type n) const;\n    reference operator[](difference_type n) const;\n\n    friend bool operator==(iterator a, iterator b) {\n      return a.iter_ == b.iter_;\n    }\n    friend StrongOrdering RIEGELI_COMPARE(iterator a, iterator b) {\n      if (a.iter_ < b.iter_) return StrongOrdering::less;\n      if (a.iter_ > b.iter_) return StrongOrdering::greater;\n      return StrongOrdering::equal;\n    }\n    friend difference_type operator-(iterator a, iterator b) {\n      return a.iter_ - b.iter_;\n    }\n    friend iterator operator+(difference_type n, iterator a) { return a + n; }\n\n   private:\n    friend class CsvHeader;\n\n    explicit iterator(std::vector<std::string>::const_iterator iter)\n        : iter_(iter) {}\n\n    std::vector<std::string>::const_iterator iter_{};\n  };\n\n  using key_type = std::string;\n  using value_type = std::string;\n  using reference = const std::string&;\n  using const_reference = reference;\n  using pointer = const std::string*;\n  using const_pointer = pointer;\n  using const_iterator = iterator;\n  using reverse_iterator = std::reverse_iterator<iterator>;\n  using const_reverse_iterator = reverse_iterator;\n  using size_type = size_t;\n  using difference_type = ptrdiff_t;\n\n  // Creates a `CsvHeader` with no field names.\n  CsvHeader() = default;\n\n  // Creates a `CsvHeader` with no field names.\n  //\n  // Field names are matched by passing them through `normalizer` first.\n  // `nullptr` is the same as the identity function.\n  //\n  // `riegeli::AsciiCaseInsensitive` is a normalizer providing case insensitive\n  // matching.\n  explicit CsvHeader(std::function<std::string(absl::string_view)> normalizer);\n\n  // Creates a set consisting of the given sequence of field names.\n  //\n  // The type of `names` must support iteration yielding `absl::string_view`:\n  // `for (const absl::string_view name : names)`,\n  // e.g. `std::vector<std::string>`.\n  //\n  // Precondition: `names` have no duplicates\n  template <typename Names,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<CsvHeader, Names>,\n                                   IsIterableOf<Names, absl::string_view>>,\n                int> = 0>\n  explicit CsvHeader(Names&& names);\n  /*implicit*/ CsvHeader(std::initializer_list<absl::string_view> names);\n  CsvHeader& operator=(std::initializer_list<absl::string_view> names);\n\n  // Creates a set consisting of the given sequence of field names.\n  //\n  // The type of `names` must support iteration yielding `absl::string_view`:\n  // `for (const absl::string_view name : names)`,\n  // e.g. `std::vector<std::string>`.\n  //\n  // Field names are matched by passing them through `normalizer` first.\n  // `nullptr` is the same as the identity function.\n  //\n  // `riegeli::AsciiCaseInsensitive` is a normalizer providing case insensitive\n  // matching.\n  //\n  // Precondition: normalized `names` have no duplicates\n  template <\n      typename Names,\n      std::enable_if_t<IsIterableOf<Names, absl::string_view>::value, int> = 0>\n  explicit CsvHeader(std::function<std::string(absl::string_view)> normalizer,\n                     Names&& names);\n  explicit CsvHeader(std::function<std::string(absl::string_view)> normalizer,\n                     std::initializer_list<absl::string_view> names);\n\n  CsvHeader(const CsvHeader& that) = default;\n  CsvHeader& operator=(const CsvHeader& that) = default;\n\n  // The source `CsvHeader` is left empty.\n  CsvHeader(CsvHeader&& that) = default;\n  CsvHeader& operator=(CsvHeader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `CsvHeader`.\n  //\n  // Precondition: like for the corresponding constructor\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n  template <typename Names,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<CsvHeader, Names>,\n                                   IsIterableOf<Names, absl::string_view>>,\n                int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Names&& names);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      std::initializer_list<absl::string_view> names);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      std::function<std::string(absl::string_view)> normalizer);\n  template <\n      typename Names,\n      std::enable_if_t<IsIterableOf<Names, absl::string_view>::value, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      std::function<std::string(absl::string_view)> normalizer, Names&& names);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      std::function<std::string(absl::string_view)> normalizer,\n      std::initializer_list<absl::string_view> names);\n\n  // Makes `*this` equivalent to a newly constructed `CsvHeader`, reporting\n  // whether construction was successful.\n  //\n  // Return values:\n  //  * `absl::OkStatus()`                 - `CsvHeader` is set to `names`\n  //  * `absl::FailedPreconditionError(_)` - `names` had duplicates,\n  //                                         `CsvHeader` is empty\n  template <typename Names,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<CsvHeader, Names>,\n                                   IsIterableOf<Names, absl::string_view>>,\n                int> = 0>\n  absl::Status TryReset(Names&& names);\n  absl::Status TryReset(std::initializer_list<absl::string_view> names);\n  template <\n      typename Names,\n      std::enable_if_t<IsIterableOf<Names, absl::string_view>::value, int> = 0>\n  absl::Status TryReset(\n      std::function<std::string(absl::string_view)> normalizer, Names&& names);\n  absl::Status TryReset(\n      std::function<std::string(absl::string_view)> normalizer,\n      std::initializer_list<absl::string_view> names);\n\n  // Reserve space for future calls to `Add()` or `TryAdd()` if the expected\n  // final number of fields is known. This improves performance.\n  void Reserve(size_t size);\n\n  // Adds the given field `name`, ordered at the end.\n  //\n  // Precondition: `name` was not already present\n  void Add(StringInitializer name);\n\n  // Equivalent to calling `Add()` for each name in order.\n  //\n  // Precondition: like for `Add()`\n  template <typename... Names,\n            std::enable_if_t<(sizeof...(Names) > 0), int> = 0>\n  void Add(StringInitializer name, Names&&... names);\n\n  // Adds the given field `name`, ordered at the end, reporting whether this was\n  // successful.\n  //\n  // Return values:\n  //  * `absl::OkStatus()`                 - `name` has been added\n  //  * `absl::FailedPreconditionError(_)` - `name` was already present,\n  //                                         `CsvHeader` is unchanged\n  absl::Status TryAdd(StringInitializer name);\n\n  // Equivalent to calling `TryAdd()` for each name in order.\n  //\n  // Returns early in case of a failure.\n  template <typename... Names,\n            std::enable_if_t<(sizeof...(Names) > 0), int> = 0>\n  absl::Status TryAdd(StringInitializer name, Names&&... names);\n\n  // Returns the sequence of field names, in the order in which they have been\n  // added.\n  absl::Span<const std::string> names() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns the normalizer used to match field names, or `nullptr` which is the\n  // same as the identity function.\n  const std::function<std::string(absl::string_view)>& normalizer() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Iterates over field names, in the order in which they have been added.\n  iterator begin() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  iterator cbegin() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return begin(); }\n  iterator end() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  iterator cend() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return end(); }\n\n  // Iterates over field names, backwards.\n  reverse_iterator rbegin() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return reverse_iterator(end());\n  }\n  reverse_iterator crbegin() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return rbegin();\n  }\n  reverse_iterator rend() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return reverse_iterator(begin());\n  }\n  reverse_iterator crend() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return rend();\n  }\n\n  // Returns `true` if there are no field names.\n  bool empty() const;\n\n  // Returns the number of field names.\n  size_t size() const;\n\n  // Returns an iterator positioned at `name`, or `end()` if `name` is not\n  // present.\n  iterator find(absl::string_view name) const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns `true` if `name` is present.\n  bool contains(absl::string_view name) const;\n\n  // Returns the position of `name` in the sequence of field names, or\n  // `std::nullopt` if `name` is not present.\n  //\n  // This can be used together with `CsvRecord::fields()` to look up the same\n  // field in multiple `CsvRecord`s sharing a `CsvHeader`.\n  std::optional<size_t> IndexOf(absl::string_view name) const;\n\n  // Compares the sequence of field names. Does not compare the normalizer.\n  friend bool operator==(const CsvHeader& a, const CsvHeader& b) {\n    return Equal(a, b);\n  }\n\n  // Renders contents in a human-readable way.\n  std::string DebugString() const;\n\n  // Default stringification by `absl::StrCat()` etc.\n  //\n  // Writes `src.DebugString()` to `dest`.\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const CsvHeader& src) {\n    StringifyWriter<Sink*> writer(&dest);\n    src.WriteDebugStringTo(writer);\n    writer.Close();\n  }\n\n  // Writes `src.DebugString()` to `dest`.\n  friend std::ostream& operator<<(std::ostream& dest, const CsvHeader& src) {\n    src.Output(dest);\n    return dest;\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const CsvHeader* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->payload_);\n  }\n\n private:\n  struct Payload {\n    Payload() = default;\n    Payload(std::function<std::string(absl::string_view)>&& normalizer)\n        : normalizer(std::move(normalizer)) {}\n    Payload(const Payload& that);\n\n    // Supports `MemoryEstimator`.\n    template <typename MemoryEstimator>\n    friend void RiegeliRegisterSubobjects(const Payload* self,\n                                          MemoryEstimator& memory_estimator) {\n      // Ignore `normalizer`. Even if not `nullptr`, usually it is stateless.\n      memory_estimator.RegisterSubobjects(&self->index_to_name);\n      memory_estimator.RegisterSubobjects(&self->name_to_index);\n    }\n\n    std::function<std::string(absl::string_view)> normalizer;\n    std::vector<std::string> index_to_name;\n    absl::flat_hash_map<std::string, size_t> name_to_index;\n\n    // Invariants:\n    //  * `!index_to_name.empty()`\n    //  * `name_to_index.size() == index_to_name.size()`\n    //  * for each `i` below `index_to_name.size()`:\n    //        `name_to_index[normalizer == nullptr\n    //                           ? index_to_name[i]\n    //                           : normalizer(index_to_name[i])] == i`\n  };\n\n  template <typename Names>\n  absl::Status TryResetInternal(\n      std::function<std::string(absl::string_view)>&& normalizer,\n      Names&& names);\n  absl::Status TryResetInternal(\n      std::function<std::string(absl::string_view)>&& normalizer,\n      std::vector<std::string>&& names);\n\n  // Handles `TryReset()` for `names` which are empty or which match the cached\n  // payload. Returns `true` if done.\n  template <typename Names>\n  bool MaybeResetToCachedPayload(Names&& names);\n\n  // Handles `TryReset()` for `names` which are not empty and which do not match\n  // the cached payload.\n  absl::Status TryResetUncached(\n      std::function<std::string(absl::string_view)>&& normalizer,\n      std::vector<std::string>&& names);\n\n  void EnsureUnique();\n\n  static bool Equal(const CsvHeader& a, const CsvHeader& b);\n\n  void WriteDebugStringTo(Writer& dest) const;\n  void Output(std::ostream& dest) const;\n\n  // A one-element cache of a recently constructed `Payload`, to reuse the\n  // `Payload` when multiple `CsvHeader` objects are created from the same\n  // iterable of field names. Its `normalizer` is always `nullptr` and its\n  // `index_to_name` is never empty.\n  //\n  // Reusing `CsvHeader` directly is more efficient but not always feasible.\n  ABSL_CONST_INIT static absl::Mutex payload_cache_mutex_;\n  ABSL_CONST_INIT static SharedPtr<Payload> payload_cache_\n      ABSL_GUARDED_BY(payload_cache_mutex_);\n\n  SharedPtr<Payload> payload_;\n};\n\n// `CsvHeaderConstant<n>` lazily constructs and stores a `CsvHeader` with `n`\n// fields, and never calls its destructor.\n//\n// It should be used as the type of a variable with static storage duration.\n//\n// By relying on CTAD the template argument can be deduced from constructor\n// arguments.\ntemplate <size_t num_fields>\nclass CsvHeaderConstant {\n public:\n  // Will create a `CsvHeader` consisting of the given sequence of field names.\n  //\n  // The number of `fields` must be `num_fields`, and all `fields` must have\n  // static storage duration.\n  template <\n      typename... Fields,\n      std::enable_if_t<std::conjunction_v<\n                           std::bool_constant<sizeof...(Fields) == num_fields>,\n                           std::is_convertible<Fields&&, absl::string_view>...>,\n                       int> = 0>\n  /*implicit*/ constexpr CsvHeaderConstant(\n      Fields&&... fields ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : fields_{std::forward<Fields>(fields)...} {}\n\n  // Will create a `CsvHeader` consisting of the given sequence of field names.\n  //\n  // The number of `fields` must be `num_fields`, and all `fields` must have\n  // static storage duration.\n  //\n  // Field names are matched by passing them through `normalizer` first.\n  // `nullptr` is the same as the identity function.\n  template <\n      typename... Fields,\n      std::enable_if_t<std::conjunction_v<\n                           std::bool_constant<sizeof...(Fields) == num_fields>,\n                           std::is_convertible<Fields&&, absl::string_view>...>,\n                       int> = 0>\n  explicit constexpr CsvHeaderConstant(\n      std::string (*normalizer)(absl::string_view),\n      Fields&&... fields ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : normalizer_(normalizer), fields_{std::forward<Fields>(fields)...} {}\n\n  // Will create a `CsvHeader` consisting of field names from `base_header`\n  // followed by the given sequence of field names.\n  //\n  // The number of fields in `base_header` plus the number of `fields` must be\n  // `num_fields`, and `base_header` and all `fields` must have static storage\n  // duration.\n  //\n  // The normalizer is the same as in `base_header`.\n  template <\n      size_t base_num_fields, typename... Fields,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::integral_constant<\n                  bool, base_num_fields + sizeof...(Fields) == num_fields>,\n              std::is_convertible<Fields&&, absl::string_view>...>,\n          int> = 0>\n  explicit constexpr CsvHeaderConstant(\n      const CsvHeaderConstant<base_num_fields>& base_header\n          ABSL_ATTRIBUTE_LIFETIME_BOUND,\n      Fields&&... fields ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : CsvHeaderConstant(base_header,\n                          std::make_index_sequence<base_num_fields>(),\n                          std::forward<Fields>(fields)...) {}\n\n  CsvHeaderConstant(const CsvHeaderConstant&) = delete;\n  CsvHeaderConstant& operator=(const CsvHeaderConstant&) = delete;\n\n  const CsvHeader* get() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const CsvHeader& operator*() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return *get();\n  }\n  const CsvHeader* operator->() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return get();\n  }\n\n private:\n  template <size_t other_num_fields>\n  friend class CsvHeaderConstant;  // For `normalizer_` and `fields_`.\n\n  template <size_t base_num_fields, size_t... base_indices, typename... Fields>\n  explicit constexpr CsvHeaderConstant(\n      const CsvHeaderConstant<base_num_fields>& base_header,\n      std::index_sequence<base_indices...>, Fields&&... fields)\n      : normalizer_(base_header.normalizer_),\n        fields_{base_header.fields_[base_indices]...,\n                std::forward<Fields>(fields)...} {}\n\n  std::string (*const normalizer_)(absl::string_view) = nullptr;\n  const absl::string_view fields_[num_fields];\n  mutable absl::once_flag once_;\n  alignas(CsvHeader) mutable char header_[sizeof(CsvHeader)] = {};\n};\n\ntemplate <\n    typename... Fields,\n    std::enable_if_t<\n        std::conjunction_v<std::is_convertible<Fields&&, absl::string_view>...>,\n        int> = 0>\n/*implicit*/ CsvHeaderConstant(Fields&&... fields)\n    -> CsvHeaderConstant<sizeof...(Fields)>;\ntemplate <\n    typename... Fields,\n    std::enable_if_t<\n        std::conjunction_v<std::is_convertible<Fields&&, absl::string_view>...>,\n        int> = 0>\nexplicit CsvHeaderConstant(std::string (*normalizer)(absl::string_view),\n                           Fields&&... fields)\n    -> CsvHeaderConstant<sizeof...(Fields)>;\ntemplate <\n    size_t base_num_fields, typename... Fields,\n    std::enable_if_t<\n        std::conjunction_v<std::is_convertible<Fields&&, absl::string_view>...>,\n        int> = 0>\nexplicit CsvHeaderConstant(\n    const CsvHeaderConstant<base_num_fields>& base_header, Fields&&... fields)\n    -> CsvHeaderConstant<base_num_fields + sizeof...(Fields)>;\n\n// A row of a CSV file, with fields accessed by name.\n//\n// This is conceptually a mapping from field names to field values, with a fixed\n// set of field names. The set of field names is expressed as `CsvHeader`.\nclass CsvRecord : public WithEqual<CsvRecord> {\n private:\n  // Implementation shared between `iterator` and `const_iterator`.\n  template <typename FieldIterator>\n  class IteratorImpl : public WithCompare<IteratorImpl<FieldIterator>> {\n   public:\n    using iterator_concept = std::random_access_iterator_tag;\n    // `iterator_category` is only `std::input_iterator_tag` because the\n    // `LegacyForwardIterator` requirement and above require `reference` to be\n    // a true reference type.\n    using iterator_category = std::input_iterator_tag;\n    using value_type = std::pair<const std::string, std::string>;\n    using reference =\n        ReferencePair<const std::string&,\n                      typename std::iterator_traits<FieldIterator>::reference>;\n    using pointer = ArrowProxy<reference>;\n    using difference_type = ptrdiff_t;\n\n    IteratorImpl() = default;\n\n    // Conversion from `iterator` to `const_iterator`.\n    template <\n        typename ThatFieldIterator,\n        std::enable_if_t<\n            std::conjunction_v<\n                std::negation<std::is_same<ThatFieldIterator, FieldIterator>>,\n                std::is_convertible<ThatFieldIterator, FieldIterator>>,\n            int> = 0>\n    /*implicit*/ IteratorImpl(IteratorImpl<ThatFieldIterator> that) noexcept;\n\n    IteratorImpl(const IteratorImpl& that) = default;\n    IteratorImpl& operator=(const IteratorImpl& that) = default;\n\n    reference operator*() const;\n    pointer operator->() const;\n    IteratorImpl& operator++();\n    IteratorImpl operator++(int);\n    IteratorImpl& operator--();\n    IteratorImpl operator--(int);\n    IteratorImpl& operator+=(difference_type n);\n    IteratorImpl operator+(difference_type n) const;\n    IteratorImpl& operator-=(difference_type n);\n    IteratorImpl operator-(difference_type n) const;\n    reference operator[](difference_type n) const;\n\n    friend bool operator==(IteratorImpl a, IteratorImpl b) {\n      return a.field_iter_ == b.field_iter_;\n    }\n    friend StrongOrdering RIEGELI_COMPARE(IteratorImpl a, IteratorImpl b) {\n      if (a.field_iter_ < b.field_iter_) return StrongOrdering::less;\n      if (a.field_iter_ > b.field_iter_) return StrongOrdering::greater;\n      return StrongOrdering::equal;\n    }\n    friend difference_type operator-(IteratorImpl a, IteratorImpl b) {\n      return a.field_iter_ - b.field_iter_;\n    }\n    friend IteratorImpl operator+(difference_type n, IteratorImpl a) {\n      return a + n;\n    }\n\n   private:\n    friend class CsvRecord;\n\n    explicit IteratorImpl(CsvHeader::iterator name_iter,\n                          FieldIterator field_iter);\n\n    // Invariant:\n    //   `name_iter_ - header_.begin() == field_iter_ - fields_.begin()`\n    CsvHeader::iterator name_iter_;\n    FieldIterator field_iter_;\n  };\n\n public:\n  using key_type = std::string;\n  using mapped_type = std::string;\n  using value_type = std::pair<const std::string, std::string>;\n  using reference = ReferencePair<const std::string&, std::string&>;\n  using const_reference = ReferencePair<const std::string&, const std::string&>;\n  using iterator = IteratorImpl<std::vector<std::string>::iterator>;\n  using const_iterator = IteratorImpl<std::vector<std::string>::const_iterator>;\n  using reverse_iterator = std::reverse_iterator<iterator>;\n  using const_reverse_iterator = std::reverse_iterator<const_iterator>;\n  using size_type = size_t;\n  using difference_type = ptrdiff_t;\n\n  // Creates a `CsvRecord` with no fields.\n  CsvRecord() = default;\n\n  // Creates a `CsvRecord` with the given field names, and with all field values\n  // empty.\n  explicit CsvRecord(CsvHeader header);\n\n  // Creates a `CsvRecord` with the given field names and field values in the\n  // corresponding order.\n  //\n  // Precondition: `header.size() == fields.size()`\n  template <\n      typename Fields,\n      std::enable_if_t<IsIterableOf<Fields, absl::string_view>::value, int> = 0>\n  explicit CsvRecord(CsvHeader header, Fields&& fields);\n  explicit CsvRecord(CsvHeader header,\n                     std::initializer_list<absl::string_view> fields);\n\n  CsvRecord(const CsvRecord& that) = default;\n  CsvRecord& operator=(const CsvRecord& that) = default;\n\n  // The source `CsvRecord` is left empty.\n  CsvRecord(CsvRecord&& that) = default;\n  CsvRecord& operator=(CsvRecord&& that) = default;\n\n  // Returns the set of field names.\n  const CsvHeader& header() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return header_;\n  }\n\n  // Makes `*this` equivalent to a newly constructed `CsvRecord`.\n  //\n  // Precondition: like for the corresponding constructor\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(CsvHeader header);\n  template <\n      typename Fields,\n      std::enable_if_t<IsIterableOf<Fields, absl::string_view>::value, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(CsvHeader header, Fields&& fields);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      CsvHeader header, std::initializer_list<absl::string_view> fields);\n\n  // Makes `*this` equivalent to a newly constructed `CsvRecord`, reporting\n  // whether construction was successful.\n  //\n  // Return values:\n  //  * `absl::OkStatus()`                 - `CsvRecord` is set to `header`\n  //                                         and `fields`\n  //  * `absl::FailedPreconditionError(_)` - lengths of `header` and `fields`\n  //                                         do not match, `CsvRecord` is empty\n  template <\n      typename Fields,\n      std::enable_if_t<IsIterableOf<Fields, absl::string_view>::value, int> = 0>\n  absl::Status TryReset(CsvHeader header, Fields&& fields);\n  absl::Status TryReset(CsvHeader header,\n                        std::initializer_list<absl::string_view> fields);\n\n  // Makes all field values empty. The number of fields is unchanged.\n  void Clear();\n\n  // Returns the sequence of field values, in the order corresponding to the\n  // order of field names in the header.\n  absl::Span<std::string> fields() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return absl::MakeSpan(fields_);\n  }\n  absl::Span<const std::string> fields() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return fields_;\n  }\n\n  // Iterates over pairs of field names and field values, in the order\n  // corresponding to the order of field names in the header.\n  iterator begin() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator begin() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator cbegin() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return begin();\n  }\n  iterator end() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator end() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator cend() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return end(); }\n\n  // Iterates over pairs of field names and field values, backwards.\n  reverse_iterator rbegin() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return reverse_iterator(end());\n  }\n  const_reverse_iterator rbegin() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return const_reverse_iterator(end());\n  }\n  const_reverse_iterator crbegin() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return rbegin();\n  }\n  reverse_iterator rend() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return reverse_iterator(begin());\n  }\n  const_reverse_iterator rend() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return const_reverse_iterator(begin());\n  }\n  const_reverse_iterator crend() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return rend();\n  }\n\n  // Returns `true` if there are no fields.\n  bool empty() const { return fields_.empty(); }\n\n  // Returns the number of field names, which is the same as the number of field\n  // values.\n  size_t size() const { return fields_.size(); }\n\n  // Returns a reference to the field value corresponding to the given field\n  // `name`.\n  //\n  // Precondition: `name` is present\n  std::string& operator[](absl::string_view name) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const std::string& operator[](absl::string_view name) const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns an iterator positioned at the pair of the given field `name` and\n  // the corresponding field value, or `end()` if `name` is not present.\n  iterator find(absl::string_view name) ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  const_iterator find(absl::string_view name) const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns `true` if `name` is present.\n  bool contains(absl::string_view name) const;\n\n  // Sets all fields resulting from iteration over another iterable of pairs of\n  // field names and field values, which can be an associative container or\n  // another `CsvRecord`.\n  //\n  // This can be used to convert a `CsvRecord` to a superset of fields, as long\n  // as fields to be preserved have the same names.\n  //\n  // Preconditions:\n  //  * all fields from `src` are present in `*this`\n  template <\n      typename Src,\n      std::enable_if_t<IsIterableOf<Src, std::pair<absl::string_view,\n                                                   absl::string_view>>::value,\n                       int> = 0>\n  void Merge(Src&& src);\n  void Merge(\n      std::initializer_list<std::pair<absl::string_view, absl::string_view>>\n          src);\n\n  // Sets all fields resulting from iteration over another iterable of pairs of\n  // field names and field values, which can be an associative container or\n  // another `CsvRecord`. Reports whether that was successful.\n  //\n  // This can be used to convert a `CsvRecord` to a different set of fields, as\n  // long as fields to be preserved have the same names.\n  //\n  // Return values:\n  //  * `absl::OkStatus()`                 - all fields from `src` have been set\n  //  * `absl::FailedPreconditionError(_)` - some fields were absent in `*this`,\n  //                                         only the intersection of fields\n  //                                         has been set\n  template <\n      typename Src,\n      std::enable_if_t<IsIterableOf<Src, std::pair<absl::string_view,\n                                                   absl::string_view>>::value,\n                       int> = 0>\n  absl::Status TryMerge(Src&& src);\n  absl::Status TryMerge(\n      std::initializer_list<std::pair<absl::string_view, absl::string_view>>\n          src);\n\n  // Assigns corresponding field values to all field values resulting from\n  // iteration over another iterable of pairs of field names and field values,\n  // which can be an associative container or another `CsvRecord`.\n  //\n  // This can be used to project a `CsvRecord` to a subset of fields, as long\n  // as fields to be preserved have the same names.\n  //\n  // Preconditions:\n  //  * all fields from `dest` are present in `*this`\n  template <typename Dest,\n            std::enable_if_t<IsIterableOfPairsWithAssignableValues<\n                                 Dest, absl::string_view, std::string>::value,\n                             int> = 0>\n  void Split(Dest& dest) const;\n\n  // Assigns corresponding field values to all field values resulting from\n  // iteration over another iterable of pairs of field names and field values,\n  // which can be an associative container or another `CsvRecord`. Reports\n  // whether that was successful.\n  //\n  // This can be used to project a `CsvRecord` to a subset of fields, as long\n  // as fields to be preserved have the same names.\n  //\n  // Return values:\n  //  * `absl::OkStatus()`                 - all fields in `dest` have been set\n  //  * `absl::FailedPreconditionError(_)` - some fields were absent in `*this`,\n  //                                         only the intersection of fields\n  //                                         has been set\n  template <typename Dest,\n            std::enable_if_t<IsIterableOfPairsWithAssignableValues<\n                                 Dest, absl::string_view, std::string>::value,\n                             int> = 0>\n  absl::Status TrySplit(Dest& dest) const;\n\n  friend bool operator==(const CsvRecord& a, const CsvRecord& b) {\n    return Equal(a, b);\n  }\n\n  // Renders contents in a human-readable way.\n  std::string DebugString() const;\n\n  // Default stringification by `absl::StrCat()` etc.\n  //\n  // Writes `src.DebugString()` to `dest`.\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const CsvRecord& src) {\n    StringifyWriter<Sink*> writer(&dest);\n    src.WriteDebugStringTo(writer);\n    writer.Close();\n  }\n\n  // Writes `src.DebugString()` to `dest`.\n  friend std::ostream& operator<<(std::ostream& dest, const CsvRecord& src) {\n    src.Output(dest);\n    return dest;\n  }\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const CsvRecord* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->header_);\n    memory_estimator.RegisterSubobjects(&self->fields_);\n  }\n\n private:\n  friend class CsvReaderBase;  // For `fields_`.\n\n  template <typename Fields>\n  absl::Status TryResetInternal(CsvHeader&& header, Fields&& fields);\n  absl::Status TryResetInternal(CsvHeader&& header,\n                                std::vector<std::string>&& fields);\n\n  absl::Status FailMissingNames(\n      absl::Span<const std::string> missing_names) const;\n\n  static bool Equal(const CsvRecord& a, const CsvRecord& b);\n\n  void WriteDebugStringTo(Writer& dest) const;\n  void Output(std::ostream& dest) const;\n\n  // Invariant: `header_.size() == fields_.size()`\n  CsvHeader header_;\n  std::vector<std::string> fields_;\n};\n\nnamespace csv_internal {\nvoid WriteDebugQuotedIfNeeded(absl::string_view src, Writer& dest);\n}  // namespace csv_internal\n\n// Implementation details follow.\n\ninline typename CsvHeader::iterator::reference CsvHeader::iterator::operator*()\n    const {\n  return *iter_;\n}\n\ninline typename CsvHeader::iterator::pointer CsvHeader::iterator::operator->()\n    const {\n  return &*iter_;\n}\n\ninline CsvHeader::iterator& CsvHeader::iterator::operator++() {\n  ++iter_;\n  return *this;\n}\n\ninline CsvHeader::iterator CsvHeader::iterator::operator++(int) {\n  const iterator tmp = *this;\n  ++*this;\n  return tmp;\n}\n\ninline CsvHeader::iterator& CsvHeader::iterator::operator--() {\n  --iter_;\n  return *this;\n}\n\ninline CsvHeader::iterator CsvHeader::iterator::operator--(int) {\n  const iterator tmp = *this;\n  --*this;\n  return tmp;\n}\n\ninline CsvHeader::iterator& CsvHeader::iterator::operator+=(difference_type n) {\n  iter_ += n;\n  return *this;\n}\n\ninline CsvHeader::iterator CsvHeader::iterator::operator+(\n    difference_type n) const {\n  return iterator(*this) += n;\n}\n\ninline CsvHeader::iterator& CsvHeader::iterator::operator-=(difference_type n) {\n  iter_ -= n;\n  return *this;\n}\n\ninline CsvHeader::iterator CsvHeader::iterator::operator-(\n    difference_type n) const {\n  return iterator(*this) -= n;\n}\n\ninline typename CsvHeader::iterator::reference CsvHeader::iterator::operator[](\n    difference_type n) const {\n  return *(*this + n);\n}\n\ntemplate <\n    typename Names,\n    std::enable_if_t<std::conjunction_v<NotSameRef<CsvHeader, Names>,\n                                        IsIterableOf<Names, absl::string_view>>,\n                     int>>\nCsvHeader::CsvHeader(Names&& names) {\n  const absl::Status status =\n      TryResetInternal(nullptr, std::forward<Names>(names));\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvHeader::CsvHeader()\";\n}\n\ntemplate <typename Names,\n          std::enable_if_t<IsIterableOf<Names, absl::string_view>::value, int>>\nCsvHeader::CsvHeader(std::function<std::string(absl::string_view)> normalizer,\n                     Names&& names) {\n  const absl::Status status =\n      TryResetInternal(std::move(normalizer), std::forward<Names>(names));\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvHeader::CsvHeader()\";\n}\n\ntemplate <\n    typename Names,\n    std::enable_if_t<std::conjunction_v<NotSameRef<CsvHeader, Names>,\n                                        IsIterableOf<Names, absl::string_view>>,\n                     int>>\nvoid CsvHeader::Reset(Names&& names) {\n  const absl::Status status =\n      TryResetInternal(nullptr, std::forward<Names>(names));\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvHeader::Reset()\";\n}\n\ntemplate <typename Names,\n          std::enable_if_t<IsIterableOf<Names, absl::string_view>::value, int>>\nvoid CsvHeader::Reset(std::function<std::string(absl::string_view)> normalizer,\n                      Names&& names) {\n  const absl::Status status =\n      TryResetInternal(std::move(normalizer), std::forward<Names>(names));\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvHeader::Reset()\";\n}\n\ntemplate <\n    typename Names,\n    std::enable_if_t<std::conjunction_v<NotSameRef<CsvHeader, Names>,\n                                        IsIterableOf<Names, absl::string_view>>,\n                     int>>\nabsl::Status CsvHeader::TryReset(Names&& names) {\n  return TryResetInternal(nullptr, std::forward<Names>(names));\n}\n\ntemplate <typename Names,\n          std::enable_if_t<IsIterableOf<Names, absl::string_view>::value, int>>\nabsl::Status CsvHeader::TryReset(\n    std::function<std::string(absl::string_view)> normalizer, Names&& names) {\n  return TryResetInternal(std::move(normalizer), std::forward<Names>(names));\n}\n\ntemplate <typename Names>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline absl::Status CsvHeader::TryResetInternal(\n    std::function<std::string(absl::string_view)>&& normalizer, Names&& names) {\n  if constexpr (IsRandomAccessIterable<Names>::value) {\n    // Iterable supports random access, which allows `std::equal()` in\n    // `MaybeResetToCachedPayload()` to compare the size first, which makes it\n    // efficient to call `MaybeResetToCachedPayload()` before converting `names`\n    // to `std::vector<std::string>`.\n    if (normalizer == nullptr && MaybeResetToCachedPayload(names)) {\n      return absl::OkStatus();\n    }\n    return TryResetUncached(\n        std::move(normalizer),\n        csv_internal::ToStringVector(std::forward<Names>(names)));\n  } else {\n    return TryResetInternal(\n        std::move(normalizer),\n        csv_internal::ToStringVector(std::forward<Names>(names)));\n  }\n}\n\nABSL_ATTRIBUTE_ALWAYS_INLINE\ninline absl::Status CsvHeader::TryResetInternal(\n    std::function<std::string(absl::string_view)>&& normalizer,\n    std::vector<std::string>&& names) {\n  if (normalizer == nullptr && MaybeResetToCachedPayload(names)) {\n    return absl::OkStatus();\n  }\n  return TryResetUncached(std::move(normalizer), std::move(names));\n}\n\ntemplate <typename Names>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool CsvHeader::MaybeResetToCachedPayload(\n    Names&& names) {\n  using std::begin;\n  auto names_iter = begin(names);\n  using std::end;\n  auto names_end_iter = end(names);\n  if (names_iter == names_end_iter) {\n    payload_.Reset();\n    return true;\n  }\n  SharedPtr<Payload> payload;\n  {\n    absl::MutexLock lock(payload_cache_mutex_);\n    payload = payload_cache_;\n  }\n  if (payload != nullptr &&\n      std::equal(payload->index_to_name.begin(), payload->index_to_name.end(),\n                 names_iter, names_end_iter)) {\n    payload_ = std::move(payload);\n    return true;\n  }\n  return false;\n}\n\ntemplate <typename... Names, std::enable_if_t<(sizeof...(Names) > 0), int>>\ninline void CsvHeader::Add(StringInitializer name, Names&&... names) {\n  Add(std::move(name));\n  Add(std::forward<Names>(names)...);\n}\n\ntemplate <typename... Names, std::enable_if_t<(sizeof...(Names) > 0), int>>\ninline absl::Status CsvHeader::TryAdd(StringInitializer name,\n                                      Names&&... names) {\n  if (absl::Status status = TryAdd(std::move(name)); !status.ok()) {\n    return status;\n  }\n  return TryAdd(std::forward<Names>(names)...);\n}\n\ninline absl::Span<const std::string> CsvHeader::names() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  if (ABSL_PREDICT_FALSE(payload_ == nullptr)) return {};\n  return payload_->index_to_name;\n}\n\ninline const std::function<std::string(absl::string_view)>&\nCsvHeader::normalizer() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  if (ABSL_PREDICT_FALSE(payload_ == nullptr)) {\n    return Global<std::function<std::string(absl::string_view)>>();\n  }\n  return payload_->normalizer;\n}\n\ninline CsvHeader::iterator CsvHeader::begin() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  if (ABSL_PREDICT_FALSE(payload_ == nullptr)) return iterator();\n  return iterator(payload_->index_to_name.cbegin());\n}\n\ninline CsvHeader::iterator CsvHeader::end() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  if (ABSL_PREDICT_FALSE(payload_ == nullptr)) return iterator();\n  return iterator(payload_->index_to_name.cend());\n}\n\ninline bool CsvHeader::empty() const {\n  return payload_ == nullptr || payload_->index_to_name.empty();\n}\n\ninline size_t CsvHeader::size() const {\n  if (ABSL_PREDICT_FALSE(payload_ == nullptr)) return 0;\n  return payload_->index_to_name.size();\n}\n\ntemplate <size_t num_fields>\ninline const CsvHeader* CsvHeaderConstant<num_fields>::get() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  absl::call_once(once_,\n                  [&] { new (header_) CsvHeader(normalizer_, fields_); });\n  return std::launder(reinterpret_cast<const CsvHeader*>(header_));\n}\n\ntemplate <typename FieldIterator>\ninline CsvRecord::IteratorImpl<FieldIterator>::IteratorImpl(\n    CsvHeader::iterator name_iter, FieldIterator field_iter)\n    : name_iter_(name_iter), field_iter_(field_iter) {}\n\ntemplate <typename FieldIterator>\ntemplate <typename ThatFieldIterator,\n          std::enable_if_t<\n              std::conjunction_v<\n                  std::negation<std::is_same<ThatFieldIterator, FieldIterator>>,\n                  std::is_convertible<ThatFieldIterator, FieldIterator>>,\n              int>>\ninline CsvRecord::IteratorImpl<FieldIterator>::IteratorImpl(\n    IteratorImpl<ThatFieldIterator> that) noexcept\n    : name_iter_(that.name_iter_), field_iter_(that.field_iter_) {}\n\ntemplate <typename FieldIterator>\ninline typename CsvRecord::IteratorImpl<FieldIterator>::reference\nCsvRecord::IteratorImpl<FieldIterator>::operator*() const {\n  return reference(*name_iter_, *field_iter_);\n}\n\ntemplate <typename FieldIterator>\ninline typename CsvRecord::IteratorImpl<FieldIterator>::pointer\nCsvRecord::IteratorImpl<FieldIterator>::operator->() const {\n  return pointer(**this);\n}\n\ntemplate <typename FieldIterator>\ninline CsvRecord::IteratorImpl<FieldIterator>&\nCsvRecord::IteratorImpl<FieldIterator>::operator++() {\n  ++name_iter_;\n  ++field_iter_;\n  return *this;\n}\n\ntemplate <typename FieldIterator>\ninline CsvRecord::IteratorImpl<FieldIterator>\nCsvRecord::IteratorImpl<FieldIterator>::operator++(int) {\n  const IteratorImpl<FieldIterator> tmp = *this;\n  ++*this;\n  return tmp;\n}\n\ntemplate <typename FieldIterator>\ninline CsvRecord::IteratorImpl<FieldIterator>&\nCsvRecord::IteratorImpl<FieldIterator>::operator--() {\n  --name_iter_;\n  --field_iter_;\n  return *this;\n}\n\ntemplate <typename FieldIterator>\ninline CsvRecord::IteratorImpl<FieldIterator>\nCsvRecord::IteratorImpl<FieldIterator>::operator--(int) {\n  const IteratorImpl<FieldIterator> tmp = *this;\n  --*this;\n  return tmp;\n}\n\ntemplate <typename FieldIterator>\ninline CsvRecord::IteratorImpl<FieldIterator>&\nCsvRecord::IteratorImpl<FieldIterator>::operator+=(difference_type n) {\n  name_iter_ += n;\n  field_iter_ += n;\n  return *this;\n}\n\ntemplate <typename FieldIterator>\ninline CsvRecord::IteratorImpl<FieldIterator>\nCsvRecord::IteratorImpl<FieldIterator>::operator+(difference_type n) const {\n  return IteratorImpl<FieldIterator>(*this) += n;\n}\n\ntemplate <typename FieldIterator>\ninline CsvRecord::IteratorImpl<FieldIterator>&\nCsvRecord::IteratorImpl<FieldIterator>::operator-=(difference_type n) {\n  name_iter_ -= n;\n  field_iter_ -= n;\n  return *this;\n}\n\ntemplate <typename FieldIterator>\ninline CsvRecord::IteratorImpl<FieldIterator>\nCsvRecord::IteratorImpl<FieldIterator>::operator-(difference_type n) const {\n  return IteratorImpl<FieldIterator>(*this) -= n;\n}\n\ntemplate <typename FieldIterator>\ninline typename CsvRecord::IteratorImpl<FieldIterator>::reference\nCsvRecord::IteratorImpl<FieldIterator>::operator[](difference_type n) const {\n  return *(*this + n);\n}\n\ninline CsvRecord::CsvRecord(CsvHeader header)\n    : header_(std::move(header)), fields_(header_.size()) {}\n\ntemplate <typename Fields,\n          std::enable_if_t<IsIterableOf<Fields, absl::string_view>::value, int>>\nCsvRecord::CsvRecord(CsvHeader header, Fields&& fields) {\n  const absl::Status status =\n      TryResetInternal(std::move(header), std::forward<Fields>(fields));\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvRecord::CsvRecord()\";\n}\n\ntemplate <typename Fields,\n          std::enable_if_t<IsIterableOf<Fields, absl::string_view>::value, int>>\nvoid CsvRecord::Reset(CsvHeader header, Fields&& fields) {\n  const absl::Status status =\n      TryResetInternal(std::move(header), std::forward<Fields>(fields));\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvRecord::Reset()\";\n}\n\ntemplate <typename Fields,\n          std::enable_if_t<IsIterableOf<Fields, absl::string_view>::value, int>>\nabsl::Status CsvRecord::TryReset(CsvHeader header, Fields&& fields) {\n  return TryResetInternal(std::move(header), std::forward<Fields>(fields));\n}\n\ntemplate <typename Fields>\ninline absl::Status CsvRecord::TryResetInternal(CsvHeader&& header,\n                                                Fields&& fields) {\n  return TryResetInternal(std::move(header), csv_internal::ToStringVector(\n                                                 std::forward<Fields>(fields)));\n}\n\ninline CsvRecord::iterator CsvRecord::begin() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return iterator(header_.begin(), fields_.begin());\n}\n\ninline CsvRecord::const_iterator CsvRecord::begin() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return const_iterator(header_.begin(), fields_.begin());\n}\n\ninline CsvRecord::iterator CsvRecord::end() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return iterator(header_.end(), fields_.end());\n}\n\ninline CsvRecord::const_iterator CsvRecord::end() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  return const_iterator(header_.end(), fields_.end());\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<IsIterableOf<Src, std::pair<absl::string_view,\n                                                 absl::string_view>>::value,\n                     int>>\nvoid CsvRecord::Merge(Src&& src) {\n  const absl::Status status = TryMerge(std::forward<Src>(src));\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvHeader::Merge()\";\n}\n\ntemplate <\n    typename Src,\n    std::enable_if_t<IsIterableOf<Src, std::pair<absl::string_view,\n                                                 absl::string_view>>::value,\n                     int>>\nabsl::Status CsvRecord::TryMerge(Src&& src) {\n  using std::begin;\n  auto src_iter = begin(src);\n  using std::end;\n  auto src_end_iter = end(src);\n  iterator this_iter = this->begin();\n  // If fields of `src` match a prefix of fields of `*this` (like when extending\n  // a `CsvRecord` with fields added at the end), avoid string lookups and just\n  // verify the assumption.\n  for (;;) {\n    if (src_iter == src_end_iter) return absl::OkStatus();\n    if (this_iter == this->end() || this_iter->first != src_iter->first) break;\n    riegeli::Reset(\n        this_iter->second,\n        StringInitializer((*MaybeMakeMoveIterator<Src>(src_iter)).second));\n    ++this_iter;\n    ++src_iter;\n  }\n  RIEGELI_ASSERT(src_iter != src_end_iter)\n      << \"The code below assumes that the code above \"\n         \"did not leave the source iterator at the end\";\n  // The assumption about matching fields no longer holds. Switch to string\n  // lookups for the remaining fields.\n  std::vector<std::string> missing_names;\n  do {\n    this_iter = find(src_iter->first);\n    if (ABSL_PREDICT_FALSE(this_iter == this->end())) {\n      missing_names.emplace_back(src_iter->first);\n    } else {\n      riegeli::Reset(\n          this_iter->second,\n          StringInitializer((*MaybeMakeMoveIterator<Src>(src_iter)).second));\n    }\n    ++src_iter;\n  } while (src_iter != src_end_iter);\n  if (ABSL_PREDICT_FALSE(!missing_names.empty())) {\n    return FailMissingNames(missing_names);\n  }\n  return absl::OkStatus();\n}\n\ntemplate <typename Dest,\n          std::enable_if_t<IsIterableOfPairsWithAssignableValues<\n                               Dest, absl::string_view, std::string>::value,\n                           int>>\nvoid CsvRecord::Split(Dest& dest) const {\n  const absl::Status status = TrySplit(dest);\n  RIEGELI_CHECK_OK(status) << \"Failed precondition of CsvHeader::Split()\";\n}\n\ntemplate <typename Dest,\n          std::enable_if_t<IsIterableOfPairsWithAssignableValues<\n                               Dest, absl::string_view, std::string>::value,\n                           int>>\nabsl::Status CsvRecord::TrySplit(Dest& dest) const {\n  using std::begin;\n  auto dest_iter = begin(dest);\n  using std::end;\n  auto dest_end_iter = end(dest);\n  const_iterator this_iter = this->begin();\n  // If fields of `dest` match a prefix of fields of `*this` (like when\n  // projecting a `CsvRecord` to its prefix), avoid string lookups and just\n  // verify the assumption.\n  for (;;) {\n    if (dest_iter == dest_end_iter) return absl::OkStatus();\n    if (this_iter == this->end() || this_iter->first != dest_iter->first) break;\n    dest_iter->second = this_iter->second;\n    ++this_iter;\n    ++dest_iter;\n  }\n  RIEGELI_ASSERT(dest_iter != dest_end_iter)\n      << \"The code below assumes that the code above \"\n         \"did not leave the destination iterator at the end\";\n  // The assumption about matching fields no longer holds. Switch to string\n  // lookups for the remaining fields.\n  std::vector<std::string> missing_names;\n  do {\n    this_iter = find(dest_iter->first);\n    if (ABSL_PREDICT_FALSE(this_iter == this->end())) {\n      missing_names.emplace_back(dest_iter->first);\n    } else {\n      dest_iter->second = this_iter->second;\n    }\n    ++dest_iter;\n  } while (dest_iter != dest_end_iter);\n  if (ABSL_PREDICT_FALSE(!missing_names.empty())) {\n    return FailMissingNames(missing_names);\n  }\n  return absl::OkStatus();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CSV_CSV_RECORD_H_\n"
  },
  {
    "path": "riegeli/csv/csv_writer.cc",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/csv/csv_writer.h\"\n\n#include <stddef.h>\n\n#include <array>\n#include <cstring>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/debug.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/csv/csv_record.h\"\n#include \"riegeli/lines/line_writing.h\"\n#include \"riegeli/lines/newline.h\"\n\nnamespace riegeli {\n\nvoid CsvWriterBase::Initialize(Writer* dest, Options&& options) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of CsvWriter: null Writer pointer\";\n  // Set `header_` early, so that `header()` is valid even in case of a failure,\n  // and because `WriteRecord(const CsvRecord&)` uses this as a precondition.\n  bool write_header = false;\n  if (options.header() != std::nullopt) {\n    RIEGELI_ASSERT(options.assumed_header() == std::nullopt)\n        << \"Failed precondition of CsvWriter: \"\n           \"header() and assumed_header() both set\";\n    has_header_ = true;\n    write_header = true;\n    header_ = *std::move(options.header());\n  } else if (options.assumed_header() != std::nullopt) {\n    has_header_ = true;\n    header_ = *std::move(options.assumed_header());\n  }\n\n  if (options.comment() != std::nullopt &&\n      ABSL_PREDICT_FALSE(*options.comment() == '\\n' ||\n                         *options.comment() == '\\r')) {\n    Fail(absl::InvalidArgumentError(\n        absl::StrCat(\"Comment character conflicts with record separator: \",\n                     riegeli::Debug(*options.comment()))));\n    return;\n  }\n  if (ABSL_PREDICT_FALSE(options.field_separator() == '\\n' ||\n                         options.field_separator() == '\\r')) {\n    Fail(absl::InvalidArgumentError(\n        absl::StrCat(\"Field separator conflicts with record separator: \",\n                     riegeli::Debug(options.field_separator()))));\n    return;\n  }\n  if (ABSL_PREDICT_FALSE(options.field_separator() == options.comment())) {\n    Fail(absl::InvalidArgumentError(\n        absl::StrCat(\"Field separator conflicts with comment character: \",\n                     riegeli::Debug(options.field_separator()))));\n    return;\n  }\n  if (options.quote() != std::nullopt) {\n    if (ABSL_PREDICT_FALSE(*options.quote() == '\\n' ||\n                           *options.quote() == '\\r')) {\n      Fail(absl::InvalidArgumentError(\n          absl::StrCat(\"Quote character conflicts with record separator: \",\n                       riegeli::Debug(*options.quote()))));\n      return;\n    }\n    if (ABSL_PREDICT_FALSE(*options.quote() == options.comment())) {\n      Fail(absl::InvalidArgumentError(\n          absl::StrCat(\"Quote character conflicts with comment character: \",\n                       riegeli::Debug(*options.quote()))));\n      return;\n    }\n    if (ABSL_PREDICT_FALSE(*options.quote() == options.field_separator())) {\n      Fail(absl::InvalidArgumentError(\n          absl::StrCat(\"Quote character conflicts with field separator: \",\n                       riegeli::Debug(*options.quote()))));\n      return;\n    }\n  }\n\n  quotes_needed_['\\n'] = true;\n  quotes_needed_['\\r'] = true;\n  if (options.comment() != std::nullopt) {\n    quotes_needed_[static_cast<unsigned char>(*options.comment())] = true;\n  }\n  quotes_needed_[static_cast<unsigned char>(options.field_separator())] = true;\n  if (options.quote() != std::nullopt) {\n    quotes_needed_[static_cast<unsigned char>(*options.quote())] = true;\n  }\n  newline_ = options.newline();\n  field_separator_ = options.field_separator();\n  quote_ = options.quote();\n\n  if (ABSL_PREDICT_FALSE(!dest->ok())) {\n    FailWithoutAnnotation(AnnotateOverDest(dest->status()));\n    return;\n  }\n  if (options.write_utf8_bom()) WriteUtf8Bom(*dest);\n\n  if (write_header) {\n    if (ABSL_PREDICT_TRUE(WriteRecord(header_.names()))) --record_index_;\n  }\n}\n\nabsl::Status CsvWriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    status = dest.AnnotateStatus(std::move(status));\n  }\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status CsvWriterBase::AnnotateOverDest(absl::Status status) {\n  if (!standalone_record_) {\n    return Annotate(status, absl::StrCat(\"at record \", record_index()));\n  }\n  return status;\n}\n\ninline bool CsvWriterBase::WriteQuoted(Writer& dest, absl::string_view field,\n                                       size_t already_scanned) {\n  RIEGELI_ASSERT_NE(quote_, std::nullopt)\n      << \"Failed precondition of CsvWriterBase::WriteQuoted(): \"\n         \"quote character not available\";\n  if (ABSL_PREDICT_FALSE(!dest.Write(*quote_))) {\n    return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n  const char* start = field.data();\n  const char* next_to_check = field.data() + already_scanned;\n  const char* const limit = field.data() + field.size();\n  // Write characters in the range [`start`..`limit`), except that if quotes are\n  // found in the range [`next_to_check`..`limit`), write them twice.\n  while (const char* const next_quote = static_cast<const char*>(std::memchr(\n             next_to_check, *quote_, PtrDistance(next_to_check, limit)))) {\n    if (ABSL_PREDICT_FALSE(!dest.Write(\n            absl::string_view(start, PtrDistance(start, next_quote + 1))))) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    }\n    start = next_quote;\n    next_to_check = next_quote + 1;\n  }\n  if (ABSL_PREDICT_FALSE(\n          !dest.Write(absl::string_view(start, PtrDistance(start, limit))))) {\n    return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n  if (ABSL_PREDICT_FALSE(!dest.Write(*quote_))) {\n    return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n  return true;\n}\n\nbool CsvWriterBase::WriteQuotes(Writer& dest) {\n  if (quote_ == std::nullopt) return true;\n  if (ABSL_PREDICT_FALSE(!dest.Write(*quote_) || !dest.Write(*quote_))) {\n    return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n  return true;\n}\n\nbool CsvWriterBase::WriteFirstField(Writer& dest, absl::string_view field) {\n  // Quote the first field if the field together with the field separator could\n  // make the line starting with UTF-8 BOM.\n  //\n  // For simplicity other fields are not considered, at the cost of unnecessary\n  // quoting in corner cases.\n  if (ABSL_PREDICT_FALSE(\n          field.empty() ? field_separator_ == kUtf8Bom[0]\n                        : field[0] == kUtf8Bom[0] &&\n                              (field.size() == 1\n                                   ? field_separator_ == kUtf8Bom[1]\n                                   : field[1] == kUtf8Bom[1] &&\n                                         (field.size() == 2\n                                              ? field_separator_ == kUtf8Bom[2]\n                                              : field[2] == kUtf8Bom[2]))) &&\n      quote_ != std::nullopt) {\n    return WriteQuoted(dest, field, 0);\n  }\n  return WriteField(dest, field);\n}\n\nbool CsvWriterBase::WriteField(Writer& dest, absl::string_view field) {\n  for (size_t i = 0; i < field.size(); ++i) {\n    if (quotes_needed_[static_cast<unsigned char>(field[i])]) {\n      if (ABSL_PREDICT_FALSE(quote_ == std::nullopt)) {\n        return Fail(absl::InvalidArgumentError(\n            absl::StrCat(\"If quoting is turned off, special characters inside \"\n                         \"fields are not \"\n                         \"expressible: \",\n                         riegeli::Debug(field[i]))));\n      }\n      return WriteQuoted(dest, field, i);\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!dest.Write(field))) {\n    return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n  return true;\n}\n\nbool CsvWriterBase::WriteRecord(const CsvRecord& record) {\n  RIEGELI_CHECK(has_header_)\n      << \"Failed precondition of CsvWriterBase::WriteRecord(CsvRecord): \"\n         \"CsvWriterBase::Options::header() is required\";\n  if (ok()) {\n    RIEGELI_CHECK_EQ(record.header(), header_)\n        << \"Failed precondition of CsvWriterBase::WriteRecord(CsvRecord): \"\n        << \"mismatched CSV header and record\";\n  }\n  return WriteRecord(record.fields());\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/csv/csv_writer.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_CSV_CSV_WRITER_H_\n#define RIEGELI_CSV_CSV_WRITER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <array>\n#include <initializer_list>\n#include <limits>\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/iterable.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/bytes/string_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/csv/csv_record.h\"\n#include \"riegeli/lines/line_writing.h\"\n#include \"riegeli/lines/newline.h\"\n\nnamespace riegeli {\n\nclass CsvWriterBase;\n\nnamespace csv_internal {\n\ntemplate <typename Fields>\nbool WriteStandaloneRecord(const Fields& record, CsvWriterBase& csv_writer);\n\n}  // namespace csv_internal\n\n// Template parameter independent part of `CsvWriter`.\nclass CsvWriterBase : public Object {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // If not `std::nullopt`, sets field names, and automatically writes them\n    // as the first record.\n    //\n    // In this case `WriteRecord(CsvRecord)` is supported. Otherwise no\n    // particular header is assumed, and only `WriteRecord()` from a sequence of\n    // fields is supported.\n    //\n    // The CSV format does not support empty records. A header with no fields\n    // will be written as an empty line, which will be read as a header\n    // consisting of one empty field, or will be skipped if\n    // `CsvReaderBase::Options::skip_empty_lines()`.\n    //\n    // Default: `std::nullopt`.\n    Options& set_header(Initializer<std::optional<CsvHeader>> header) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      riegeli::Reset(header_, std::move(header));\n      return *this;\n    }\n    Options&& set_header(Initializer<std::optional<CsvHeader>> header) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_header(std::move(header)));\n    }\n    Options& set_header(std::initializer_list<absl::string_view> names) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_header(Initializer<std::optional<CsvHeader>>(names));\n    }\n    Options&& set_header(std::initializer_list<absl::string_view> names) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_header(names));\n    }\n    std::optional<CsvHeader>& header() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return header_;\n    }\n    const std::optional<CsvHeader>& header() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return header_;\n    }\n\n    // If not `std::nullopt`, a header is not written to the file, but\n    // `WriteRecord(CsvRecord&)` is supported as if this header was written as\n    // the first record.\n    //\n    // `header()` and `assumed_header()` must not be both set.\n    //\n    // Default: `std::nullopt`.\n    Options& set_assumed_header(Initializer<std::optional<CsvHeader>> header) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      riegeli::Reset(assumed_header_, std::move(header));\n      return *this;\n    }\n    Options&& set_assumed_header(\n        Initializer<std::optional<CsvHeader>> header) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_header(std::move(header)));\n    }\n    Options& set_assumed_header(\n        std::initializer_list<absl::string_view> names) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return set_assumed_header(Initializer<std::optional<CsvHeader>>(names));\n    }\n    Options&& set_assumed_header(\n        std::initializer_list<absl::string_view> names) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_header(names));\n    }\n    std::optional<CsvHeader>& assumed_header() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return assumed_header_;\n    }\n    const std::optional<CsvHeader>& assumed_header() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return assumed_header_;\n    }\n\n    // If `false`, does not write initial UTF-8 BOM. This conforms to RFC4180\n    // and UTF-8 BOM is normally not used on Unix.\n    //\n    // If `true`, writes initial UTF-8 BOM. Microsoft Excel by default expects\n    // UTF-8 BOM in order to recognize UTF-8 contents without prompting for the\n    // encoding.\n    //\n    // By default `CsvReader` will understand files written with any option.\n    //\n    // Default: `false`.\n    Options& set_write_utf8_bom(bool write_utf8_bom) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      write_utf8_bom_ = write_utf8_bom;\n      return *this;\n    }\n    Options&& set_write_utf8_bom(bool write_utf8_bom) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_write_utf8_bom(write_utf8_bom));\n    }\n    bool write_utf8_bom() const { return write_utf8_bom_; }\n\n    // Record terminator.\n    //\n    // RFC4180 requires `WriteNewline::kCrLf` but Unix normally uses `kLf`.\n    // `CsvReader` will understand files written with any option.\n    //\n    // Default: `WriteNewline::kNative`.\n    Options& set_newline(WriteNewline newline) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      newline_ = newline;\n      return *this;\n    }\n    Options&& set_newline(WriteNewline newline) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_newline(newline));\n    }\n    WriteNewline newline() const { return newline_; }\n\n    // Comment character.\n    //\n    // If not `std::nullopt`, fields containing this character will be quoted.\n    // This is not covered by RFC4180.\n    //\n    // Often used: '#'.\n    //\n    // Default: `std::nullopt`.\n    Options& set_comment(std::optional<char> comment) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      comment_ = comment;\n      return *this;\n    }\n    Options&& set_comment(std::optional<char> comment) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_comment(comment));\n    }\n    std::optional<char> comment() const { return comment_; }\n\n    // Field separator.\n    //\n    // Default: ','.\n    Options& set_field_separator(char field_separator) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      field_separator_ = field_separator;\n      return *this;\n    }\n    Options&& set_field_separator(char field_separator) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_field_separator(field_separator));\n    }\n    char field_separator() const { return field_separator_; }\n\n    // Quote character.\n    //\n    // Quotes around a field allow expressing special characters inside the\n    // field: LF, CR, comment character, field separator, or quote character\n    // itself.\n    //\n    // To express a quote inside a quoted field, it must be written twice.\n    //\n    // Quotes are also used for unambiguous interpretation of a record\n    // consisting of a single empty field or beginning with UTF-8 BOM.\n    //\n    // If `std::nullopt`, special characters inside fields are not expressible,\n    // and `CsvWriter` fails if they are encountered, except that potential\n    // ambiguities above skip quoting instead. In this case, reading a record\n    // consisting of a single empty field is incompatible with\n    // `CsvReaderBase::Options::skip_empty_lines()`, and reading the first\n    // record beginning with UTF-8 BOM requires\n    // `CsvReaderBase::Options::set_preserve_utf8_bom()`.\n    //\n    // Default: '\"'.\n    Options& set_quote(std::optional<char> quote) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      quote_ = quote;\n      return *this;\n    }\n    Options&& set_quote(std::optional<char> quote) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_quote(quote));\n    }\n    std::optional<char> quote() const { return quote_; }\n\n   private:\n    std::optional<CsvHeader> header_;\n    std::optional<CsvHeader> assumed_header_;\n    bool write_utf8_bom_ = false;\n    WriteNewline newline_ = WriteNewline::kNative;\n    std::optional<char> comment_;\n    char field_separator_ = ',';\n    std::optional<char> quote_ = '\"';\n  };\n\n  // Returns the byte `Writer` being written to. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns `true` if writing the header was requested, i.e. if\n  // `Options::header() != std::nullopt`.\n  //\n  // In this case `WriteRecord(CsvRecord)` is supported. Otherwise no particular\n  // header is assumed, and only `WriteRecord()` from a sequence of fields is\n  // supported.\n  bool has_header() const { return has_header_; }\n\n  // If `has_header()`, returns field names set by `Options::header()` and\n  // written to the first record.\n  //\n  // If `!has_header()`, returns an empty header.\n  const CsvHeader& header() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return header_;\n  }\n\n  // Writes the next record expressed as `CsvRecord`, with named fields.\n  //\n  // The CSV format does not support empty records. A record with no fields will\n  // be written as an empty line, which will be read as a record consisting of\n  // one empty field, or will be skipped if\n  // `CsvReaderBase::Options::skip_empty_lines()`.\n  //\n  // Preconditions:\n  //  * `has_header()`, i.e. `Options::header() != std::nullopt`\n  //  * `record.header() == header()`\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool WriteRecord(const CsvRecord& record);\n\n  // Writes the next record expressed as a sequence of fields.\n  //\n  // The type of `record` must support iteration yielding `absl::string_view`:\n  // `for (const absl::string_view field : record)`,\n  // e.g. `std::vector<std::string>`.\n  //\n  // By a common convention each record should consist of the same number of\n  // fields, but this is not enforced.\n  //\n  // The CSV format does not support empty records. A record with no fields will\n  // be written as an empty line, which will be read as a record consisting of\n  // one empty field, or will be skipped if\n  // `CsvReaderBase::Options::skip_empty_lines()`.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  template <\n      typename Record,\n      std::enable_if_t<IsIterableOf<Record, absl::string_view>::value, int> = 0>\n  bool WriteRecord(const Record& record);\n  bool WriteRecord(std::initializer_list<absl::string_view> record);\n\n  // The index of the most recently written record, starting from 0.\n  //\n  // The record count does not include any header written with\n  // `Options::header()`.\n  //\n  // `last_record_index()` is unchanged by `Close()`.\n  //\n  // Precondition: some record was successfully written (`record_index() > 0`).\n  uint64_t last_record_index() const;\n\n  // The index of the next record, starting from 0.\n  //\n  // The record count does not include any header written with\n  // `Options::header()`.\n  //\n  // `record_index()` is unchanged by `Close()`.\n  uint64_t record_index() const { return record_index_; }\n\n protected:\n  using Object::Object;\n\n  CsvWriterBase(CsvWriterBase&& that) noexcept;\n  CsvWriterBase& operator=(CsvWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(Writer* dest, Options&& options);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n\n private:\n  template <typename Record>\n  friend bool csv_internal::WriteStandaloneRecord(const Record& record,\n                                                  CsvWriterBase& csv_writer);\n\n  bool WriteQuoted(Writer& dest, absl::string_view field,\n                   size_t already_scanned);\n  bool WriteQuotes(Writer& dest);\n  bool WriteFirstField(Writer& dest, absl::string_view field);\n  bool WriteField(Writer& dest, absl::string_view field);\n  template <typename Record>\n  bool WriteRecordInternal(const Record& record);\n\n  bool standalone_record_ = false;\n  bool has_header_ = false;\n  CsvHeader header_;\n  // Lookup table for checking whether quotes are needed if the given character\n  // is present in a field.\n  //\n  // Using `std::bitset` instead would make `CsvWriter` about 20% slower because\n  // of a more complicated lookup code.\n  std::array<bool, std::numeric_limits<unsigned char>::max() + 1>\n      quotes_needed_{};\n  WriteNewline newline_ = WriteNewline::kNative;\n  char field_separator_ = '\\0';\n  std::optional<char> quote_;\n  uint64_t record_index_ = 0;\n};\n\n// `CsvWriter` writes records to a CSV (comma-separated values) file.\n//\n// A basic variant of CSV is specified in https://tools.ietf.org/html/rfc4180,\n// and some common extensions are described in\n// https://specs.frictionlessdata.io/csv-dialect/.\n//\n// `CsvWriter` writes RFC4180-compliant CSV files if\n// `Options::newline() == WriteNewline::kCrLf`, and also supports some\n// extensions.\n//\n// By a common convention the first record consists of field names. This is\n// supported by `Options::header()` and `WriteRecord(CsvRecord)`.\n//\n// A record is terminated by a newline: LF or CR-LF (\"\\n\" or \"\\r\\n\").\n//\n// A record consists of a sequence of fields separated by a field separator\n// (usually ',' or '\\t'). Each record contains at least one field.\n//\n// Quotes (usually '\"') around a field allow expressing special characters\n// inside the field: LF, CR, comment character, field separator, or quote\n// character itself.\n//\n// To express a quote inside a quoted field, it must be written twice.\n//\n// Quotes are also used for unambiguous interpretation of a record consisting of\n// a single empty field or beginning with UTF-8 BOM.\n//\n// If quoting is turned off, special characters inside fields are not\n// expressible, and `CsvWriter` fails if they are encountered, except that\n// potential ambiguities above skip quoting instead. In this case, reading\n// a record consisting of a single empty field is incompatible with\n// `CsvReaderBase::Options::skip_empty_lines()`, and reading the first record\n// beginning with UTF-8 BOM requires\n// `CsvReaderBase::Options::set_preserve_utf8_bom()`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the byte `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The current position is synchronized with the byte `Writer` between records.\ntemplate <typename Dest = Writer*>\nclass CsvWriter : public CsvWriterBase {\n public:\n  // Creates a closed `CsvWriter`.\n  explicit CsvWriter(Closed) noexcept : CsvWriterBase(kClosed) {}\n\n  // Will write to the byte `Writer` provided by `dest`.\n  explicit CsvWriter(Initializer<Dest> dest, Options options = Options());\n\n  CsvWriter(CsvWriter&& that) = default;\n  CsvWriter& operator=(CsvWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `CsvWriter`. This avoids\n  // constructing a temporary `CsvWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the byte `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n\n private:\n  // The object providing and possibly owning the byte `Writer`.\n  Dependency<Writer*, Dest> dest_;\n};\n\nexplicit CsvWriter(Closed) -> CsvWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit CsvWriter(Dest&& dest,\n                   CsvWriterBase::Options options = CsvWriterBase::Options())\n    -> CsvWriter<TargetT<Dest>>;\n\n// Writes a single record to a CSV string.\n//\n// A record terminator will not be included.\n//\n// The type of `record` must support iteration yielding `absl::string_view`:\n// `for (const absl::string_view field : record)`,\n// e.g. `std::vector<std::string>`.\n//\n// Preconditions:\n//  * `options.header() == std::nullopt`\n//  * if `options.quote() == std::nullopt`, fields do not include inexpressible\n//    characters: LF, CR, comment character, field separator.\ntemplate <\n    typename Record,\n    std::enable_if_t<IsIterableOf<Record, absl::string_view>::value, int> = 0>\nstd::string WriteCsvRecordToString(\n    const Record& record,\n    CsvWriterBase::Options options = CsvWriterBase::Options());\nstd::string WriteCsvRecordToString(\n    std::initializer_list<absl::string_view> record,\n    CsvWriterBase::Options options = CsvWriterBase::Options());\n\n// Implementation details follow.\n\ninline CsvWriterBase::CsvWriterBase(CsvWriterBase&& that) noexcept\n    : Object(static_cast<Object&&>(that)),\n      standalone_record_(that.standalone_record_),\n      has_header_(that.has_header_),\n      header_(std::move(that.header_)),\n      quotes_needed_(that.quotes_needed_),\n      newline_(that.newline_),\n      field_separator_(that.field_separator_),\n      quote_(that.quote_),\n      record_index_(std::exchange(that.record_index_, 0)) {}\n\ninline CsvWriterBase& CsvWriterBase::operator=(CsvWriterBase&& that) noexcept {\n  Object::operator=(static_cast<Object&&>(that));\n  standalone_record_ = that.standalone_record_;\n  has_header_ = that.has_header_;\n  header_ = std::move(that.header_);\n  quotes_needed_ = that.quotes_needed_;\n  newline_ = that.newline_;\n  field_separator_ = that.field_separator_;\n  quote_ = that.quote_;\n  record_index_ = std::exchange(that.record_index_, 0);\n  return *this;\n}\n\ninline void CsvWriterBase::Reset(Closed) {\n  Object::Reset(kClosed);\n  standalone_record_ = false;\n  has_header_ = false;\n  header_.Reset();\n  record_index_ = 0;\n}\n\ninline void CsvWriterBase::Reset() {\n  Object::Reset();\n  standalone_record_ = false;\n  has_header_ = false;\n  header_.Reset();\n  quotes_needed_ = {};\n  record_index_ = 0;\n}\n\nnamespace csv_internal {\n\ntemplate <typename Record>\ninline bool WriteStandaloneRecord(const Record& record,\n                                  CsvWriterBase& csv_writer) {\n  csv_writer.standalone_record_ = true;\n  return csv_writer.WriteRecordInternal(record);\n}\n\n}  // namespace csv_internal\n\ntemplate <typename Record,\n          std::enable_if_t<IsIterableOf<Record, absl::string_view>::value, int>>\ninline bool CsvWriterBase::WriteRecord(const Record& record) {\n  return WriteRecordInternal(record);\n}\n\ninline bool CsvWriterBase::WriteRecord(\n    std::initializer_list<absl::string_view> record) {\n  return WriteRecord<>(record);\n}\n\ntemplate <typename Record>\ninline bool CsvWriterBase::WriteRecordInternal(const Record& record) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (standalone_record_) {\n    RIEGELI_ASSERT_EQ(record_index_, 0u)\n        << \"Failed precondition of CsvWriterBase::WriteRecordInternal(): \"\n           \"called more than once by WriteCsvRecordToString()\";\n  }\n  Writer& dest = *DestWriter();\n  using std::begin;\n  auto iter = begin(record);\n  using std::end;\n  auto end_iter = end(record);\n  if (iter != end_iter) {\n    const absl::string_view field = *iter;\n    if (ABSL_PREDICT_FALSE(!WriteFirstField(dest, field))) return false;\n    if (++iter != end_iter) {\n      do {\n        if (ABSL_PREDICT_FALSE(!dest.Write(field_separator_))) {\n          return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n        }\n        const absl::string_view field = *iter;\n        if (ABSL_PREDICT_FALSE(!WriteField(dest, field))) return false;\n      } while (++iter != end_iter);\n    } else if (field.empty() &&\n               ABSL_PREDICT_TRUE(field_separator_ != kUtf8Bom[0])) {\n      // Quote a single empty field if not already quoted, to avoid writing an\n      // empty line which might be skipped by some readers.\n      if (ABSL_PREDICT_FALSE(!WriteQuotes(dest))) return false;\n    }\n  }\n  if (!standalone_record_) {\n    if (ABSL_PREDICT_FALSE(!WriteLine(dest, newline_))) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    }\n  }\n  ++record_index_;\n  return true;\n}\n\ninline uint64_t CsvWriterBase::last_record_index() const {\n  RIEGELI_ASSERT_GT(record_index_, 0u)\n      << \"Failed precondition of CsvWriterBase::last_record_index(): \"\n         \"no record was written\";\n  return record_index_ - 1;\n}\n\ntemplate <typename Dest>\ninline CsvWriter<Dest>::CsvWriter(Initializer<Dest> dest, Options options)\n    : dest_(std::move(dest)) {\n  Initialize(dest_.get(), std::move(options));\n}\n\ntemplate <typename Dest>\ninline void CsvWriter<Dest>::Reset(Closed) {\n  CsvWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void CsvWriter<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  CsvWriterBase::Reset();\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), std::move(options));\n}\n\ntemplate <typename Dest>\nvoid CsvWriter<Dest>::Done() {\n  CsvWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <typename Record,\n          std::enable_if_t<IsIterableOf<Record, absl::string_view>::value, int>>\nstd::string WriteCsvRecordToString(const Record& record,\n                                   CsvWriterBase::Options options) {\n  RIEGELI_ASSERT(options.header() == std::nullopt)\n      << \"Failed precondition of WriteCsvRecordToString(): \"\n         \"options.header() != std::nullopt not applicable\";\n  std::string dest;\n  CsvWriter csv_writer(riegeli::Maker<StringWriter>(&dest), std::move(options));\n  csv_internal::WriteStandaloneRecord(record, csv_writer);\n  // This can fail if `std::string` overflows, or if quoting is turned off and\n  // fields include inexpressible characters.\n  RIEGELI_CHECK(csv_writer.Close()) << csv_writer.status();\n  return dest;\n}\n\ninline std::string WriteCsvRecordToString(\n    std::initializer_list<absl::string_view> record,\n    CsvWriterBase::Options options) {\n  return WriteCsvRecordToString<>(record, std::move(options));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_CSV_CSV_WRITER_H_\n"
  },
  {
    "path": "riegeli/digests/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"digest_converter\",\n    hdrs = [\"digest_converter.h\"],\n    deps = [\n        \"//riegeli/endian:endian_reading\",\n        \"//riegeli/endian:endian_writing\",\n        \"@com_google_absl//absl/meta:type_traits\",\n        \"@com_google_absl//absl/numeric:int128\",\n    ],\n)\n\ncc_library(\n    name = \"digester_handle\",\n    srcs = [\"digester_handle.cc\"],\n    hdrs = [\"digester_handle.h\"],\n    deps = [\n        \":digest_converter\",\n        \"//riegeli/base:any\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:type_erased_ref\",\n        \"//riegeli/base:type_traits\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:int128\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"wrapping_digester\",\n    hdrs = [\"wrapping_digester.h\"],\n    deps = [\n        \":digest_converter\",\n        \":digester_handle\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:type_traits\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"digesting_reader\",\n    srcs = [\"digesting_reader.cc\"],\n    hdrs = [\"digesting_reader.h\"],\n    deps = [\n        \":digest_converter\",\n        \":digester_handle\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"digesting_writer\",\n    srcs = [\"digesting_writer.cc\"],\n    hdrs = [\"digesting_writer.h\"],\n    deps = [\n        \":digest_converter\",\n        \":digester_handle\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:type_traits\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:null_writer\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:stringify\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"crc32c_digester\",\n    hdrs = [\"crc32c_digester.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:byte_fill\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/crc:crc32c\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"crc32_digester\",\n    srcs = [\"crc32_digester.cc\"],\n    hdrs = [\"crc32_digester.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@zlib\",\n    ],\n)\n\ncc_library(\n    name = \"adler32_digester\",\n    srcs = [\"adler32_digester.cc\"],\n    hdrs = [\"adler32_digester.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@zlib\",\n    ],\n)\n\ncc_library(\n    name = \"highwayhash_digester\",\n    srcs = [\"highwayhash_digester.cc\"],\n    hdrs = [\"highwayhash_digester.h\"],\n    deps = [\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/meta:type_traits\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@highwayhash\",\n        \"@highwayhash//:arch_specific\",\n        \"@highwayhash//:hh_types\",\n    ],\n)\n\ncc_library(\n    name = \"openssl_digester\",\n    hdrs = [\"openssl_digester.h\"],\n    deps = [\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\n# Warning: MD5 as a cryptographic hash function is broken.\n# Use this only if a preexisting format has already decided to use MD5.\ncc_library(\n    name = \"md5_digester\",\n    hdrs = [\"md5_digester.h\"],\n    deps = [\n        \":openssl_digester\",\n        \"@boringssl//:crypto\",\n    ],\n)\n\n# Warning: SHA-1 as a cryptographic hash function is broken.\n# Use this only if a preexisting format has already decided to use SHA-1.\ncc_library(\n    name = \"sha1_digester\",\n    hdrs = [\"sha1_digester.h\"],\n    deps = [\n        \":openssl_digester\",\n        \"@boringssl//:crypto\",\n    ],\n)\n\ncc_library(\n    name = \"sha256_digester\",\n    hdrs = [\"sha256_digester.h\"],\n    deps = [\n        \":openssl_digester\",\n        \"@boringssl//:crypto\",\n    ],\n)\n\ncc_library(\n    name = \"sha512_digester\",\n    hdrs = [\"sha512_digester.h\"],\n    deps = [\n        \":openssl_digester\",\n        \"@boringssl//:crypto\",\n    ],\n)\n\ncc_library(\n    name = \"sha512_256_digester\",\n    hdrs = [\"sha512_256_digester.h\"],\n    deps = [\n        \":openssl_digester\",\n        \"@boringssl//:crypto\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/digests/adler32_digester.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/digests/adler32_digester.h\"\n\n#include <stdint.h>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"zconf.h\"\n#include \"zlib.h\"\n\nnamespace riegeli {\n\nAdler32Digester::Adler32Digester(uint32_t seed) : adler_(seed) {\n  // This checks CPU features.\n  adler32_z(0, nullptr, 0);\n}\n\nvoid Adler32Digester::Write(absl::string_view src) {\n  if (ABSL_PREDICT_FALSE(src.empty())) {\n    // `adler32_z(state, nullptr, 0)` exceptionally returns 1, not `state`.\n    return;\n  }\n  adler_ = IntCast<uint32_t>(adler32_z(\n      IntCast<uLong>(adler_), reinterpret_cast<const Bytef*>(src.data()),\n      IntCast<z_size_t>(src.size())));\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/digests/adler32_digester.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_ADLER32_DIGESTER_H_\n#define RIEGELI_DIGESTS_ADLER32_DIGESTER_H_\n\n#include <stdint.h>\n\n#include \"absl/strings/string_view.h\"\n\nnamespace riegeli {\n\n// A digester computing Adler32 checksums, for `DigestingReader` and\n// `DigestingWriter`.\nclass Adler32Digester {\n public:\n  Adler32Digester() : Adler32Digester(1) {}\n\n  explicit Adler32Digester(uint32_t seed);\n\n  Adler32Digester(const Adler32Digester& that) = default;\n  Adler32Digester& operator=(const Adler32Digester& that) = default;\n\n  void Write(absl::string_view src);\n  uint32_t Digest() { return adler_; }\n\n private:\n  uint32_t adler_;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_ADLER32_DIGESTER_H_\n"
  },
  {
    "path": "riegeli/digests/crc32_digester.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/digests/crc32_digester.h\"\n\n#include <stdint.h>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"zconf.h\"\n#include \"zlib.h\"\n\nnamespace riegeli {\n\nCrc32Digester::Crc32Digester(uint32_t seed) : crc_(seed) {\n  // This checks CPU features.\n  crc32_z(0, nullptr, 0);\n}\n\nvoid Crc32Digester::Write(absl::string_view src) {\n  if (ABSL_PREDICT_FALSE(src.empty())) {\n    // `crc32_z(state, nullptr, 0)` exceptionally returns 0, not `state`.\n    return;\n  }\n  crc_ = IntCast<uint32_t>(crc32_z(IntCast<uLong>(crc_),\n                                   reinterpret_cast<const Bytef*>(src.data()),\n                                   IntCast<z_size_t>(src.size())));\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/digests/crc32_digester.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_CRC32_DIGESTER_H_\n#define RIEGELI_DIGESTS_CRC32_DIGESTER_H_\n\n#include <stdint.h>\n\n#include \"absl/strings/string_view.h\"\n\nnamespace riegeli {\n\n// A digester computing CRC32 checksums, for `DigestingReader` and\n// `DigestingWriter`.\n//\n// This uses the polynomial x^32 + x^26 + x^23 + x^22 + x^16 + x^12 + x^11 +\n// x^10 + x^8 + x^7 + x^5 + x^4 + x^2 + x + 1 (0x104c11db7).\n//\n// This polynomial is used e.g. by gzip, zip, and png:\n// https://en.wikipedia.org/wiki/Cyclic_redundancy_check#Polynomial_representations_of_cyclic_redundancy_checks\nclass Crc32Digester {\n public:\n  Crc32Digester() : Crc32Digester(0) {}\n\n  explicit Crc32Digester(uint32_t seed);\n\n  Crc32Digester(const Crc32Digester& that) = default;\n  Crc32Digester& operator=(const Crc32Digester& that) = default;\n\n  void Write(absl::string_view src);\n  uint32_t Digest() { return crc_; }\n\n private:\n  uint32_t crc_;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_CRC32_DIGESTER_H_\n"
  },
  {
    "path": "riegeli/digests/crc32c_digester.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_CRC32C_DIGESTER_H_\n#define RIEGELI_DIGESTS_CRC32C_DIGESTER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <cstring>\n#include <limits>\n#include <optional>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/crc/crc32c.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/byte_fill.h\"\n\nnamespace riegeli {\n\n// A digester computing CRC32C checksums, for `DigestingReader` and\n// `DigestingWriter`.\n//\n// This uses the polynomial x^32 + x^28 + x^27 + x^26 + x^25 + x^23 + x^22 +\n// x^20 + x^19 + x^18 + x^14 + x^13 + x^11 + x^10 + x^9 + x^8 + x^6 + 1\n// (0x11edc6f41).\n//\n// This polynomial is used e.g. by SSE4.2:\n// https://en.wikipedia.org/wiki/Cyclic_redundancy_check#Polynomial_representations_of_cyclic_redundancy_checks\nclass Crc32cDigester {\n public:\n  Crc32cDigester() : Crc32cDigester(0) {}\n\n  explicit Crc32cDigester(uint32_t seed) : crc_(seed) {}\n\n  Crc32cDigester(const Crc32cDigester& that) = default;\n  Crc32cDigester& operator=(const Crc32cDigester& that) = default;\n\n  void Write(absl::string_view src);\n  void Write(const absl::Cord& src);\n  void Write(ByteFill src);\n  uint32_t Digest() { return crc_; }\n\n private:\n  uint32_t crc_;\n};\n\n// A common way to mask CRC32C values for storage along with the data.\n// These constants are used e.g. by Framed Snappy and TFRecord.\n\ntemplate <uint32_t delta = 0xa282ead8, int ror_bits = 15>\nconstexpr uint32_t MaskCrc32c(uint32_t unmasked) {\n  const uint32_t rotated =\n      (unmasked << (32 - ror_bits)) | (unmasked >> ror_bits);\n  return rotated + delta;\n}\n\ntemplate <uint32_t delta = 0xa282ead8, int ror_bits = 15>\nconstexpr uint32_t UnmaskCrc32c(uint32_t masked) {\n  const uint32_t rotated = masked - delta;\n  return (rotated << ror_bits) | (rotated >> (32 - ror_bits));\n}\n\n// Implementation details follow.\n\ninline void Crc32cDigester::Write(absl::string_view src) {\n  crc_ = static_cast<uint32_t>(absl::ExtendCrc32c(absl::crc32c_t{crc_}, src));\n}\n\ninline void Crc32cDigester::Write(const absl::Cord& src) {\n  if (const std::optional<uint32_t> src_crc = src.ExpectedChecksum();\n      src_crc != std::nullopt) {\n    crc_ = static_cast<uint32_t>(absl::ConcatCrc32c(\n        absl::crc32c_t{crc_}, absl::crc32c_t{*src_crc}, src.size()));\n    return;\n  }\n  if (const std::optional<absl::string_view> flat = src.TryFlat();\n      flat != std::nullopt) {\n    crc_ =\n        static_cast<uint32_t>(absl::ExtendCrc32c(absl::crc32c_t{crc_}, *flat));\n    return;\n  }\n  for (const absl::string_view fragment : src.Chunks()) {\n    crc_ = static_cast<uint32_t>(\n        absl::ExtendCrc32c(absl::crc32c_t{crc_}, fragment));\n  }\n}\n\ninline void Crc32cDigester::Write(ByteFill src) {\n  if (src.fill() == '\\0') {\n    while (\n        ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max())) {\n      crc_ = static_cast<uint32_t>(absl::ExtendCrc32cByZeroes(\n          absl::crc32c_t{crc_}, std::numeric_limits<size_t>::max()));\n      src.Extract(std::numeric_limits<size_t>::max());\n    }\n    crc_ = static_cast<uint32_t>(absl::ExtendCrc32cByZeroes(\n        absl::crc32c_t{crc_}, IntCast<size_t>(src.size())));\n    return;\n  }\n  for (const absl::string_view fragment : src.blocks()) {\n    crc_ = static_cast<uint32_t>(\n        absl::ExtendCrc32c(absl::crc32c_t{crc_}, fragment));\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_CRC32C_DIGESTER_H_\n"
  },
  {
    "path": "riegeli/digests/digest_converter.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_DIGEST_CONVERTER_H_\n#define RIEGELI_DIGESTS_DIGEST_CONVERTER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <array>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/meta/type_traits.h\"\n#include \"absl/numeric/int128.h\"\n#include \"riegeli/endian/endian_reading.h\"\n#include \"riegeli/endian/endian_writing.h\"\n\nnamespace riegeli {\n\n// `DigestConverterImpl<From, To>::Convert(From)` specifies how to convert a\n// digest to another supported type.\n//\n// This template is specialized but does not have a primary definition.\ntemplate <typename From, typename To, typename Enable = void>\nstruct DigestConverterImpl;\n\n// `HasDigestConverterImpl<From, To>::value` is `true` when\n// `DigestConverterImpl<From, To>` is defined.\n\ntemplate <typename From, typename To, typename Enable = void>\nstruct HasDigestConverterImpl : std::false_type {};\n\ntemplate <typename From, typename To>\nstruct HasDigestConverterImpl<\n    From, To,\n    std::void_t<decltype(DigestConverterImpl<From, To>::Convert(\n        std::declval<From>()))>> : std::true_type {};\n\n// `DigestConverterImpl<From, To>` tries to transform `From` from types with\n// more information to types with less information, or from integers or arrays\n// of integers of larger sizes to arrays of integers of smaller sizes, and then\n// considers further transformations, until reaching a type which can be\n// natively explicitly converted to `To`.\n//\n// To prevent infinite recursion, in conversions from arrays of integers of\n// smaller sizes to integers or arrays of integers of larger sizes, further\n// conversions using `DigestConverterImpl` are not considered, only types which\n// can be natively explicitly converted.\n\n// A digest can be converted if its type can be natively explicitly converted.\n// This includes the case when `To` is the same as `From`.\ntemplate <typename From, typename To>\nstruct DigestConverterImpl<\n    From, To, std::enable_if_t<std::is_constructible_v<To, From>>> {\n  template <\n      typename DependentFrom = From,\n      std::enable_if_t<!std::is_rvalue_reference_v<DependentFrom>, int> = 0>\n  static To Convert(const From& digest) {\n    return To(digest);\n  }\n  template <\n      typename DependentFrom = From,\n      std::enable_if_t<!std::is_lvalue_reference_v<DependentFrom>, int> = 0>\n  static To Convert(From&& digest) {\n    return To(std::forward<From>(digest));\n  }\n};\n\n// `std::array<char, size>` can be converted to `std::string`.\ntemplate <size_t size, typename To>\nstruct DigestConverterImpl<\n    std::array<char, size>, To,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<std::is_constructible<To, std::array<char, size>>>,\n        HasDigestConverterImpl<std::string, To>>>> {\n  static To Convert(const std::array<char, size>& digest) {\n    return DigestConverterImpl<std::string, To>::Convert(\n        std::string(digest.data(), digest.size()));\n  }\n};\n\n// `uint32_t`, `uint64_t`, and `absl::uint128` can be converted to\n// `std::array<char, sizeof(T)>`, using Big Endian.\n\ntemplate <typename To>\nstruct DigestConverterImpl<\n    uint32_t, To,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<std::is_constructible<To, uint32_t>>,\n        HasDigestConverterImpl<std::array<char, sizeof(uint32_t)>, To>>>> {\n  static To Convert(uint32_t digest) {\n    std::array<char, sizeof(uint32_t)> result;\n    riegeli::WriteBigEndian<uint32_t>(digest, result.data());\n    return DigestConverterImpl<std::array<char, sizeof(uint32_t)>, To>::Convert(\n        result);\n  }\n};\n\ntemplate <typename To>\nstruct DigestConverterImpl<\n    uint64_t, To,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<std::is_constructible<To, uint64_t>>,\n        HasDigestConverterImpl<std::array<char, sizeof(uint64_t)>, To>>>> {\n  static To Convert(uint64_t digest) {\n    std::array<char, sizeof(uint64_t)> result;\n    riegeli::WriteBigEndian<uint64_t>(digest, result.data());\n    return DigestConverterImpl<std::array<char, sizeof(uint64_t)>, To>::Convert(\n        result);\n  }\n};\n\ntemplate <typename To>\nstruct DigestConverterImpl<\n    absl::uint128, To,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<std::is_constructible<To, absl::uint128>>,\n        HasDigestConverterImpl<std::array<char, sizeof(absl::uint128)>, To>>>> {\n  static To Convert(absl::uint128 digest) {\n    std::array<char, sizeof(absl::uint128)> result;\n    riegeli::WriteBigEndian<absl::uint128>(digest, result.data());\n    return DigestConverterImpl<std::array<char, sizeof(absl::uint128)>,\n                               To>::Convert(result);\n  }\n};\n\n// `std::array<char, sizeof(T)>` can be converted to `uint32_t`, `uint64_t`,\n// and `absl::uint128`, using Big Endian.\n//\n// To prevent infinite recursion, further conversions using\n// `DigestConverterImpl` are not considered, only types which can be natively\n// explicitly converted.\n\ntemplate <typename To>\nstruct DigestConverterImpl<std::array<char, sizeof(uint32_t)>, To,\n                           std::enable_if_t<std::conjunction_v<\n                               std::negation<std::is_constructible<\n                                   To, std::array<char, sizeof(uint32_t)>>>,\n                               std::is_constructible<To, uint32_t>>>> {\n  static To Convert(std::array<char, sizeof(uint32_t)> digest) {\n    return To(riegeli::ReadBigEndian<uint32_t>(digest.data()));\n  }\n};\n\ntemplate <typename To>\nstruct DigestConverterImpl<std::array<char, sizeof(uint64_t)>, To,\n                           std::enable_if_t<std::conjunction_v<\n                               std::negation<std::is_constructible<\n                                   To, std::array<char, sizeof(uint64_t)>>>,\n                               std::is_constructible<To, uint64_t>>>> {\n  static To Convert(std::array<char, sizeof(uint64_t)> digest) {\n    return To(riegeli::ReadBigEndian<uint64_t>(digest.data()));\n  }\n};\n\ntemplate <typename To>\nstruct DigestConverterImpl<\n    std::array<char, sizeof(absl::uint128)>, To,\n    std::enable_if_t<\n        std::conjunction_v<std::negation<std::is_constructible<\n                               To, std::array<char, sizeof(absl::uint128)>>>,\n                           std::is_constructible<To, absl::uint128>>>> {\n  static To Convert(std::array<char, sizeof(absl::uint128)> digest) {\n    return To(riegeli::ReadBigEndian<absl::uint128>(digest.data()));\n  }\n};\n\n// `std::array<uint64_t, size>` can be converted to\n// `std::array<char, size * sizeof(uint64_t)>`, using Big Endian.\ntemplate <size_t size, typename To>\nstruct DigestConverterImpl<\n    std::array<uint64_t, size>, To,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<std::is_constructible<To, std::array<uint64_t, size>>>,\n        HasDigestConverterImpl<std::array<char, size * sizeof(uint64_t)>,\n                               To>>>> {\n  static To Convert(const std::array<uint64_t, size>& digest) {\n    std::array<char, size * sizeof(uint64_t)> result;\n    riegeli::WriteBigEndians<uint64_t>(absl::MakeConstSpan(digest.data(), size),\n                                       result.data());\n    return DigestConverterImpl<std::array<char, size * sizeof(uint64_t)>,\n                               To>::Convert(result);\n  }\n};\n\n// `std::array<char, size * sizeof(uint64_t)>` can be converted to\n// `std::array<uint64_t, size>`, using Big Endian.\n//\n// To prevent infinite recursion, further conversions using\n// `DigestConverterImpl` are not considered, only types which can be natively\n// explicitly converted.\ntemplate <size_t size, typename To>\nstruct DigestConverterImpl<\n    std::array<char, size>, To,\n    std::enable_if_t<std::conjunction_v<\n        std::bool_constant<size % sizeof(uint64_t) == 0>,\n        std::negation<std::is_constructible<To, std::array<char, size>>>,\n        std::is_constructible<\n            To, std::array<uint64_t, size / sizeof(uint64_t)>>>>> {\n  static To Convert(const std::array<char, size>& digest) {\n    std::array<uint64_t, size / sizeof(uint64_t)> result;\n    riegeli::ReadBigEndians<uint64_t>(\n        digest.data(), absl::MakeSpan(result.data(), size / sizeof(uint64_t)));\n    return To(result);\n  }\n};\n\n// `DigestConverter<From, To>` extends `DigestConverterImpl<From, To>` with the\n// case of `To` being a reference.\n\ntemplate <typename From, typename To, typename Enable = void>\nstruct DigestConverter;\n\ntemplate <typename From, typename To>\nstruct DigestConverter<From&, To&,\n                       std::enable_if_t<std::is_convertible_v<From*, To*>>> {\n  static To& Convert(From& digest) { return digest; }\n};\n\ntemplate <typename From, typename To>\nstruct DigestConverter<\n    From, To,\n    std::enable_if_t<std::conjunction_v<\n        std::negation<std::is_reference<To>>,\n        HasDigestConverterImpl<absl::remove_cvref_t<From>, To>>>>\n    : DigestConverterImpl<absl::remove_cvref_t<From>, To> {\n  static_assert(\n      std::is_convertible_v<\n          decltype(DigestConverterImpl<absl::remove_cvref_t<From>, To>::Convert(\n              std::declval<From>())),\n          To>,\n      \"DigestConverterImpl<From, To>::Convert() must return To\");\n};\n\n// `HasDigestConverter<From, To>::value` is `true` when\n// `DigestConverter<From, To>` is defined or when `To` is `void`.\ntemplate <typename From, typename To>\nstruct HasDigestConverter\n    : std::disjunction<\n          std::is_void<To>,\n          std::conjunction<std::is_lvalue_reference<To>,\n                           std::is_lvalue_reference<From>,\n                           std::is_convertible<std::remove_reference_t<From>*,\n                                               std::remove_reference_t<To>*>>,\n          std::conjunction<\n              std::negation<std::is_reference<To>>,\n              HasDigestConverterImpl<absl::remove_cvref_t<From>, To>>> {};\n\n// Converts a digest returned by `digest_function` to another supported type.\n//\n// The digest is passed as `digest_function` to support `void`.\n\ntemplate <\n    typename To, typename DigestFunction,\n    std::enable_if_t<\n        std::conjunction_v<\n            std::negation<std::is_void<To>>,\n            HasDigestConverter<decltype(std::declval<DigestFunction>()()), To>>,\n        int> = 0>\ninline To ConvertDigest(DigestFunction&& digest_function) {\n  return DigestConverter<decltype(std::declval<DigestFunction>()()), To>::\n      Convert(std::forward<DigestFunction>(digest_function)());\n}\n\ntemplate <typename To, typename DigestFunction,\n          std::enable_if_t<std::is_void_v<To>, int> = 0>\ninline void ConvertDigest(DigestFunction&& digest_function) {\n  std::forward<DigestFunction>(digest_function)();\n}\n\nnamespace digest_converter_internal {\n\n// A placeholder type to support an optional template parameter specifying the\n// desired digest type, which precedes other template parameters to be deduced\n// from function arguments.\n//\n// The optional template parameter should default to the digest type deduced\n// from later template parameters, but this cannot be written directly because\n// they are not in scope yet. Instead, the optional template parameter defaults\n// to `NoConversion` and is resolved later.\nclass NoConversion;\n\ntemplate <typename From, typename To>\nstruct ResolveNoConversionImpl {\n  using type = To;\n};\n\ntemplate <typename From>\nstruct ResolveNoConversionImpl<From, NoConversion> {\n  using type = From;\n};\n\ntemplate <typename From, typename To>\nusing ResolveNoConversion = typename ResolveNoConversionImpl<From, To>::type;\n\ntemplate <typename From, typename To>\nstruct HasDigestConverterOrNoConversion : HasDigestConverter<From, To> {};\n\ntemplate <typename From>\nstruct HasDigestConverterOrNoConversion<From, NoConversion> : std::true_type {};\n\n}  // namespace digest_converter_internal\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_DIGEST_CONVERTER_H_\n"
  },
  {
    "path": "riegeli/digests/digester_handle.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/digests/digester_handle.h\"\n\n#include <stddef.h>\n\n#include <optional>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/type_erased_ref.h\"\n#include \"riegeli/base/types.h\"\n\nnamespace riegeli {\n\nvoid DigesterBaseHandle::FailedDigestMethodDefault() {\n  RIEGELI_CHECK_UNREACHABLE()\n      << \"DigesterHandle::Digest() called on a default-constructed \"\n         \"DigesterHandle with a non-void DigesterHandle::DigestType\";\n}\n\nvoid DigesterBaseHandle::SetWriteSizeHintMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target,\n    ABSL_ATTRIBUTE_UNUSED std::optional<Position> write_size_hint) {}\n\nbool DigesterBaseHandle::WriteMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target,\n    ABSL_ATTRIBUTE_UNUSED absl::string_view src) {\n  return true;\n}\n\nbool DigesterBaseHandle::WriteChainMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target,\n    ABSL_ATTRIBUTE_UNUSED const Chain& src) {\n  return true;\n}\n\nbool DigesterBaseHandle::WriteCordMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target,\n    ABSL_ATTRIBUTE_UNUSED const absl::Cord& src) {\n  return true;\n}\n\nbool DigesterBaseHandle::WriteByteFillMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target,\n    ABSL_ATTRIBUTE_UNUSED ByteFill src) {\n  return true;\n}\n\nbool DigesterBaseHandle::CloseMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target) {\n  return true;\n}\n\nabsl::Status DigesterBaseHandle::StatusMethodDefault(\n    ABSL_ATTRIBUTE_UNUSED TypeErasedRef target) {\n  return absl::OkStatus();\n}\n\nbool DigesterBaseHandle::WriteChainFallback(\n    TypeErasedRef target, const Chain& src,\n    bool (*write)(TypeErasedRef target, absl::string_view src)) {\n  for (const absl::string_view fragment : src.blocks()) {\n    if (ABSL_PREDICT_FALSE(!write(target, fragment))) return false;\n  }\n  return true;\n}\n\nvoid DigesterBaseHandle::WriteChainFallback(\n    TypeErasedRef target, const Chain& src,\n    void (*write)(TypeErasedRef target, absl::string_view src)) {\n  for (const absl::string_view fragment : src.blocks()) write(target, fragment);\n}\n\nbool DigesterBaseHandle::WriteCordFallback(\n    TypeErasedRef target, const absl::Cord& src,\n    bool (*write)(TypeErasedRef target, absl::string_view src)) {\n  if (const std::optional<absl::string_view> flat = src.TryFlat();\n      flat != std::nullopt) {\n    return write(target, *flat);\n  }\n  for (const absl::string_view fragment : src.Chunks()) {\n    if (ABSL_PREDICT_FALSE(!write(target, fragment))) return false;\n  }\n  return true;\n}\n\nvoid DigesterBaseHandle::WriteCordFallback(\n    TypeErasedRef target, const absl::Cord& src,\n    void (*write)(TypeErasedRef target, absl::string_view src)) {\n  if (const std::optional<absl::string_view> flat = src.TryFlat();\n      flat != std::nullopt) {\n    write(target, *flat);\n    return;\n  }\n  for (const absl::string_view fragment : src.Chunks()) write(target, fragment);\n}\n\nbool DigesterBaseHandle::WriteByteFillFallback(\n    TypeErasedRef target, ByteFill src,\n    bool (*write)(TypeErasedRef target, absl::string_view src)) {\n  for (const absl::string_view fragment : src.blocks()) {\n    if (ABSL_PREDICT_FALSE(!write(target, fragment))) return false;\n  }\n  return true;\n}\n\nvoid DigesterBaseHandle::WriteByteFillFallback(\n    TypeErasedRef target, ByteFill src,\n    void (*write)(TypeErasedRef target, absl::string_view src)) {\n  for (const absl::string_view fragment : src.blocks()) write(target, fragment);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/digests/digester_handle.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_DIGESTER_HANDLE_H_\n#define RIEGELI_DIGESTS_DIGESTER_HANDLE_H_\n\n#include <stddef.h>\n\n#include <cstddef>\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/int128.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/has_absl_stringify.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/any.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/dependency_manager.h\"\n#include \"riegeli/base/type_erased_ref.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/digests/digest_converter.h\"\n\nnamespace riegeli {\n\n// `SupportsDigesterBaseHandle<T>::value` is `true` when `T&` is a valid\n// constructor argument for `DigesterBaseHandle`.\n\ntemplate <typename T, typename Enable = void>\nstruct SupportsDigesterBaseHandle : std::false_type {};\n\ntemplate <typename T>\nstruct SupportsDigesterBaseHandle<\n    T, std::enable_if_t<!std::is_const_v<T>,\n                        std::void_t<decltype(std::declval<T&>().Write(\n                            std::declval<absl::string_view>()))>>>\n    : std::true_type {};\n\n// Type-erased pointer to a target object called a digester, which observes data\n// being read or written.\n//\n// The target should support:\n//\n// ```\n//   // All of the following methods returning `bool` return `true` on success.\n//   // They may also return `void` which is treated as `true`.\n//\n//   // If `write_size_hint` is not `std::nullopt`, hints that this amount of\n//   // data will be written from the current position. This may improve\n//   // performance and memory usage.\n//   //\n//   // If the hint turns out to not match reality, nothing breaks. It is better\n//   // if `write_size_hint` is slightly too large than slightly too small.\n//   //\n//   // Optional. If absent, does nothing.\n//   void SetWriteSizeHint(std::optional<Position> write_size_hint);\n//\n//   // Called with consecutive fragments of data.\n//   //\n//   // Precondition: the digester is open.\n//   bool Write(absl::string_view src);\n//\n//   // Called with consecutive fragments of data.\n//   //\n//   // Precondition: the digester is open.\n//   //\n//   // Optional. If absent, implemented in terms of `Write(absl::string_view)`.\n//   bool Write(const Chain& src);\n//\n//   // Called with consecutive fragments of data.\n//   //\n//   // Precondition: the digester is open.\n//   //\n//   // Optional. If absent, implemented in terms of `Write(absl::string_view)`.\n//   bool Write(const absl::Cord& src);\n//\n//   // Can be called instead of `Write()` when data consists of zeros.\n//   //\n//   // Precondition: the digester is open.\n//   //\n//   // Optional. If absent, implemented in terms of `Write(absl::string_view)`.\n//   bool Write(ByteFill src);\n//\n//   // Optionally called when nothing more will be digested. This can make\n//   // `Digest()` more efficient. Resources can be freed. Marks the digester\n//   // as not open.\n//   //\n//   // Does nothing if the digester is not open.\n//   //\n//   // Optional. If absent, does nothing.\n//   bool Close();\n//\n//   // Returns an `absl::Status` describing the failure if the digester is\n//   // failed.\n//   //\n//   // Can return `absl::OkStatus()` if tracking the status is not supported.\n//   //\n//   // Optional. If absent, `absl::OkStatus()` is assumed.\n//   absl::Status status() const;\n// ```\n//\n// `DigesterHandle<DigestType>` extends `DigesterBaseHandle` with `Digest()`\n// returning `DigestType`.\n//\n// For digesting many small values it is better to use `DigestingWriter` which\n// adds a buffering layer.\nclass DigesterBaseHandle : public WithEqual<DigesterBaseHandle> {\n public:\n  // Creates a `DigesterBaseHandle` which does not refer to a target.\n  DigesterBaseHandle() = default;\n  /*implicit*/ DigesterBaseHandle(std::nullptr_t) {}\n\n  // Creates a `DigesterBaseHandle` which refers to `target`.\n  template <\n      typename T,\n      std::enable_if_t<std::conjunction_v<NotSameRef<DigesterBaseHandle, T&>,\n                                          SupportsDigesterBaseHandle<T>>,\n                       int> = 0>\n  /*implicit*/ DigesterBaseHandle(T& target ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : methods_(&kMethods<T>), target_(target) {}\n\n  DigesterBaseHandle(const DigesterBaseHandle& that) = default;\n  DigesterBaseHandle& operator=(const DigesterBaseHandle& that) = default;\n\n  friend bool operator==(DigesterBaseHandle a, std::nullptr_t) {\n    return a.target().empty();\n  }\n\n  // If `write_size_hint` is not `std::nullopt`, hints that this amount of data\n  // will be written from the current position. This may improve performance and\n  // memory usage.\n  //\n  // If the hint turns out to not match reality, nothing breaks. It is better if\n  // `write_size_hint` is slightly too large than slightly too small.\n  void SetWriteSizeHint(std::optional<Position> write_size_hint) {\n    methods_->set_write_size_hint(target_, write_size_hint);\n  }\n\n  // Called with consecutive fragments of data.\n  //\n  // Precondition: the digester is open.\n  bool Write(char src) { return Write(absl::string_view(&src, 1)); }\n#if __cpp_char8_t\n  bool Write(char8_t src) { return Write(static_cast<char>(src)); }\n#endif\n  bool Write(BytesRef src) { return methods()->write(target(), src); }\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  bool Write(const char* src) { return Write(absl::string_view(src)); }\n  bool Write(const Chain& src) { return methods()->write_chain(target(), src); }\n  bool Write(const absl::Cord& src) {\n    return methods()->write_cord(target(), src);\n  }\n  bool Write(ByteFill src) { return methods()->write_byte_fill(target(), src); }\n  template <\n      typename Src,\n      std::enable_if_t<\n          std::conjunction_v<\n              absl::HasAbslStringify<Src>,\n              std::negation<std::is_convertible<Src&&, BytesRef>>,\n              std::negation<std::is_convertible<Src&&, const Chain&>>,\n              std::negation<std::is_convertible<Src&&, const absl::Cord&>>,\n              std::negation<std::is_convertible<Src&&, ByteFill>>>,\n          int> = 0>\n  bool Write(Src&& src);\n\n  // Numeric types supported by `Writer::Write()` are not supported by\n  // `DigesterBaseHandle::Write()`. Use `DigestingWriter` instead or convert\n  // them to strings.\n  bool Write(signed char) = delete;\n  bool Write(unsigned char) = delete;\n  bool Write(short) = delete;\n  bool Write(unsigned short) = delete;\n  bool Write(int) = delete;\n  bool Write(unsigned) = delete;\n  bool Write(long) = delete;\n  bool Write(unsigned long) = delete;\n  bool Write(long long) = delete;\n  bool Write(unsigned long long) = delete;\n  bool Write(absl::int128) = delete;\n  bool Write(absl::uint128) = delete;\n  bool Write(float) = delete;\n  bool Write(double) = delete;\n  bool Write(long double) = delete;\n  bool Write(bool) = delete;\n  bool Write(wchar_t) = delete;\n  bool Write(char16_t) = delete;\n  bool Write(char32_t) = delete;\n\n  // Called when nothing more will be digested. This can make `Digest()` more\n  // efficient. Resources can be freed. Marks the digester as not open.\n  //\n  // Does nothing if the digester is not open.\n  bool Close() { return methods()->close(target()); }\n\n  // Returns an `absl::Status` describing the failure if the digester is\n  // failed.\n  //\n  // Can return `absl::OkStatus()` if tracking the status is not supported.\n  absl::Status status() const { return methods()->status(target()); }\n\n protected:\n  ABSL_ATTRIBUTE_NORETURN static void FailedDigestMethodDefault();\n\n private:\n  template <typename Function,\n            std::enable_if_t<\n                std::is_same_v<decltype(std::declval<Function&&>()()), bool>,\n                int> = 0>\n  static bool ConvertToBool(Function&& function) {\n    return std::forward<Function>(function)();\n  }\n  template <\n      typename Function,\n      std::enable_if_t<std::is_void_v<decltype(std::declval<Function&&>()())>,\n                       int> = 0>\n  static bool ConvertToBool(Function&& function) {\n    std::forward<Function>(function)();\n    return true;\n  }\n\n  template <typename T, typename Enable = void>\n  struct DigesterTargetHasSetWriteSizeHint : std::false_type {};\n  template <typename T>\n  struct DigesterTargetHasSetWriteSizeHint<\n      T, std::void_t<decltype(std::declval<T&>().SetWriteSizeHint(\n             std::declval<std::optional<Position>>()))>> : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct DigesterTargetHasWriteChain : std::false_type {};\n  template <typename T>\n  struct DigesterTargetHasWriteChain<\n      T, std::void_t<decltype(std::declval<T&>().Write(\n             std::declval<const Chain&>()))>> : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct DigesterTargetHasWriteCord : std::false_type {};\n  template <typename T>\n  struct DigesterTargetHasWriteCord<\n      T, std::void_t<decltype(std::declval<T&>().Write(\n             std::declval<const absl::Cord&>()))>> : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct DigesterTargetHasWriteByteFill : std::false_type {};\n  template <typename T>\n  struct DigesterTargetHasWriteByteFill<\n      T,\n      std::void_t<decltype(std::declval<T&>().Write(std::declval<ByteFill>()))>>\n      : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct DigesterTargetHasClose : std::false_type {};\n  template <typename T>\n  struct DigesterTargetHasClose<\n      T, std::void_t<decltype(std::declval<T&>().Close())>> : std::true_type {};\n\n  template <typename T, typename Enable = void>\n  struct DigesterTargetHasStatus : std::false_type {};\n  template <typename T>\n  struct DigesterTargetHasStatus<\n      T, std::enable_if_t<std::is_convertible_v<\n             decltype(std::declval<const T&>().status()), absl::Status>>>\n      : std::true_type {};\n\n  static void SetWriteSizeHintMethodDefault(\n      TypeErasedRef target, std::optional<Position> write_size_hint);\n  static bool WriteMethodDefault(TypeErasedRef target, absl::string_view src);\n  static bool WriteChainMethodDefault(TypeErasedRef target, const Chain& src);\n  static bool WriteCordMethodDefault(TypeErasedRef target,\n                                     const absl::Cord& src);\n  static bool WriteByteFillMethodDefault(TypeErasedRef target, ByteFill src);\n  static bool CloseMethodDefault(TypeErasedRef target);\n  static absl::Status StatusMethodDefault(TypeErasedRef target);\n\n  template <typename T>\n  static void SetWriteSizeHintMethod(TypeErasedRef target,\n                                     std::optional<Position> write_size_hint) {\n    if constexpr (DigesterTargetHasSetWriteSizeHint<T>::value) {\n      target.Cast<T&>().SetWriteSizeHint(write_size_hint);\n    }\n  }\n\n  template <typename T>\n  static auto RawWriteMethod(TypeErasedRef target, absl::string_view src) {\n    return target.Cast<T&>().Write(src);\n  }\n\n  template <typename T>\n  static bool WriteMethod(TypeErasedRef target, absl::string_view src) {\n    return ConvertToBool([&] { return RawWriteMethod<T>(target, src); });\n  }\n\n  static bool WriteChainFallback(TypeErasedRef target, const Chain& src,\n                                 bool (*write)(TypeErasedRef target,\n                                               absl::string_view src));\n  static void WriteChainFallback(TypeErasedRef target, const Chain& src,\n                                 void (*write)(TypeErasedRef target,\n                                               absl::string_view src));\n\n  template <typename T>\n  static bool WriteChainMethod(TypeErasedRef target, const Chain& src) {\n    return ConvertToBool([&] {\n      if constexpr (DigesterTargetHasWriteChain<T>::value) {\n        return target.Cast<T&>().Write(src);\n      } else {\n        return WriteChainFallback(target, src, RawWriteMethod<T>);\n      }\n    });\n  }\n\n  static bool WriteCordFallback(TypeErasedRef target, const absl::Cord& src,\n                                bool (*write)(TypeErasedRef target,\n                                              absl::string_view src));\n  static void WriteCordFallback(TypeErasedRef target, const absl::Cord& src,\n                                void (*write)(TypeErasedRef target,\n                                              absl::string_view src));\n\n  template <typename T>\n  static bool WriteCordMethod(TypeErasedRef target, const absl::Cord& src) {\n    return ConvertToBool([&] {\n      if constexpr (DigesterTargetHasWriteCord<T>::value) {\n        return target.Cast<T&>().Write(src);\n      } else {\n        return WriteCordFallback(target, src, RawWriteMethod<T>);\n      }\n    });\n  }\n\n  static bool WriteByteFillFallback(TypeErasedRef target, ByteFill src,\n                                    bool (*write)(TypeErasedRef target,\n                                                  absl::string_view src));\n  static void WriteByteFillFallback(TypeErasedRef target, ByteFill src,\n                                    void (*write)(TypeErasedRef target,\n                                                  absl::string_view src));\n\n  template <typename T>\n  static bool WriteByteFillMethod(TypeErasedRef target, ByteFill src) {\n    return ConvertToBool([&] {\n      if constexpr (DigesterTargetHasWriteByteFill<T>::value) {\n        return target.Cast<T&>().Write(src);\n      } else {\n        return WriteByteFillFallback(target, src, RawWriteMethod<T>);\n      }\n    });\n  }\n\n  template <typename T>\n  static bool CloseMethod(TypeErasedRef target) {\n    if constexpr (DigesterTargetHasClose<T>::value) {\n      return ConvertToBool([&] { return target.Cast<T&>().Close(); });\n    } else {\n      return true;\n    }\n  }\n\n  template <typename T>\n  static absl::Status StatusMethod(TypeErasedRef target) {\n    if constexpr (DigesterTargetHasStatus<T>::value) {\n      return target.Cast<const T&>().status();\n    } else {\n      return absl::OkStatus();\n    }\n  }\n\n protected:\n  struct Methods {\n    void (*set_write_size_hint)(TypeErasedRef target,\n                                std::optional<Position> write_size_hint);\n    bool (*write)(TypeErasedRef target, absl::string_view src);\n    bool (*write_chain)(TypeErasedRef target, const Chain& src);\n    bool (*write_cord)(TypeErasedRef target, const absl::Cord& src);\n    bool (*write_byte_fill)(TypeErasedRef target, ByteFill src);\n    bool (*close)(TypeErasedRef target);\n    absl::Status (*status)(TypeErasedRef target);\n  };\n\n  static constexpr Methods kMethodsDefault = {SetWriteSizeHintMethodDefault,\n                                              WriteMethodDefault,\n                                              WriteChainMethodDefault,\n                                              WriteCordMethodDefault,\n                                              WriteByteFillMethodDefault,\n                                              CloseMethodDefault,\n                                              StatusMethodDefault};\n\n  template <typename T>\n  static constexpr Methods kMethods = {SetWriteSizeHintMethod<T>,\n                                       WriteMethod<T>,\n                                       WriteChainMethod<T>,\n                                       WriteCordMethod<T>,\n                                       WriteByteFillMethod<T>,\n                                       CloseMethod<T>,\n                                       StatusMethod<T>};\n\n  explicit DigesterBaseHandle(const Methods* methods, TypeErasedRef target)\n      : methods_(methods), target_(target) {}\n\n  const Methods* methods() const { return methods_; }\n  TypeErasedRef target() const { return target_; }\n\n private:\n  class DigesterStringifySink;\n\n  const Methods* methods_ = &kMethodsDefault;\n  TypeErasedRef target_;\n};\n\nnamespace digester_handle_internal {\n\ntemplate <typename T, typename Enable = void>\nstruct DigestOfDigesterTarget {\n  using type = void;\n};\n\ntemplate <typename T>\nstruct DigestOfDigesterTarget<\n    T, std::void_t<decltype(std::declval<T&>().Digest())>> {\n  using type = decltype(std::declval<T&>().Digest());\n};\n\n}  // namespace digester_handle_internal\n\n// `SupportsDigesterHandle<T, DigestType>::value` is `true` when `T&` is a valid\n// constructor argument for `DigesterHandle<DigestType>`.\n\ntemplate <typename T, typename DigestType, typename Enable = void>\nstruct SupportsDigesterHandle : std::false_type {};\n\ntemplate <typename T, typename DigestType>\nstruct SupportsDigesterHandle<\n    T, DigestType,\n    std::enable_if_t<std::conjunction_v<\n        SupportsDigesterBaseHandle<T>,\n        HasDigestConverter<\n            typename digester_handle_internal::DigestOfDigesterTarget<T>::type,\n            DigestType>>>> : std::true_type {};\n\n// `DigesterHandle<DigestType>` extends `DigesterBaseHandle` with `Digest()`\n// returning some data of type `DigestType` called a digest, e.g. a checksum.\n//\n// The digester should support:\n//\n// ```\n//   // Returns the digest of data written so far. Its type and meaning depends\n//   // on the concrete digester. Unchanged by `Close()`.\n//   //\n//   // `OriginalDigestType` can be any type convertible to `DigestType` using\n//   // `DigestConverter`.\n//   //\n//   // Depending on the digester, `Digest()` can be more efficient if `Close()`\n//   // is called before.\n//   //\n//   // Many digesters support calling `Digest()` and then accepting more data\n//   // or calling `Digest()` again, but this is not guaranteed.\n//   OriginalDigestType Digest();\n// ```\n//\n// `DigestType` can be `void` for digesters used for their side effects.\ntemplate <typename DigestTypeParam>\nclass DigesterHandle : public DigesterBaseHandle {\n public:\n  // The type of the digest.\n  using DigestType = DigestTypeParam;\n\n  // Creates a `DigesterHandle` which does not refer to a target.\n  DigesterHandle() noexcept\n      : DigesterBaseHandle(&kMethodsDefault, TypeErasedRef()) {}\n  /*implicit*/ DigesterHandle(std::nullptr_t) : DigesterHandle() {}\n\n  // Creates a `DigesterHandle` which refers to `target`.\n  template <typename T,\n            std::enable_if_t<\n                std::conjunction_v<NotSameRef<DigesterHandle, T&>,\n                                   SupportsDigesterHandle<T, DigestType>>,\n                int> = 0>\n  /*implicit*/ DigesterHandle(T& target ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : DigesterBaseHandle(&kMethods<T>, TypeErasedRef(target)) {}\n\n  DigesterHandle(const DigesterHandle& that) = default;\n  DigesterHandle& operator=(const DigesterHandle& that) = default;\n\n  // Returns the digest of data written so far. Its type and meaning depends on\n  // the concrete digester. Unchanged by `Close()`.\n  //\n  // The digest is converted to `DesiredDigestType` using `DigestConverter`.\n  template <\n      typename DesiredDigestType = DigestType,\n      std::enable_if_t<HasDigestConverter<DigestType, DesiredDigestType>::value,\n                       int> = 0>\n  DesiredDigestType Digest() {\n    return ConvertDigest<DesiredDigestType>(\n        [&]() -> DigestType { return methods()->digest(target()); });\n  }\n\n private:\n  template <typename T, typename Enable = void>\n  struct DigesterTargetHasDigest : std::false_type {};\n  template <typename T>\n  struct DigesterTargetHasDigest<\n      T, std::void_t<decltype(std::declval<T&>().Digest())>> : std::true_type {\n  };\n\n  static DigestType DigestMethodDefault(\n      ABSL_ATTRIBUTE_UNUSED TypeErasedRef target) {\n    if constexpr (!std::is_void_v<DigestType>) FailedDigestMethodDefault();\n  }\n\n  template <typename T>\n  static DigestType DigestMethod(TypeErasedRef target) {\n    if constexpr (DigesterTargetHasDigest<T>::value) {\n      return ConvertDigest<DigestType>(\n          [&]() -> decltype(auto) { return target.Cast<T&>().Digest(); });\n    } else {\n      static_assert(std::is_void_v<DigestType>,\n                    \"DigestType must be void for digesters without Digest().\");\n    }\n  }\n\n  struct Methods : DigesterBaseHandle::Methods {\n    // MSVC does not like the `DigestType` alias here for some reason.\n    DigestTypeParam (*digest)(TypeErasedRef target);\n  };\n\n  static constexpr Methods kMethodsDefault = {\n      DigesterBaseHandle::kMethodsDefault, DigestMethodDefault};\n\n  template <typename T>\n  static constexpr Methods kMethods = {DigesterBaseHandle::kMethods<T>,\n                                       DigestMethod<T>};\n\n  const Methods* methods() const {\n    return static_cast<const Methods*>(DigesterBaseHandle::methods());\n  }\n};\n\nDigesterHandle() -> DigesterHandle<DeleteCtad<>>;\nDigesterHandle(std::nullptr_t) -> DigesterHandle<DeleteCtad<std::nullptr_t>>;\ntemplate <typename T,\n          std::enable_if_t<SupportsDigesterBaseHandle<T>::value, int> = 0>\nexplicit DigesterHandle(T& target) -> DigesterHandle<\n    typename digester_handle_internal::DigestOfDigesterTarget<T>::type>;\n\n// Specialization of `DependencyImpl<DigesterBaseHandle, Manager>` when\n// `DependencyManagerRef<Manager>` is a valid digester target.\n//\n// Specialized separately for `get()` to return `DigesterHandle<DigestType>`.\ntemplate <typename Manager>\nclass DependencyImpl<\n    DigesterBaseHandle, Manager,\n    std::enable_if_t<std::conjunction_v<\n        std::is_pointer<DependencyManagerPtr<Manager>>,\n        SupportsDigesterBaseHandle<DependencyManagerRef<Manager>>>>>\n    : public DependencyManager<Manager> {\n public:\n  using DependencyImpl::DependencyManager::DependencyManager;\n\n  // Returns `DigesterHandle<DigestType>` rather than `DigesterBaseHandle`.\n  DigesterHandle<typename digester_handle_internal::DigestOfDigesterTarget<\n      DependencyManagerRef<Manager>>::type>\n  get() const {\n    return DigesterHandle<\n        typename digester_handle_internal::DigestOfDigesterTarget<\n            DependencyManagerRef<Manager>>::type>(*this->ptr());\n  }\n\n protected:\n  DependencyImpl(const DependencyImpl& that) = default;\n  DependencyImpl& operator=(const DependencyImpl& that) = default;\n\n  DependencyImpl(DependencyImpl&& that) = default;\n  DependencyImpl& operator=(DependencyImpl&& that) = default;\n\n  ~DependencyImpl() = default;\n};\n\n// The type of the digest returned by the digester provided by `Digester`.\ntemplate <typename Digester>\nusing DigestOf =\n    typename Dependency<DigesterBaseHandle, Digester>::Subhandle::DigestType;\n\n// Type-erased digester returning a digest of type `DigestType`.\ntemplate <typename DigestType>\nusing AnyDigester = Any<DigesterHandle<DigestType>>;\n\n// Implementation details follow.\n\nclass DigesterBaseHandle::DigesterStringifySink {\n public:\n  explicit DigesterStringifySink(DigesterBaseHandle digester)\n      : digester_(digester) {}\n\n  void Append(size_t length, char fill) {\n    if (ABSL_PREDICT_FALSE(!digester_.Write(ByteFill(length, fill)))) {\n      ok_ = false;\n    }\n  }\n  void Append(absl::string_view src) {\n    if (ABSL_PREDICT_FALSE(!digester_.Write(src))) ok_ = false;\n  }\n  friend void AbslFormatFlush(DigesterStringifySink* dest,\n                              absl::string_view src) {\n    dest->Append(src);\n  }\n\n  bool ok() const { return ok_; }\n\n private:\n  DigesterBaseHandle digester_;\n  bool ok_ = true;\n};\n\ntemplate <typename Src,\n          std::enable_if_t<\n              std::conjunction_v<\n                  absl::HasAbslStringify<Src>,\n                  std::negation<std::is_convertible<Src&&, BytesRef>>,\n                  std::negation<std::is_convertible<Src&&, const Chain&>>,\n                  std::negation<std::is_convertible<Src&&, const absl::Cord&>>,\n                  std::negation<std::is_convertible<Src&&, ByteFill>>>,\n              int>>\nbool DigesterBaseHandle::Write(Src&& src) {\n  DigesterStringifySink sink(*this);\n  AbslStringify(sink, std::forward<Src>(src));\n  return sink.ok();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_DIGESTER_HANDLE_H_\n"
  },
  {
    "path": "riegeli/digests/digesting_reader.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/digests/digesting_reader.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/digests/digester_handle.h\"\n\nnamespace riegeli {\n\nnamespace {\n\nABSL_ATTRIBUTE_COLD absl::Status FailedStatus(DigesterBaseHandle digester) {\n  absl::Status status = digester.status();\n  if (status.ok()) status = absl::UnknownError(\"Digester failed\");\n  return status;\n}\n\n}  // namespace\n\nbool DigestingReaderBase::FailFromDigester() {\n  const DigesterBaseHandle digester = GetDigester();\n  return Fail(FailedStatus(digester));\n}\n\nvoid DigestingReaderBase::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    Reader& src = *SrcReader();\n    SyncBuffer(src);\n  }\n  Reader::Done();\n}\n\nabsl::Status DigestingReaderBase::AnnotateStatusImpl(absl::Status status) {\n  // Fully delegate annotations to `*SrcReader()`.\n  if (is_open()) {\n    Reader& src = *SrcReader();\n    const bool sync_buffer_ok = SyncBuffer(src);\n    status = src.AnnotateStatus(std::move(status));\n    if (ABSL_PREDICT_TRUE(sync_buffer_ok)) MakeBuffer(src);\n  }\n  return status;\n}\n\nbool DigestingReaderBase::PullSlow(size_t min_length,\n                                   size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::PullSlow(): \"\n         \"enough data available, use Pull() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(src))) return false;\n  const bool pull_ok = src.Pull(min_length, recommended_length);\n  MakeBuffer(src);\n  return pull_ok;\n}\n\nbool DigestingReaderBase::ReadSlow(size_t length, char* dest) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of Reader::ReadSlow(char*): \"\n         \"enough data available, use Read(char*) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(src))) return false;\n  size_t length_read;\n  bool read_ok = src.Read(length, dest, &length_read);\n  if (length_read > 0) {\n    if (ABSL_PREDICT_FALSE(\n            !WriteToDigester(absl::string_view(dest, length_read)))) {\n      RIEGELI_EVAL_ASSERT(!FailFromDigester());\n      read_ok = false;\n    }\n  }\n  MakeBuffer(src);\n  return read_ok;\n}\n\nbool DigestingReaderBase::ReadSlow(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"Chain size overflow\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(src))) return false;\n  Chain data;\n  bool read_ok = src.Read(length, data);\n  if (!data.empty()) {\n    DigesterBaseHandle digester = GetDigester();\n    if (ABSL_PREDICT_FALSE(!digester.Write(data))) {\n      RIEGELI_EVAL_ASSERT(!FailFromDigester());\n      read_ok = false;\n    } else {\n      dest.Append(std::move(data));\n    }\n  }\n  MakeBuffer(src);\n  return read_ok;\n}\n\nbool DigestingReaderBase::ReadSlow(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"Cord size overflow\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(src))) return false;\n  absl::Cord data;\n  bool read_ok = src.Read(length, data);\n  if (!data.empty()) {\n    DigesterBaseHandle digester = GetDigester();\n    if (ABSL_PREDICT_FALSE(!digester.Write(data))) {\n      RIEGELI_EVAL_ASSERT(!FailFromDigester());\n      read_ok = false;\n    } else {\n      dest.Append(std::move(data));\n    }\n  }\n  MakeBuffer(src);\n  return read_ok;\n}\n\nbool DigestingReaderBase::ReadSomeSlow(size_t max_length, char* dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"nothing to read, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"some data available, use ReadSome(char*) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(src))) return false;\n  size_t length_read;\n  bool read_ok = src.ReadSome(max_length, dest, &length_read);\n  if (ABSL_PREDICT_TRUE(read_ok)) {\n    if (ABSL_PREDICT_FALSE(\n            !WriteToDigester(absl::string_view(dest, length_read)))) {\n      RIEGELI_EVAL_ASSERT(!FailFromDigester());\n      read_ok = false;\n    }\n  }\n  MakeBuffer(src);\n  return read_ok;\n}\n\nvoid DigestingReaderBase::ReadHintSlow(size_t min_length,\n                                       size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::ReadHintSlow(): \"\n         \"enough data available, use ReadHint() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  Reader& src = *SrcReader();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(src))) return;\n  src.ReadHint(min_length, recommended_length);\n  MakeBuffer(src);\n}\n\nbool DigestingReaderBase::SupportsSize() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsSize();\n}\n\nstd::optional<Position> DigestingReaderBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  Reader& src = *SrcReader();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(src))) return std::nullopt;\n  const std::optional<Position> size = src.Size();\n  MakeBuffer(src);\n  return size;\n}\n\nbool DigestingReaderBase::SupportsNewReader() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsNewReader();\n}\n\nstd::unique_ptr<Reader> DigestingReaderBase::NewReaderImpl(\n    Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point\n  // if `SrcReader()->SupportsNewReader()`.\n  Reader& src = *SrcReader();\n  std::unique_ptr<Reader> reader = src.NewReader(initial_pos);\n  if (ABSL_PREDICT_FALSE(reader == nullptr)) {\n    FailWithoutAnnotation(src.status());\n  }\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/digests/digesting_reader.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_DIGESTING_READER_H_\n#define RIEGELI_DIGESTS_DIGESTING_READER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/digests/digest_converter.h\"\n#include \"riegeli/digests/digester_handle.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `DigestingReader`.\nclass DigestingReaderBase : public Reader {\n public:\n  // Returns the `DigesterBaseHandle`. Unchanged by `Close()`.\n  virtual DigesterBaseHandle GetDigester() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the original `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsSize() override;\n  bool SupportsNewReader() override;\n\n protected:\n  using Reader::Reader;\n\n  DigestingReaderBase(DigestingReaderBase&& that) noexcept;\n  DigestingReaderBase& operator=(DigestingReaderBase&& that) noexcept;\n\n  void Initialize(Reader* src, DigesterBaseHandle digester);\n  ABSL_ATTRIBUTE_COLD bool FailFromDigester();\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n\n  virtual bool WriteToDigester(absl::string_view src) = 0;\n\n  // Sets cursor of `src` to cursor of `*this`, digesting what has been read\n  // from the buffer (until `cursor()`).\n  bool SyncBuffer(Reader& src);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `src`, adjusting\n  // `start()` to hide data already digested. Fails `*this` if `src` failed.\n  void MakeBuffer(Reader& src);\n\n  bool PullSlow(size_t min_length, size_t recommended_length) override;\n  using Reader::ReadSlow;\n  bool ReadSlow(size_t length, char* dest) override;\n  bool ReadSlow(size_t length, Chain& dest) override;\n  bool ReadSlow(size_t length, absl::Cord& dest) override;\n  using Reader::ReadSomeSlow;\n  bool ReadSomeSlow(size_t max_length, char* dest) override;\n  void ReadHintSlow(size_t min_length, size_t recommended_length) override;\n  std::optional<Position> SizeImpl() override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n  // Invariants if `is_open()`:\n  //   `start() == SrcReader()->cursor()`\n  //   `limit() == SrcReader()->limit()`\n  //   `limit_pos() == SrcReader()->limit_pos()`\n};\n\n// A `Reader` which reads from another `Reader`, and lets another object observe\n// data being read and return some data called a digest, e.g. a checksum.\n//\n// The `Digester` template parameter specifies the type of the object providing\n// and possibly owning the digester. `Digester` must support\n// `Dependency<DigesterBaseHandle, Digester>`, e.g.\n// `DigesterHandle<uint32_t>` (not owned), `Crc32cDigester` (owned),\n// `AnyDigester<uint32_t>` (maybe owned).\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the original `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the `Digester` template argument can be deduced as\n// `TargetT` of the type of the `digester` constructor argument, and the `Src`\n// template argument can be deduced as `TargetT` of the type of the `src`\n// constructor argument.\n//\n// The original `Reader` must not be accessed until the `DigestingReader` is\n// closed or no longer used.\ntemplate <typename Digester, typename Src = Reader*>\nclass DigestingReader : public DigestingReaderBase {\n public:\n  // The type of the digest.\n  using DigestType = DigestOf<Digester>;\n\n  // Creates a closed `DigestingReader`.\n  explicit DigestingReader(Closed) noexcept : DigestingReaderBase(kClosed) {}\n\n  // Will read from the original `Reader` provided by `src`, using the\n  // digester provided by `digester`.\n  explicit DigestingReader(Initializer<Src> src,\n                           Initializer<Digester> digester = riegeli::Maker());\n\n  DigestingReader(DigestingReader&& that) = default;\n  DigestingReader& operator=(DigestingReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `DigestingReader`. This\n  // avoids constructing a temporary `DigestingReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      Initializer<Src> src, Initializer<Digester> digester = riegeli::Maker());\n\n  // Digests buffered data if needed, and returns the digest.\n  //\n  // The digest is converted to `DesiredDigestType` using `DigestConverter`.\n  //\n  // Depending on the digester, `Digest()` can be more efficient if `Close()` is\n  // called before.\n  //\n  // Many digesters support calling `Digest()` and then accepting more data or\n  // calling `Digest()` again, but this is not guaranteed.\n  template <\n      typename DesiredDigestType = DigestType,\n      std::enable_if_t<HasDigestConverter<DigestType, DesiredDigestType>::value,\n                       int> = 0>\n  DesiredDigestType Digest() {\n    if (start_to_cursor() > 0) {\n      if (ABSL_PREDICT_FALSE(!digester_.get().Write(\n              absl::string_view(start(), start_to_cursor())))) {\n        FailFromDigester();\n      } else {\n        set_buffer(cursor(), available());\n        src_->set_cursor(cursor());\n      }\n    }\n    return digester_.get().template Digest<DesiredDigestType>();\n  }\n\n  // Returns the object providing and possibly owning the digester. Unchanged by\n  // `Close()`.\n  Digester& digester() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return digester_.manager();\n  }\n  const Digester& digester() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return digester_.manager();\n  }\n  DigesterBaseHandle GetDigester() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return digester_.get();\n  }\n\n  // Returns the object providing and possibly owning the original `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  bool WriteToDigester(absl::string_view src) override {\n    return digester_.get().Write(src);\n  }\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  void VerifyEndImpl() override;\n  bool SyncImpl(SyncType sync_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the digester.\n  Dependency<DigesterBaseHandle, Digester> digester_;\n  // The object providing and possibly owning the original `Reader`.\n  MovingDependency<Reader*, Src, Mover> src_;\n};\n\nexplicit DigestingReader(Closed) -> DigestingReader<void, DeleteCtad<Closed>>;\ntemplate <typename Digester, typename Src>\nexplicit DigestingReader(Src&& src, Digester&& digester)\n    -> DigestingReader<TargetT<Digester>, TargetT<Src>>;\n\n// Reads all remaining data from `src` and returns their digest.\n//\n// If `length_read != nullptr` then sets `*length_read` to the length read.\n// This is equal to the difference between `src.pos()` after and before the\n// call.\n//\n// The `Digester` template parameter specifies the type of the object providing\n// and possibly owning the digester. `Digester` must support\n// `DependencyRef<DigesterBaseHandle, Digester>` and must provide a member\n// function `DigestType Digest()` for some `DigestType`, e.g.\n// `DigesterHandle<uint32_t>` (not owned), `Crc32cDigester` (owned),\n// `AnyDigester<uint32_t>` (maybe owned).\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `Reader`. `Src` must support\n// `Dependency<Reader*, Src&&>`, e.g. `Reader&` (not owned),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// The digest is converted to `DesiredDigestType` using `DigestConverter`.\ntemplate <typename DesiredDigestType = digest_converter_internal::NoConversion,\n          typename Digester, typename Src,\n          std::enable_if_t<\n              std::conjunction_v<\n                  TargetRefSupportsDependency<DigesterBaseHandle, Digester>,\n                  TargetRefSupportsDependency<Reader*, Src>,\n                  digest_converter_internal::HasDigestConverterOrNoConversion<\n                      DigestOf<TargetRefT<Digester>>, DesiredDigestType>>,\n              int> = 0>\nStatusOrMakerT<digest_converter_internal::ResolveNoConversion<\n    DigestOf<TargetRefT<Digester>>, DesiredDigestType>>\nDigestFromReader(Src&& src, Digester&& digester,\n                 Position* length_read = nullptr);\n\n// Implementation details follow.\n\ninline DigestingReaderBase::DigestingReaderBase(\n    DigestingReaderBase&& that) noexcept\n    : Reader(static_cast<Reader&&>(that)) {}\n\ninline DigestingReaderBase& DigestingReaderBase::operator=(\n    DigestingReaderBase&& that) noexcept {\n  Reader::operator=(static_cast<Reader&&>(that));\n  return *this;\n}\n\ninline void DigestingReaderBase::Initialize(Reader* src,\n                                            DigesterBaseHandle digester) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of DigestingReader: null Reader pointer\";\n  MakeBuffer(*src);\n  absl::Status status = digester.status();\n  if (ABSL_PREDICT_FALSE(!status.ok())) Fail(std::move(status));\n}\n\ninline bool DigestingReaderBase::SyncBuffer(Reader& src) {\n  RIEGELI_ASSERT_EQ(start(), src.cursor())\n      << \"Failed invariant of DigestingReaderBase: \"\n         \"cursor of the original Reader changed unexpectedly\";\n  if (start_to_cursor() > 0) {\n    if (ABSL_PREDICT_FALSE(\n            !WriteToDigester(absl::string_view(start(), start_to_cursor())))) {\n      RIEGELI_EVAL_ASSERT(!FailFromDigester());\n      return false;\n    }\n    src.set_cursor(cursor());\n  }\n  return true;\n}\n\ninline void DigestingReaderBase::MakeBuffer(Reader& src) {\n  set_buffer(src.cursor(), src.available());\n  set_limit_pos(src.limit_pos());\n  if (ABSL_PREDICT_FALSE(!src.ok())) FailWithoutAnnotation(src.status());\n}\n\ntemplate <typename Digester, typename Dest>\nclass DigestingReader<Digester, Dest>::Mover {\n public:\n  static auto member() { return &DigestingReader::src_; }\n\n  explicit Mover(DigestingReader& self, DigestingReader& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `src_` is not moved yet so `src_` is taken from `that`.\n    if (uses_buffer_) {\n      if (ABSL_PREDICT_FALSE(!self.SyncBuffer(*that.src_))) {\n        uses_buffer_ = false;\n      }\n    }\n  }\n\n  void Done(DigestingReader& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.src_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Digester, typename Src>\ninline DigestingReader<Digester, Src>::DigestingReader(\n    Initializer<Src> src, Initializer<Digester> digester)\n    : digester_(std::move(digester)), src_(std::move(src)) {\n  Initialize(src_.get(), digester_.get());\n}\n\ntemplate <typename Digester, typename Src>\ninline void DigestingReader<Digester, Src>::Reset(Closed) {\n  DigestingReaderBase::Reset(kClosed);\n  digester_.Reset();\n  src_.Reset();\n}\n\ntemplate <typename Digester, typename Src>\ninline void DigestingReader<Digester, Src>::Reset(\n    Initializer<Src> src, Initializer<Digester> digester) {\n  DigestingReaderBase::Reset();\n  digester_.Reset(std::move(digester));\n  src_.Reset(std::move(src));\n  Initialize(src_.get(), digester_.get());\n}\n\ntemplate <typename Digester, typename Src>\nvoid DigestingReader<Digester, Src>::Done() {\n  DigestingReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(src_->status());\n    }\n  }\n  if (digester_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!digester_.get().Close())) FailFromDigester();\n  }\n}\n\ntemplate <typename Digester, typename Src>\nvoid DigestingReader<Digester, Src>::SetReadAllHintImpl(bool read_all_hint) {\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!SyncBuffer(*src_))) return;\n    src_->SetReadAllHint(read_all_hint);\n    MakeBuffer(*src_);\n  }\n}\n\ntemplate <typename Digester, typename Src>\nvoid DigestingReader<Digester, Src>::VerifyEndImpl() {\n  if (!src_.IsOwning()) {\n    DigestingReaderBase::VerifyEndImpl();\n  } else if (ABSL_PREDICT_TRUE(ok())) {\n    if (ABSL_PREDICT_FALSE(!SyncBuffer(*src_))) return;\n    src_->VerifyEnd();\n    MakeBuffer(*src_);\n  }\n}\n\ntemplate <typename Digester, typename Src>\nbool DigestingReader<Digester, Src>::SyncImpl(SyncType sync_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(*src_))) return false;\n  bool sync_ok = true;\n  if (sync_type != SyncType::kFromObject || src_.IsOwning()) {\n    sync_ok = src_->Sync(sync_type);\n  }\n  MakeBuffer(*src_);\n  return sync_ok;\n}\n\ntemplate <typename DesiredDigestType, typename Digester, typename Src,\n          std::enable_if_t<\n              std::conjunction_v<\n                  TargetRefSupportsDependency<DigesterBaseHandle, Digester>,\n                  TargetRefSupportsDependency<Reader*, Src>,\n                  digest_converter_internal::HasDigestConverterOrNoConversion<\n                      DigestOf<TargetRefT<Digester>>, DesiredDigestType>>,\n              int>>\ninline StatusOrMakerT<digest_converter_internal::ResolveNoConversion<\n    DigestOf<TargetRefT<Digester>>, DesiredDigestType>>\nDigestFromReader(Src&& src, Digester&& digester, Position* length_read) {\n  using DigestType = digest_converter_internal::ResolveNoConversion<\n      DigestOf<TargetRefT<Digester>>, DesiredDigestType>;\n  DigestingReader<TargetRefT<Digester>, TargetRefT<Src>> reader(\n      std::forward<Src>(src), std::forward<Digester>(digester));\n  reader.SetReadAllHint(true);\n  const Position pos_before = reader.pos();\n  do {\n    reader.move_cursor(reader.available());\n  } while (reader.Pull());\n  RIEGELI_ASSERT_GE(reader.pos(), pos_before)\n      << \"DigestingReader decreased src.pos()\";\n  if (length_read != nullptr) *length_read = reader.pos() - pos_before;\n  if (ABSL_PREDICT_FALSE(!reader.VerifyEndAndClose())) {\n    return StatusOrMaker<DigestType>::FromStatus(reader.status());\n  }\n  return StatusOrMaker<DigestType>::FromWork(\n      [&]() -> DigestType { return reader.template Digest<DigestType>(); });\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_DIGESTING_READER_H_\n"
  },
  {
    "path": "riegeli/digests/digesting_writer.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/digests/digesting_writer.h\"\n\n#include <stddef.h>\n\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/digests/digester_handle.h\"\n\nnamespace riegeli {\n\nnamespace digesting_writer_internal {\n\nabsl::Status FailedStatus(DigesterBaseHandle digester) {\n  absl::Status status = digester.status();\n  if (status.ok()) status = absl::UnknownError(\"Digester failed\");\n  return status;\n}\n\n}  // namespace digesting_writer_internal\n\nbool DigestingWriterBase::FailFromDigester() {\n  const DigesterBaseHandle digester = GetDigester();\n  return Fail(digesting_writer_internal::FailedStatus(digester));\n}\n\nvoid DigestingWriterBase::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) {\n    Writer& dest = *DestWriter();\n    SyncBuffer(dest);\n  }\n  Writer::Done();\n}\n\nabsl::Status DigestingWriterBase::AnnotateStatusImpl(absl::Status status) {\n  // Fully delegate annotations to `*DestWriter()`.\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    const bool sync_buffer_ok = SyncBuffer(dest);\n    status = dest.AnnotateStatus(std::move(status));\n    if (ABSL_PREDICT_TRUE(sync_buffer_ok)) MakeBuffer(dest);\n  }\n  return status;\n}\n\nbool DigestingWriterBase::PushSlow(size_t min_length,\n                                   size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return false;\n  const bool push_ok = dest.Push(min_length, recommended_length);\n  MakeBuffer(dest);\n  return push_ok;\n}\n\nbool DigestingWriterBase::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of Writer::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return false;\n  if (ABSL_PREDICT_FALSE(!WriteToDigester(src))) return FailFromDigester();\n  const bool write_ok = dest.Write(src);\n  MakeBuffer(dest);\n  return write_ok;\n}\n\nbool DigestingWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool DigestingWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  return WriteInternal(src);\n}\n\nbool DigestingWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool DigestingWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  return WriteInternal(src);\n}\n\nbool DigestingWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  return WriteInternal(std::move(src));\n}\n\nbool DigestingWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  return WriteInternal(src);\n}\n\ntemplate <typename Src>\ninline bool DigestingWriterBase::WriteInternal(Src&& src) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return false;\n  DigesterBaseHandle digester = GetDigester();\n  if (ABSL_PREDICT_FALSE(!digester.Write(src))) return FailFromDigester();\n  const bool write_ok = dest.Write(std::forward<Src>(src));\n  MakeBuffer(dest);\n  return write_ok;\n}\n\nbool DigestingWriterBase::SupportsReadMode() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsReadMode();\n}\n\nReader* DigestingWriterBase::ReadModeImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(dest))) return nullptr;\n  Reader* const reader = dest.ReadMode(initial_pos);\n  MakeBuffer(dest);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/digests/digesting_writer.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_DIGESTING_WRITER_H_\n#define RIEGELI_DIGESTS_DIGESTING_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <tuple>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/null_writer.h\"\n#include \"riegeli/bytes/stringify.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/digests/digest_converter.h\"\n#include \"riegeli/digests/digester_handle.h\"\n\nnamespace riegeli {\n\nclass Reader;\n\n// Template parameter independent part of `DigestingWriter`.\nclass DigestingWriterBase : public Writer {\n public:\n  // Returns the `DigesterBaseHandle`. Unchanged by `Close()`.\n  virtual DigesterBaseHandle GetDigester() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the original `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsReadMode() override;\n\n protected:\n  using Writer::Writer;\n\n  DigestingWriterBase(DigestingWriterBase&& that) noexcept;\n  DigestingWriterBase& operator=(DigestingWriterBase&& that) noexcept;\n\n  void Initialize(Writer* dest, DigesterBaseHandle digester);\n  ABSL_ATTRIBUTE_COLD bool FailFromDigester();\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n\n  virtual bool WriteToDigester(absl::string_view src) = 0;\n\n  // Sets cursor of `dest` to cursor of `*this`, digesting what has been written\n  // to the buffer (until `cursor()`).\n  bool SyncBuffer(Writer& dest);\n\n  // Sets buffer pointers of `*this` to buffer pointers of `dest`, adjusting\n  // `start()` to hide data already digested. Fails `*this` if `dest` failed.\n  void MakeBuffer(Writer& dest);\n\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  Reader* ReadModeImpl(Position initial_pos) override;\n\n private:\n  // This template is defined and used only in digesting_writer.cc.\n  template <typename Src>\n  bool WriteInternal(Src&& src);\n\n  // Invariants if `ok()`:\n  //   `start() == DestWriter()->cursor()`\n  //   `limit() == DestWriter()->limit()`\n  //   `start_pos() == DestWriter()->pos()`\n};\n\n// A `Writer` which writes to another `Writer`, and lets another object observe\n// data being written and return some data called a digest, e.g. a checksum.\n//\n// The `Digester` template parameter specifies the type of the object providing\n// and possibly owning the digester. `Digester` must support\n// `Dependency<DigesterBaseHandle, Digester>`, e.g.\n// `DigesterHandle<uint32_t>` (not owned), `Crc32cDigester` (owned),\n// `AnyDigester<uint32_t>>` (maybe owned).\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the original `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the `Digester` template argument can be deduced as\n// `TargetT` of the type of the `digester` constructor argument, and the `Dest`\n// template argument can be deduced as `TargetT` of the type of the `dest`\n// constructor argument, or as `NullWriter` if the `dest` constructor argument\n// is omitted.\n//\n// The original `Writer` must not be accessed until the `DigestingWriter` is\n// closed or no longer used, except that it is allowed to read the destination\n// of the original `Writer` immediately after `Flush()`.\ntemplate <typename Digester, typename Dest = Writer*>\nclass DigestingWriter : public DigestingWriterBase {\n public:\n  // The type of the digest.\n  using DigestType = DigestOf<Digester>;\n\n  // Creates a closed `DigestingWriter`.\n  explicit DigestingWriter(Closed) noexcept : DigestingWriterBase(kClosed) {}\n\n  // Will write to the original `Writer` provided by `dest`, using the\n  // digester provided by `digester`.\n  //\n  // If `Dest` is `NullWriter` and one constructor parameter is given,\n  // it is interpreted as `digester` (overload below), not `dest`.\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<!std::is_same_v<DependentDest, NullWriter>, int> = 0>\n  explicit DigestingWriter(Initializer<Dest> dest);\n  explicit DigestingWriter(Initializer<Dest> dest,\n                           Initializer<Digester> digester);\n\n  // In the common case of `DigestingWriter<Digester, NullWriter>`,\n  // an initializer for the `NullWriter` can be omitted.\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<std::is_same_v<DependentDest, NullWriter>, int> = 0>\n  explicit DigestingWriter(Initializer<Digester> digester = riegeli::Maker());\n\n  DigestingWriter(DigestingWriter&& that) = default;\n  DigestingWriter& operator=(DigestingWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `DigestingWriter`. This\n  // avoids constructing a temporary `DigestingWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<!std::is_same_v<DependentDest, NullWriter>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Initializer<Digester> digester);\n  template <\n      typename DependentDest = Dest,\n      std::enable_if_t<std::is_same_v<DependentDest, NullWriter>, int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      Initializer<Digester> digester = riegeli::Maker());\n\n  // Digests buffered data if needed, and returns the digest.\n  //\n  // The digest is converted to `DesiredDigestType` using `DigestConverter`.\n  //\n  // Depending on the digester, `Digest()` can be more efficient if `Close()` is\n  // called before.\n  //\n  // Many digesters support calling `Digest()` and then accepting more data or\n  // calling `Digest()` again, but this is not guaranteed.\n  template <\n      typename DesiredDigestType = DigestType,\n      std::enable_if_t<HasDigestConverter<DigestType, DesiredDigestType>::value,\n                       int> = 0>\n  DesiredDigestType Digest() {\n    if (start_to_cursor() > 0) {\n      if (ABSL_PREDICT_FALSE(!digester_.get().Write(\n              absl::string_view(start(), start_to_cursor())))) {\n        FailFromDigester();\n      } else {\n        set_start_pos(pos());\n        set_buffer(cursor(), available());\n        dest_->set_cursor(cursor());\n      }\n    }\n    return digester_.get().template Digest<DesiredDigestType>();\n  }\n\n  // Returns the object providing and possibly owning the digester. Unchanged by\n  // `Close()`.\n  Digester& digester() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return digester_.manager();\n  }\n  const Digester& digester() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return digester_.manager();\n  }\n  DigesterBaseHandle GetDigester() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return digester_.get();\n  }\n\n  // Returns the object providing and possibly owning the original `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  bool WriteToDigester(absl::string_view src) override {\n    return digester_.get().Write(src);\n  }\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the digester.\n  Dependency<DigesterBaseHandle, Digester> digester_;\n  // The object providing and possibly owning the original `Writer`.\n  MovingDependency<Writer*, Dest, Mover> dest_;\n};\n\nexplicit DigestingWriter(Closed) -> DigestingWriter<void, DeleteCtad<Closed>>;\ntemplate <typename Digester, typename Dest>\nexplicit DigestingWriter(Dest&& dest, Digester&& digester)\n    -> DigestingWriter<TargetT<Digester>, TargetT<Dest>>;\ntemplate <typename Digester>\nexplicit DigestingWriter(Digester&& digester)\n    -> DigestingWriter<TargetT<Digester>, NullWriter>;\n\n// Returns the digest of the concatenation of stringifiable values.\n//\n// The last argument is the digester of some type `Digester`. The remaining\n// arguments are the values.\n//\n// `Digester` specifies the type of the object providing and possibly owning\n// the digester. `Digester` must support\n// `DependencyRef<DigesterBaseHandle, Digester>` and must provide a member\n// function `DigestType Digest()` for some `DigestType`, e.g.\n// `DigesterHandle<uint32_t>` (not owned), `Crc32cDigester` (owned),\n// `AnyDigester<uint32_t>>` (maybe owned).\n//\n// The digester should not be expected to fail. If it fails, the process\n// terminates.\n//\n// The digest is converted to `DesiredDigestType` using `DigestConverter`.\ntemplate <typename DesiredDigestType = digest_converter_internal::NoConversion,\n          typename... Args,\n          std::enable_if_t<\n              std::conjunction_v<\n                  TargetRefSupportsDependency<DigesterBaseHandle,\n                                              GetTypeFromEndT<1, Args...>>,\n                  TupleElementsSatisfy<RemoveTypesFromEndT<1, Args&&...>,\n                                       IsStringifiable>,\n                  digest_converter_internal::HasDigestConverterOrNoConversion<\n                      DigestOf<TargetRefT<GetTypeFromEndT<1, Args...>>>,\n                      DesiredDigestType>>,\n              int> = 0>\ndigest_converter_internal::ResolveNoConversion<\n    DigestOf<TargetRefT<GetTypeFromEndT<1, Args...>>>, DesiredDigestType>\nDigestFrom(Args&&... args);\n\n// Implementation details follow.\n\ninline DigestingWriterBase::DigestingWriterBase(\n    DigestingWriterBase&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)) {}\n\ninline DigestingWriterBase& DigestingWriterBase::operator=(\n    DigestingWriterBase&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  return *this;\n}\n\ninline void DigestingWriterBase::Initialize(Writer* dest,\n                                            DigesterBaseHandle digester) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of DigestingWriter: null Writer pointer\";\n  MakeBuffer(*dest);\n  absl::Status status = digester.status();\n  if (ABSL_PREDICT_FALSE(!status.ok())) Fail(std::move(status));\n}\n\ninline bool DigestingWriterBase::SyncBuffer(Writer& dest) {\n  RIEGELI_ASSERT_EQ(start(), dest.cursor())\n      << \"Failed invariant of DigestingWriterBase: \"\n         \"cursor of the original Writer changed unexpectedly\";\n  if (start_to_cursor() > 0) {\n    if (ABSL_PREDICT_FALSE(\n            !WriteToDigester(absl::string_view(start(), start_to_cursor())))) {\n      RIEGELI_EVAL_ASSERT(!FailFromDigester());\n      return false;\n    }\n    dest.set_cursor(cursor());\n  }\n  return true;\n}\n\ninline void DigestingWriterBase::MakeBuffer(Writer& dest) {\n  set_buffer(dest.cursor(), dest.available());\n  set_start_pos(dest.pos());\n  if (ABSL_PREDICT_FALSE(!dest.ok())) FailWithoutAnnotation(dest.status());\n}\n\ntemplate <typename Digester, typename Dest>\nclass DigestingWriter<Digester, Dest>::Mover {\n public:\n  static auto member() { return &DigestingWriter::dest_; }\n\n  explicit Mover(DigestingWriter& self, DigestingWriter& that)\n      : uses_buffer_(self.start() != nullptr) {\n    // Buffer pointers are already moved so `SyncBuffer()` is called on `self`.\n    // `dest_` is not moved yet so `dest_` is taken from `that`.\n    if (uses_buffer_) {\n      if (ABSL_PREDICT_FALSE(!self.SyncBuffer(*that.dest_))) {\n        uses_buffer_ = false;\n      }\n    }\n  }\n\n  void Done(DigestingWriter& self) {\n    if (uses_buffer_) self.MakeBuffer(*self.dest_);\n  }\n\n private:\n  bool uses_buffer_;\n};\n\ntemplate <typename Digester, typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<!std::is_same_v<DependentDest, NullWriter>, int>>\ninline DigestingWriter<Digester, Dest>::DigestingWriter(Initializer<Dest> dest)\n    : DigestingWriter(std::move(dest), riegeli::Maker()) {}\n\ntemplate <typename Digester, typename Dest>\ninline DigestingWriter<Digester, Dest>::DigestingWriter(\n    Initializer<Dest> dest, Initializer<Digester> digester)\n    : digester_(std::move(digester)), dest_(std::move(dest)) {\n  Initialize(dest_.get(), digester_.get());\n}\n\ntemplate <typename Digester, typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_same_v<DependentDest, NullWriter>, int>>\ninline DigestingWriter<Digester, Dest>::DigestingWriter(\n    Initializer<Digester> digester)\n    : DigestingWriter(riegeli::Maker(), std::move(digester)) {}\n\ntemplate <typename Digester, typename Dest>\ninline void DigestingWriter<Digester, Dest>::Reset(Closed) {\n  DigestingWriterBase::Reset(kClosed);\n  digester_.Reset();\n  dest_.Reset();\n}\n\ntemplate <typename Digester, typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<!std::is_same_v<DependentDest, NullWriter>, int>>\ninline void DigestingWriter<Digester, Dest>::Reset(Initializer<Dest> dest) {\n  Reset(std::move(dest), riegeli::Maker());\n}\n\ntemplate <typename Digester, typename Dest>\ninline void DigestingWriter<Digester, Dest>::Reset(\n    Initializer<Dest> dest, Initializer<Digester> digester) {\n  DigestingWriterBase::Reset();\n  digester_.Reset(std::move(digester));\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), digester_.get());\n}\n\ntemplate <typename Digester, typename Dest>\ntemplate <typename DependentDest,\n          std::enable_if_t<std::is_same_v<DependentDest, NullWriter>, int>>\ninline void DigestingWriter<Digester, Dest>::Reset(\n    Initializer<Digester> digester) {\n  Reset(riegeli::Maker(), std::move(digester));\n}\n\ntemplate <typename Digester, typename Dest>\nvoid DigestingWriter<Digester, Dest>::Done() {\n  DigestingWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(dest_->status());\n    }\n  }\n  if (digester_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!digester_.get().Close())) FailFromDigester();\n  }\n}\n\ntemplate <typename Digester, typename Dest>\nvoid DigestingWriter<Digester, Dest>::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!SyncBuffer(*dest_))) return;\n    dest_->SetWriteSizeHint(write_size_hint);\n    if (digester_.IsOwning()) digester_.get().SetWriteSizeHint(write_size_hint);\n    MakeBuffer(*dest_);\n  } else if (digester_.IsOwning()) {\n    digester_.get().SetWriteSizeHint(\n        write_size_hint == std::nullopt\n            ? std::nullopt\n            : std::make_optional(SaturatingAdd(Position{start_to_cursor()},\n                                               *write_size_hint)));\n  }\n}\n\ntemplate <typename Digester, typename Dest>\nbool DigestingWriter<Digester, Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!SyncBuffer(*dest_))) return false;\n  bool flush_ok = true;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    flush_ok = dest_->Flush(flush_type);\n  }\n  MakeBuffer(*dest_);\n  return flush_ok;\n}\n\nnamespace digesting_writer_internal {\n\nABSL_ATTRIBUTE_COLD absl::Status FailedStatus(DigesterBaseHandle digester);\n\ntemplate <typename T, typename Enable = void>\nstruct IsDigestible : std::false_type {};\n\ntemplate <typename T>\nstruct IsDigestible<\n    T, std::void_t<decltype(std::declval<DigesterBaseHandle&>().Write(\n           std::declval<const T&>()))>> : std::true_type {};\n\ntemplate <typename DesiredDigestType, typename Digester, typename... Srcs,\n          std::enable_if_t<std::conjunction_v<IsDigestible<Srcs>...>, int> = 0>\ninline DesiredDigestType DigestFromImpl(std::tuple<Srcs...> srcs,\n                                        Digester&& digester) {\n  DependencyRef<DigesterBaseHandle, Digester> digester_dep(\n      std::forward<Digester>(digester));\n  if constexpr (HasStringifiedSize<Srcs...>::value) {\n    if (digester_dep.IsOwning()) {\n      digester_dep.get().SetWriteSizeHint(std::apply(\n          [](const Srcs&... srcs) { return riegeli::StringifiedSize(srcs...); },\n          srcs));\n    }\n  }\n  bool ok = std::apply(\n      [&](const Srcs&... srcs) {\n        return (digester_dep.get().Write(srcs) && ...);\n      },\n      srcs);\n  if (digester_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!digester_dep.get().Close())) ok = false;\n  }\n  RIEGELI_CHECK(ok) << FailedStatus(digester_dep.get());\n  return digester_dep.get().template Digest<DesiredDigestType>();\n}\n\ntemplate <typename DesiredDigestType, typename Digester, typename... Srcs,\n          std::enable_if_t<!std::conjunction_v<IsDigestible<Srcs>...>, int> = 0>\ninline DesiredDigestType DigestFromImpl(std::tuple<Srcs...> srcs,\n                                        Digester&& digester) {\n  DigestingWriter<TargetRefT<Digester>, NullWriter> writer(\n      std::forward<Digester>(digester));\n  if constexpr (HasStringifiedSize<Srcs...>::value) {\n    writer.SetWriteSizeHint(std::apply(\n        [](const Srcs&... srcs) { return riegeli::StringifiedSize(srcs...); },\n        srcs));\n  }\n  std::apply([&](const Srcs&... srcs) { writer.Write(srcs...); }, srcs);\n  RIEGELI_CHECK(writer.Close()) << writer.status();\n  return writer.template Digest<DesiredDigestType>();\n}\n\n}  // namespace digesting_writer_internal\n\ntemplate <typename DesiredDigestType, typename... Args,\n          std::enable_if_t<\n              std::conjunction_v<\n                  TargetRefSupportsDependency<DigesterBaseHandle,\n                                              GetTypeFromEndT<1, Args...>>,\n                  TupleElementsSatisfy<RemoveTypesFromEndT<1, Args&&...>,\n                                       IsStringifiable>,\n                  digest_converter_internal::HasDigestConverterOrNoConversion<\n                      DigestOf<TargetRefT<GetTypeFromEndT<1, Args...>>>,\n                      DesiredDigestType>>,\n              int>>\ndigest_converter_internal::ResolveNoConversion<\n    DigestOf<TargetRefT<GetTypeFromEndT<1, Args...>>>, DesiredDigestType>\nDigestFrom(Args&&... args) {\n  return digesting_writer_internal::DigestFromImpl<\n      digest_converter_internal::ResolveNoConversion<\n          DigestOf<TargetRefT<GetTypeFromEndT<1, Args...>>>,\n          DesiredDigestType>>(RemoveFromEnd<1>(std::forward<Args>(args)...),\n                              GetFromEnd<1>(std::forward<Args>(args)...));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_DIGESTING_WRITER_H_\n"
  },
  {
    "path": "riegeli/digests/highwayhash_digester.cc",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/digests/highwayhash_digester.h\"\n\n#include \"highwayhash/hh_types.h\"\n\nnamespace riegeli {\n\ntemplate <>\nalignas(32) const HighwayHashKey\n    HighwayHashDigester<highwayhash::HHResult64>::kDefaultKey = {\n        0x4ea9929a25d561c6,\n        0x98470d187b523e8f,\n        0x592040a2da3c4b53,\n        0xbff8b246e3c587a2,\n};\n\ntemplate <>\nalignas(32) const HighwayHashKey\n    HighwayHashDigester<highwayhash::HHResult128>::kDefaultKey = {\n        0x025ed8a16fb5f783,\n        0xb44bc74d89d26c86,\n        0x111ea964039fa769,\n        0x6f7d7159e15612b6,\n};\n\ntemplate <>\nalignas(32) const HighwayHashKey\n    HighwayHashDigester<highwayhash::HHResult256>::kDefaultKey = {\n        0x93fee04321119357,\n        0x21e397ea62c264b6,\n        0x9d856914f2ad0e15,\n        0x64dca6f86247f384,\n};\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/digests/highwayhash_digester.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_HIGHWAYHASH_DIGESTER_H_\n#define RIEGELI_DIGESTS_HIGHWAYHASH_DIGESTER_H_\n\n#include <stddef.h>\n\n#include <array>\n#include <type_traits>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/meta/type_traits.h\"\n#include \"absl/strings/string_view.h\"\n#include \"highwayhash/arch_specific.h\"\n#include \"highwayhash/hh_types.h\"\n#include \"highwayhash/highwayhash.h\"\n\nnamespace riegeli {\n\n// `uint64_t[4]`; prefer `alignas(32)` for constants of this type.\nusing HighwayHashKey = highwayhash::HHKey;\n\ntemplate <typename ResultType>\nclass HighwayHashDigester {\n private:\n  using DigestType =\n      std::conditional_t<std::is_array_v<ResultType>,\n                         const std::array<std::remove_extent_t<ResultType>,\n                                          std::extent_v<ResultType>>&,\n                         ResultType>;\n\n public:\n  // The default keys were chosen once with `openssl rand`.\n  alignas(32) static const HighwayHashKey kDefaultKey;\n\n  explicit HighwayHashDigester(const HighwayHashKey& key = kDefaultKey)\n      : cat_(key) {}\n\n  HighwayHashDigester(const HighwayHashDigester& that) = default;\n  HighwayHashDigester& operator=(const HighwayHashDigester& that) = default;\n\n  ABSL_ATTRIBUTE_REINITIALIZES\n  void Reset(const HighwayHashKey& key = kDefaultKey) {\n    cat_.Reset(key);\n    is_open_ = true;\n  }\n\n  void Write(absl::string_view chunk) {\n    cat_.Append(chunk.data(), chunk.size());\n  }\n\n  void Close() {\n    if (is_open_) {\n      cat_.Finalize(reinterpret_cast<ResultType*>(&digest_));\n      is_open_ = false;\n    }\n  }\n\n  DigestType Digest() {\n    if (is_open_) {\n      highwayhash::HighwayHashCatT<HH_TARGET>(cat_).Finalize(\n          reinterpret_cast<ResultType*>(&digest_));\n    }\n    return digest_;\n  }\n\n private:\n  highwayhash::HighwayHashCatT<HH_TARGET> cat_;\n  absl::remove_cvref_t<DigestType> digest_;\n  bool is_open_ = true;\n};\n\nusing HighwayHash64Digester = HighwayHashDigester<highwayhash::HHResult64>;\nusing HighwayHash128Digester = HighwayHashDigester<highwayhash::HHResult128>;\nusing HighwayHash256Digester = HighwayHashDigester<highwayhash::HHResult256>;\n\n// Implementation details follow.\n\ntemplate <>\nalignas(32) const HighwayHashKey\n    HighwayHashDigester<highwayhash::HHResult64>::kDefaultKey;\n\ntemplate <>\nalignas(32) const HighwayHashKey\n    HighwayHashDigester<highwayhash::HHResult128>::kDefaultKey;\n\ntemplate <>\nalignas(32) const HighwayHashKey\n    HighwayHashDigester<highwayhash::HHResult256>::kDefaultKey;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_HIGHWAYHASH_DIGESTER_H_\n"
  },
  {
    "path": "riegeli/digests/md5_digester.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_MD5_DIGESTER_H_\n#define RIEGELI_DIGESTS_MD5_DIGESTER_H_\n\n#include \"openssl/base.h\"\n#include \"openssl/md5.h\"\n#include \"riegeli/digests/openssl_digester.h\"\n\nnamespace riegeli {\n\n// A digester computing MD5 checksums, for `DigestingReader` and\n// `DigestingWriter`.\n//\n// Warning: MD5 as a cryptographic hash function is broken.\n// Use this only if a preexisting format has already decided to use MD5.\n// Please contact ise-team@ in case of doubt.\nusing Md5Digester = OpenSslDigester<MD5_CTX, MD5_Init, MD5_Update, MD5_Final,\n                                    MD5_DIGEST_LENGTH>;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_MD5_DIGESTER_H_\n"
  },
  {
    "path": "riegeli/digests/openssl_digester.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_OPENSSL_DIGESTER_H_\n#define RIEGELI_DIGESTS_OPENSSL_DIGESTER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <array>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/string_view.h\"\n\nnamespace riegeli {\n\n// A digester template computing checksums implemented by OpenSSL, for\n// `DigestingReader` and `DigestingWriter`.\ntemplate <typename H, int (*init)(H*), int (*update)(H*, const void*, size_t),\n          int (*final)(uint8_t*, H*), int digest_size>\nclass OpenSslDigester {\n public:\n  OpenSslDigester() { init(&ctx_); }\n\n  OpenSslDigester(const OpenSslDigester& that) = default;\n  OpenSslDigester& operator=(const OpenSslDigester& that) = default;\n\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset() {\n    init(&ctx_);\n    is_open_ = true;\n  }\n\n  void Write(absl::string_view src) { update(&ctx_, src.data(), src.size()); }\n\n  void Close() {\n    if (is_open_) {\n      final(reinterpret_cast<uint8_t*>(digest_.data()), &ctx_);\n      is_open_ = false;\n    }\n  }\n\n  std::array<char, digest_size> Digest() {\n    if (is_open_) {\n      H copy = ctx_;\n      final(reinterpret_cast<uint8_t*>(digest_.data()), &copy);\n    }\n    return digest_;\n  }\n\n private:\n  H ctx_;\n  std::array<char, digest_size> digest_;\n  bool is_open_ = true;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_OPENSSL_DIGESTER_H_\n"
  },
  {
    "path": "riegeli/digests/sha1_digester.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_SHA1_DIGESTER_H_\n#define RIEGELI_DIGESTS_SHA1_DIGESTER_H_\n\n#include \"openssl/base.h\"\n#include \"openssl/sha.h\"\n#include \"riegeli/digests/openssl_digester.h\"\n\nnamespace riegeli {\n\n// A digester computing SHA-1 checksums, for `DigestingReader` and\n// `DigestingWriter`.\n//\n// Warning: SHA-1 as a cryptographic hash function is broken.\n// Use this only if a preexisting format has already decided to use SHA-1.\n// Please contact ise-team@ in case of doubt.\nusing Sha1Digester = OpenSslDigester<SHA_CTX, SHA1_Init, SHA1_Update,\n                                     SHA1_Final, SHA_DIGEST_LENGTH>;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_SHA1_DIGESTER_H_\n"
  },
  {
    "path": "riegeli/digests/sha256_digester.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_SHA256_DIGESTER_H_\n#define RIEGELI_DIGESTS_SHA256_DIGESTER_H_\n\n#include \"openssl/base.h\"\n#include \"openssl/sha.h\"\n#include \"riegeli/digests/openssl_digester.h\"\n\nnamespace riegeli {\n\n// A digester computing SHA-256 checksums, for `DigestingReader` and\n// `DigestingWriter`.\nusing Sha256Digester = OpenSslDigester<SHA256_CTX, SHA256_Init, SHA256_Update,\n                                       SHA256_Final, SHA256_DIGEST_LENGTH>;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_SHA256_DIGESTER_H_\n"
  },
  {
    "path": "riegeli/digests/sha512_256_digester.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_SHA512_256_DIGESTER_H_\n#define RIEGELI_DIGESTS_SHA512_256_DIGESTER_H_\n\n#include \"openssl/base.h\"\n#include \"openssl/sha.h\"\n#include \"riegeli/digests/openssl_digester.h\"\n\nnamespace riegeli {\n\n// A digester computing SHA-512/256 checksums, for `DigestingReader` and\n// `DigestingWriter`.\nusing Sha512_256Digester =\n    OpenSslDigester<SHA512_CTX, SHA512_256_Init, SHA512_256_Update,\n                    SHA512_256_Final, SHA512_256_DIGEST_LENGTH>;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_SHA512_256_DIGESTER_H_\n"
  },
  {
    "path": "riegeli/digests/sha512_digester.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_SHA512_DIGESTER_H_\n#define RIEGELI_DIGESTS_SHA512_DIGESTER_H_\n\n#include \"openssl/base.h\"\n#include \"openssl/sha.h\"\n#include \"riegeli/digests/openssl_digester.h\"\n\nnamespace riegeli {\n\n// A digester computing SHA-512 checksums, for `DigestingReader` and\n// `DigestingWriter`.\nusing Sha512Digester = OpenSslDigester<SHA512_CTX, SHA512_Init, SHA512_Update,\n                                       SHA512_Final, SHA512_DIGEST_LENGTH>;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_SHA512_DIGESTER_H_\n"
  },
  {
    "path": "riegeli/digests/wrapping_digester.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_DIGESTS_WRAPPING_DIGESTER_H_\n#define RIEGELI_DIGESTS_WRAPPING_DIGESTER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/digests/digest_converter.h\"\n#include \"riegeli/digests/digester_handle.h\"\n\nnamespace riegeli {\n\nnamespace wrapping_digester_internal {\n\n// The type of a function converting a digest, taking it by value, except that\n// conversion from `void` takes no parameters.\n\ntemplate <typename From, typename To, typename Enable = void>\nstruct DigestConverterFunction {\n  using type = To (*)(From);\n};\n\ntemplate <typename From, typename To>\nstruct DigestConverterFunction<From, To,\n                               std::enable_if_t<std::is_void_v<From>>> {\n  using type = To (*)();\n};\n\n}  // namespace wrapping_digester_internal\n\n// Wraps an object providing and possibly owning a digester in a concrete\n// digester type. Propagates calls to `Close()` if the base digester is owned.\n// Possibly converts the type of the digest returned by `Digest()`.\n//\n// `BaseDigester` must support `Dependency<DigesterBaseHandle, BaseDigester>`.\n//\n// `DigestType` is the new digest type, by default `DigestOf<BaseDigester>`,\n// i.e. unchanged.\n//\n// `digest_converter` is a function used to convert a digest, by default using\n// `DigestConverter`.\ntemplate <typename BaseDigester, typename DigestType = DigestOf<BaseDigester>,\n          typename wrapping_digester_internal::DigestConverterFunction<\n              DigestOf<BaseDigester>, DigestType>::type digest_converter =\n              nullptr>\nclass WrappingDigester {\n public:\n  // Default-constructs the `BaseDigester`.\n  template <\n      typename DependentBaseDigester = BaseDigester,\n      std::enable_if_t<std::is_default_constructible_v<DependentBaseDigester>,\n                       int> = 0>\n  WrappingDigester() : base_(riegeli::Maker()) {}\n\n  // Forwards constructor arguments to the `BaseDigester`.\n  template <\n      typename... Args,\n      std::enable_if_t<\n          std::conjunction_v<NotSameRef<WrappingDigester, TargetT<Args>...>,\n                             std::is_constructible<BaseDigester, Args&&...>>,\n          int> = 0>\n  explicit WrappingDigester(Args&&... args)\n      : base_(riegeli::Maker(std::forward<Args>(args)...)) {}\n\n  WrappingDigester(const WrappingDigester& that) = default;\n  WrappingDigester& operator=(const WrappingDigester& that) = default;\n\n  WrappingDigester(WrappingDigester&& that) = default;\n  WrappingDigester& operator=(WrappingDigester&& that) = default;\n\n  template <\n      typename... Args,\n      std::enable_if_t<\n          std::conjunction_v<NotSameRef<WrappingDigester, TargetT<Args>...>,\n                             std::is_constructible<BaseDigester, Args&&...>>,\n          int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Args&&... args) {\n    base_.Reset(riegeli::Maker(std::forward<Args>(args)...));\n  }\n\n  void SetWriteSizeHint(std::optional<Position> write_size_hint) {\n    if (base_.IsOwning()) base_.get().SetWriteSizeHint(write_size_hint);\n  }\n\n  bool Write(absl::string_view src) { return base_.get().Write(src); }\n  bool Write(const Chain& src) { return base_.get().Write(src); }\n  bool Write(const absl::Cord& src) { return base_.get().Write(src); }\n  bool Write(ByteFill src) { return base_.get().Write(src); }\n  bool Close() { return !base_.IsOwning() || base_.get().Close(); }\n\n  template <\n      typename DependentBaseDigester = BaseDigester,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::bool_constant<digest_converter == nullptr>,\n              HasDigestConverter<DigestOf<DependentBaseDigester>, DigestType>>,\n          int> = 0>\n  DigestType Digest() {\n    return base_.get().template Digest<DigestType>();\n  }\n  template <\n      typename DependentBaseDigester = BaseDigester,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::bool_constant<digest_converter != nullptr>,\n              std::negation<std::is_void<DigestOf<DependentBaseDigester>>>>,\n          int> = 0>\n  DigestType Digest() {\n    return digest_converter(base_.get().Digest());\n  }\n  template <\n      typename DependentBaseDigester = BaseDigester,\n      std::enable_if_t<\n          std::conjunction_v<std::bool_constant<digest_converter != nullptr>,\n                             std::is_void<DigestOf<DependentBaseDigester>>>,\n          int> = 0>\n  DigestType Digest() {\n    base_.get().Digest();\n    return digest_converter();\n  }\n\n private:\n  Dependency<DigesterBaseHandle, BaseDigester> base_;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_DIGESTS_WRAPPING_DIGESTER_H_\n"
  },
  {
    "path": "riegeli/endian/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"endian_reading\",\n    hdrs = [\"endian_reading.h\"],\n    deps = [\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:type_traits\",\n        \"//riegeli/bytes:reader\",\n        \"@com_google_absl//absl/base\",\n        \"@com_google_absl//absl/base:config\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:int128\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n\ncc_library(\n    name = \"endian_writing\",\n    hdrs = [\"endian_writing.h\"],\n    deps = [\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:type_traits\",\n        \"//riegeli/bytes:backward_writer\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base\",\n        \"@com_google_absl//absl/base:config\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:int128\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/endian/endian_reading.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_ENDIAN_ENDIAN_READING_H_\n#define RIEGELI_ENDIAN_ENDIAN_READING_H_\n\n#include <stdint.h>\n\n#include <cstring>\n\n#include \"absl/base/casts.h\"\n#include \"absl/base/config.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/int128.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\n// Reads a number in a fixed width Little/Big Endian encoding from an array.\n// The width of the encoding is determined by the template argument, which\n// must be one of: `{u,}int{8,16,32,64}_t`, `absl::{u,}int128`, `float`, or\n// `double`.\n//\n// Reads `sizeof(T)` bytes from `src[]`.\ntemplate <typename T>\nT ReadLittleEndian(const char* src);\ntemplate <typename T>\nT ReadBigEndian(const char* src);\n\n// Reads an array of numbers in a fixed width Little/Big Endian encoding from an\n// array. The width of the encoding is determined by the template argument,\n// which must be one of: `{u,}int{8,16,32,64}_t`, `absl::{u,}int128`, `float`,\n// or `double`.\n//\n// This is faster than reading them individually if the endianness matches the\n// native one.\n//\n// Reads `dest.size() * sizeof(T)` bytes from `src[]`.\ntemplate <typename T>\nvoid ReadLittleEndians(const char* src, absl::Span<type_identity_t<T>> dest);\ntemplate <typename T>\nvoid ReadBigEndians(const char* src, absl::Span<type_identity_t<T>> dest);\n\n// Reads a number in a fixed width Little/Big Endian encoding. The width of\n// the encoding is determined by the template argument, which must be one of:\n// `{u,}int{8,16,32,64}_t`, `absl::{u,}int128`, `float`, or `double`.\n//\n// Return values:\n//  * `true`                     - success (`dest` is set)\n//  * `false` (when `src.ok()`)  - source ends\n//                                 (`src` position is unchanged,\n//                                 `dest` is undefined)\n//  * `false` (when `!src.ok()`) - failure\n//                                 (`src` position is unchanged,\n//                                 `dest` is undefined)\ntemplate <typename T>\nbool ReadLittleEndian(Reader& src, type_identity_t<T>& dest);\ntemplate <typename T>\nbool ReadBigEndian(Reader& src, type_identity_t<T>& dest);\n\n// Reads an array of numbers in a fixed width Little/Big Endian encoding.\n// The width of the encoding is determined by the template argument, which\n// must be one of: `{u,}int{8,16,32,64}_t`, `absl::{u,}int128`, `float`, or\n// `double`.\n//\n// This is faster than reading them individually if the endianness matches the\n// native one.\n//\n// Return values:\n//  * `true`                     - success (`dest[]` is filled)\n//  * `false` (when `src.ok()`)  - source ends\n//                                 (`src` position is undefined,\n//                                 `dest[]` is undefined)\n//  * `false` (when `!src.ok()`) - failure\n//                                 (`src` position is undefined,\n//                                 `dest[]` is undefined)\ntemplate <typename T>\nbool ReadLittleEndians(Reader& src, absl::Span<type_identity_t<T>> dest);\ntemplate <typename T>\nbool ReadBigEndians(Reader& src, absl::Span<type_identity_t<T>> dest);\n\n// Implementation details follow.\n\ntemplate <>\ninline uint8_t ReadLittleEndian<uint8_t>(const char* src) {\n  return static_cast<uint8_t>(*src);\n}\n\ntemplate <>\ninline uint16_t ReadLittleEndian<uint16_t>(const char* src) {\n#if ABSL_IS_LITTLE_ENDIAN\n  uint16_t dest;\n  std::memcpy(&dest, src, sizeof(uint16_t));\n  return dest;\n#else\n  // `static_cast<uint16_t>` avoids triggering `-Wimplicit-int-conversion`:\n  // the result of `uint16_t | uint16_t` is `int` (if `uint16_t` is narrower\n  // than `int`).\n  return static_cast<uint16_t>(uint16_t{static_cast<uint8_t>(src[0])} |\n                               (uint16_t{static_cast<uint8_t>(src[1])} << 8));\n#endif\n}\n\ntemplate <>\ninline uint32_t ReadLittleEndian<uint32_t>(const char* src) {\n#if ABSL_IS_LITTLE_ENDIAN\n  uint32_t dest;\n  std::memcpy(&dest, src, sizeof(uint32_t));\n  return dest;\n#else\n  return uint32_t{static_cast<uint8_t>(src[0])} |\n         (uint32_t{static_cast<uint8_t>(src[1])} << 8) |\n         (uint32_t{static_cast<uint8_t>(src[2])} << (2 * 8)) |\n         (uint32_t{static_cast<uint8_t>(src[3])} << (3 * 8));\n#endif\n}\n\ntemplate <>\ninline uint64_t ReadLittleEndian<uint64_t>(const char* src) {\n#if ABSL_IS_LITTLE_ENDIAN\n  uint64_t dest;\n  std::memcpy(&dest, src, sizeof(uint64_t));\n  return dest;\n#else\n  return uint64_t{static_cast<uint8_t>(src[0])} |\n         (uint64_t{static_cast<uint8_t>(src[1])} << 8) |\n         (uint64_t{static_cast<uint8_t>(src[2])} << (2 * 8)) |\n         (uint64_t{static_cast<uint8_t>(src[3])} << (3 * 8)) |\n         (uint64_t{static_cast<uint8_t>(src[4])} << (4 * 8)) |\n         (uint64_t{static_cast<uint8_t>(src[5])} << (5 * 8)) |\n         (uint64_t{static_cast<uint8_t>(src[6])} << (6 * 8)) |\n         (uint64_t{static_cast<uint8_t>(src[7])} << (7 * 8));\n#endif\n}\n\ntemplate <>\ninline absl::uint128 ReadLittleEndian<absl::uint128>(const char* src) {\n#if ABSL_IS_LITTLE_ENDIAN\n  absl::uint128 dest;\n  std::memcpy(&dest, src, sizeof(absl::uint128));\n  return dest;\n#else\n  const uint64_t low = ReadLittleEndian<uint64_t>(src);\n  const uint64_t high = ReadLittleEndian<uint64_t>(src + sizeof(uint64_t));\n  return absl::MakeUint128(high, low);\n#endif\n}\n\ntemplate <>\ninline int8_t ReadLittleEndian<int8_t>(const char* src) {\n  return static_cast<int8_t>(*src);\n}\n\ntemplate <>\ninline int16_t ReadLittleEndian<int16_t>(const char* src) {\n  return static_cast<int16_t>(ReadLittleEndian<uint16_t>(src));\n}\n\ntemplate <>\ninline int32_t ReadLittleEndian<int32_t>(const char* src) {\n  return static_cast<int32_t>(ReadLittleEndian<uint32_t>(src));\n}\n\ntemplate <>\ninline int64_t ReadLittleEndian<int64_t>(const char* src) {\n  return static_cast<int64_t>(ReadLittleEndian<uint64_t>(src));\n}\n\ntemplate <>\ninline absl::int128 ReadLittleEndian<absl::int128>(const char* src) {\n  return static_cast<absl::int128>(ReadLittleEndian<absl::uint128>(src));\n}\n\ntemplate <>\ninline float ReadLittleEndian<float>(const char* src) {\n  return absl::bit_cast<float>(ReadLittleEndian<uint32_t>(src));\n}\n\ntemplate <>\ninline double ReadLittleEndian<double>(const char* src) {\n  return absl::bit_cast<double>(ReadLittleEndian<uint64_t>(src));\n}\n\ntemplate <>\ninline uint8_t ReadBigEndian<uint8_t>(const char* src) {\n  return static_cast<uint8_t>(*src);\n}\n\ntemplate <>\ninline uint16_t ReadBigEndian<uint16_t>(const char* src) {\n#if ABSL_IS_BIG_ENDIAN\n  uint16_t dest;\n  std::memcpy(&dest, src, sizeof(uint16_t));\n  return dest;\n#else\n  // `static_cast<uint16_t>` avoids triggering `-Wimplicit-int-conversion`:\n  // the result of `uint16_t | uint16_t` is `int` (if `uint16_t` is narrower\n  // than `int`).\n  return static_cast<uint16_t>((uint16_t{static_cast<uint8_t>(src[0])} << 8) |\n                               uint16_t{static_cast<uint8_t>(src[1])});\n#endif\n}\n\ntemplate <>\ninline uint32_t ReadBigEndian<uint32_t>(const char* src) {\n#if ABSL_IS_BIG_ENDIAN\n  uint32_t dest;\n  std::memcpy(&dest, src, sizeof(uint32_t));\n  return dest;\n#else\n  return (uint32_t{static_cast<uint8_t>(src[0])} << (3 * 8)) |\n         (uint32_t{static_cast<uint8_t>(src[1])} << (2 * 8)) |\n         (uint32_t{static_cast<uint8_t>(src[2])} << 8) |\n         uint32_t{static_cast<uint8_t>(src[3])};\n#endif\n}\n\ntemplate <>\ninline uint64_t ReadBigEndian<uint64_t>(const char* src) {\n#if ABSL_IS_BIG_ENDIAN\n  uint64_t dest;\n  std::memcpy(&dest, src, sizeof(uint64_t));\n  return dest;\n#else\n  return (uint64_t{static_cast<uint8_t>(src[0])} << (7 * 8)) |\n         (uint64_t{static_cast<uint8_t>(src[1])} << (6 * 8)) |\n         (uint64_t{static_cast<uint8_t>(src[2])} << (5 * 8)) |\n         (uint64_t{static_cast<uint8_t>(src[3])} << (4 * 8)) |\n         (uint64_t{static_cast<uint8_t>(src[4])} << (3 * 8)) |\n         (uint64_t{static_cast<uint8_t>(src[5])} << (2 * 8)) |\n         (uint64_t{static_cast<uint8_t>(src[6])} << 8) |\n         uint64_t{static_cast<uint8_t>(src[7])};\n#endif\n}\n\ntemplate <>\ninline absl::uint128 ReadBigEndian<absl::uint128>(const char* src) {\n#if ABSL_IS_BIG_ENDIAN\n  absl::uint128 dest;\n  std::memcpy(&dest, src, sizeof(absl::uint128));\n  return dest;\n#else\n  const uint64_t high = ReadBigEndian<uint64_t>(src);\n  const uint64_t low = ReadBigEndian<uint64_t>(src + sizeof(uint64_t));\n  return absl::MakeUint128(high, low);\n#endif\n}\n\ntemplate <>\ninline int8_t ReadBigEndian<int8_t>(const char* src) {\n  return static_cast<int8_t>(*src);\n}\n\ntemplate <>\ninline int16_t ReadBigEndian<int16_t>(const char* src) {\n  return static_cast<int16_t>(ReadBigEndian<uint16_t>(src));\n}\n\ntemplate <>\ninline int32_t ReadBigEndian<int32_t>(const char* src) {\n  return static_cast<int32_t>(ReadBigEndian<uint32_t>(src));\n}\n\ntemplate <>\ninline int64_t ReadBigEndian<int64_t>(const char* src) {\n  return static_cast<int64_t>(ReadBigEndian<uint64_t>(src));\n}\n\ntemplate <>\ninline absl::int128 ReadBigEndian<absl::int128>(const char* src) {\n  return static_cast<absl::int128>(ReadBigEndian<absl::uint128>(src));\n}\n\ntemplate <>\ninline float ReadBigEndian<float>(const char* src) {\n  return absl::bit_cast<float>(ReadBigEndian<uint32_t>(src));\n}\n\ntemplate <>\ninline double ReadBigEndian<double>(const char* src) {\n  return absl::bit_cast<double>(ReadBigEndian<uint64_t>(src));\n}\n\ntemplate <typename T>\ninline void ReadLittleEndians(const char* src,\n                              absl::Span<type_identity_t<T>> dest) {\n#if ABSL_IS_LITTLE_ENDIAN\n  riegeli::null_safe_memcpy(dest.data(), src, dest.size() * sizeof(T));\n#else\n  for (T& value : dest) {\n    value = ReadLittleEndian<T>(src);\n    src += sizeof(T);\n  }\n#endif\n}\n\ntemplate <>\ninline void ReadLittleEndians<uint8_t>(const char* src,\n                                       absl::Span<uint8_t> dest) {\n  riegeli::null_safe_memcpy(dest.data(), src, dest.size());\n}\n\ntemplate <>\ninline void ReadLittleEndians<int8_t>(const char* src,\n                                      absl::Span<int8_t> dest) {\n  riegeli::null_safe_memcpy(dest.data(), src, dest.size());\n}\n\ntemplate <typename T>\ninline void ReadBigEndians(const char* src,\n                           absl::Span<type_identity_t<T>> dest) {\n#if ABSL_IS_BIG_ENDIAN\n  riegeli::null_safe_memcpy(dest.data(), src, dest.size() * sizeof(T));\n#else\n  if constexpr (sizeof(T) == 1) {\n    riegeli::null_safe_memcpy(dest.data(), src, dest.size());\n  } else {\n    for (T& value : dest) {\n      value = ReadBigEndian<T>(src);\n      src += sizeof(T);\n    }\n  }\n#endif\n}\n\ntemplate <>\ninline void ReadBigEndians<uint8_t>(const char* src, absl::Span<uint8_t> dest) {\n  riegeli::null_safe_memcpy(dest.data(), src, dest.size());\n}\n\ntemplate <>\ninline void ReadBigEndians<int8_t>(const char* src, absl::Span<int8_t> dest) {\n  riegeli::null_safe_memcpy(dest.data(), src, dest.size());\n}\n\ntemplate <typename T>\ninline bool ReadLittleEndian(Reader& src, type_identity_t<T>& dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(sizeof(T)))) return false;\n  dest = ReadLittleEndian<T>(src.cursor());\n  src.move_cursor(sizeof(T));\n  return true;\n}\n\ntemplate <>\ninline bool ReadLittleEndian<uint8_t>(Reader& src, uint8_t& dest) {\n  return src.ReadByte(dest);\n}\n\ntemplate <>\ninline bool ReadLittleEndian<int8_t>(Reader& src, int8_t& dest) {\n  uint8_t byte;\n  if (ABSL_PREDICT_FALSE(!src.ReadByte(byte))) return false;\n  dest = static_cast<int8_t>(byte);\n  return true;\n}\n\ntemplate <typename T>\ninline bool ReadBigEndian(Reader& src, type_identity_t<T>& dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(sizeof(T)))) return false;\n  dest = ReadBigEndian<T>(src.cursor());\n  src.move_cursor(sizeof(T));\n  return true;\n}\n\ntemplate <>\ninline bool ReadBigEndian<uint8_t>(Reader& src, uint8_t& dest) {\n  return src.ReadByte(dest);\n}\n\ntemplate <>\ninline bool ReadBigEndian<int8_t>(Reader& src, int8_t& dest) {\n  uint8_t byte;\n  if (ABSL_PREDICT_FALSE(!src.ReadByte(byte))) return false;\n  dest = static_cast<int8_t>(byte);\n  return true;\n}\n\ntemplate <typename T>\ninline bool ReadLittleEndians(Reader& src,\n                              absl::Span<type_identity_t<T>> dest) {\n#if ABSL_IS_LITTLE_ENDIAN\n  return src.Read(dest.size() * sizeof(T),\n                  reinterpret_cast<char*>(dest.data()));\n#else\n  for (T& dest_value : dest) {\n    if (ABSL_PREDICT_FALSE(!ReadLittleEndian<T>(src, dest_value))) return false;\n  }\n  return true;\n#endif\n}\n\ntemplate <>\ninline bool ReadLittleEndians<uint8_t>(Reader& src, absl::Span<uint8_t> dest) {\n  return src.Read(dest.size(), reinterpret_cast<char*>(dest.data()));\n}\n\ntemplate <>\ninline bool ReadLittleEndians<int8_t>(Reader& src, absl::Span<int8_t> dest) {\n  return src.Read(dest.size(), reinterpret_cast<char*>(dest.data()));\n}\n\ntemplate <typename T>\ninline bool ReadBigEndians(Reader& src, absl::Span<type_identity_t<T>> dest) {\n#if ABSL_IS_BIG_ENDIAN\n  return src.Read(dest.size() * sizeof(T),\n                  reinterpret_cast<char*>(dest.data()));\n#else\n  for (T& dest_value : dest) {\n    if (ABSL_PREDICT_FALSE(!ReadBigEndian<T>(src, dest_value))) return false;\n  }\n  return true;\n#endif\n}\n\ntemplate <>\ninline bool ReadBigEndians<uint8_t>(Reader& src, absl::Span<uint8_t> dest) {\n  return src.Read(dest.size(), reinterpret_cast<char*>(dest.data()));\n}\n\ntemplate <>\ninline bool ReadBigEndians<int8_t>(Reader& src, absl::Span<int8_t> dest) {\n  return src.Read(dest.size(), reinterpret_cast<char*>(dest.data()));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_ENDIAN_ENDIAN_READING_H_\n"
  },
  {
    "path": "riegeli/endian/endian_writing.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_ENDIAN_ENDIAN_WRITING_H_\n#define RIEGELI_ENDIAN_ENDIAN_WRITING_H_\n\n#include <stdint.h>\n\n#include <cstring>\n\n#include \"absl/base/casts.h\"\n#include \"absl/base/config.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/int128.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// Writes a number in a fixed width Little/Big Endian encoding to an array.\n// The width of the encoding is determined by the template argument, which\n// must be one of: `{u,}int{8,16,32,64}_t`, `absl::{u,}int128`, `float`, or\n// `double`.\n//\n// Writes `sizeof(T)` bytes to `dest[]`.\ntemplate <typename T>\nvoid WriteLittleEndian(type_identity_t<T> data, char* dest);\ntemplate <typename T>\nvoid WriteBigEndian(type_identity_t<T> data, char* dest);\n\n// Writes an array of numbers in a fixed width Little/Big Endian encoding.\n// The width of the encoding is determined by the template argument, which\n// must be one of: `{u,}int{8,16,32,64}_t`, `absl::{u,}int128`, `float`, or\n// `double`.\n//\n// This is faster than writing them individually if the endianness matches the\n// native one.\n//\n// Writes `data.size() * sizeof(T)` bytes to `dest[]`.\ntemplate <typename T>\nvoid WriteLittleEndians(absl::Span<const type_identity_t<T>> data, char* dest);\ntemplate <typename T>\nvoid WriteBigEndians(absl::Span<const type_identity_t<T>> data, char* dest);\n\n// Writes a number in a fixed width Little/Big Endian encoding. The width of\n// the encoding is determined by the template argument, which must be one of:\n// `{u,}int{8,16,32,64}_t`, `absl::{u,}int128`, `float`, or `double`.\n//\n// Return values:\n//  * `true`  - success (`dest.ok()`)\n//  * `false` - failure (`!dest.ok()`)\ntemplate <typename T>\nbool WriteLittleEndian(type_identity_t<T> data, Writer& dest);\ntemplate <typename T>\nbool WriteBigEndian(type_identity_t<T> data, Writer& dest);\n\n// Writes an array of numbers in a fixed width Little/Big Endian encoding.\n// The width of the encoding is determined by the template argument, which\n// must be one of: `{u,}int{8,16,32,64}_t`, `absl::{u,}int128`, `float`, or\n// `double`.\n//\n// This is faster than writing them individually if the endianness matches the\n// native one.\n//\n// Return values:\n//  * `true`  - success (`dest.ok()`)\n//  * `false` - failure (`!dest.ok()`)\ntemplate <typename T>\nbool WriteLittleEndians(absl::Span<const type_identity_t<T>> data,\n                        Writer& dest);\ntemplate <typename T>\nbool WriteBigEndians(absl::Span<const type_identity_t<T>> data, Writer& dest);\n\n// Writes a number in a fixed width Little/Big Endian encoding to a\n// `BackwardWriter`. The width of the encoding is determined by the template\n// argument, which must be one of: `{u,}int{8,16,32,64}_t`, `absl::{u,}int128`,\n// `float`, or `double`.\n//\n// Return values:\n//  * `true`  - success (`dest.ok()`)\n//  * `false` - failure (`!dest.ok()`)\ntemplate <typename T>\nbool WriteLittleEndian(type_identity_t<T> data, BackwardWriter& dest);\ntemplate <typename T>\nbool WriteBigEndian(type_identity_t<T> data, BackwardWriter& dest);\n\n// Writes an array of numbers in a fixed width Little/Big Endian encoding to a\n// `BackwardWriter`. The width of the encoding is determined by the template\n// argument, which must be one of: `{u,}int{8,16,32,64}_t`, `absl::{u,}int128`,\n// `float`, or `double`.\n//\n// This is faster than writing them individually if the endianness matches the\n// native one.\n//\n// Return values:\n//  * `true`  - success (`dest.ok()`)\n//  * `false` - failure (`!dest.ok()`)\ntemplate <typename T>\nbool WriteLittleEndians(absl::Span<const type_identity_t<T>> data,\n                        BackwardWriter& dest);\ntemplate <typename T>\nbool WriteBigEndians(absl::Span<const type_identity_t<T>> data,\n                     BackwardWriter& dest);\n\n// Implementation details follow.\n\ntemplate <>\ninline void WriteLittleEndian<uint8_t>(uint8_t data, char* dest) {\n  *dest = static_cast<char>(data);\n}\n\ntemplate <>\ninline void WriteLittleEndian<uint16_t>(uint16_t data, char* dest) {\n#if ABSL_IS_LITTLE_ENDIAN\n  std::memcpy(dest, &data, sizeof(uint16_t));\n#else\n  dest[0] = static_cast<char>(data);\n  dest[1] = static_cast<char>(data >> 8);\n#endif\n}\n\ntemplate <>\ninline void WriteLittleEndian<uint32_t>(uint32_t data, char* dest) {\n#if ABSL_IS_LITTLE_ENDIAN\n  std::memcpy(dest, &data, sizeof(uint32_t));\n#else\n  dest[0] = static_cast<char>(data);\n  dest[1] = static_cast<char>(data >> 8);\n  dest[2] = static_cast<char>(data >> (2 * 8));\n  dest[3] = static_cast<char>(data >> (3 * 8));\n#endif\n}\n\ntemplate <>\ninline void WriteLittleEndian<uint64_t>(uint64_t data, char* dest) {\n#if ABSL_IS_LITTLE_ENDIAN\n  std::memcpy(dest, &data, sizeof(uint64_t));\n#else\n  dest[0] = static_cast<char>(data);\n  dest[1] = static_cast<char>(data >> 8);\n  dest[2] = static_cast<char>(data >> (2 * 8));\n  dest[3] = static_cast<char>(data >> (3 * 8));\n  dest[4] = static_cast<char>(data >> (4 * 8));\n  dest[5] = static_cast<char>(data >> (5 * 8));\n  dest[6] = static_cast<char>(data >> (6 * 8));\n  dest[7] = static_cast<char>(data >> (7 * 8));\n#endif\n}\n\ntemplate <>\ninline void WriteLittleEndian<absl::uint128>(absl::uint128 data, char* dest) {\n#if ABSL_IS_LITTLE_ENDIAN\n  std::memcpy(dest, &data, sizeof(absl::uint128));\n#else\n  WriteLittleEndian<uint64_t>(absl::Uint128Low64(data), dest);\n  WriteLittleEndian<uint64_t>(absl::Uint128High64(data),\n                              dest + sizeof(uint64_t));\n#endif\n}\n\ntemplate <>\ninline void WriteLittleEndian<int8_t>(int8_t data, char* dest) {\n  *dest = static_cast<char>(data);\n}\n\ntemplate <>\ninline void WriteLittleEndian<int16_t>(int16_t data, char* dest) {\n  WriteLittleEndian<uint16_t>(static_cast<uint16_t>(data), dest);\n}\n\ntemplate <>\ninline void WriteLittleEndian<int32_t>(int32_t data, char* dest) {\n  WriteLittleEndian<uint32_t>(static_cast<uint32_t>(data), dest);\n}\n\ntemplate <>\ninline void WriteLittleEndian<int64_t>(int64_t data, char* dest) {\n  WriteLittleEndian<uint64_t>(static_cast<uint64_t>(data), dest);\n}\n\ntemplate <>\ninline void WriteLittleEndian<absl::int128>(absl::int128 data, char* dest) {\n  WriteLittleEndian<absl::uint128>(static_cast<absl::uint128>(data), dest);\n}\n\ntemplate <>\ninline void WriteLittleEndian<float>(float data, char* dest) {\n  WriteLittleEndian<uint32_t>(absl::bit_cast<uint32_t>(data), dest);\n}\n\ntemplate <>\ninline void WriteLittleEndian<double>(double data, char* dest) {\n  WriteLittleEndian<uint64_t>(absl::bit_cast<uint64_t>(data), dest);\n}\n\ntemplate <>\ninline void WriteBigEndian<uint8_t>(uint8_t data, char* dest) {\n  *dest = static_cast<char>(data);\n}\n\ntemplate <>\ninline void WriteBigEndian<uint16_t>(uint16_t data, char* dest) {\n#if ABSL_IS_BIG_ENDIAN\n  std::memcpy(dest, &data, sizeof(uint16_t));\n#else\n  dest[0] = static_cast<char>(data >> 8);\n  dest[1] = static_cast<char>(data);\n#endif\n}\n\ntemplate <>\ninline void WriteBigEndian<uint32_t>(uint32_t data, char* dest) {\n#if ABSL_IS_BIG_ENDIAN\n  std::memcpy(dest, &data, sizeof(uint32_t));\n#else\n  dest[0] = static_cast<char>(data >> (3 * 8));\n  dest[1] = static_cast<char>(data >> (2 * 8));\n  dest[2] = static_cast<char>(data >> 8);\n  dest[3] = static_cast<char>(data);\n#endif\n}\n\ntemplate <>\ninline void WriteBigEndian<uint64_t>(uint64_t data, char* dest) {\n#if ABSL_IS_BIG_ENDIAN\n  std::memcpy(dest, &data, sizeof(uint64_t));\n#else\n  dest[0] = static_cast<char>(data >> (7 * 8));\n  dest[1] = static_cast<char>(data >> (6 * 8));\n  dest[2] = static_cast<char>(data >> (5 * 8));\n  dest[3] = static_cast<char>(data >> (4 * 8));\n  dest[4] = static_cast<char>(data >> (3 * 8));\n  dest[5] = static_cast<char>(data >> (2 * 8));\n  dest[6] = static_cast<char>(data >> 8);\n  dest[7] = static_cast<char>(data);\n#endif\n}\n\ntemplate <>\ninline void WriteBigEndian<absl::uint128>(absl::uint128 data, char* dest) {\n#if ABSL_IS_BIG_ENDIAN\n  std::memcpy(dest, &data, sizeof(absl::uint128));\n#else\n  WriteBigEndian<uint64_t>(absl::Uint128High64(data), dest);\n  WriteBigEndian<uint64_t>(absl::Uint128Low64(data), dest + sizeof(uint64_t));\n#endif\n}\n\ntemplate <>\ninline void WriteBigEndian<int8_t>(int8_t data, char* dest) {\n  *dest = static_cast<char>(data);\n}\n\ntemplate <>\ninline void WriteBigEndian<int16_t>(int16_t data, char* dest) {\n  WriteBigEndian<uint16_t>(static_cast<uint16_t>(data), dest);\n}\n\ntemplate <>\ninline void WriteBigEndian<int32_t>(int32_t data, char* dest) {\n  WriteBigEndian<uint32_t>(static_cast<uint32_t>(data), dest);\n}\n\ntemplate <>\ninline void WriteBigEndian<int64_t>(int64_t data, char* dest) {\n  WriteBigEndian<uint64_t>(static_cast<uint64_t>(data), dest);\n}\n\ntemplate <>\ninline void WriteBigEndian<absl::int128>(absl::int128 data, char* dest) {\n  WriteBigEndian<absl::uint128>(static_cast<absl::uint128>(data), dest);\n}\n\ntemplate <>\ninline void WriteBigEndian<float>(float data, char* dest) {\n  WriteBigEndian<uint32_t>(absl::bit_cast<uint32_t>(data), dest);\n}\n\ntemplate <>\ninline void WriteBigEndian<double>(double data, char* dest) {\n  WriteBigEndian<uint64_t>(absl::bit_cast<uint64_t>(data), dest);\n}\n\ntemplate <typename T>\ninline void WriteLittleEndians(absl::Span<const type_identity_t<T>> data,\n                               char* dest) {\n#if ABSL_IS_LITTLE_ENDIAN\n  riegeli::null_safe_memcpy(dest, data.data(), data.size() * sizeof(T));\n#else\n  for (const T value : data) {\n    WriteLittleEndian<T>(value, dest);\n    dest += sizeof(T);\n  }\n#endif\n}\n\ntemplate <>\ninline void WriteLittleEndians<uint8_t>(absl::Span<const uint8_t> data,\n                                        char* dest) {\n  riegeli::null_safe_memcpy(dest, data.data(), data.size());\n}\n\ntemplate <>\ninline void WriteLittleEndians<int8_t>(absl::Span<const int8_t> data,\n                                       char* dest) {\n  riegeli::null_safe_memcpy(dest, data.data(), data.size());\n}\n\ntemplate <typename T>\ninline void WriteBigEndians(absl::Span<const type_identity_t<T>> data,\n                            char* dest) {\n#if ABSL_IS_BIG_ENDIAN\n  riegeli::null_safe_memcpy(dest, data.data(), data.size() * sizeof(T));\n#else\n  for (const T value : data) {\n    WriteBigEndian<T>(value, dest);\n    dest += sizeof(T);\n  }\n#endif\n}\n\ntemplate <>\ninline void WriteBigEndians<uint8_t>(absl::Span<const uint8_t> data,\n                                     char* dest) {\n  riegeli::null_safe_memcpy(dest, data.data(), data.size());\n}\n\ntemplate <>\ninline void WriteBigEndians<int8_t>(absl::Span<const int8_t> data, char* dest) {\n  riegeli::null_safe_memcpy(dest, data.data(), data.size());\n}\n\ntemplate <typename T>\ninline bool WriteLittleEndian(type_identity_t<T> data, Writer& dest) {\n  if (ABSL_PREDICT_FALSE(!dest.Push(sizeof(T)))) return false;\n  WriteLittleEndian<T>(data, dest.cursor());\n  dest.move_cursor(sizeof(T));\n  return true;\n}\n\ntemplate <>\ninline bool WriteLittleEndian<uint8_t>(uint8_t data, Writer& dest) {\n  return dest.WriteByte(data);\n}\n\ntemplate <>\ninline bool WriteLittleEndian<int8_t>(int8_t data, Writer& dest) {\n  return dest.WriteByte(static_cast<uint8_t>(data));\n}\n\ntemplate <typename T>\ninline bool WriteBigEndian(type_identity_t<T> data, Writer& dest) {\n  if (ABSL_PREDICT_FALSE(!dest.Push(sizeof(T)))) return false;\n  WriteBigEndian<T>(data, dest.cursor());\n  dest.move_cursor(sizeof(T));\n  return true;\n}\n\ntemplate <>\ninline bool WriteBigEndian<uint8_t>(uint8_t data, Writer& dest) {\n  return dest.WriteByte(data);\n}\n\ntemplate <>\ninline bool WriteBigEndian<int8_t>(int8_t data, Writer& dest) {\n  return dest.WriteByte(static_cast<uint8_t>(data));\n}\n\ntemplate <typename T>\ninline bool WriteLittleEndians(absl::Span<const type_identity_t<T>> data,\n                               Writer& dest) {\n#if ABSL_IS_LITTLE_ENDIAN\n  return dest.Write(absl::string_view(\n      reinterpret_cast<const char*>(data.data()), data.size() * sizeof(T)));\n#else\n  for (const T value : data) {\n    if (ABSL_PREDICT_FALSE(!WriteLittleEndian<T>(value, dest))) return false;\n  }\n  return true;\n#endif\n}\n\ntemplate <>\ninline bool WriteLittleEndians<uint8_t>(absl::Span<const uint8_t> data,\n                                        Writer& dest) {\n  return dest.Write(absl::string_view(\n      reinterpret_cast<const char*>(data.data()), data.size()));\n}\n\ntemplate <>\ninline bool WriteLittleEndians<int8_t>(absl::Span<const int8_t> data,\n                                       Writer& dest) {\n  return dest.Write(absl::string_view(\n      reinterpret_cast<const char*>(data.data()), data.size()));\n}\n\ntemplate <typename T>\ninline bool WriteBigEndians(absl::Span<const type_identity_t<T>> data,\n                            Writer& dest) {\n#if ABSL_IS_BIG_ENDIAN\n  return dest.Write(absl::string_view(\n      reinterpret_cast<const char*>(data.data()), data.size() * sizeof(T)));\n#else\n  for (const T value : data) {\n    if (ABSL_PREDICT_FALSE(!WriteBigEndian<T>(value, dest))) return false;\n  }\n  return true;\n#endif\n}\n\ntemplate <>\ninline bool WriteBigEndians<uint8_t>(absl::Span<const uint8_t> data,\n                                     Writer& dest) {\n  return dest.Write(absl::string_view(\n      reinterpret_cast<const char*>(data.data()), data.size()));\n}\n\ntemplate <>\ninline bool WriteBigEndians<int8_t>(absl::Span<const int8_t> data,\n                                    Writer& dest) {\n  return dest.Write(absl::string_view(\n      reinterpret_cast<const char*>(data.data()), data.size()));\n}\n\ntemplate <typename T>\ninline bool WriteLittleEndian(type_identity_t<T> data, BackwardWriter& dest) {\n  if (ABSL_PREDICT_FALSE(!dest.Push(sizeof(T)))) return false;\n  dest.move_cursor(sizeof(T));\n  WriteLittleEndian<T>(data, dest.cursor());\n  return true;\n}\n\ntemplate <>\ninline bool WriteLittleEndian<uint8_t>(uint8_t data, BackwardWriter& dest) {\n  return dest.WriteByte(data);\n}\n\ntemplate <>\ninline bool WriteLittleEndian<int8_t>(int8_t data, BackwardWriter& dest) {\n  return dest.WriteByte(static_cast<uint8_t>(data));\n}\n\ntemplate <typename T>\ninline bool WriteBigEndian(type_identity_t<T> data, BackwardWriter& dest) {\n  if (ABSL_PREDICT_FALSE(!dest.Push(sizeof(T)))) return false;\n  dest.move_cursor(sizeof(T));\n  WriteBigEndian<T>(data, dest.cursor());\n  return true;\n}\n\ntemplate <>\ninline bool WriteBigEndian<uint8_t>(uint8_t data, BackwardWriter& dest) {\n  return dest.WriteByte(data);\n}\n\ntemplate <>\ninline bool WriteBigEndian<int8_t>(int8_t data, BackwardWriter& dest) {\n  return dest.WriteByte(static_cast<uint8_t>(data));\n}\n\ntemplate <typename T>\ninline bool WriteLittleEndians(absl::Span<const type_identity_t<T>> data,\n                               BackwardWriter& dest) {\n#if ABSL_IS_LITTLE_ENDIAN\n  return dest.Write(absl::string_view(\n      reinterpret_cast<const char*>(data.data()), data.size() * sizeof(T)));\n#else\n  for (auto iter = data.crbegin(); iter != data.crend(); ++iter) {\n    if (ABSL_PREDICT_FALSE(!WriteLittleEndian<T>(*iter, dest))) return false;\n  }\n  return true;\n#endif\n}\n\ntemplate <>\ninline bool WriteLittleEndians<uint8_t>(absl::Span<const uint8_t> data,\n                                        BackwardWriter& dest) {\n  return dest.Write(absl::string_view(\n      reinterpret_cast<const char*>(data.data()), data.size()));\n}\n\ntemplate <>\ninline bool WriteLittleEndians<int8_t>(absl::Span<const int8_t> data,\n                                       BackwardWriter& dest) {\n  return dest.Write(absl::string_view(\n      reinterpret_cast<const char*>(data.data()), data.size()));\n}\n\ntemplate <typename T>\ninline bool WriteBigEndians(absl::Span<const type_identity_t<T>> data,\n                            BackwardWriter& dest) {\n#if ABSL_IS_BIG_ENDIAN\n  return dest.Write(absl::string_view(\n      reinterpret_cast<const char*>(data.data()), data.size() * sizeof(T)));\n#else\n  for (auto iter = data.crbegin(); iter != data.crend(); ++iter) {\n    if (ABSL_PREDICT_FALSE(!WriteBigEndian<T>(*iter, dest))) return false;\n  }\n  return true;\n#endif\n}\n\ntemplate <>\ninline bool WriteBigEndians<uint8_t>(absl::Span<const uint8_t> data,\n                                     BackwardWriter& dest) {\n  return dest.Write(absl::string_view(\n      reinterpret_cast<const char*>(data.data()), data.size()));\n}\n\ntemplate <>\ninline bool WriteBigEndians<int8_t>(absl::Span<const int8_t> data,\n                                    BackwardWriter& dest) {\n  return dest.Write(absl::string_view(\n      reinterpret_cast<const char*>(data.data()), data.size()));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_ENDIAN_ENDIAN_WRITING_H_\n"
  },
  {
    "path": "riegeli/gcs/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"gcs_object\",\n    srcs = [\"gcs_object.cc\"],\n    hdrs = [\"gcs_object.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:debug\",\n        \"//riegeli/base:global\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:string_ref\",\n        \"//riegeli/bytes:ostream_writer\",\n        \"//riegeli/bytes:string_writer\",\n        \"//riegeli/bytes:stringify\",\n        \"//riegeli/bytes:stringify_writer\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"gcs_reader\",\n    srcs = [\n        \"gcs_internal.h\",\n        \"gcs_reader.cc\",\n    ],\n    hdrs = [\"gcs_reader.h\"],\n    deps = [\n        \":gcs_object\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:type_traits\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:istream_reader\",\n        \"//riegeli/bytes:reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@google_cloud_cpp//:common\",\n        \"@google_cloud_cpp//:storage\",\n    ],\n)\n\ncc_library(\n    name = \"gcs_writer\",\n    srcs = [\n        \"gcs_internal.h\",\n        \"gcs_writer.cc\",\n    ],\n    hdrs = [\"gcs_writer.h\"],\n    deps = [\n        \":gcs_object\",\n        \":gcs_reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:string_ref\",\n        \"//riegeli/base:type_traits\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:ostream_writer\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@google_cloud_cpp//:common\",\n        \"@google_cloud_cpp//:storage\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/gcs/gcs_internal.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_GCS_GCS_INTERNAL_H_\n#define RIEGELI_GCS_GCS_INTERNAL_H_\n\n#include \"absl/base/attributes.h\"\n#include \"absl/status/status.h\"\n#include \"google/cloud/status.h\"\n\nnamespace riegeli::gcs_internal {\n\ninline absl::Status FromCloudStatus(const google::cloud::Status& status) {\n  return absl::Status(static_cast<absl::StatusCode>(status.code()),\n                      status.message());\n}\n\ntemplate <typename T>\nT GetOption() {\n  return T();\n}\ntemplate <typename T>\nconst T& GetOption(const T& option) {\n  return option;\n}\ntemplate <typename T, typename... Options>\nconst T& GetOption(ABSL_ATTRIBUTE_UNUSED const T& previous_option,\n                   const T& option, const Options&... options) {\n  return GetOption<T>(option, options...);\n}\ntemplate <typename T, typename Other, typename... Options>\nconst T& GetOption(const T& option,\n                   ABSL_ATTRIBUTE_UNUSED const Other& other_option,\n                   const Options&... options) {\n  return GetOption<T>(option, options...);\n}\ntemplate <typename T, typename Other, typename... Options>\nauto GetOption(ABSL_ATTRIBUTE_UNUSED const Other& other_option,\n               const Options&... options) {\n  return GetOption<T>(options...);\n}\n\n}  // namespace riegeli::gcs_internal\n\n#endif  // RIEGELI_GCS_GCS_INTERNAL_H_\n"
  },
  {
    "path": "riegeli/gcs/gcs_object.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/gcs/gcs_object.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <optional>\n#include <ostream>\n#include <string>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/match.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/debug.h\"\n#include \"riegeli/base/global.h\"\n#include \"riegeli/bytes/ostream_writer.h\"\n#include \"riegeli/bytes/string_writer.h\"\n#include \"riegeli/bytes/stringify.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nnamespace {\n\nconstexpr absl::string_view kUriPrefix = \"gs://\";\n\ninline bool IsUtf8Tail(char ch, unsigned char start = 0x80,\n                       unsigned char end = 0xbf) {\n  return static_cast<unsigned char>(ch) >= start &&\n         static_cast<unsigned char>(ch) <= end;\n}\n\n}  // namespace\n\nvoid GcsObject::ParseUri(absl::string_view uri) {\n  if (ABSL_PREDICT_FALSE(!absl::StartsWith(uri, kUriPrefix))) {\n    FailParsing(\"GCS URI does not begin with \\\"gs://\\\"\", uri);\n    return;\n  }\n  const absl::string_view after_prefix(uri.data() + kUriPrefix.size(),\n                                       uri.size() - kUriPrefix.size());\n\n  const size_t slash = after_prefix.find('/');\n  if (ABSL_PREDICT_FALSE(slash == absl::string_view::npos)) {\n    FailParsing(\n        \"GCS URI does not include \\\"/\\\" \"\n        \"separating bucket name from object name\",\n        uri);\n    return;\n  }\n  const absl::string_view after_bucket_name(after_prefix.data() + (slash + 1),\n                                            after_prefix.size() - (slash + 1));\n\n  size_t hash = after_bucket_name.rfind('#');\n  std::optional<int64_t> generation;\n  if (hash == absl::string_view::npos) {\n    hash = after_bucket_name.size();\n  } else {\n    const char* ptr = after_bucket_name.data() + (hash + 1);\n    const char* const limit =\n        after_bucket_name.data() + after_bucket_name.size();\n    if (ABSL_PREDICT_FALSE(ptr == limit)) {\n      FailParsing(\"Empty generation number\", uri);\n      return;\n    }\n    generation = 0;\n    do {\n      if (ABSL_PREDICT_FALSE(*ptr < '0' || *ptr > '9')) {\n        FailParsing(\"Invalid generation number\", uri);\n        return;\n      }\n      const int64_t digit = *ptr - '0';\n      if (ABSL_PREDICT_FALSE(*generation >\n                             std::numeric_limits<int64_t>::max() / 10)) {\n        FailParsing(\"Generation number out of range\", uri);\n        return;\n      }\n      *generation *= 10;\n      if (ABSL_PREDICT_FALSE(*generation >\n                             std::numeric_limits<int64_t>::max() - digit)) {\n        FailParsing(\"Generation number out of range\", uri);\n        return;\n      }\n      *generation += digit;\n      ++ptr;\n    } while (ptr != limit);\n  }\n\n  bucket_name_.assign(after_prefix.data(), slash);\n  object_name_.assign(after_bucket_name.data(), hash);\n  generation_ = generation;\n  ValidateNames();\n}\n\nconst absl::Status& GcsObject::DefaultStatus() {\n  return Global([] {\n    return absl::InvalidArgumentError(\"Default-constructed GcsObject\");\n  });\n}\n\nconst absl::Status& GcsObject::MovedFromStatus() {\n  return Global(\n      [] { return absl::InvalidArgumentError(\"Moved-from GcsObject\"); });\n}\n\ninline bool GcsObject::FailParsing(absl::string_view message,\n                                   absl::string_view context) {\n  status_ = absl::InvalidArgumentError(\n      absl::StrCat(message, \": \", riegeli::Debug(context)));\n  return false;\n}\n\nvoid GcsObject::ValidateNames() {\n  if (ABSL_PREDICT_FALSE(!ValidateBucketName() || !ValidateObjectName())) {\n    bucket_name_ = std::string();\n    object_name_ = std::string();\n    generation_ = std::nullopt;\n  }\n}\n\ninline bool GcsObject::ValidateBucketName() {\n  const absl::string_view bucket_name = bucket_name_;\n  if (ABSL_PREDICT_FALSE(bucket_name.size() < 3)) {\n    return FailParsing(\"Bucket name must not be shorter than 3 characters\",\n                       bucket_name);\n  }\n  if (ABSL_PREDICT_FALSE(bucket_name.size() > 222)) {\n    return FailParsing(\"Bucket name must not be longer than 222 characters\",\n                       bucket_name);\n  }\n\n  // Whether there were any '.' characters.\n  bool has_dots = false;\n  // Whether there were any '_' characters.\n  bool has_underscores = false;\n  // Whether the current dot-separated component contains characters other than\n  // digits.\n  bool component_has_non_digits = false;\n  // Whether the previous character was a letter or digit.\n  bool after_alphanumeric = false;\n  // The beginning of the current dot-separated component.\n  const char* component_begin = bucket_name.data();\n\n  const char* const limit = bucket_name.data() + bucket_name.size();\n  for (const char* cursor = bucket_name.data(); cursor < limit; ++cursor) {\n    const char ch = *cursor;\n    if (ch >= 'a' && ch <= 'z') {\n      component_has_non_digits = true;\n      after_alphanumeric = true;\n    } else if (ch >= '0' && ch <= '9') {\n      after_alphanumeric = true;\n    } else {\n      if (ABSL_PREDICT_FALSE(cursor == component_begin)) {\n        return FailParsing(\n            component_begin == bucket_name.data()\n                ? absl::string_view(\n                      \"Bucket name must begin with a letter or digit\")\n                : absl::string_view(\"Bucket name dot-separated component must \"\n                                    \"begin with a letter or digit\"),\n            bucket_name);\n      }\n      if (ch == '-') {\n        component_has_non_digits = true;\n        after_alphanumeric = false;\n      } else if (ch == '_') {\n        has_underscores = true;\n        component_has_non_digits = true;\n        after_alphanumeric = false;\n      } else if (ch == '.') {\n        if (ABSL_PREDICT_FALSE(PtrDistance(component_begin, cursor) > 63)) {\n          return FailParsing(\n              \"Bucket name dot-separated component must not \"\n              \"be longer than 63 characters\",\n              bucket_name);\n        }\n        if (ABSL_PREDICT_FALSE(!after_alphanumeric)) {\n          return FailParsing(\n              \"Bucket name dot-separated component must \"\n              \"end with a letter or digit\",\n              bucket_name);\n        }\n        has_dots = true;\n        component_has_non_digits = false;\n        after_alphanumeric = false;\n        component_begin = cursor + 1;\n      } else {\n        return FailParsing(\n            \"Bucket name must consist of \"\n            \"letters, digits, hyphens, underscores, and dots\",\n            bucket_name);\n      }\n    }\n  }\n  if (ABSL_PREDICT_FALSE(PtrDistance(component_begin, limit) > 63)) {\n    return FailParsing(\n        component_begin == bucket_name.data()\n            ? absl::string_view(\"Bucket name with no dots must not \"\n                                \"be longer than 63 characters\")\n            : absl::string_view(\"Bucket name dot-separated component must not \"\n                                \"be longer than 63 characters\"),\n        bucket_name);\n  }\n  if (ABSL_PREDICT_FALSE(!after_alphanumeric)) {\n    return FailParsing(\"Bucket name must end with a letter or digit\",\n                       bucket_name);\n  }\n  if (has_dots) {\n    if (ABSL_PREDICT_FALSE(has_underscores)) {\n      return FailParsing(\"Bucket name with dots must not contain underscores\",\n                         bucket_name);\n    }\n    if (ABSL_PREDICT_FALSE(!component_has_non_digits)) {\n      return FailParsing(\n          \"Bucket name last dot-separated component must not \"\n          \"consist of only digits\",\n          bucket_name);\n    }\n  }\n  return true;\n}\n\ninline bool GcsObject::ValidateObjectName() {\n  const absl::string_view object_name = object_name_;\n  if (ABSL_PREDICT_FALSE(object_name.empty())) {\n    return FailParsing(\"Object name must not be empty\", object_name);\n  }\n  if (ABSL_PREDICT_FALSE(object_name.size() > 1024)) {\n    return FailParsing(\"Object name must not be longer than 1024 bytes\",\n                       object_name);\n  }\n\n  const char* const limit = object_name.data() + object_name.size();\n  for (const char* cursor = object_name.data(); cursor < limit;) {\n    const unsigned char byte = static_cast<unsigned char>(*cursor);\n    if (ABSL_PREDICT_TRUE(byte <= 0x7f)) {\n      if (ABSL_PREDICT_FALSE(byte == '\\n' || byte == '\\r')) {\n        return FailParsing(\"Object name must not contain newlines\",\n                           object_name);\n      }\n      ++cursor;\n    } else {\n      if (ABSL_PREDICT_FALSE(byte < 0xc2 || byte > 0xf4)) {\n      invalid:\n        return FailParsing(\"Object name must be valid UTF-8\", object_name);\n      }\n      const size_t remaining = PtrDistance(cursor, limit);\n      if (byte <= 0xdf) {\n        if (ABSL_PREDICT_FALSE(remaining < 2 || !IsUtf8Tail(cursor[1]))) {\n          goto invalid;\n        }\n        cursor += 2;\n      } else if (byte <= 0xef) {\n        if (ABSL_PREDICT_FALSE(\n                remaining < 3 ||\n                (byte == 0xe0   ? !IsUtf8Tail(cursor[1], 0xa0, 0xbf)\n                 : byte == 0xed ? !IsUtf8Tail(cursor[1], 0x80, 0x9f)\n                                : !IsUtf8Tail(cursor[1])) ||\n                !IsUtf8Tail(cursor[2]))) {\n          goto invalid;\n        }\n        cursor += 3;\n      } else {\n        if (ABSL_PREDICT_FALSE(\n                remaining < 4 ||\n                (byte == 0xf0   ? !IsUtf8Tail(cursor[1], 0x90, 0xbf)\n                 : byte == 0xf4 ? !IsUtf8Tail(cursor[1], 0x80, 0x8f)\n                                : !IsUtf8Tail(cursor[1]))) ||\n            !IsUtf8Tail(cursor[2]) || !IsUtf8Tail(cursor[3])) {\n          goto invalid;\n        }\n        cursor += 4;\n      }\n    }\n  }\n\n  if (ABSL_PREDICT_FALSE(object_name == \".\")) {\n    return FailParsing(\"Object name must not be \\\".\\\"\", object_name);\n  }\n  if (ABSL_PREDICT_FALSE(object_name == \"..\")) {\n    return FailParsing(\"Object name must not be \\\"..\\\"\", object_name);\n  }\n  if (ABSL_PREDICT_FALSE(\n          absl::StartsWith(object_name, \".well-known/acme-challenge/\"))) {\n    return FailParsing(\n        \"Object name must not start with \\\".well-known/acme-challenge/\\\"\",\n        object_name);\n  }\n  return true;\n}\n\nbool GcsObject::Equal(const GcsObject& a, const GcsObject& b) {\n  if (ABSL_PREDICT_FALSE(!a.ok())) return a.status() == b.status();\n  if (ABSL_PREDICT_FALSE(!b.ok())) return false;\n  return a.bucket_name() == b.bucket_name() &&\n         a.object_name() == b.object_name() && a.generation() == b.generation();\n}\n\nvoid GcsObject::Output(std::ostream& dest) const {\n  OStreamWriter<> writer(&dest);\n  WriteTo(writer);\n  writer.Close();\n}\n\nvoid GcsObject::WriteTo(Writer& dest) const {\n  if (ABSL_PREDICT_FALSE(!ok())) {\n    dest.Write('<', status().message(), '>');\n    return;\n  }\n  dest.Write(kUriPrefix, bucket_name(), '/', object_name());\n  if (generation() != std::nullopt) dest.Write('#', *generation());\n}\n\nstd::string GcsObject::uri() const {\n  std::string uri;\n  if (ABSL_PREDICT_TRUE(ok())) {\n    StringWriter<> writer(&uri);\n    writer.SetWriteSizeHint(\n        kUriPrefix.size() + bucket_name().size() + 1 + object_name().size() +\n        (generation() == std::nullopt\n             ? 0\n             : 1 + riegeli::StringifiedSize(*generation())));\n    WriteTo(writer);\n    writer.Close();\n  }\n  return uri;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/gcs/gcs_object.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_GCS_GCS_OBJECT_H_\n#define RIEGELI_GCS_GCS_OBJECT_H_\n\n#include <stdint.h>\n\n#include <iosfwd>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/string_ref.h\"\n#include \"riegeli/bytes/stringify_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// Identifies a GCS object by storing the bucket name, object name, and optional\n// generation.\n//\n// It provides parsing and producing URIs of the form\n// \"gs://bucket_name/object_name#generation\" (generation is optional).\n//\n// The syntax of bucket names is validated against\n// https://cloud.google.com/storage/docs/naming#requirements and\n// https://datatracker.ietf.org/doc/html/rfc3696#section-2.\n// The syntax of object names is validated against\n// https://cloud.google.com/storage/docs/objects#naming and\n// https://datatracker.ietf.org/doc/html/rfc3629#section-4.\n//\n// No detailed verification of the domain name is performed in bucket names\n// containing dots (e.g. that the top-level domain is currently recognized),\n// nor that the bucket name does not begin with \"goog\" or contain \"google\"\n// (this applies only to bucket creation).\nclass GcsObject : public WithEqual<GcsObject> {\n public:\n  // Constructs a dummy `GcsObject` which is `!ok()`.\n  GcsObject() noexcept : status_(DefaultStatus()) {}\n\n  // Constructs `GcsObject` from bucket name, object name, and\n  // optional generation.\n  explicit GcsObject(StringInitializer bucket_name,\n                     StringInitializer object_name,\n                     std::optional<int64_t> generation = std::nullopt);\n\n  // Constructs `GcsObject` from a URI of the form\n  // \"gs://bucket_name/object_name#generation\" (generation is optional).\n  //\n  // If the URI is not in the expected format, `ok()` is `false` and `status()`\n  // explains the reason.\n  explicit GcsObject(absl::string_view uri);\n\n  GcsObject(const GcsObject& that);\n  GcsObject& operator=(const GcsObject& that);\n\n  GcsObject(GcsObject&& that) noexcept;\n  GcsObject& operator=(GcsObject&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `GcsObject`. This avoids\n  // constructing a temporary `GcsObject` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      StringInitializer bucket_name, StringInitializer object_name,\n      std::optional<int64_t> generation = std::nullopt);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(absl::string_view uri);\n\n  // Returns `true` if the `GcsObject` is valid (the URI was\n  // successfully parsed).\n  bool ok() const { return status_.ok(); }\n  // If `ok()`, returns `absl::OkStatus()`, otherwise returns the status\n  // explaining the reason why the `GcsObject` is invalid.\n  const absl::Status& status() const { return status_; }\n\n  const std::string& bucket_name() const { return bucket_name_; }\n  const std::string& object_name() const { return object_name_; }\n  std::optional<int64_t> generation() const { return generation_; }\n\n  // Returns the URI of the form \"gs://bucket_name/object_name#generation\"\n  // (generation is optional), or empty string if `!ok()`.\n  std::string uri() const;\n\n  friend bool operator==(const GcsObject& a, const GcsObject& b) {\n    return Equal(a, b);\n  }\n\n  // Default stringification by `absl::StrCat()` etc.\n  //\n  // Writes `src.uri()` to `dest`, or \"<status_message>\" if `!ok()`.\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const GcsObject& src) {\n    StringifyWriter<Sink*> writer(&dest);\n    src.WriteTo(writer);\n    writer.Close();\n  }\n\n  // Writes `src.uri()` to `dest`, or \"<status_message>\" if `!ok()`.\n  friend std::ostream& operator<<(std::ostream& dest, const GcsObject& src) {\n    src.Output(dest);\n    return dest;\n  }\n\n private:\n  static const absl::Status& DefaultStatus();\n  static const absl::Status& MovedFromStatus();\n\n  void ParseUri(absl::string_view uri);\n  ABSL_ATTRIBUTE_COLD bool FailParsing(absl::string_view message,\n                                       absl::string_view context);\n  void ValidateNames();\n  bool ValidateBucketName();\n  bool ValidateObjectName();\n\n  static bool Equal(const GcsObject& a, const GcsObject& b);\n\n  void WriteTo(Writer& dest) const;\n  void Output(std::ostream& dest) const;\n\n  absl::Status status_;\n  std::string bucket_name_;\n  std::string object_name_;\n  std::optional<int64_t> generation_;\n};\n\n// Implementation details follow.\n\ninline GcsObject::GcsObject(const GcsObject& that)\n    : status_(that.status()),\n      bucket_name_(that.bucket_name()),\n      object_name_(that.object_name()),\n      generation_(that.generation()) {}\n\ninline GcsObject& GcsObject::operator=(const GcsObject& that) {\n  status_ = that.status();\n  bucket_name_ = that.bucket_name();\n  object_name_ = that.object_name();\n  generation_ = that.generation();\n  return *this;\n}\n\ninline GcsObject::GcsObject(GcsObject&& that) noexcept\n    : status_(std::exchange(that.status_, MovedFromStatus())),\n      bucket_name_(std::exchange(that.bucket_name_, std::string())),\n      object_name_(std::exchange(that.object_name_, std::string())),\n      generation_(std::exchange(that.generation_, std::nullopt)) {}\n\ninline GcsObject& GcsObject::operator=(GcsObject&& that) noexcept {\n  status_ = std::exchange(that.status_, MovedFromStatus());\n  bucket_name_ = std::exchange(that.bucket_name_, std::string());\n  object_name_ = std::exchange(that.object_name_, std::string());\n  generation_ = std::exchange(that.generation_, std::nullopt);\n  return *this;\n}\n\ninline GcsObject::GcsObject(StringInitializer bucket_name,\n                            StringInitializer object_name,\n                            std::optional<int64_t> generation)\n    : bucket_name_(std::move(bucket_name)),\n      object_name_(std::move(object_name)),\n      generation_(generation) {\n  ValidateNames();\n}\n\ninline GcsObject::GcsObject(absl::string_view uri) { ParseUri(uri); }\n\ninline void GcsObject::Reset() {\n  status_ = DefaultStatus();\n  bucket_name_.clear();\n  object_name_.clear();\n  generation_ = std::nullopt;\n}\n\ninline void GcsObject::Reset(StringInitializer bucket_name,\n                             StringInitializer object_name,\n                             std::optional<int64_t> generation) {\n  status_ = absl::OkStatus();\n  riegeli::Reset(bucket_name_, std::move(bucket_name));\n  riegeli::Reset(object_name_, std::move(object_name));\n  generation_ = generation;\n  ValidateNames();\n}\n\ninline void GcsObject::Reset(absl::string_view uri) {\n  status_ = absl::OkStatus();\n  bucket_name_.clear();\n  object_name_.clear();\n  generation_ = std::nullopt;\n  ParseUri(uri);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_GCS_GCS_OBJECT_H_\n"
  },
  {
    "path": "riegeli/gcs/gcs_reader.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/gcs/gcs_reader.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <functional>\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"google/cloud/status.h\"\n#include \"google/cloud/storage/client.h\"\n#include \"google/cloud/storage/object_read_stream.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/istream_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/gcs/gcs_internal.h\"\n#include \"riegeli/gcs/gcs_object.h\"\n\nnamespace riegeli {\n\nGcsReader::GcsReader(\n    NewReaderTag, const google::cloud::storage::Client& client,\n    const GcsObject& object,\n    const std::function<google::cloud::storage::ObjectReadStream(\n        GcsReader&, int64_t)>& read_object,\n    BufferOptions buffer_options, Position read_from_offset)\n    : IStreamReader(kClosed),\n      client_(client),\n      object_(object),\n      read_object_(read_object) {\n  IStreamReader::Reset(read_object(*this, IntCast<int64_t>(read_from_offset)),\n                       IStreamReaderBase::Options()\n                           .set_assumed_pos(read_from_offset)\n                           .set_buffer_options(buffer_options));\n  PropagateStatus();\n  set_limit_pos(read_from_offset);\n}\n\nvoid GcsReader::Reset(Closed) {\n  IStreamReader::Reset(kClosed);\n  client_ = std::nullopt;\n  object_ = GcsObject();\n  read_object_ = nullptr;\n}\n\nvoid GcsReader::Initialize(google::cloud::storage::ObjectReadStream stream,\n                           BufferOptions buffer_options,\n                           const RangeOptions& range_options) {\n  uint64_t initial_pos = 0;\n  if (ABSL_PREDICT_TRUE(range_options.initial_pos >= 0)) {\n    switch (range_options.origin) {\n      case Origin::kBegin:\n        initial_pos = IntCast<uint64_t>(range_options.initial_pos);\n        break;\n      case Origin::kEnd:\n        if (stream.size() != std::nullopt &&\n            ABSL_PREDICT_TRUE(*stream.size() <=\n                              uint64_t{std::numeric_limits<int64_t>::max()})) {\n          initial_pos = SaturatingSub(\n              *stream.size(), IntCast<uint64_t>(range_options.initial_pos));\n        }\n        break;\n    }\n  }\n  IStreamReader::Reset(std::move(stream),\n                       IStreamReaderBase::Options()\n                           .set_assumed_pos(initial_pos)\n                           .set_buffer_options(buffer_options));\n  PropagateStatus();\n  if (src().size() != std::nullopt &&\n      ABSL_PREDICT_TRUE(*src().size() <=\n                        uint64_t{std::numeric_limits<int64_t>::max()})) {\n    set_exact_size(*src().size());\n    if (range_options.max_size != std::nullopt &&\n        ABSL_PREDICT_TRUE(*range_options.max_size >= 0)) {\n      set_exact_size(UnsignedMin(*exact_size(),\n                                 IntCast<uint64_t>(*range_options.max_size)));\n    }\n  }\n}\n\nvoid GcsReader::Done() {\n  IStreamReader::Done();\n  src().Close();\n  PropagateStatus();\n  client_ = std::nullopt;\n  read_object_ = nullptr;\n}\n\nabsl::Status GcsReader::AnnotateStatusImpl(absl::Status status) {\n  if (object_.ok()) {\n    status = Annotate(status, absl::StrCat(\"reading \", object_));\n  }\n  return IStreamReader::AnnotateStatusImpl(std::move(status));\n}\n\ninline void GcsReader::PropagateStatus() {\n  if (ABSL_PREDICT_FALSE(!src().status().ok())) PropagateStatusSlow();\n}\n\nvoid GcsReader::PropagateStatusSlow() {\n  RIEGELI_ASSERT(!src().status().ok())\n      << \"Failed precondition of GcsReader::PropagateStatusSlow(): \"\n         \"ObjectReadStream not failed\";\n  MarkNotFailed();\n  Fail(gcs_internal::FromCloudStatus(src().status()));\n}\n\nbool GcsReader::ReadInternal(size_t min_length, size_t max_length, char* dest) {\n  RIEGELI_ASSERT_GT(min_length, 0u)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"nothing to read\";\n  RIEGELI_ASSERT_GE(max_length, min_length)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"max_length < min_length\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::ReadInternal()\";\n  const bool result = IStreamReader::ReadInternal(min_length, max_length, dest);\n  PropagateStatus();\n  return result;\n}\n\nbool GcsReader::SupportsRandomAccess() { return exact_size() != std::nullopt; }\n\nbool GcsReader::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(exact_size() == std::nullopt)) {\n    return IStreamReader::SeekBehindBuffer(new_pos);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  // `ObjectReadStream` does not support seeking to the very end.\n  const Position read_from_offset =\n      UnsignedMin(new_pos, SaturatingSub(*exact_size(), Position{1}));\n  src() = read_object_(*this, IntCast<int64_t>(read_from_offset));\n  PropagateStatus();\n  set_limit_pos(read_from_offset);\n  if (new_pos > read_from_offset) {\n    if (ABSL_PREDICT_FALSE(!Pull())) return false;\n    move_cursor(1);\n  } else {\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n  }\n  return new_pos <= *exact_size();\n}\n\nbool GcsReader::SupportsNewReader() { return exact_size() != std::nullopt; }\n\nstd::unique_ptr<Reader> GcsReader::NewReaderImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!GcsReader::SupportsNewReader())) {\n    // Delegate to the base class to avoid repeating the error message.\n    return IStreamReader::NewReaderImpl(initial_pos);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point.\n\n  // `ObjectReadStream` does not support seeking to the very end.\n  const Position read_from_offset =\n      UnsignedMin(initial_pos, SaturatingSub(*exact_size(), Position{1}));\n  std::unique_ptr<GcsReader> reader(\n      new GcsReader(NewReaderTag(), client(), object(), read_object_,\n                    buffer_options(), read_from_offset));\n  reader->set_exact_size(exact_size());\n  if (initial_pos > read_from_offset) {\n    if (ABSL_PREDICT_TRUE(reader->Pull())) reader->move_cursor(1);\n  }\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/gcs/gcs_reader.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_GCS_GCS_READER_H_\n#define RIEGELI_GCS_GCS_READER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <functional>\n#include <memory>\n#include <optional>\n#include <tuple>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/cloud/storage/client.h\"\n#include \"google/cloud/storage/download_options.h\"\n#include \"google/cloud/storage/object_read_stream.h\"\n#include \"google/cloud/storage/well_known_parameters.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/istream_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/gcs/gcs_internal.h\"\n#include \"riegeli/gcs/gcs_object.h\"\n\nnamespace riegeli {\n\n// A `Reader` which reads from a Google Cloud Storage object.\nclass GcsReader\n    : public IStreamReader<google::cloud::storage::ObjectReadStream> {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    static constexpr size_t kDefaultMinBufferSize = size_t{64} << 10;\n    static constexpr size_t kDefaultMaxBufferSize = size_t{1} << 20;\n  };\n\n  // Creates a closed `GcsReader`.\n  explicit GcsReader(Closed) : IStreamReader(kClosed) {}\n\n  // Will read from `object`.\n  template <typename... ReadObjectOptions>\n  explicit GcsReader(const google::cloud::storage::Client& client,\n                     Initializer<GcsObject> object, Options options,\n                     ReadObjectOptions&&... read_object_options);\n  template <typename... ReadObjectOptions>\n  explicit GcsReader(const google::cloud::storage::Client& client,\n                     Initializer<GcsObject> object,\n                     ReadObjectOptions&&... read_object_options);\n\n  GcsReader(GcsReader&& that) noexcept;\n  GcsReader& operator=(GcsReader&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `GcsReader`. This avoids\n  // constructing a temporary `GcsReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  template <typename... ReadObjectOptions>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      const google::cloud::storage::Client& client,\n      Initializer<GcsObject> object, Options options,\n      ReadObjectOptions&&... read_object_options);\n  template <typename... ReadObjectOptions>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      const google::cloud::storage::Client& client,\n      Initializer<GcsObject> object,\n      ReadObjectOptions&&... read_object_options);\n\n  google::cloud::storage::Client& client() {\n    RIEGELI_ASSERT(client_ != std::nullopt)\n        << \"Failed precondition of GcsReader::client(): \"\n           \"default-constructed GcsReader\";\n    return *client_;\n  }\n  const GcsObject& object() const { return object_; }\n\n  // Returns the object's generation at the time of the download, if known.\n  std::optional<int64_t> generation() const { return src().generation(); }\n\n  // Returns the object's metageneration at the time of the download, if known.\n  std::optional<int64_t> metageneration() const {\n    return src().metageneration();\n  }\n\n  // Returns the object's storage class at the time of the download, if known.\n  std::optional<absl::string_view> storage_class() const {\n    return src().storage_class();\n  }\n\n  bool SupportsRandomAccess() override;\n  bool SupportsNewReader() override;\n\n protected:\n  void Done() override;\n  absl::Status AnnotateStatusImpl(absl::Status status) override;\n  bool ReadInternal(size_t min_length, size_t max_length, char* dest) override;\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  friend class GcsWriter;  // For `set_exact_size()`.\n\n  struct NewReaderTag {};\n\n  enum class Origin { kBegin, kEnd };\n  struct RangeOptions {\n    Origin origin;\n    int64_t initial_pos;\n    std::optional<int64_t> max_size;\n  };\n\n  explicit GcsReader(\n      NewReaderTag, const google::cloud::storage::Client& client,\n      const GcsObject& object,\n      const std::function<google::cloud::storage::ObjectReadStream(\n          GcsReader&, int64_t)>& read_object,\n      BufferOptions buffer_options, Position read_from_offset);\n\n  template <typename... ReadObjectOptions>\n  static RangeOptions GetRangeOptions(\n      const ReadObjectOptions&... read_object_options);\n\n  template <typename ReadObjectOption>\n  struct NewReadObjectOptionPredicate\n      : std::negation<std::disjunction<\n            std::is_same<std::decay_t<ReadObjectOption>,\n                         google::cloud::storage::ReadFromOffset>,\n            std::is_same<std::decay_t<ReadObjectOption>,\n                         google::cloud::storage::ReadRange>,\n            std::is_same<std::decay_t<ReadObjectOption>,\n                         google::cloud::storage::ReadLast>>> {};\n\n  template <typename... ReadObjectOptions, size_t... indices,\n            typename OffsetOption>\n  google::cloud::storage::ObjectReadStream ApplyReadObject(\n      const std::tuple<ReadObjectOptions...>& read_object_options,\n      std::index_sequence<indices...>, OffsetOption&& offset_option) {\n    return object_.generation() == std::nullopt\n               ? client_->ReadObject(object_.bucket_name(),\n                                     object_.object_name(),\n                                     std::get<indices>(read_object_options)...,\n                                     std::forward<OffsetOption>(offset_option))\n               : client_->ReadObject(\n                     object_.bucket_name(), object_.object_name(),\n                     google::cloud::storage::Generation(*object_.generation()),\n                     std::get<indices>(read_object_options)...,\n                     std::forward<OffsetOption>(offset_option));\n  }\n\n  template <typename... ReadObjectOptions>\n  void Initialize(const Options& options,\n                  ReadObjectOptions&&... read_object_options);\n  void Initialize(google::cloud::storage::ObjectReadStream stream,\n                  BufferOptions buffer_options,\n                  const RangeOptions& range_options);\n  template <typename... ReadObjectOptions>\n  void SetReadObject(const RangeOptions& range_options,\n                     const ReadObjectOptions&... read_object_options);\n  void PropagateStatus();\n  ABSL_ATTRIBUTE_COLD void PropagateStatusSlow();\n\n  std::optional<google::cloud::storage::Client> client_;\n  GcsObject object_;\n  std::function<google::cloud::storage::ObjectReadStream(GcsReader&, int64_t)>\n      read_object_;\n};\n\n// Implementation details follow.\n\ntemplate <typename... ReadObjectOptions>\nGcsReader::GcsReader(const google::cloud::storage::Client& client,\n                     Initializer<GcsObject> object, Options options,\n                     ReadObjectOptions&&... read_object_options)\n    : IStreamReader(kClosed), client_(client), object_(std::move(object)) {\n  Initialize(options, std::forward<ReadObjectOptions>(read_object_options)...);\n}\n\ntemplate <typename... ReadObjectOptions>\nGcsReader::GcsReader(const google::cloud::storage::Client& client,\n                     Initializer<GcsObject> object,\n                     ReadObjectOptions&&... read_object_options)\n    : GcsReader(client, std::move(object), Options(),\n                std::forward<ReadObjectOptions>(read_object_options)...) {}\n\ninline GcsReader::GcsReader(GcsReader&& that) noexcept\n    : IStreamReader(static_cast<IStreamReader&&>(that)),\n      client_(std::move(that.client_)),\n      object_(std::move(that.object_)),\n      read_object_(std::move(that.read_object_)) {}\n\ninline GcsReader& GcsReader::operator=(GcsReader&& that) noexcept {\n  IStreamReader::operator=(static_cast<IStreamReader&&>(that));\n  client_ = std::move(that.client_);\n  object_ = std::move(that.object_);\n  read_object_ = std::move(that.read_object_);\n  return *this;\n}\n\ntemplate <typename... ReadObjectOptions>\nvoid GcsReader::Reset(const google::cloud::storage::Client& client,\n                      Initializer<GcsObject> object, Options options,\n                      ReadObjectOptions&&... read_object_options) {\n  client_ = client;\n  riegeli::Reset(object_, std::move(object));\n  Initialize(options, std::forward<ReadObjectOptions>(read_object_options)...);\n}\n\ntemplate <typename... ReadObjectOptions>\nvoid GcsReader::Reset(const google::cloud::storage::Client& client,\n                      Initializer<GcsObject> object,\n                      ReadObjectOptions&&... read_object_options) {\n  Reset(client, std::move(object), Options(),\n        std::forward<ReadObjectOptions>(read_object_options)...);\n}\n\ntemplate <typename... ReadObjectOptions>\ninline void GcsReader::Initialize(const Options& options,\n                                  ReadObjectOptions&&... read_object_options) {\n  if (ABSL_PREDICT_FALSE(!object_.ok())) {\n    Fail(object_.status());\n    return;\n  }\n  const RangeOptions range_options = GetRangeOptions(read_object_options...);\n  SetReadObject(range_options, read_object_options...);\n  Initialize(\n      object_.generation() == std::nullopt\n          ? client_->ReadObject(\n                object_.bucket_name(), object_.object_name(),\n                std::forward<ReadObjectOptions>(read_object_options)...)\n          : client_->ReadObject(\n                object_.bucket_name(), object_.object_name(),\n                google::cloud::storage::Generation(*object_.generation()),\n                std::forward<ReadObjectOptions>(read_object_options)...),\n      options.buffer_options(), range_options);\n}\n\ntemplate <typename... ReadObjectOptions>\ninline GcsReader::RangeOptions GcsReader::GetRangeOptions(\n    const ReadObjectOptions&... read_object_options) {\n  const google::cloud::storage::ReadFromOffset& read_from_offset =\n      gcs_internal::GetOption<google::cloud::storage::ReadFromOffset>(\n          read_object_options...);\n  const google::cloud::storage::ReadRange& read_range =\n      gcs_internal::GetOption<google::cloud::storage::ReadRange>(\n          read_object_options...);\n  if (read_range.has_value()) {\n    return RangeOptions{\n        Origin::kBegin,\n        read_from_offset.has_value()\n            ? SignedMax(read_range.value().begin, read_from_offset.value())\n            : read_range.value().begin,\n        read_range.value().end};\n  }\n  const google::cloud::storage::ReadLast& read_last =\n      gcs_internal::GetOption<google::cloud::storage::ReadLast>(\n          read_object_options...);\n  if (read_last.has_value()) {\n    return RangeOptions{Origin::kEnd, read_last.value(), std::nullopt};\n  }\n  return RangeOptions{Origin::kBegin, read_from_offset.value_or(0),\n                      std::nullopt};\n}\n\ntemplate <typename... ReadObjectOptions>\ninline void GcsReader::SetReadObject(\n    const RangeOptions& range_options,\n    const ReadObjectOptions&... read_object_options) {\n  if (range_options.max_size == std::nullopt) {\n    read_object_ =\n        [new_read_object_options = DecayTuple(\n             Filter<NewReadObjectOptionPredicate>(read_object_options...))](\n            GcsReader& self, int64_t read_from_offset) mutable {\n          return self.ApplyReadObject(\n              new_read_object_options,\n              std::make_index_sequence<\n                  std::tuple_size_v<decltype(new_read_object_options)>>(),\n              google::cloud::storage::ReadFromOffset(read_from_offset));\n        };\n  } else {\n    read_object_ =\n        [new_read_object_options = DecayTuple(\n             Filter<NewReadObjectOptionPredicate>(read_object_options...)),\n         max_size = *range_options.max_size](GcsReader& self,\n                                             int64_t read_from_offset) mutable {\n          return self.ApplyReadObject(\n              new_read_object_options,\n              std::make_index_sequence<\n                  std::tuple_size_v<decltype(new_read_object_options)>>(),\n              google::cloud::storage::ReadRange(read_from_offset, max_size));\n        };\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_GCS_GCS_READER_H_\n"
  },
  {
    "path": "riegeli/gcs/gcs_writer.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/gcs/gcs_writer.h\"\n\n#include <stdint.h>\n\n#include <functional>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/cloud/status.h\"\n#include \"google/cloud/status_or.h\"\n#include \"google/cloud/storage/object_metadata.h\"\n#include \"google/cloud/storage/object_write_stream.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/ostream_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/gcs/gcs_internal.h\"\n#include \"riegeli/gcs/gcs_object.h\"\n#include \"riegeli/gcs/gcs_reader.h\"\n\nnamespace riegeli {\n\nvoid GcsWriter::Reset(Closed) {\n  OStreamWriter::Reset(kClosed);\n  client_ = std::nullopt;\n  object_ = GcsObject();\n  temp_object_name_ = std::nullopt;\n  read_mode_buffer_options_ = BufferOptions();\n  write_temp_object_ = nullptr;\n  compose_object_ = nullptr;\n  delete_temp_object_ = nullptr;\n  get_object_size_ = nullptr;\n  make_gcs_reader_ = nullptr;\n  writing_temp_object_ = false;\n  initial_temp_object_pos_ = 0;\n  metadata_ = std::nullopt;\n  associated_reader_.Reset();\n}\n\nbool GcsWriter::CheckPreconditions(bool append, bool resumable_upload_session) {\n  if (ABSL_PREDICT_FALSE(object_.generation() != std::nullopt && !append)) {\n    return Fail(absl::FailedPreconditionError(\n        \"Specifying a generation requires GcsWriter::Options::append()\"));\n  }\n  if (temp_object_name_ == std::nullopt) {\n    if (ABSL_PREDICT_FALSE(append)) {\n      return Fail(absl::FailedPreconditionError(\n          \"GcsWriter::Options::append() requires \"\n          \"GcsWriter::Options::temp_object_name()\"));\n    }\n  } else {\n    if (ABSL_PREDICT_FALSE(*temp_object_name_ == object_.object_name())) {\n      return Fail(absl::FailedPreconditionError(\n          absl::StrCat(\"GcsWriter::Options::temp_object_name() must differ \"\n                       \"from object name: \\\"\",\n                       object_.object_name(), \"\\\"\")));\n    }\n    if (ABSL_PREDICT_FALSE(resumable_upload_session)) {\n      return Fail(absl::FailedPreconditionError(\n          \"UseResumableUploadSession() is incompatible with \"\n          \"GcsWriter::Options::temp_object_name()\"));\n    }\n  }\n  return true;\n}\n\nvoid GcsWriter::Initialize(google::cloud::storage::ObjectWriteStream stream,\n                           BufferOptions buffer_options, bool append) {\n  uint64_t initial_pos;\n  if (append) {\n    stream.Close();\n    if (stream.last_status().code() ==\n        google::cloud::StatusCode::kFailedPrecondition) {\n      // Precondition `IfGenerationMatch(0)` failed: object already exists.\n    } else if (ABSL_PREDICT_FALSE(stream.fail())) {\n      FailOperation(\"ObjectWriteStream::Close()\");\n      PropagateStatus();\n      return;\n    }\n    const google::cloud::StatusOr<google::cloud::storage::ObjectMetadata>\n        metadata = get_object_size_(*this);\n    if (ABSL_PREDICT_FALSE(!metadata.ok())) {\n      Fail(gcs_internal::FromCloudStatus(metadata.status()));\n      return;\n    }\n    initial_pos = metadata->size();\n    stream = write_temp_object_(*this);\n    writing_temp_object_ = true;\n    initial_temp_object_pos_ = initial_pos;\n  } else {\n    initial_pos = stream.next_expected_byte();\n  }\n  OStreamWriter::Reset(std::move(stream),\n                       OStreamWriterBase::Options()\n                           .set_assumed_pos(initial_pos)\n                           .set_buffer_options(buffer_options));\n  PropagateStatus();\n  if (ABSL_PREDICT_FALSE(!dest().IsOpen()) && ABSL_PREDICT_TRUE(ok())) {\n    // A resumable upload is already finished. Mark `GcsWriter` closed without\n    // clearing `dest()`.\n    OStreamWriterBase::Reset(kClosed);\n  }\n}\n\nvoid GcsWriter::Done() {\n  OStreamWriter::Done();\n  dest().Close();\n  if (ABSL_PREDICT_FALSE(dest().fail())) {\n    FailOperation(\"ObjectWriteStream::Close()\");\n    PropagateStatus();\n  }\n  if (writing_temp_object_ && ABSL_PREDICT_TRUE(ok())) {\n    if (start_pos() > initial_temp_object_pos_) {\n      google::cloud::StatusOr<google::cloud::storage::ObjectMetadata> metadata =\n          compose_object_(*this);\n      if (ABSL_PREDICT_FALSE(!metadata.ok())) {\n        Fail(gcs_internal::FromCloudStatus(metadata.status()));\n        goto failed;\n      }\n      metadata_ = *std::move(metadata);\n    }\n    if (const google::cloud::Status status = delete_temp_object_(*this);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      Fail(gcs_internal::FromCloudStatus(status));\n    }\n  }\nfailed:\n  client_ = std::nullopt;\n  write_temp_object_ = nullptr;\n  compose_object_ = nullptr;\n  delete_temp_object_ = nullptr;\n  get_object_size_ = nullptr;\n  make_gcs_reader_ = nullptr;\n  associated_reader_.Reset();\n}\n\nabsl::Status GcsWriter::AnnotateStatusImpl(absl::Status status) {\n  if (object_.ok()) {\n    status = Annotate(status, absl::StrCat(\"writing \", object_));\n  }\n  return OStreamWriter::AnnotateStatusImpl(std::move(status));\n}\n\ninline void GcsWriter::PropagateStatus() {\n  const google::cloud::Status status = dest().last_status();\n  if (ABSL_PREDICT_FALSE(!status.ok())) PropagateStatusSlow(status);\n}\n\nvoid GcsWriter::PropagateStatusSlow(const google::cloud::Status& status) {\n  RIEGELI_ASSERT(!status.ok())\n      << \"Failed precondition of GcsWriter::PropagateStatusSlow(): \"\n         \"ObjectWriteStream not failed\";\n  MarkNotFailed();\n  Fail(gcs_internal::FromCloudStatus(status));\n}\n\nbool GcsWriter::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of BufferedWriter::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedWriter::WriteInternal()\";\n  const bool result = OStreamWriter::WriteInternal(src);\n  PropagateStatus();\n  return result;\n}\n\nbool GcsWriter::FlushImpl(FlushType flush_type) {\n  const bool flush_ok = OStreamWriter::FlushImpl(flush_type);\n  PropagateStatus();\n  if (ABSL_PREDICT_FALSE(!flush_ok)) return false;\n  if (temp_object_name_ == std::nullopt ||\n      (writing_temp_object_ && start_pos() == initial_temp_object_pos_)) {\n    return true;\n  }\n  dest().Close();\n  if (ABSL_PREDICT_FALSE(dest().fail())) {\n    FailOperation(\"ObjectWriteStream::Close()\");\n    PropagateStatus();\n    return false;\n  }\n  if (writing_temp_object_) {\n    google::cloud::StatusOr<google::cloud::storage::ObjectMetadata> metadata =\n        compose_object_(*this);\n    if (ABSL_PREDICT_FALSE(!metadata.ok())) {\n      return Fail(gcs_internal::FromCloudStatus(metadata.status()));\n    }\n    metadata_ = *std::move(metadata);\n  }\n  dest() = write_temp_object_(*this);\n  writing_temp_object_ = true;\n  initial_temp_object_pos_ = start_pos();\n  if (ABSL_PREDICT_FALSE(dest().fail())) {\n    FailOperation(\"Client::WriteObject()\");\n    PropagateStatus();\n    return false;\n  }\n  return true;\n}\n\nbool GcsWriter::SupportsReadMode() { return temp_object_name_ != std::nullopt; }\n\nReader* GcsWriter::ReadModeBehindBuffer(Position initial_pos) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::ReadModeBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!GcsWriter::SupportsReadMode())) {\n    // Delegate to the base class to avoid repeating the error message.\n    return OStreamWriter::ReadModeBehindBuffer(initial_pos);\n  }\n  if (ABSL_PREDICT_FALSE(!GcsWriter::FlushImpl(FlushType::kFromObject))) {\n    return nullptr;\n  }\n  // `ObjectReadStream` does not support seeking to the very end.\n  const Position read_from_offset =\n      UnsignedMin(initial_pos, SaturatingSub(start_pos(), Position{1}));\n  GcsReader* const reader = make_gcs_reader_(*this, read_from_offset);\n  reader->set_exact_size(start_pos());\n  if (initial_pos > read_from_offset) {\n    if (ABSL_PREDICT_TRUE(reader->Pull())) reader->move_cursor(1);\n  }\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/gcs/gcs_writer.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_GCS_GCS_WRITER_H_\n#define RIEGELI_GCS_GCS_WRITER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <functional>\n#include <optional>\n#include <string>\n#include <tuple>\n#include <type_traits>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/cloud/status.h\"\n#include \"google/cloud/status_or.h\"\n#include \"google/cloud/storage/auto_finalize.h\"\n#include \"google/cloud/storage/client.h\"\n#include \"google/cloud/storage/download_options.h\"\n#include \"google/cloud/storage/object_metadata.h\"\n#include \"google/cloud/storage/object_write_stream.h\"\n#include \"google/cloud/storage/upload_options.h\"\n#include \"google/cloud/storage/user_ip_option.h\"\n#include \"google/cloud/storage/well_known_headers.h\"\n#include \"google/cloud/storage/well_known_parameters.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/string_ref.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/ostream_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/gcs/gcs_internal.h\"\n#include \"riegeli/gcs/gcs_object.h\"\n#include \"riegeli/gcs/gcs_reader.h\"\n\nnamespace riegeli {\n\n// A `Writer` which writes to a Google Cloud Storage object.\nclass GcsWriter\n    : public OStreamWriter<google::cloud::storage::ObjectWriteStream> {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // The effective buffer size is always a multiple of 256 KiB, and is at\n    // least the `UploadBufferSize` setting in the client (which defaults to\n    // 8 MiB).\n    static constexpr size_t kDefaultMinBufferSize = size_t{8} << 20;\n    static constexpr size_t kDefaultMaxBufferSize = size_t{8} << 20;\n\n    // If `std::nullopt`, writing os performed directly to the destination\n    // object. `Flush()` is not effective, `append()` and `ReadMode()` are not\n    // supported, but `UseResumableUploadSession()` is supported.\n    //\n    // If not `std::nullopt`, writing is performed either directly to the\n    // destination object, or to a temporary object with this name, which is\n    // then composed with the destination object and deleted. `Flush()` is\n    // effective, `append()` and `ReadMode()` are supported, but\n    // `UseResumableUploadSession()` is not supported.\n    //\n    // Default: `std::nullopt`.\n    Options& set_temp_object_name(StringInitializer temp_object_name) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      riegeli::Reset(temp_object_name_, std::move(temp_object_name));\n      return *this;\n    }\n    Options&& set_temp_object_name(StringInitializer temp_object_name) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_temp_object_name(std::move(temp_object_name)));\n    }\n    std::optional<std::string>& temp_object_name()\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return temp_object_name_;\n    }\n    const std::optional<std::string>& temp_object_name() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return temp_object_name_;\n    }\n\n    // If `false`, replaces existing contents of the destination object,\n    // clearing it first.\n    //\n    // If `true`, appends to existing contents of the destination object.\n    // This requires `temp_object_name() != std::nullopt`.\n    //\n    // Default: `false`.\n    Options& set_append(bool append) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      append_ = append;\n      return *this;\n    }\n    Options&& set_append(bool append) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_append(append));\n    }\n    bool append() const { return append_; }\n\n    // Buffer options for `GcsReader` returned by `ReadMode()`.\n    //\n    // Default: like in `GcsReader::Options`, i.e.\n    //   `BufferOptions()\n    //        .set_min_buffer_size(size_t{64} << 10)\n    //        .set_max_buffer_size(size_t{1} << 20)`.\n    Options& set_read_mode_buffer_options(\n        BufferOptions read_mode_buffer_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      read_mode_buffer_options_ = read_mode_buffer_options;\n      return *this;\n    }\n    Options&& set_read_mode_buffer_options(\n        BufferOptions read_mode_buffer_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_read_mode_buffer_options(read_mode_buffer_options));\n    }\n    BufferOptions read_mode_buffer_options() const {\n      return read_mode_buffer_options_;\n    }\n\n   private:\n    std::optional<std::string> temp_object_name_;\n    bool append_ = false;\n    BufferOptions read_mode_buffer_options_ =\n        BufferOptions()\n            .set_min_buffer_size(size_t{64} << 10)\n            .set_max_buffer_size(size_t{1} << 20);\n  };\n\n  // Creates a closed `GcsWriter`.\n  explicit GcsWriter(Closed) : OStreamWriter(kClosed) {}\n\n  // Will write to `object`.\n  //\n  // If an upload session is being resumed, `pos()` corresponds to the next\n  // position to write. `GcsWriter` is left closed if the resumed upload session\n  // is finalized.\n  template <typename... WriteObjectOptions>\n  explicit GcsWriter(const google::cloud::storage::Client& client,\n                     Initializer<GcsObject> object, Options options,\n                     WriteObjectOptions&&... write_object_options);\n  template <typename... WriteObjectOptions>\n  explicit GcsWriter(const google::cloud::storage::Client& client,\n                     Initializer<GcsObject> object,\n                     WriteObjectOptions&&... write_object_options);\n\n  GcsWriter(GcsWriter&& that) noexcept;\n  GcsWriter& operator=(GcsWriter&& that) noexcept;\n\n  // Makes `*this` equivalent to a newly constructed `GcsWriter`. This avoids\n  // constructing a temporary `GcsWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  template <typename... WriteObjectOptions>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      const google::cloud::storage::Client& client,\n      Initializer<GcsObject> object, Options options,\n      WriteObjectOptions&&... write_object_options);\n  template <typename... WriteObjectOptions>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      const google::cloud::storage::Client& client,\n      Initializer<GcsObject> object,\n      WriteObjectOptions&&... write_object_options);\n\n  google::cloud::storage::Client& client() {\n    RIEGELI_ASSERT(client_ != std::nullopt)\n        << \"Failed precondition of GcsWriter::client(): \"\n           \"default-constructed GcsWriter\";\n    return *client_;\n  }\n  const GcsObject& object() const { return object_; }\n  std::optional<absl::string_view> temp_object_name() const {\n    return temp_object_name_;\n  }\n\n  // Returns metadata about upload results.\n  //\n  // Precondition: `Close()` succeeded or `GcsWriter` was created with a resumed\n  // upload session which was finalized.\n  const google::cloud::storage::ObjectMetadata& metadata() const&;\n  google::cloud::storage::ObjectMetadata&& metadata() &&;\n\n  // Returns the resumable upload session id for this upload, or an empty string\n  // for uploads that do not use resumable upload session ids.\n  absl::string_view resumable_session_id() const {\n    return dest().resumable_session_id();\n  }\n\n  bool SupportsReadMode() override;\n\n protected:\n  void Done() override;\n  absl::Status AnnotateStatusImpl(absl::Status status) override;\n  bool WriteInternal(absl::string_view src) override;\n  bool FlushImpl(FlushType flush_type) override;\n  Reader* ReadModeBehindBuffer(Position initial_pos) override;\n\n private:\n  template <typename WriteObjectOption>\n  struct CommonOptionPredicate\n      : std::disjunction<std::is_same<std::decay_t<WriteObjectOption>,\n                                      google::cloud::storage::QuotaUser>,\n                         std::is_same<std::decay_t<WriteObjectOption>,\n                                      google::cloud::storage::UserIp>,\n                         std::is_same<std::decay_t<WriteObjectOption>,\n                                      google::cloud::storage::UserProject>> {};\n\n  template <typename WriteObjectOption>\n  struct ComposeObjectOptionPredicate\n      : std::disjunction<CommonOptionPredicate<WriteObjectOption>,\n                         std::is_same<std::decay_t<WriteObjectOption>,\n                                      google::cloud::storage::EncryptionKey>,\n                         std::is_same<std::decay_t<WriteObjectOption>,\n                                      google::cloud::storage::Fields>,\n                         std::is_same<std::decay_t<WriteObjectOption>,\n                                      google::cloud::storage::KmsKeyName>> {};\n\n  template <typename WriteObjectOption>\n  struct DeleteTempObjectOptionPredicate\n      : CommonOptionPredicate<WriteObjectOption> {};\n\n  template <typename WriteObjectOption>\n  struct GetObjectSizeOptionPredicate\n      : CommonOptionPredicate<WriteObjectOption> {};\n\n  template <typename WriteObjectOption>\n  struct MakeGcsReaderOptionPredicate\n      : CommonOptionPredicate<WriteObjectOption> {};\n\n  template <typename... ComposeObjectOptions, size_t... indices>\n  google::cloud::StatusOr<google::cloud::storage::ObjectMetadata>\n  ApplyComposeObject(\n      const std::tuple<ComposeObjectOptions...>& compose_object_options,\n      std::index_sequence<indices...>) {\n    std::vector<google::cloud::storage::ComposeSourceObject> source_objects;\n    source_objects.reserve(2);\n#if __cpp_aggregate_paren_init\n    source_objects.emplace_back(\n        object_.object_name(),\n        already_composed_ ? std::nullopt : object_.generation(), std::nullopt);\n    source_objects.emplace_back(*temp_object_name_, std::nullopt, std::nullopt);\n#else\n    source_objects.push_back(google::cloud::storage::ComposeSourceObject{\n        object_.object_name(),\n        already_composed_ ? std::nullopt : object_.generation(), std::nullopt});\n    source_objects.push_back(google::cloud::storage::ComposeSourceObject{\n        *temp_object_name_, std::nullopt, std::nullopt});\n#endif\n    already_composed_ = true;\n    return client_->ComposeObject(object_.bucket_name(), source_objects,\n                                  object_.object_name(),\n                                  std::get<indices>(compose_object_options)...);\n  }\n\n  template <typename... WriteObjectOptions, size_t... indices,\n            typename... ExtraOptions>\n  google::cloud::storage::ObjectWriteStream ApplyWriteTempObject(\n      const std::tuple<WriteObjectOptions...>& write_object_options,\n      std::index_sequence<indices...>, ExtraOptions&&... extra_options) {\n    return client_->WriteObject(object_.bucket_name(), *temp_object_name_,\n                                std::get<indices>(write_object_options)...,\n                                std::forward<ExtraOptions>(extra_options)...,\n                                google::cloud::storage::Fields(\"\"),\n                                google::cloud::storage::AutoFinalizeDisabled());\n  }\n\n  template <typename... DeleteObjectOptions, size_t... indices>\n  google::cloud::Status ApplyDeleteTempObject(\n      const std::tuple<DeleteObjectOptions...>& delete_object_options,\n      std::index_sequence<indices...>) {\n    return client_->DeleteObject(object_.bucket_name(), *temp_object_name_,\n                                 std::get<indices>(delete_object_options)...,\n                                 google::cloud::storage::Fields(\"\"));\n  }\n\n  template <typename... GetObjectMetadataOptions, size_t... indices>\n  google::cloud::StatusOr<google::cloud::storage::ObjectMetadata>\n  ApplyGetObjectSize(const std::tuple<GetObjectMetadataOptions...>&\n                         get_object_metadata_options,\n                     std::index_sequence<indices...>) {\n    return object_.generation() == std::nullopt\n               ? client_->GetObjectMetadata(\n                     object_.bucket_name(), object_.object_name(),\n                     std::get<indices>(get_object_metadata_options)...,\n                     google::cloud::storage::Fields(\"size\"))\n               : client_->GetObjectMetadata(\n                     object_.bucket_name(), object_.object_name(),\n                     google::cloud::storage::Generation(*object_.generation()),\n                     std::get<indices>(get_object_metadata_options)...,\n                     google::cloud::storage::Fields(\"size\"));\n  }\n\n  template <typename... ReadObjectOptions, size_t... indices>\n  GcsReader* ApplyMakeGcsReader(\n      const std::tuple<ReadObjectOptions...>& read_object_options,\n      std::index_sequence<indices...>, int64_t read_from_offset) {\n    return associated_reader_.ResetReader(\n        client(),\n        already_composed_\n            ? GcsObject(object().bucket_name(), object().object_name())\n            : object(),\n        GcsReader::Options().set_buffer_options(read_mode_buffer_options_),\n        std::get<indices>(read_object_options)...,\n        google::cloud::storage::ReadFromOffset(read_from_offset));\n  }\n\n  bool CheckPreconditions(bool append, bool resumable_upload_session);\n  template <typename... WriteObjectOptions>\n  void Initialize(const Options& options,\n                  WriteObjectOptions&&... write_object_options);\n  template <typename... WriteObjectOptions>\n  void SetFunctions(const WriteObjectOptions&... write_object_options);\n  void Initialize(google::cloud::storage::ObjectWriteStream stream,\n                  BufferOptions buffer_options, bool append);\n  void PropagateStatus();\n  ABSL_ATTRIBUTE_COLD void PropagateStatusSlow(\n      const google::cloud::Status& status);\n\n  std::optional<google::cloud::storage::Client> client_;\n  GcsObject object_;\n  std::optional<std::string> temp_object_name_;\n  BufferOptions read_mode_buffer_options_;\n  std::function<google::cloud::storage::ObjectWriteStream(GcsWriter&)>\n      write_temp_object_;\n  std::function<google::cloud::StatusOr<google::cloud::storage::ObjectMetadata>(\n      GcsWriter&)>\n      compose_object_;\n  std::function<google::cloud::Status(GcsWriter&)> delete_temp_object_;\n  std::function<google::cloud::StatusOr<google::cloud::storage::ObjectMetadata>(\n      GcsWriter&)>\n      get_object_size_;\n  std::function<GcsReader*(GcsWriter&, int64_t)> make_gcs_reader_;\n  bool writing_temp_object_ = false;\n  bool already_composed_ = false;\n  Position initial_temp_object_pos_ = 0;\n  std::optional<google::cloud::storage::ObjectMetadata> metadata_;\n\n  AssociatedReader<GcsReader> associated_reader_;\n};\n\n// Implementation details follow.\n\ntemplate <typename... WriteObjectOptions>\nGcsWriter::GcsWriter(const google::cloud::storage::Client& client,\n                     Initializer<GcsObject> object, Options options,\n                     WriteObjectOptions&&... write_object_options)\n    : OStreamWriter(kClosed),\n      client_(client),\n      object_(std::move(object)),\n      temp_object_name_(std::move(options.temp_object_name())),\n      read_mode_buffer_options_(options.read_mode_buffer_options()) {\n  Initialize(options,\n             std::forward<WriteObjectOptions>(write_object_options)...);\n}\n\ntemplate <typename... WriteObjectOptions>\nGcsWriter::GcsWriter(const google::cloud::storage::Client& client,\n                     Initializer<GcsObject> object,\n                     WriteObjectOptions&&... write_object_options)\n    : GcsWriter(client, std::move(object), Options(),\n                std::forward<WriteObjectOptions>(write_object_options)...) {}\n\ninline GcsWriter::GcsWriter(GcsWriter&& that) noexcept\n    : OStreamWriter(static_cast<OStreamWriter&&>(that)),\n      client_(std::move(that.client_)),\n      object_(std::move(that.object_)),\n      temp_object_name_(std::exchange(that.temp_object_name_, std::nullopt)),\n      read_mode_buffer_options_(that.read_mode_buffer_options_),\n      write_temp_object_(std::move(that.write_temp_object_)),\n      compose_object_(std::move(that.compose_object_)),\n      delete_temp_object_(std::move(that.delete_temp_object_)),\n      get_object_size_(std::move(that.get_object_size_)),\n      make_gcs_reader_(std::move(that.make_gcs_reader_)),\n      writing_temp_object_(that.writing_temp_object_),\n      already_composed_(that.already_composed_),\n      initial_temp_object_pos_(that.initial_temp_object_pos_),\n      metadata_(std::exchange(that.metadata_, std::nullopt)),\n      associated_reader_(std::move(that.associated_reader_)) {}\n\ninline GcsWriter& GcsWriter::operator=(GcsWriter&& that) noexcept {\n  OStreamWriter::operator=(static_cast<OStreamWriter&&>(that));\n  client_ = std::move(that.client_);\n  object_ = std::move(that.object_);\n  temp_object_name_ = std::exchange(that.temp_object_name_, std::nullopt);\n  read_mode_buffer_options_ = that.read_mode_buffer_options_;\n  write_temp_object_ = std::move(that.write_temp_object_);\n  compose_object_ = std::move(that.compose_object_);\n  delete_temp_object_ = std::move(that.delete_temp_object_);\n  get_object_size_ = std::move(that.get_object_size_);\n  make_gcs_reader_ = std::move(that.make_gcs_reader_);\n  writing_temp_object_ = that.writing_temp_object_;\n  already_composed_ = that.already_composed_;\n  initial_temp_object_pos_ = that.initial_temp_object_pos_;\n  metadata_ = std::exchange(that.metadata_, std::nullopt);\n  associated_reader_ = std::move(that.associated_reader_);\n  return *this;\n}\n\ntemplate <typename... WriteObjectOptions>\nvoid GcsWriter::Reset(const google::cloud::storage::Client& client,\n                      Initializer<GcsObject> object, Options options,\n                      WriteObjectOptions&&... write_object_options) {\n  OStreamWriter::Reset(kClosed);\n  client_ = client;\n  riegeli::Reset(object_, std::move(object));\n  temp_object_name_ = std::move(options.temp_object_name());\n  read_mode_buffer_options_ = options.read_mode_buffer_options();\n  write_temp_object_ = nullptr;\n  compose_object_ = nullptr;\n  delete_temp_object_ = nullptr;\n  get_object_size_ = nullptr;\n  make_gcs_reader_ = nullptr;\n  writing_temp_object_ = false;\n  already_composed_ = false;\n  initial_temp_object_pos_ = 0;\n  metadata_ = std::nullopt;\n  associated_reader_.Reset();\n  Initialize(options,\n             std::forward<WriteObjectOptions>(write_object_options)...);\n}\n\ntemplate <typename... WriteObjectOptions>\nvoid GcsWriter::Reset(const google::cloud::storage::Client& client,\n                      Initializer<GcsObject> object,\n                      WriteObjectOptions&&... write_object_options) {\n  Reset(client, std::move(object), Options(),\n        std::forward<WriteObjectOptions>(write_object_options)...);\n}\n\ntemplate <typename... WriteObjectOptions>\ninline void GcsWriter::Initialize(\n    const Options& options, WriteObjectOptions&&... write_object_options) {\n  if (ABSL_PREDICT_FALSE(!object_.ok())) {\n    Fail(object_.status());\n    return;\n  }\n  if (ABSL_PREDICT_FALSE(!CheckPreconditions(\n          options.append(),\n          gcs_internal::GetOption<\n              google::cloud::storage::UseResumableUploadSession>(\n              write_object_options...)\n              .has_value()))) {\n    return;\n  }\n  if (temp_object_name_ != std::nullopt) SetFunctions(write_object_options...);\n  Initialize(\n      options.append()\n          ? client_->WriteObject(\n                object_.bucket_name(), object_.object_name(),\n                std::forward<WriteObjectOptions>(write_object_options)...,\n                google::cloud::storage::IfGenerationMatch(0),\n                google::cloud::storage::AutoFinalizeDisabled())\n          : client_->WriteObject(\n                object_.bucket_name(), object_.object_name(),\n                std::forward<WriteObjectOptions>(write_object_options)...,\n                google::cloud::storage::AutoFinalizeDisabled()),\n      options.buffer_options(), options.append());\n}\n\ntemplate <typename... WriteObjectOptions>\ninline void GcsWriter::SetFunctions(\n    const WriteObjectOptions&... write_object_options) {\n  write_temp_object_ =\n      [write_temp_object_options = std::tuple<WriteObjectOptions...>(\n           write_object_options...)](GcsWriter& self) mutable {\n        return self.ApplyWriteTempObject(\n            write_temp_object_options,\n            std::make_index_sequence<\n                std::tuple_size_v<decltype(write_temp_object_options)>>());\n      };\n  compose_object_ =\n      [compose_object_options = DecayTuple(Filter<ComposeObjectOptionPredicate>(\n           write_object_options...))](GcsWriter& self) mutable {\n        return self.ApplyComposeObject(\n            compose_object_options,\n            std::make_index_sequence<\n                std::tuple_size_v<decltype(compose_object_options)>>());\n      };\n  delete_temp_object_ =\n      [delete_temp_object_options = DecayTuple(\n           Filter<DeleteTempObjectOptionPredicate>(write_object_options...))](\n          GcsWriter& self) mutable {\n        return self.ApplyDeleteTempObject(\n            delete_temp_object_options,\n            std::make_index_sequence<\n                std::tuple_size_v<decltype(delete_temp_object_options)>>());\n      };\n  get_object_size_ =\n      [get_object_size_options = DecayTuple(\n           Filter<GetObjectSizeOptionPredicate>(write_object_options...))](\n          GcsWriter& self) mutable {\n        return self.ApplyGetObjectSize(\n            get_object_size_options,\n            std::make_index_sequence<\n                std::tuple_size_v<decltype(get_object_size_options)>>());\n      };\n  make_gcs_reader_ =\n      [gcs_reader_options = DecayTuple(\n           Filter<MakeGcsReaderOptionPredicate>(write_object_options...))](\n          GcsWriter& self, int64_t read_from_offset) mutable {\n        return self.ApplyMakeGcsReader(\n            gcs_reader_options,\n            std::make_index_sequence<\n                std::tuple_size_v<decltype(gcs_reader_options)>>(),\n            read_from_offset);\n      };\n}\n\ninline const google::cloud::storage::ObjectMetadata& GcsWriter::metadata()\n    const& {\n  if (metadata_ != std::nullopt) return *metadata_;\n  const google::cloud::StatusOr<google::cloud::storage::ObjectMetadata>&\n      metadata = dest().metadata();\n  RIEGELI_ASSERT_OK(metadata)\n      << \"Failed precondition of GcsWriter::metadata(): \"\n         \"Close() must have succeeded\";\n  return *metadata;\n}\n\ninline google::cloud::storage::ObjectMetadata&& GcsWriter::metadata() && {\n  if (metadata_ != std::nullopt) return *std::move(metadata_);\n  google::cloud::StatusOr<google::cloud::storage::ObjectMetadata>&& metadata =\n      std::move(dest()).metadata();\n  RIEGELI_ASSERT_OK(metadata)\n      << \"Failed precondition of GcsWriter::metadata(): \"\n         \"Close() must have succeeded\";\n  return *std::move(metadata);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_GCS_GCS_WRITER_H_\n"
  },
  {
    "path": "riegeli/lines/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"newline\",\n    hdrs = [\"newline.h\"],\n    deps = [\"@com_google_absl//absl/strings:string_view\"],\n)\n\ncc_library(\n    name = \"line_reading\",\n    srcs = [\"line_reading.cc\"],\n    hdrs = [\"line_reading.h\"],\n    deps = [\n        \":newline\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/bytes:reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"line_writing\",\n    hdrs = [\"line_writing.h\"],\n    deps = [\n        \":newline\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:type_traits\",\n        \"//riegeli/bytes:stringify\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n    ],\n)\n\ncc_library(\n    name = \"text_reader\",\n    srcs = [\"text_reader.cc\"],\n    hdrs = [\"text_reader.h\"],\n    deps = [\n        \":newline\",\n        \"//riegeli/base:any\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_reader\",\n        \"//riegeli/bytes:prefix_limiting_reader\",\n        \"//riegeli/bytes:reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n    ],\n)\n\ncc_library(\n    name = \"text_writer\",\n    srcs = [\"text_writer.cc\"],\n    hdrs = [\"text_writer.h\"],\n    deps = [\n        \":line_writing\",\n        \":newline\",\n        \"//riegeli/base:any\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_writer\",\n        \"//riegeli/bytes:prefix_limiting_writer\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/lines/line_reading.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/lines/line_reading.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <string>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/lines/newline.h\"\n\nnamespace riegeli {\n\nnamespace {\n\n// Reads `length_to_read` bytes from `src`, writes their prefix of\n// `length_to_write` bytes to `dest`, appending to existing contents\n// (unless `Dest` is `absl::string_view`).\n//\n// The data to read must be already available in the buffer.\n\ntemplate <typename Dest>\ninline void ReadFlatAndSkip(Reader& src, size_t length_to_read,\n                            size_t length_to_write, Dest& dest) {\n  RIEGELI_ASSERT_LE(length_to_read, src.available())\n      << \"Failed precondition of ReadFlatAndSkip(): \"\n         \"reading more than buffered\";\n  RIEGELI_ASSERT_LE(length_to_write, length_to_read)\n      << \"Failed precondition of ReadFlatAndSkip(): \"\n         \"writing more than reading\";\n  src.ReadAndAppend(length_to_write, dest);\n  src.Skip(length_to_read - length_to_write);\n}\n\ninline void ReadFlatAndSkip(Reader& src, size_t length_to_read,\n                            size_t length_to_write, absl::string_view& dest) {\n  RIEGELI_ASSERT_LE(length_to_read, src.available())\n      << \"Failed precondition of ReadFlatAndSkip(): \"\n         \"reading more than buffered\";\n  RIEGELI_ASSERT_LE(length_to_write, length_to_read)\n      << \"Failed precondition of ReadFlatAndSkip(): \"\n         \"writing more than reading\";\n  dest = absl::string_view(src.cursor(), length_to_write);\n  src.move_cursor(length_to_read);\n}\n\ninline void ReadFlatAndSkip(Reader& src, size_t length_to_read,\n                            size_t length_to_write, std::string& dest) {\n  RIEGELI_ASSERT_LE(length_to_read, src.available())\n      << \"Failed precondition of ReadFlatAndSkip(): \"\n         \"reading more than buffered\";\n  RIEGELI_ASSERT_LE(length_to_write, length_to_read)\n      << \"Failed precondition of ReadFlatAndSkip(): \"\n         \"writing more than reading\";\n  dest.append(src.cursor(), length_to_write);\n  src.move_cursor(length_to_read);\n}\n\ntemplate <typename Dest>\ninline void ReadFlat(Reader& src, size_t length, Dest& dest) {\n  return ReadFlatAndSkip(src, length, length, dest);\n}\n\ntemplate <typename Dest>\nABSL_ATTRIBUTE_COLD bool FailMaxLineLengthExceeded(Reader& src, Dest& dest,\n                                                   size_t max_length) {\n  ReadFlat(src, max_length, dest);\n  return src.Fail(absl::ResourceExhaustedError(\n      absl::StrCat(\"Maximum line length exceeded: \", max_length)));\n}\n\ntemplate <typename Dest>\ninline bool FoundNewline(Reader& src, Dest& dest, ReadLineOptions options,\n                         size_t length, size_t newline_length) {\n  const size_t length_with_newline = length + newline_length;\n  if (options.keep_newline()) length = length_with_newline;\n  if (ABSL_PREDICT_FALSE(length > options.max_length())) {\n    return FailMaxLineLengthExceeded(src, dest, options.max_length());\n  }\n  ReadFlatAndSkip(src, length_with_newline, length, dest);\n  return true;\n}\n\ntemplate <typename Dest>\ninline bool ReadLineInternal(Reader& src, Dest& dest, ReadLineOptions options) {\n  if (ABSL_PREDICT_FALSE(!src.Pull())) return false;\n  size_t length;\n  do {\n    switch (options.newline()) {\n      case ReadNewline::kLf: {\n        const char* const newline = static_cast<const char*>(\n            std::memchr(src.cursor(), '\\n', src.available()));\n        if (ABSL_PREDICT_TRUE(newline != nullptr)) {\n          return FoundNewline(src, dest, options,\n                              PtrDistance(src.cursor(), newline), 1);\n        }\n        length = src.available();\n        goto continue_reading;\n      }\n      case ReadNewline::kCrLfOrLf: {\n        const char* newline = static_cast<const char*>(\n            std::memchr(src.cursor(), '\\n', src.available()));\n        for (;;) {\n          if (ABSL_PREDICT_TRUE(newline != nullptr)) {\n            length = PtrDistance(src.cursor(), newline);\n            if (length > 0 && newline[-1] == '\\r') {\n              return FoundNewline(src, dest, options, length - 1, 2);\n            }\n            return FoundNewline(src, dest, options, length, 1);\n          }\n          if (ABSL_PREDICT_TRUE(src.limit()[-1] != '\\r')) {\n            length = src.available();\n            goto continue_reading;\n          }\n          // The buffer ends with CR.\n          length = src.available() - 1;\n          if (ABSL_PREDICT_TRUE(length > 0)) {\n            // The CR is not first in the buffer. Move line read so far to\n            // `dest` to avoid copying that part during flattening of the CR\n            // together with the next buffer.\n            goto continue_reading;\n          }\n          // The buffer contains only CR.\n          if (ABSL_PREDICT_FALSE(!src.Pull(2))) {\n            // The CR is the final character and is not a part of a line\n            // terminator.\n            if (ABSL_PREDICT_FALSE(options.max_length() < 1)) {\n              return FailMaxLineLengthExceeded(src, dest, options.max_length());\n            }\n            ReadFlat(src, 1, dest);\n            return src.ok();\n          }\n          // The buffer begins with CR.\n          if (ABSL_PREDICT_TRUE(src.cursor()[1] == '\\n')) {\n            return FoundNewline(src, dest, options, 0, 2);\n          }\n          // The CR is not a part of a line terminator. Search for LF again.\n          newline = static_cast<const char*>(\n              std::memchr(src.cursor() + 2, '\\n', src.available() - 2));\n        }\n      }\n    }\n    RIEGELI_ASSUME_UNREACHABLE()\n        << \"Unknown newline: \" << static_cast<int>(options.newline());\n  continue_reading:\n    if (ABSL_PREDICT_FALSE(length > options.max_length())) {\n      return FailMaxLineLengthExceeded(src, dest, options.max_length());\n    }\n    options.set_max_length(options.max_length() - length);\n    ReadFlat(src, length, dest);\n  } while (src.Pull());\n  return src.ok();\n}\n\n}  // namespace\n\nbool ReadLine(Reader& src, absl::string_view& dest, ReadLineOptions options) {\n  size_t length = 0;\n  if (ABSL_PREDICT_FALSE(!src.Pull())) {\n    dest = absl::string_view();\n    return false;\n  }\n  do {\n    switch (options.newline()) {\n      case ReadNewline::kLf: {\n        const char* const newline = static_cast<const char*>(\n            std::memchr(src.cursor() + length, '\\n', src.available() - length));\n        if (ABSL_PREDICT_TRUE(newline != nullptr)) {\n          return FoundNewline(src, dest, options,\n                              PtrDistance(src.cursor(), newline), 1);\n        }\n        goto continue_reading;\n      }\n      case ReadNewline::kCrLfOrLf:\n        for (;;) {\n          const char* const newline = static_cast<const char*>(std::memchr(\n              src.cursor() + length, '\\n', src.available() - length));\n          if (ABSL_PREDICT_TRUE(newline != nullptr)) {\n            length = PtrDistance(src.cursor(), newline);\n            if (length > 0 && newline[-1] == '\\r') {\n              return FoundNewline(src, dest, options, length - 1, 2);\n            }\n            return FoundNewline(src, dest, options, length, 1);\n          }\n          if (ABSL_PREDICT_TRUE(src.limit()[-1] != '\\r')) goto continue_reading;\n          // The buffer ends with CR.\n          length = src.available() - 1;\n          if (ABSL_PREDICT_FALSE(length > options.max_length())) {\n            return FailMaxLineLengthExceeded(src, dest, options.max_length());\n          }\n          if (ABSL_PREDICT_FALSE(!src.Pull(length + 2))) {\n            // The CR is the final character and is not a part of a line\n            // terminator.\n            if (ABSL_PREDICT_FALSE(src.available() > options.max_length())) {\n              return FailMaxLineLengthExceeded(src, dest, options.max_length());\n            }\n            dest = absl::string_view(src.cursor(), src.available());\n            src.move_cursor(src.available());\n            return src.ok();\n          }\n          if (ABSL_PREDICT_TRUE(src.cursor()[length + 1] == '\\n')) {\n            return FoundNewline(src, dest, options, length, 2);\n          }\n          // The CR at `src.cursor()[length]` is not a part of a line\n          // terminator. Search for LF again.\n          length += 2;\n        }\n    }\n    RIEGELI_ASSUME_UNREACHABLE()\n        << \"Unknown newline: \" << static_cast<int>(options.newline());\n  continue_reading:\n    length = src.available();\n    if (ABSL_PREDICT_FALSE(length > options.max_length())) {\n      return FailMaxLineLengthExceeded(src, dest, options.max_length());\n    }\n  } while (src.Pull(length + 1));\n  dest = absl::string_view(src.cursor(), src.available());\n  src.move_cursor(src.available());\n  return src.ok();\n}\n\nbool ReadLine(Reader& src, std::string& dest, ReadLineOptions options) {\n  dest.clear();\n  return ReadLineInternal(src, dest, options);\n}\n\nbool ReadLine(Reader& src, Chain& dest, ReadLineOptions options) {\n  dest.Clear();\n  return ReadLineInternal(src, dest, options);\n}\n\nbool ReadLine(Reader& src, absl::Cord& dest, ReadLineOptions options) {\n  dest.Clear();\n  return ReadLineInternal(src, dest, options);\n}\n\nvoid SkipUtf8Bom(Reader& src) {\n  if (src.Pull(kUtf8Bom.size()) &&\n      absl::string_view(src.cursor(), kUtf8Bom.size()) == kUtf8Bom) {\n    src.move_cursor(kUtf8Bom.size());\n  }\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/lines/line_reading.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_LINES_LINE_READING_H_\n#define RIEGELI_LINES_LINE_READING_H_\n\n#include <stddef.h>\n\n#include <limits>\n#include <string>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/lines/newline.h\"\n\nnamespace riegeli {\n\n// Options for `ReadLine()`.\nclass ReadLineOptions {\n public:\n  ReadLineOptions() noexcept {}\n\n  // Options can also be specified by the line terminator alone.\n  /*implicit*/ ReadLineOptions(ReadNewline newline) : newline_(newline) {}\n\n  // Line terminator representations to recognize.\n  //\n  // Default: `ReadNewline::kCrLfOrLf`.\n  ReadLineOptions& set_newline(ReadNewline newline) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    newline_ = newline;\n    return *this;\n  }\n  ReadLineOptions&& set_newline(ReadNewline newline) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_newline(newline));\n  }\n  ReadNewline newline() const { return newline_; }\n\n  // If `false`, line terminators are stripped.\n  //\n  // If `true`, each returned line includes its terminator if it was present\n  // (it can be absent in the last line).\n  //\n  // Default: `false`.\n  ReadLineOptions& set_keep_newline(bool keep_newline) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    keep_newline_ = keep_newline;\n    return *this;\n  }\n  ReadLineOptions&& set_keep_newline(bool keep_newline) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_keep_newline(keep_newline));\n  }\n  bool keep_newline() const { return keep_newline_; }\n\n  // Expected maximum line length.\n  //\n  // If this length is exceeded, reading fails with\n  // `absl::ResourceExhaustedError()`.\n  //\n  // Default: `std::numeric_limits<size_t>::max()`.\n  ReadLineOptions& set_max_length(size_t max_length) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    max_length_ = max_length;\n    return *this;\n  }\n  ReadLineOptions&& set_max_length(size_t max_length) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_max_length(max_length));\n  }\n  size_t max_length() const { return max_length_; }\n\n private:\n  ReadNewline newline_ = ReadNewline::kCrLfOrLf;\n  bool keep_newline_ = false;\n  size_t max_length_ = std::numeric_limits<size_t>::max();\n};\n\n// Reads a line.\n//\n// Line terminator after the last line is optional.\n//\n// Return values:\n//  * `true`                     - success (`dest` is set)\n//  * `false` (when `src.ok()`)  - source ends (`dest` is empty)\n//  * `false` (when `!src.ok()`) - failure (`dest` is set to the partial line\n//                                 read before the failure)\nbool ReadLine(Reader& src, absl::string_view& dest,\n              ReadLineOptions options = ReadLineOptions());\nbool ReadLine(Reader& src, std::string& dest,\n              ReadLineOptions options = ReadLineOptions());\nbool ReadLine(Reader& src, Chain& dest,\n              ReadLineOptions options = ReadLineOptions());\nbool ReadLine(Reader& src, absl::Cord& dest,\n              ReadLineOptions options = ReadLineOptions());\n\n// Skips UTF-8 BOM if it is present.\nvoid SkipUtf8Bom(Reader& src);\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_LINES_LINE_READING_H_\n"
  },
  {
    "path": "riegeli/lines/line_writing.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_LINES_LINE_WRITING_H_\n#define RIEGELI_LINES_LINE_WRITING_H_\n\n#include <stddef.h>\n\n#include <tuple>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/bytes/stringify.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/lines/newline.h\"\n\nnamespace riegeli {\n\n// Options for `WriteLine()`.\nclass WriteLineOptions {\n public:\n  WriteLineOptions() noexcept {}\n\n  // Options can also be specified by the line terminator alone.\n  /*implicit*/ WriteLineOptions(WriteNewline newline) : newline_(newline) {}\n\n  // Line terminator representation to write.\n  //\n  // Default: `WriteNewline::kNative`.\n  WriteLineOptions& set_newline(WriteNewline newline) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    newline_ = newline;\n    return *this;\n  }\n  WriteLineOptions&& set_newline(WriteNewline newline) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_newline(newline));\n  }\n  WriteNewline newline() const { return newline_; }\n\n private:\n  WriteNewline newline_ = WriteNewline::kNative;\n};\n\n// Writes stringified values, then a line terminator.\n//\n// The last one or two arguments are the `Writer&`, optionally followed by\n// `WriteLineOptions`. The remaining arguments are the values.\n//\n// Return values:\n//  * `true`  - success\n//  * `false` - failure (`!ok()`)\ntemplate <typename... Args,\n          std::enable_if_t<\n              std::conjunction_v<\n                  std::is_convertible<GetTypeFromEndT<1, Args&&...>, Writer&>,\n                  TupleElementsSatisfy<RemoveTypesFromEndT<1, Args&&...>,\n                                       IsStringifiable>>,\n              int> = 0>\nbool WriteLine(Args&&... args);\ntemplate <typename... Args,\n          std::enable_if_t<\n              std::conjunction_v<\n                  std::is_convertible<GetTypeFromEndT<1, Args&&...>,\n                                      WriteLineOptions>,\n                  std::is_convertible<GetTypeFromEndT<2, Args&&...>, Writer&>,\n                  TupleElementsSatisfy<RemoveTypesFromEndT<2, Args&&...>,\n                                       IsStringifiable>>,\n              int> = 0>\nbool WriteLine(Args&&... args);\n\n// Writes UTF-8 BOM.\nvoid WriteUtf8Bom(Writer& dest);\n\n// Implementation details follow.\n\nnamespace write_line_internal {\n\ntemplate <typename... Srcs>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool WriteLineInternal(\n    ABSL_ATTRIBUTE_UNUSED std::tuple<Srcs...> srcs, Writer& dest,\n    WriteLineOptions options) {\n  if (ABSL_PREDICT_FALSE(!std::apply(\n          [&](Srcs&&... srcs) {\n            return dest.Write(std::forward<Srcs>(srcs)...);\n          },\n          std::move(srcs)))) {\n    return false;\n  }\n  switch (options.newline()) {\n    case WriteNewline::kLf:\n      return dest.Write('\\n');\n    case WriteNewline::kCrLf:\n      return dest.Write(\"\\r\\n\");\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown newline: \" << static_cast<int>(options.newline());\n}\n\n}  // namespace write_line_internal\n\ntemplate <typename... Args,\n          std::enable_if_t<\n              std::conjunction_v<\n                  std::is_convertible<GetTypeFromEndT<1, Args&&...>, Writer&>,\n                  TupleElementsSatisfy<RemoveTypesFromEndT<1, Args&&...>,\n                                       IsStringifiable>>,\n              int>>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool WriteLine(Args&&... args) {\n  return write_line_internal::WriteLineInternal(\n      RemoveFromEnd<1>(std::forward<Args>(args)...),\n      GetFromEnd<1>(std::forward<Args>(args)...), WriteLineOptions());\n}\n\ntemplate <typename... Args,\n          std::enable_if_t<\n              std::conjunction_v<\n                  std::is_convertible<GetTypeFromEndT<1, Args&&...>,\n                                      WriteLineOptions>,\n                  std::is_convertible<GetTypeFromEndT<2, Args&&...>, Writer&>,\n                  TupleElementsSatisfy<RemoveTypesFromEndT<2, Args&&...>,\n                                       IsStringifiable>>,\n              int>>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool WriteLine(Args&&... args) {\n  return write_line_internal::WriteLineInternal(\n      RemoveFromEnd<2>(std::forward<Args>(args)...),\n      GetFromEnd<2>(std::forward<Args>(args)...),\n      GetFromEnd<1>(std::forward<Args>(args)...));\n}\n\ninline void WriteUtf8Bom(Writer& dest) { dest.Write(kUtf8Bom); }\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_LINES_LINE_WRITING_H_\n"
  },
  {
    "path": "riegeli/lines/newline.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_LINES_NEWLINE_H_\n#define RIEGELI_LINES_NEWLINE_H_\n\n#include \"absl/strings/string_view.h\"\n\nnamespace riegeli {\n\n// Line terminator representations to recognize.\nenum class ReadNewline {\n  kLf,        //         LF          (\"\\n\")\n  kCrLfOrLf,  // CR-LF | LF (\"\\r\\n\" | \"\\n\")\n};\n\n// Line terminator representation to write.\nenum class WriteNewline {\n  kLf,    // LF    (\"\\n\")\n  kCrLf,  // CR-LF (\"\\r\\n\")\n\n#ifndef _WIN32\n  kNative = kLf,\n#else\n  kNative = kCrLf,\n#endif\n};\n\n// Native line representation as a string.\n#ifndef _WIN32\ninline constexpr absl::string_view kNewline = \"\\n\";\n#else\ninline constexpr absl::string_view kNewline = \"\\r\\n\";\n#endif\n\n// UTF-8 BOM representation as a string.\ninline constexpr absl::string_view kUtf8Bom = \"\\xef\\xbb\\xbf\";\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_LINES_NEWLINE_H_\n"
  },
  {
    "path": "riegeli/lines/text_reader.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/lines/text_reader.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/lines/newline.h\"\n\nnamespace riegeli {\n\nvoid TextReaderBase::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of TextReader: null Reader pointer\";\n  if (ABSL_PREDICT_FALSE(!src->ok()) && src->available() == 0) {\n    FailWithoutAnnotation(AnnotateOverSrc(src->status()));\n    return;\n  }\n  initial_original_pos_ = src->pos();\n}\n\nabsl::Status TextReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Reader& src = *SrcReader();\n    status = src.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `src` with the original position.\n  // Clarify that the current position is the position with LF newlines instead\n  // of delegating to `BufferedReader::AnnotateStatusImpl()`.\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status TextReaderBase::AnnotateOverSrc(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"with LF newlines at byte \", pos()));\n  }\n  return status;\n}\n\nbool TextReaderBase::ToleratesReadingAhead() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->ToleratesReadingAhead();\n}\n\nbool TextReaderBase::SupportsRewind() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRewind();\n}\n\nbool TextReaderBase::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (new_pos <= limit_pos()) {\n    // Seeking backwards.\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    Reader& src = *SrcReader();\n    set_buffer();\n    set_limit_pos(0);\n    if (ABSL_PREDICT_FALSE(!src.Seek(initial_original_pos_))) {\n      return FailWithoutAnnotation(AnnotateOverSrc(src.StatusOrAnnotate(\n          absl::DataLossError(\"Zstd-compressed stream got truncated\"))));\n    }\n    if (new_pos == 0) return true;\n  }\n  return BufferedReader::SeekBehindBuffer(new_pos);\n}\n\nnamespace text_reader_internal {\n\nvoid TextReaderImpl<ReadNewline::kCrLfOrLf>::Initialize(Reader* src) {\n  pending_cr_ = false;\n  TextReaderBase::Initialize(src);\n}\n\nbool TextReaderImpl<ReadNewline::kCrLfOrLf>::ReadInternal(size_t min_length,\n                                                          size_t max_length,\n                                                          char* dest) {\n  RIEGELI_ASSERT_GT(min_length, 0u)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"nothing to read\";\n  RIEGELI_ASSERT_GE(max_length, min_length)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"max_length < min_length\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::ReadInternal()\";\n  Reader& src = *SrcReader();\n  for (;;) {\n    if (ABSL_PREDICT_FALSE(!src.Pull(1, max_length))) {\n      if (ABSL_PREDICT_FALSE(pending_cr_)) {\n        pending_cr_ = false;\n        dest[0] = '\\r';\n        move_limit_pos(1);\n        return min_length <= 1;\n      }\n      return false;\n    }\n    size_t length;\n    if (ABSL_PREDICT_FALSE(pending_cr_)) {\n      pending_cr_ = false;\n      if (src.cursor()[0] == '\\n') {\n        src.move_cursor(1);\n        dest[0] = '\\n';\n      } else {\n        dest[0] = '\\r';\n      }\n      ++dest;\n      length = 1;\n    } else {\n      length = UnsignedMin(src.available(), max_length);\n      const char* cr_ptr =\n          static_cast<const char*>(std::memchr(src.cursor(), '\\r', length));\n      if (cr_ptr == nullptr) {\n        std::memcpy(dest, src.cursor(), length);\n        src.move_cursor(length);\n        dest += length;\n      } else {\n        length = PtrDistance(src.cursor(), cr_ptr);\n        std::memcpy(dest, src.cursor(), length);\n        if (ABSL_PREDICT_FALSE(cr_ptr == src.limit() - 1)) {\n          src.move_cursor(length + 1);\n          dest += length;\n          pending_cr_ = true;\n        } else {\n          ++length;\n          src.move_cursor(length);\n          dest += length;\n          if (cr_ptr[1] == '\\n') {\n            src.move_cursor(1);\n            dest[-1] = '\\n';\n          } else {\n            dest[-1] = '\\r';\n          }\n        }\n      }\n    }\n    move_limit_pos(length);\n    if (length >= min_length) return true;\n    min_length -= length;\n    max_length -= length;\n  }\n}\n\nbool TextReaderImpl<ReadNewline::kCrLfOrLf>::SeekBehindBuffer(\n    Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (new_pos <= limit_pos()) {\n    // Seeking backwards.\n    pending_cr_ = false;\n  }\n  return TextReaderBase::SeekBehindBuffer(new_pos);\n}\n\n}  // namespace text_reader_internal\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/lines/text_reader.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_LINES_TEXT_READER_H_\n#define RIEGELI_LINES_TEXT_READER_H_\n\n#include <stddef.h>\n\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/any.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/prefix_limiting_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/lines/newline.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `TextReader<newline, Src>` when\n// `newline != ReadNewline::kLf`.\nclass TextReaderBase : public BufferedReader {\n public:\n  using Options = BufferOptions;\n\n  // Returns the original `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRewind() override;\n\n protected:\n  using BufferedReader::BufferedReader;\n\n  TextReaderBase(TextReaderBase&& that) = default;\n  TextReaderBase& operator=(TextReaderBase&& that) = default;\n\n  void Initialize(Reader* src);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool SeekBehindBuffer(Position new_pos) override;\n\n private:\n  Position initial_original_pos_ = 0;\n};\n\nnamespace text_reader_internal {\n\ntemplate <ReadNewline newline>\nclass TextReaderImpl;\n\ntemplate <>\nclass TextReaderImpl<ReadNewline::kCrLfOrLf> : public TextReaderBase {\n protected:\n  using TextReaderBase::TextReaderBase;\n\n  TextReaderImpl(TextReaderImpl&& that) = default;\n  TextReaderImpl& operator=(TextReaderImpl&& that) = default;\n\n  void Initialize(Reader* src);\n\n  bool ReadInternal(size_t min_length, size_t max_length, char* dest) override;\n  bool SeekBehindBuffer(Position new_pos) override;\n\n private:\n  // If `true`, a CR at the end of a buffer has been read from the source.\n  // If LF follows in the source, it will be skipped and LF will be written to\n  // the destination, otherwise CR will be written to the destination.\n  bool pending_cr_ = false;\n};\n\n}  // namespace text_reader_internal\n\n// A `Reader` which converts line terminators from the given representation to\n// LF after getting data from another `Reader`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the original `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the second template argument can be deduced as `TargetT`\n// of the type of the first constructor argument.\n//\n// The original `Reader` must not be accessed until the `TextReader` is closed\n// or no longer used.\n//\n// This primary class template is used when `newline != ReadNewline::kLf`.\ntemplate <ReadNewline newline = ReadNewline::kCrLfOrLf, typename Src = Reader*>\nclass TextReader : public text_reader_internal::TextReaderImpl<newline> {\n public:\n  using Options = TextReaderBase::Options;\n\n  // Creates a closed `TextReader`.\n  explicit TextReader(Closed) noexcept : TextReader::TextReaderImpl(kClosed) {}\n\n  // Will read from the original `Reader` provided by `src`.\n  explicit TextReader(Initializer<Src> src, Options options = Options());\n\n  TextReader(TextReader&& that) = default;\n  TextReader& operator=(TextReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `TextReader`. This avoids\n  // constructing a temporary `TextReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the original `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  void VerifyEndImpl() override;\n\n private:\n  // The object providing and possibly owning the original `Reader`.\n  Dependency<Reader*, Src> src_;\n};\n\n// Specialization of `TextReader<newline, Src>` when\n// `newline == ReadNewline::kLf`.\n//\n// In contrast to the primary class template, this specialization exposes\n// optional functionality of the original `Reader` (e.g. random access) and\n// avoids adding a buffering layer.\ntemplate <typename Src>\nclass TextReader<ReadNewline::kLf, Src> : public PrefixLimitingReader<Src> {\n public:\n  using Options = TextReaderBase::Options;\n\n  // Creates a closed `TextReader`.\n  explicit TextReader(Closed) noexcept\n      : TextReader::PrefixLimitingReader(kClosed) {}\n\n  // Will read from the original `Reader` provided by `src`.\n  //\n  // `options` are ignored in this class template specialization.\n  explicit TextReader(Initializer<Src> src, Options options = Options());\n\n  TextReader(TextReader&& that) = default;\n  TextReader& operator=(TextReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `TextReader`. This avoids\n  // constructing a temporary `TextReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n};\n\nexplicit TextReader(Closed)\n    -> TextReader<ReadNewline::kCrLfOrLf, DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit TextReader(Src&& src,\n                    TextReaderBase::Options options = TextReaderBase::Options())\n    -> TextReader<ReadNewline::kCrLfOrLf, TargetT<Src>>;\n\n// Wraps a `TextReader` for a line terminator specified at runtime.\ntemplate <typename Src = Reader*>\nusing AnyTextReader =\n    Any<Reader*>::Inlining<TextReader<ReadNewline::kLf, Src>,\n                           TextReader<ReadNewline::kCrLfOrLf, Src>>;\n\n// Options for `MakeAnyTextReader()`.\nclass AnyTextReaderOptions : public BufferOptionsBase<AnyTextReaderOptions> {\n public:\n  AnyTextReaderOptions() noexcept {}\n\n  // Line terminator representation to translate from LF.\n  //\n  // Default: `ReadNewline::kCrLfOrLf`.\n  AnyTextReaderOptions& set_newline(ReadNewline newline) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    newline_ = newline;\n    return *this;\n  }\n  AnyTextReaderOptions&& set_newline(ReadNewline newline) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_newline(newline));\n  }\n  ReadNewline newline() const { return newline_; }\n\n private:\n  ReadNewline newline_ = ReadNewline::kCrLfOrLf;\n};\n\n// Factory function for `AnyTextReader`.\n//\n// `src` supports `riegeli::Maker<Src>(args...)` to construct `Src` in-place.\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetSupportsDependency<Reader*, Src>::value, int> = 0>\nAnyTextReader<TargetT<Src>> MakeAnyTextReader(\n    Src&& src, AnyTextReaderOptions options = AnyTextReaderOptions());\n\n// Implementation details below.\n\ntemplate <ReadNewline newline, typename Src>\ninline TextReader<newline, Src>::TextReader(Initializer<Src> src,\n                                            Options options)\n    : TextReader::TextReaderImpl(options), src_(std::move(src)) {\n  this->Initialize(src_.get());\n}\n\ntemplate <ReadNewline newline, typename Src>\ninline void TextReader<newline, Src>::Reset(Closed) {\n  TextReader::TextReaderImpl::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <ReadNewline newline, typename Src>\ninline void TextReader<newline, Src>::Reset(Initializer<Src> src,\n                                            Options options) {\n  TextReader::TextReaderImpl::Reset(options);\n  src_.Reset(std::move(src));\n  this->Initialize(src_.get());\n}\n\ntemplate <ReadNewline newline, typename Src>\nvoid TextReader<newline, Src>::Done() {\n  TextReader::TextReaderImpl::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      this->FailWithoutAnnotation(this->AnnotateOverSrc(src_->status()));\n    }\n  }\n}\n\ntemplate <ReadNewline newline, typename Src>\nvoid TextReader<newline, Src>::SetReadAllHintImpl(bool read_all_hint) {\n  TextReader::TextReaderImpl::SetReadAllHintImpl(read_all_hint);\n  if (src_.IsOwning()) src_->SetReadAllHint(read_all_hint);\n}\n\ntemplate <ReadNewline newline, typename Src>\nvoid TextReader<newline, Src>::VerifyEndImpl() {\n  TextReader::TextReaderImpl::VerifyEndImpl();\n  if (src_.IsOwning() && ABSL_PREDICT_TRUE(this->ok())) src_->VerifyEnd();\n}\n\ntemplate <typename Src>\ninline TextReader<ReadNewline::kLf, Src>::TextReader(\n    Initializer<Src> src, ABSL_ATTRIBUTE_UNUSED Options options)\n    : TextReader::PrefixLimitingReader(std::move(src)) {}\n\ntemplate <typename Src>\ninline void TextReader<ReadNewline::kLf, Src>::Reset(Closed) {\n  TextReader::PrefixLimitingReader::Reset(kClosed);\n}\n\ntemplate <typename Src>\ninline void TextReader<ReadNewline::kLf, Src>::Reset(\n    Initializer<Src> src, ABSL_ATTRIBUTE_UNUSED Options options) {\n  TextReader::PrefixLimitingReader::Reset(std::move(src));\n}\n\ntemplate <typename Src,\n          std::enable_if_t<TargetSupportsDependency<Reader*, Src>::value, int>>\nAnyTextReader<TargetT<Src>> MakeAnyTextReader(Src&& src,\n                                              AnyTextReaderOptions options) {\n  switch (options.newline()) {\n    case ReadNewline::kLf:\n      return riegeli::Maker<TextReader<ReadNewline::kLf, TargetT<Src>>>(\n          std::forward<Src>(src), options.buffer_options());\n    case ReadNewline::kCrLfOrLf:\n      return riegeli::Maker<TextReader<ReadNewline::kCrLfOrLf, TargetT<Src>>>(\n          std::forward<Src>(src), options.buffer_options());\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown newline: \" << static_cast<int>(options.newline());\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_LINES_TEXT_READER_H_\n"
  },
  {
    "path": "riegeli/lines/text_writer.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/lines/text_writer.h\"\n\n#include <stddef.h>\n\n#include <cstring>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/lines/line_writing.h\"\n#include \"riegeli/lines/newline.h\"\n\nnamespace riegeli {\n\nabsl::Status TextWriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    status = dest.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `dest` with the original position.\n  // Clarify that the current position is the position with LF newlines instead\n  // of delegating to `BufferedWriter::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status TextWriterBase::AnnotateOverDest(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"with LF newlines at byte \", pos()));\n  }\n  return status;\n}\n\nnamespace text_writer_internal {\n\ntemplate <WriteNewline newline>\nbool TextWriterImpl<newline>::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of BufferedWriter::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedWriter::WriteInternal()\";\n  Writer& dest = *DestWriter();\n  for (;;) {\n    const char* const lf_ptr =\n        static_cast<const char*>(std::memchr(src.data(), '\\n', src.size()));\n    if (lf_ptr == nullptr) break;\n    const size_t length = PtrDistance(src.data(), lf_ptr);\n    if (ABSL_PREDICT_FALSE(\n            !WriteLine(absl::string_view(src.data(), length), dest, newline))) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    }\n    src.remove_prefix(length + 1);\n    move_start_pos(length + 1);\n  }\n  if (ABSL_PREDICT_FALSE(!dest.Write(src))) {\n    return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n  move_start_pos(src.size());\n  return true;\n}\n\ntemplate class TextWriterImpl<WriteNewline::kCrLf>;\n\n}  // namespace text_writer_internal\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/lines/text_writer.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_LINES_TEXT_WRITER_H_\n#define RIEGELI_LINES_TEXT_WRITER_H_\n\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/any.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/prefix_limiting_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/lines/newline.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `TextWriter<newline, Dest>` when\n// `newline != WriteNewline::kLf`.\nclass TextWriterBase : public BufferedWriter {\n public:\n  using Options = BufferOptions;\n\n  // Returns the original `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n protected:\n  using BufferedWriter::BufferedWriter;\n\n  TextWriterBase(TextWriterBase&& that) = default;\n  TextWriterBase& operator=(TextWriterBase&& that) = default;\n\n  void Initialize(Writer* dest);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n};\n\nnamespace text_writer_internal {\n\ntemplate <WriteNewline newline>\nclass TextWriterImpl : public TextWriterBase {\n protected:\n  using TextWriterBase::TextWriterBase;\n\n  TextWriterImpl(TextWriterImpl&& that) = default;\n  TextWriterImpl& operator=(TextWriterImpl&& that) = default;\n\n  bool WriteInternal(absl::string_view src) override;\n};\n\nextern template class TextWriterImpl<WriteNewline::kCrLf>;\n\n}  // namespace text_writer_internal\n\ntemplate <WriteNewline newline = WriteNewline::kNative, typename Dest = Writer*>\nclass TextWriter : public text_writer_internal::TextWriterImpl<newline> {\n public:\n  using Options = TextWriterBase::Options;\n\n  // Creates a closed `TextWriter`.\n  explicit TextWriter(Closed) noexcept : TextWriter::TextWriterImpl(kClosed) {}\n\n  // Will write to the original `Writer` provided by `dest`.\n  explicit TextWriter(Initializer<Dest> dest, Options options = Options());\n\n  TextWriter(TextWriter&& that) = default;\n  TextWriter& operator=(TextWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `TextWriter`. This avoids\n  // constructing a temporary `TextWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the original `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  // The object providing and possibly owning the original `Writer`.\n  Dependency<Writer*, Dest> dest_;\n};\n\n// Specialization of `TextWriter<newline, Dest>` when\n// `newline == WriteNewline::kLf`.\n//\n// In contrast to the primary class template, this specialization exposes\n// optional functionality of the original `Writer` (e.g. random access) and\n// avoids adding a buffering layer.\ntemplate <typename Dest>\nclass TextWriter<WriteNewline::kLf, Dest> : public PrefixLimitingWriter<Dest> {\n public:\n  using Options = TextWriterBase::Options;\n\n  // Creates a closed `TextWriter`.\n  explicit TextWriter(Closed) noexcept\n      : TextWriter::PrefixLimitingWriter(kClosed) {}\n\n  // Will write to the original `Writer` provided by `dest`.\n  //\n  // `options` are ignored in this class template specialization.\n  explicit TextWriter(Initializer<Dest> dest, Options options = Options());\n\n  TextWriter(TextWriter&& that) = default;\n  TextWriter& operator=(TextWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `TextWriter`. This avoids\n  // constructing a temporary `TextWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n};\n\nexplicit TextWriter(Closed)\n    -> TextWriter<WriteNewline::kNative, DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit TextWriter(Dest&& dest,\n                    TextWriterBase::Options options = TextWriterBase::Options())\n    -> TextWriter<WriteNewline::kNative, TargetT<Dest>>;\n\n// Wraps a `TextWriter` for a line terminator specified at runtime.\ntemplate <typename Dest = Writer*>\nusing AnyTextWriter =\n    Any<Writer*>::Inlining<TextWriter<WriteNewline::kLf, Dest>,\n                           TextWriter<WriteNewline::kCrLf, Dest>>;\n\n// Options for `MakeAnyTextWriter()`.\nclass AnyTextWriterOptions : public BufferOptionsBase<AnyTextWriterOptions> {\n public:\n  AnyTextWriterOptions() noexcept {}\n\n  // Line terminator representation to translate from LF.\n  //\n  // Default: `WriteNewline::kNative`.\n  AnyTextWriterOptions& set_newline(WriteNewline newline) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    newline_ = newline;\n    return *this;\n  }\n  AnyTextWriterOptions&& set_newline(WriteNewline newline) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_newline(newline));\n  }\n  WriteNewline newline() const { return newline_; }\n\n private:\n  WriteNewline newline_ = WriteNewline::kNative;\n};\n\n// Factory function for `AnyTextWriter`.\n//\n// `dest` supports `riegeli::Maker<Dest>(args...)` to construct `Dest` in-place.\ntemplate <\n    typename Dest,\n    std::enable_if_t<TargetSupportsDependency<Writer*, Dest>::value, int> = 0>\nAnyTextWriter<TargetT<Dest>> MakeAnyTextWriter(\n    Dest&& dest, AnyTextWriterOptions options = AnyTextWriterOptions());\n\n// Implementation details below.\n\ninline void TextWriterBase::Initialize(Writer* dest) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of TextWriter: null Writer pointer\";\n  if (ABSL_PREDICT_FALSE(!dest->ok())) {\n    FailWithoutAnnotation(AnnotateOverDest(dest->status()));\n  }\n}\n\ntemplate <WriteNewline newline, typename Dest>\ninline TextWriter<newline, Dest>::TextWriter(Initializer<Dest> dest,\n                                             Options options)\n    : TextWriter::TextWriterImpl(options), dest_(std::move(dest)) {\n  this->Initialize(dest_.get());\n}\n\ntemplate <WriteNewline newline, typename Dest>\ninline void TextWriter<newline, Dest>::Reset(Closed) {\n  TextWriter::TextWriterImpl::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <WriteNewline newline, typename Dest>\ninline void TextWriter<newline, Dest>::Reset(Initializer<Dest> dest,\n                                             Options options) {\n  TextWriter::TextWriterImpl::Reset(options);\n  dest_.Reset(std::move(dest));\n  this->Initialize(dest_.get());\n}\n\ntemplate <WriteNewline newline, typename Dest>\nvoid TextWriter<newline, Dest>::Done() {\n  TextWriter::TextWriterImpl::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      this->FailWithoutAnnotation(this->AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <WriteNewline newline, typename Dest>\nbool TextWriter<newline, Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!TextWriter::TextWriterImpl::FlushImpl(flush_type))) {\n    return false;\n  }\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Flush(flush_type))) {\n      return this->FailWithoutAnnotation(\n          this->AnnotateOverDest(dest_->status()));\n    }\n  }\n  return true;\n}\n\ntemplate <typename Dest>\ninline TextWriter<WriteNewline::kLf, Dest>::TextWriter(\n    Initializer<Dest> dest, ABSL_ATTRIBUTE_UNUSED Options options)\n    : TextWriter::PrefixLimitingWriter(std::move(dest)) {}\n\ntemplate <typename Dest>\ninline void TextWriter<WriteNewline::kLf, Dest>::Reset(Closed) {\n  TextWriter::PrefixLimitingWriter::Reset(kClosed);\n}\n\ntemplate <typename Dest>\ninline void TextWriter<WriteNewline::kLf, Dest>::Reset(\n    Initializer<Dest> dest, ABSL_ATTRIBUTE_UNUSED Options options) {\n  TextWriter::PrefixLimitingWriter::Reset(std::move(dest));\n}\n\ntemplate <typename Dest,\n          std::enable_if_t<TargetSupportsDependency<Writer*, Dest>::value, int>>\nAnyTextWriter<TargetT<Dest>> MakeAnyTextWriter(Dest&& dest,\n                                               AnyTextWriterOptions options) {\n  switch (options.newline()) {\n    case WriteNewline::kLf:\n      return riegeli::Maker<TextWriter<WriteNewline::kLf, TargetT<Dest>>>(\n          std::forward<Dest>(dest), options.buffer_options());\n    case WriteNewline::kCrLf:\n      return riegeli::Maker<TextWriter<WriteNewline::kCrLf, TargetT<Dest>>>(\n          std::forward<Dest>(dest), options.buffer_options());\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown newline: \" << static_cast<int>(options.newline());\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_LINES_TEXT_WRITER_H_\n"
  },
  {
    "path": "riegeli/lz4/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"lz4_reader\",\n    srcs = [\"lz4_reader.cc\"],\n    hdrs = [\"lz4_reader.h\"],\n    # zstd_reader.cc has #define before #include to influence what the included\n    # files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \":lz4_dictionary\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_reader\",\n        \"//riegeli/bytes:reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@lz4//:lz4_frame\",\n    ],\n)\n\ncc_library(\n    name = \"lz4_writer\",\n    srcs = [\"lz4_writer.cc\"],\n    hdrs = [\"lz4_writer.h\"],\n    # lz4_writer.cc has #define before #include to influence what the included\n    # files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \":lz4_dictionary\",\n        \":lz4_reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_writer\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@lz4\",\n        \"@lz4//:lz4_frame\",\n    ],\n)\n\ncc_library(\n    name = \"lz4_dictionary\",\n    srcs = [\"lz4_dictionary.cc\"],\n    hdrs = [\"lz4_dictionary.h\"],\n    # lz4_dictionary.cc has #define before #include to influence what the\n    # included files provide.\n    features = [\"-use_header_modules\"],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:shared_ptr\",\n        \"@com_google_absl//absl/base\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@lz4//:lz4_frame\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/lz4/lz4_dictionary.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// Enables the experimental lz4 API:\n//  * `LZ4F_createCDict()`\n//  * `LZ4F_freeCDict()`\n#define LZ4F_STATIC_LINKING_ONLY\n\n#include \"riegeli/lz4/lz4_dictionary.h\"\n\n#include <memory>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/call_once.h\"\n#include \"absl/strings/string_view.h\"\n#include \"lz4frame.h\"\n#include \"riegeli/base/shared_ptr.h\"\n\nnamespace riegeli {\n\nvoid Lz4Dictionary::Repr::LZ4F_CDictDeleter::operator()(LZ4F_CDict* ptr) const {\n  LZ4F_freeCDict(ptr);\n}\n\ninline const LZ4F_CDict* Lz4Dictionary::Repr::PrepareCompressionDictionary()\n    const {\n  absl::call_once(compression_once_, [&] {\n    compression_dictionary_.reset(LZ4F_createCDict(data_.data(), data_.size()));\n  });\n  return compression_dictionary_.get();\n}\n\nconst LZ4F_CDict* Lz4Dictionary::PrepareCompressionDictionary() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  if (repr_ == nullptr) return nullptr;\n  return repr_->PrepareCompressionDictionary();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/lz4/lz4_dictionary.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_LZ4_LZ4_DICTIONARY_H_\n#define RIEGELI_LZ4_LZ4_DICTIONARY_H_\n\n// IWYU pragma: private, include \"riegeli/lz4/lz4_reader.h\"\n// IWYU pragma: private, include \"riegeli/lz4/lz4_writer.h\"\n\n#include <stdint.h>\n\n#include <memory>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/call_once.h\"\n#include \"absl/strings/string_view.h\"\n#include \"lz4frame.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/shared_ptr.h\"\n\n// Copied here because the definition in `lz4frame.h` is guarded with\n// `LZ4F_STATIC_LINKING_ONLY` which should not leak to the header.\ntypedef struct LZ4F_CDict_s LZ4F_CDict;\n\nnamespace riegeli {\n\n// Stores an optional Lz4 dictionary for compression and decompression.\n//\n// An empty dictionary is equivalent to having no dictionary.\n//\n// A `Lz4Dictionary` object can own the dictionary data, or can hold a pointer\n// to unowned dictionary data which must not be changed until the last\n// `Lz4Reader` or `Lz4Writer` using this dictionary is closed or no longer used.\n// A `Lz4Dictionary` object also holds prepared structures derived from\n// dictionary data. If the same dictionary is needed for multiple compression\n// or decompression sessions, the `Lz4Dictionary` object can be reused to avoid\n// preparing them again for compression.\n//\n// Copying a `Lz4Dictionary` object is cheap, sharing the actual dictionary.\nclass Lz4Dictionary {\n public:\n  // Creates an empty `Lz4Dictionary`.\n  Lz4Dictionary() = default;\n\n  Lz4Dictionary(const Lz4Dictionary& that) = default;\n  Lz4Dictionary& operator=(const Lz4Dictionary& that) = default;\n\n  Lz4Dictionary(Lz4Dictionary&& that) = default;\n  Lz4Dictionary& operator=(Lz4Dictionary&& that) = default;\n\n  // Resets the `Lz4Dictionary` to the empty state.\n  Lz4Dictionary& Reset() & ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  Lz4Dictionary&& Reset() && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(Reset());\n  }\n\n  // Sets a dictionary.\n  //\n  // Dictionary id can help to detect whether the correct dictionary is used.\n  // 0 means unspecified.\n  Lz4Dictionary& set_data(BytesInitializer data, uint32_t dict_id = 0) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  Lz4Dictionary&& set_data(BytesInitializer data, uint32_t dict_id = 0) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_data(std::move(data), dict_id));\n  }\n\n  // Like `set_data()`, but does not take ownership of `data`, which must not\n  // be changed until the last `Lz4Reader` or `Lz4Writer` using this dictionary\n  // is closed or no longer used.\n  Lz4Dictionary& set_data_unowned(absl::string_view data\n                                      ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                                  uint32_t dict_id = 0) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  Lz4Dictionary&& set_data_unowned(absl::string_view data\n                                       ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                                   uint32_t dict_id = 0) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_data_unowned(data, dict_id));\n  }\n\n  // Returns `true` if no dictionary is present.\n  bool empty() const;\n\n  // Returns the dictionary data.\n  absl::string_view data() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns the dictionary id.\n  //\n  // Dictionary id can help to detect whether the correct dictionary is used.\n  // 0 means unspecified.\n  uint32_t dict_id() const;\n\n  // Returns the compression dictionary in the prepared form, or `nullptr` if\n  // no dictionary is present or `LZ4F_createCDict()` failed.\n  //\n  // The dictionary is owned by `*this`.\n  const LZ4F_CDict* PrepareCompressionDictionary() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n private:\n  enum class Ownership { kCopied, kUnowned };\n\n  class Repr;\n\n  SharedPtr<const Repr> repr_;\n};\n\n// Implementation details follow.\n\nclass Lz4Dictionary::Repr {\n public:\n  // Owns a copy of `data`.\n  explicit Repr(BytesInitializer data,\n                std::integral_constant<Ownership, Ownership::kCopied>,\n                uint32_t dict_id)\n      : owned_data_(std::move(data)), data_(owned_data_), dict_id_(dict_id) {}\n\n  // Does not take ownership of `data`, which must not be changed until the\n  // last `Lz4Reader` or `Lz4Writer` using this dictionary is closed or no\n  // longer used.\n  explicit Repr(absl::string_view data,\n                std::integral_constant<Ownership, Ownership::kUnowned>,\n                uint32_t dict_id)\n      : data_(data), dict_id_(dict_id) {}\n\n  Repr(const Repr&) = delete;\n  Repr& operator=(const Repr&) = delete;\n\n  // Returns the compression dictionary in the prepared form, or `nullptr` if\n  // no dictionary is present or `LZ4F_createCDict()` failed.\n  //\n  // The dictionary is owned by `*this`.\n  const LZ4F_CDict* PrepareCompressionDictionary() const;\n\n  absl::string_view data() const { return data_; }\n  uint32_t dict_id() const { return dict_id_; }\n\n private:\n  struct LZ4F_CDictDeleter {\n    void operator()(LZ4F_CDict* ptr) const;\n  };\n\n  std::string owned_data_;\n  absl::string_view data_;\n  uint32_t dict_id_;\n\n  mutable absl::once_flag compression_once_;\n  mutable std::unique_ptr<LZ4F_CDict, LZ4F_CDictDeleter>\n      compression_dictionary_;\n};\n\ninline Lz4Dictionary& Lz4Dictionary::Reset() & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  repr_.Reset();\n  return *this;\n}\n\ninline Lz4Dictionary& Lz4Dictionary::set_data(BytesInitializer data,\n                                              uint32_t dict_id) &\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  repr_.Reset(riegeli::Maker(\n      std::move(data), std::integral_constant<Ownership, Ownership::kCopied>(),\n      dict_id));\n  return *this;\n}\n\ninline Lz4Dictionary& Lz4Dictionary::set_data_unowned(\n    absl::string_view data ABSL_ATTRIBUTE_LIFETIME_BOUND, uint32_t dict_id) &\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  repr_.Reset(riegeli::Maker(\n      data, std::integral_constant<Ownership, Ownership::kUnowned>(), dict_id));\n  return *this;\n}\n\ninline bool Lz4Dictionary::empty() const {\n  return repr_ == nullptr || repr_->data().empty();\n}\n\ninline absl::string_view Lz4Dictionary::data() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  if (repr_ == nullptr) return absl::string_view();\n  return repr_->data();\n}\n\ninline uint32_t Lz4Dictionary::dict_id() const {\n  if (repr_ == nullptr) return 0;\n  return repr_->dict_id();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_LZ4_LZ4_DICTIONARY_H_\n"
  },
  {
    "path": "riegeli/lz4/lz4_reader.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// Enables the experimental lz4 API:\n//  * `LZ4F_decompress_usingDict()`\n#define LZ4F_STATIC_LINKING_ONLY\n\n#include \"riegeli/lz4/lz4_reader.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"lz4frame.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nvoid Lz4ReaderBase::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of Lz4Reader: null Reader pointer\";\n  if (ABSL_PREDICT_FALSE(!src->ok()) && src->available() == 0) {\n    FailWithoutAnnotation(AnnotateOverSrc(src->status()));\n    return;\n  }\n  initial_compressed_pos_ = src->pos();\n  InitializeDecompressor(*src);\n}\n\ninline void Lz4ReaderBase::InitializeDecompressor(Reader& src) {\n  LZ4F_errorCode_t result = 0;\n  decompressor_ = RecyclingPool<LZ4F_dctx, LZ4F_dctxDeleter>::global(\n                      recycling_pool_options_)\n                      .Get(\n                          [&result] {\n                            LZ4F_dctx* decompressor = nullptr;\n                            result = LZ4F_createDecompressionContext(\n                                &decompressor, LZ4F_VERSION);\n                            return std::unique_ptr<LZ4F_dctx, LZ4F_dctxDeleter>(\n                                decompressor);\n                          },\n                          [](LZ4F_dctx* decompressor) {\n                            LZ4F_resetDecompressionContext(decompressor);\n                          });\n  if (ABSL_PREDICT_FALSE(LZ4F_isError(result))) {\n    Fail(absl::InternalError(\n        absl::StrCat(\"LZ4F_createDecompressionContext() failed: \",\n                     LZ4F_getErrorName(result))));\n    return;\n  }\n  ReadHeader(src);\n}\n\ninline bool Lz4ReaderBase::ReadHeader(Reader& src) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(LZ4F_MIN_SIZE_TO_KNOW_HEADER_LENGTH,\n                                   LZ4F_HEADER_SIZE_MAX))) {\n    if (ABSL_PREDICT_FALSE(!src.ok())) {\n      return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    }\n    if (ABSL_PREDICT_FALSE(!concatenate_)) {\n      if (!growing_source_) {\n        Fail(absl::InvalidArgumentError(\"Truncated Lz4-compressed stream\"));\n      }\n      truncated_ = true;\n    }\n    return false;\n  }\n  const size_t header_size = LZ4F_headerSize(src.cursor(), src.available());\n  if (ABSL_PREDICT_FALSE(LZ4F_isError(header_size))) {\n    return Fail(absl::InvalidArgumentError(absl::StrCat(\n        \"LZ4F_headerSize() failed: \", LZ4F_getErrorName(header_size))));\n  }\n  if (ABSL_PREDICT_FALSE(!src.Pull(header_size))) {\n    if (ABSL_PREDICT_FALSE(!src.ok())) {\n      return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    }\n    if (ABSL_PREDICT_FALSE(!concatenate_)) {\n      if (!growing_source_) {\n        Fail(absl::InvalidArgumentError(\"Truncated Lz4-compressed stream\"));\n      }\n      truncated_ = true;\n    }\n    return false;\n  }\n  LZ4F_frameInfo_t frame_info;\n  size_t length = src.available();\n  const size_t result = LZ4F_getFrameInfo(decompressor_.get(), &frame_info,\n                                          src.cursor(), &length);\n  if (ABSL_PREDICT_FALSE(LZ4F_isError(result))) {\n    return Fail(absl::InvalidArgumentError(absl::StrCat(\n        \"LZ4F_getFrameInfo() failed: \", LZ4F_getErrorName(result))));\n  }\n  src.move_cursor(length);\n  header_read_ = true;\n\n  if (!concatenate_ && frame_info.contentSize > 0) {\n    set_exact_size(frame_info.contentSize);\n  }\n  if (frame_info.dictID > 0 &&\n      ABSL_PREDICT_FALSE(frame_info.dictID != dictionary_.dict_id())) {\n    if (dictionary_.empty()) {\n      return Fail(absl::InvalidArgumentError(absl::StrCat(\n          \"Missing dictionary: expected dict_id \", frame_info.dictID)));\n    }\n    if (dictionary_.dict_id() > 0) {\n      return Fail(absl::InvalidArgumentError(\n          absl::StrCat(\"Wrong dictionary: expected dict_id \", frame_info.dictID,\n                       \", have dict_id \", dictionary_.dict_id())));\n    }\n    // Dictionary is present but has `dict_id() == 0`. Hopefully it is the right\n    // dictionary.\n  }\n  return true;\n}\n\nvoid Lz4ReaderBase::Done() {\n  if (ABSL_PREDICT_FALSE(truncated_) && growing_source_) {\n    Reader& src = *SrcReader();\n    FailWithoutAnnotation(AnnotateOverSrc(src.AnnotateStatus(\n        absl::InvalidArgumentError(\"Truncated Lz4-compressed stream\"))));\n  }\n  BufferedReader::Done();\n  decompressor_.reset();\n  dictionary_ = Lz4Dictionary();\n}\n\nabsl::Status Lz4ReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    if (ABSL_PREDICT_FALSE(truncated_)) {\n      status = Annotate(status, \"reading truncated Lz4-compressed stream\");\n    }\n    Reader& src = *SrcReader();\n    status = src.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `src` with the compressed position.\n  // Clarify that the current position is the uncompressed position instead of\n  // delegating to `BufferedReader::AnnotateStatusImpl()`.\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status Lz4ReaderBase::AnnotateOverSrc(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nbool Lz4ReaderBase::ReadInternal(size_t min_length, size_t max_length,\n                                 char* dest) {\n  RIEGELI_ASSERT_GT(min_length, 0u)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"nothing to read\";\n  RIEGELI_ASSERT_GE(max_length, min_length)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"max_length < min_length\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::ReadInternal()\";\n  Reader& src = *SrcReader();\n  truncated_ = false;\n  if (ABSL_PREDICT_FALSE(!header_read_)) {\n    if (ABSL_PREDICT_FALSE(!ReadHeader(src))) return false;\n  }\n  LZ4F_decompressOptions_t decompress_options{};\n  size_t effective_min_length = min_length;\n  if (!growing_source_ && exact_size() != std::nullopt &&\n      max_length >= SaturatingSub(*exact_size(), limit_pos())) {\n    // Avoid a memory copy from an internal buffer of the Lz4 engine to `dest`\n    // by promising to decompress all remaining data to `dest`.\n    decompress_options.stableDst = 1;\n    effective_min_length = std::numeric_limits<size_t>::max();\n  }\n  max_length = UnsignedMin(max_length,\n                           std::numeric_limits<Position>::max() - limit_pos());\n  for (;;) {\n    size_t src_length = src.available();\n    size_t dest_length = max_length;\n    const size_t result = LZ4F_decompress_usingDict(\n        decompressor_.get(), dest, &dest_length, src.cursor(), &src_length,\n        dictionary_.data().data(), dictionary_.data().size(),\n        &decompress_options);\n    src.move_cursor(src_length);\n    move_limit_pos(dest_length);\n    if (ABSL_PREDICT_FALSE(result == 0)) {\n      if (concatenate_) {\n        header_read_ = false;\n        if (dest_length >= min_length) return true;\n        dest += dest_length;\n        min_length -= dest_length;\n        max_length -= dest_length;\n        effective_min_length -= dest_length;\n        if (ABSL_PREDICT_FALSE(!ReadHeader(src))) return false;\n        continue;\n      }\n      decompressor_.reset();\n      // Avoid `BufferedReader` allocating another buffer.\n      set_exact_size(limit_pos());\n      return dest_length >= min_length;\n    }\n    if (ABSL_PREDICT_FALSE(LZ4F_isError(result))) {\n      Fail(absl::InvalidArgumentError(absl::StrCat(\n          \"LZ4F_decompress_usingDict() failed: \", LZ4F_getErrorName(result))));\n      return dest_length >= min_length;\n    }\n    if (dest_length >= effective_min_length) return true;\n    if (ABSL_PREDICT_FALSE(src.available() > 0)) {\n      RIEGELI_ASSERT_EQ(dest_length, max_length)\n          << \"LZ4F_decompress_usingDict() returned but there are still \"\n             \"input data and output space\";\n      RIEGELI_ASSERT_EQ(dest_length,\n                        std::numeric_limits<Position>::max() - limit_pos())\n          << \"The position does not overflow but the output buffer is full, \"\n             \"while less than min_length was output, which implies that \"\n             \"LZ4F_decompress_usingDict() wants to output more than the \"\n             \"expected decompressed size to a flat buffer\";\n      return FailOverflow();\n    }\n    if (ABSL_PREDICT_FALSE(!src.Pull(1, result))) {\n      if (ABSL_PREDICT_FALSE(!src.ok())) {\n        FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n      } else {\n        if (!growing_source_) {\n          Fail(absl::InvalidArgumentError(\"Truncated Lz4-compressed stream\"));\n        }\n        truncated_ = true;\n      }\n      return dest_length >= min_length;\n    }\n    dest += dest_length;\n    min_length = SaturatingSub(min_length, dest_length);\n    max_length -= dest_length;\n    effective_min_length -= dest_length;\n  }\n}\n\nvoid Lz4ReaderBase::ExactSizeReached() {\n  if (decompressor_ == nullptr) return;\n  char buffer[1];\n  if (ABSL_PREDICT_FALSE(Lz4ReaderBase::ReadInternal(1, 1, buffer))) {\n    decompressor_.reset();\n    Fail(absl::FailedPreconditionError(\n        \"Uncompressed size reached but more data can be decompressed, \"\n        \"which implies that seeking back and reading again encountered \"\n        \"changed Lz4-compressed data\"));\n  }\n}\n\nbool Lz4ReaderBase::ToleratesReadingAhead() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->ToleratesReadingAhead();\n}\n\nbool Lz4ReaderBase::SupportsRewind() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRewind();\n}\n\nbool Lz4ReaderBase::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (new_pos <= limit_pos()) {\n    // Seeking backwards.\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    Reader& src = *SrcReader();\n    truncated_ = false;\n    set_buffer();\n    set_limit_pos(0);\n    decompressor_.reset();\n    if (ABSL_PREDICT_FALSE(!src.Seek(initial_compressed_pos_))) {\n      return FailWithoutAnnotation(AnnotateOverSrc(src.StatusOrAnnotate(\n          absl::DataLossError(\"Lz4-compressed stream got truncated\"))));\n    }\n    InitializeDecompressor(src);\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    if (new_pos == 0) return true;\n  }\n  return BufferedReader::SeekBehindBuffer(new_pos);\n}\n\nbool Lz4ReaderBase::SupportsNewReader() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsNewReader();\n}\n\nstd::unique_ptr<Reader> Lz4ReaderBase::NewReaderImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point\n  // if `SrcReader()->SupportsNewReader()`.\n  Reader& src = *SrcReader();\n  std::unique_ptr<Reader> compressed_reader =\n      src.NewReader(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    return nullptr;\n  }\n  std::unique_ptr<Reader> reader =\n      std::make_unique<Lz4Reader<std::unique_ptr<Reader>>>(\n          std::move(compressed_reader),\n          Lz4ReaderBase::Options()\n              .set_growing_source(growing_source_)\n              .set_concatenate(concatenate_)\n              .set_dictionary(dictionary_)\n              .set_buffer_options(buffer_options())\n              .set_recycling_pool_options(recycling_pool_options_));\n  reader->Seek(initial_pos);\n  return reader;\n}\n\nnamespace lz4_internal {\n\ninline bool GetFrameInfo(Reader& src, LZ4F_frameInfo_t& frame_info,\n                         const RecyclingPoolOptions& recycling_pool_options) {\n  using LZ4F_dctxDeleter = Lz4ReaderBase::LZ4F_dctxDeleter;\n  RecyclingPool<LZ4F_dctx, LZ4F_dctxDeleter>::Handle decompressor;\n  {\n    LZ4F_errorCode_t result = 0;\n    decompressor =\n        RecyclingPool<LZ4F_dctx, LZ4F_dctxDeleter>::global(\n            recycling_pool_options)\n            .Get(\n                [&result] {\n                  LZ4F_dctx* decompressor = nullptr;\n                  result = LZ4F_createDecompressionContext(&decompressor,\n                                                           LZ4F_VERSION);\n                  return std::unique_ptr<LZ4F_dctx, LZ4F_dctxDeleter>(\n                      decompressor);\n                },\n                [](LZ4F_dctx* decompressor) {\n                  LZ4F_resetDecompressionContext(decompressor);\n                });\n    if (ABSL_PREDICT_FALSE(LZ4F_isError(result))) return false;\n  }\n  if (ABSL_PREDICT_FALSE(!src.Pull(LZ4F_MIN_SIZE_TO_KNOW_HEADER_LENGTH,\n                                   LZ4F_HEADER_SIZE_MAX))) {\n    return false;\n  }\n  const size_t header_size = LZ4F_headerSize(src.cursor(), src.available());\n  if (ABSL_PREDICT_FALSE(LZ4F_isError(header_size))) return false;\n  if (ABSL_PREDICT_FALSE(!src.Pull(header_size))) return false;\n  size_t length;\n  const size_t result =\n      LZ4F_getFrameInfo(decompressor.get(), &frame_info, src.cursor(), &length);\n  return !LZ4F_isError(result);\n}\n\n}  // namespace lz4_internal\n\nbool RecognizeLz4(Reader& src,\n                  const RecyclingPoolOptions& recycling_pool_options) {\n  LZ4F_frameInfo_t frame_info;\n  return lz4_internal::GetFrameInfo(src, frame_info, recycling_pool_options);\n}\n\nstd::optional<Position> Lz4UncompressedSize(\n    Reader& src, const RecyclingPoolOptions& recycling_pool_options) {\n  LZ4F_frameInfo_t frame_info;\n  if (!lz4_internal::GetFrameInfo(src, frame_info, recycling_pool_options)) {\n    return std::nullopt;\n  }\n  if (frame_info.contentSize > 0) return frame_info.contentSize;\n  return std::nullopt;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/lz4/lz4_reader.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_LZ4_LZ4_READER_H_\n#define RIEGELI_LZ4_LZ4_READER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"lz4.h\"\n#include \"lz4frame.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/lz4/lz4_dictionary.h\"  // IWYU pragma: export\n\nnamespace riegeli {\n\nnamespace lz4_internal {\n\nbool GetFrameInfo(Reader& src, LZ4F_frameInfo_t& frame_info,\n                  const RecyclingPoolOptions& recycling_pool_options);\n\n}  // namespace lz4_internal\n\n// Template parameter independent part of `Lz4Reader`.\nclass Lz4ReaderBase : public BufferedReader {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // If `true`, supports decompressing as much as possible from a truncated\n    // source, then retrying when the source has grown. This has a small\n    // performance penalty.\n    //\n    // Default: `false`.\n    Options& set_growing_source(bool growing_source) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      growing_source_ = growing_source;\n      return *this;\n    }\n    Options&& set_growing_source(bool growing_source) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_growing_source(growing_source));\n    }\n    bool growing_source() const { return growing_source_; }\n\n    // If `true`, concatenated compressed frames are decoded to concatenation\n    // of their decompressed contents. An empty compressed stream is decoded to\n    // empty decompressed contents.\n    //\n    // If `false`, exactly one compressed frame is consumed.\n    //\n    // Default: `false`.\n    Options& set_concatenate(bool concatenate) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      concatenate_ = concatenate;\n      return *this;\n    }\n    Options&& set_concatenate(bool concatenate) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_concatenate(concatenate));\n    }\n    bool concatenate() const { return concatenate_; }\n\n    // Lz4 dictionary. The same dictionary must have been used for compression,\n    // except that it is allowed to supply a dictionary even if no dictionary\n    // was used for compression.\n    //\n    // Default: `Lz4Dictionary()`.\n    Options& set_dictionary(Lz4Dictionary dictionary) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      dictionary_ = std::move(dictionary);\n      return *this;\n    }\n    Options&& set_dictionary(Lz4Dictionary dictionary) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_dictionary(std::move(dictionary)));\n    }\n    Lz4Dictionary& dictionary() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n    const Lz4Dictionary& dictionary() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n\n    // Options for a global `RecyclingPool` of decompression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // decompression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    Options& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    Options&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    bool growing_source_ = false;\n    bool concatenate_ = false;\n    Lz4Dictionary dictionary_;\n    RecyclingPoolOptions recycling_pool_options_\n#if LZ4_VERSION_NUMBER <= 10904\n        // Workaround for https://github.com/lz4/lz4/issues/1227.\n        = RecyclingPoolOptions().set_max_size(0)\n#endif\n        ;\n  };\n\n  // Returns the compressed `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns `true` if the source is truncated (without a clean end of the\n  // compressed stream) at the current position. In such case, if the source\n  // does not grow, `Close()` will fail.\n  //\n  // Precondition: `Options::concatenate()` was `false`.\n  bool truncated() const {\n    RIEGELI_ASSERT(!concatenate_)\n        << \"Failed precondition of Lz4ReaderBase::truncated(): \"\n           \"Options::concatenate() is true\";\n    return truncated_ && available() == 0;\n  }\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRewind() override;\n  bool SupportsNewReader() override;\n\n protected:\n  explicit Lz4ReaderBase(Closed) noexcept : BufferedReader(kClosed) {}\n\n  explicit Lz4ReaderBase(BufferOptions buffer_options, bool growing_source,\n                         bool concatenate, Lz4Dictionary&& dictionary,\n                         const RecyclingPoolOptions& recycling_pool_options);\n\n  Lz4ReaderBase(Lz4ReaderBase&& that) noexcept;\n  Lz4ReaderBase& operator=(Lz4ReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, bool growing_source,\n             bool concatenate, Lz4Dictionary&& dictionary,\n             const RecyclingPoolOptions& recycling_pool_options);\n  void Initialize(Reader* src);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool ReadInternal(size_t min_length, size_t max_length, char* dest) override;\n  void ExactSizeReached() override;\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  // For `LZ4F_dctxDeleter`.\n  friend bool lz4_internal::GetFrameInfo(\n      Reader& src, LZ4F_frameInfo_t& frame_info,\n      const RecyclingPoolOptions& recycling_pool_options);\n\n  struct LZ4F_dctxDeleter {\n    void operator()(LZ4F_dctx* ptr) const {\n      const LZ4F_errorCode_t result = LZ4F_freeDecompressionContext(ptr);\n      RIEGELI_ASSERT(!LZ4F_isError(result))\n          << \"LZ4F_freeDecompressionContext() failed: \"\n          << LZ4F_getErrorName(result);\n    }\n  };\n\n  void InitializeDecompressor(Reader& src);\n  bool ReadHeader(Reader& src);\n\n  // If `true`, supports decompressing as much as possible from a truncated\n  // source, then retrying when the source has grown.\n  bool growing_source_ = false;\n  bool concatenate_ = false;\n  Position initial_compressed_pos_ = 0;\n  // If `true`, the source is truncated (without a clean end of the compressed\n  // stream) at the current position. If the source does not grow, `Close()`\n  // will fail.\n  bool truncated_ = false;\n  // If `true`, the frame header has been read.\n  bool header_read_ = false;\n  Lz4Dictionary dictionary_;\n  RecyclingPoolOptions recycling_pool_options_;\n  // If `ok()` but `decompressor_ == nullptr` then all data have been\n  // decompressed, `exact_size() == limit_pos()`, and `ReadInternal()` must not\n  // be called again.\n  RecyclingPool<LZ4F_dctx, LZ4F_dctxDeleter>::Handle decompressor_;\n};\n\n// A `Reader` which decompresses data with Lz4 after getting it from another\n// `Reader`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Reader` must not be accessed until the `Lz4Reader` is closed\n// or no longer used.\ntemplate <typename Src = Reader*>\nclass Lz4Reader : public Lz4ReaderBase {\n public:\n  // Creates a closed `Lz4Reader`.\n  explicit Lz4Reader(Closed) noexcept : Lz4ReaderBase(kClosed) {}\n\n  // Will read from the compressed `Reader` provided by `src`.\n  explicit Lz4Reader(Initializer<Src> src, Options options = Options());\n\n  Lz4Reader(Lz4Reader&& that) = default;\n  Lz4Reader& operator=(Lz4Reader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `Lz4Reader`. This avoids\n  // constructing a temporary `Lz4Reader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  void VerifyEndImpl() override;\n\n private:\n  // The object providing and possibly owning the compressed `Reader`.\n  Dependency<Reader*, Src> src_;\n};\n\nexplicit Lz4Reader(Closed) -> Lz4Reader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit Lz4Reader(Src&& src,\n                   Lz4ReaderBase::Options options = Lz4ReaderBase::Options())\n    -> Lz4Reader<TargetT<Src>>;\n\n// Returns `true` if the data look like they have been Lz4-compressed.\n//\n// The current position of `src` is unchanged.\nbool RecognizeLz4(Reader& src,\n                  const RecyclingPoolOptions& recycling_pool_options =\n                      RecyclingPoolOptions());\n\n// Returns the claimed uncompressed size of Lz4-compressed data.\n//\n// If the data consists of multiple frames, only the first frame is considered.\n//\n// Returns `std::nullopt` if the size was not stored or on failure. The size is\n// stored if `Lz4WriterBase::Options::pledged_size() != std::nullopt`.\n//\n// The current position of `src` is unchanged.\nstd::optional<Position> Lz4UncompressedSize(\n    Reader& src, const RecyclingPoolOptions& recycling_pool_options =\n                     RecyclingPoolOptions());\n\n// Implementation details follow.\n\ninline Lz4ReaderBase::Lz4ReaderBase(\n    BufferOptions buffer_options, bool growing_source, bool concatenate,\n    Lz4Dictionary&& dictionary,\n    const RecyclingPoolOptions& recycling_pool_options)\n    : BufferedReader(buffer_options),\n      growing_source_(growing_source),\n      concatenate_(concatenate),\n      dictionary_(std::move(dictionary)),\n      recycling_pool_options_(recycling_pool_options) {}\n\ninline Lz4ReaderBase::Lz4ReaderBase(Lz4ReaderBase&& that) noexcept\n    : BufferedReader(static_cast<BufferedReader&&>(that)),\n      growing_source_(that.growing_source_),\n      concatenate_(that.concatenate_),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      truncated_(that.truncated_),\n      header_read_(that.header_read_),\n      dictionary_(std::move(that.dictionary_)),\n      recycling_pool_options_(that.recycling_pool_options_),\n      decompressor_(std::move(that.decompressor_)) {}\n\ninline Lz4ReaderBase& Lz4ReaderBase::operator=(Lz4ReaderBase&& that) noexcept {\n  BufferedReader::operator=(static_cast<BufferedReader&&>(that));\n  growing_source_ = that.growing_source_;\n  concatenate_ = that.concatenate_;\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  truncated_ = that.truncated_;\n  header_read_ = that.header_read_;\n  dictionary_ = std::move(that.dictionary_);\n  recycling_pool_options_ = that.recycling_pool_options_;\n  decompressor_ = std::move(that.decompressor_);\n  return *this;\n}\n\ninline void Lz4ReaderBase::Reset(Closed) {\n  BufferedReader::Reset(kClosed);\n  growing_source_ = false;\n  concatenate_ = false;\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  header_read_ = false;\n  recycling_pool_options_ = RecyclingPoolOptions();\n  decompressor_.reset();\n  dictionary_ = Lz4Dictionary();\n}\n\ninline void Lz4ReaderBase::Reset(\n    BufferOptions buffer_options, bool growing_source, bool concatenate,\n    Lz4Dictionary&& dictionary,\n    const RecyclingPoolOptions& recycling_pool_options) {\n  BufferedReader::Reset(buffer_options);\n  growing_source_ = growing_source;\n  concatenate_ = concatenate;\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  header_read_ = false;\n  recycling_pool_options_ = recycling_pool_options;\n  decompressor_.reset();\n  dictionary_ = std::move(dictionary);\n}\n\ntemplate <typename Src>\ninline Lz4Reader<Src>::Lz4Reader(Initializer<Src> src, Options options)\n    : Lz4ReaderBase(options.buffer_options(), options.growing_source(),\n                    options.concatenate(), std::move(options.dictionary()),\n                    options.recycling_pool_options()),\n      src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void Lz4Reader<Src>::Reset(Closed) {\n  Lz4ReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void Lz4Reader<Src>::Reset(Initializer<Src> src, Options options) {\n  Lz4ReaderBase::Reset(options.buffer_options(), options.growing_source(),\n                       options.concatenate(), std::move(options.dictionary()),\n                       options.recycling_pool_options());\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\nvoid Lz4Reader<Src>::Done() {\n  Lz4ReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(AnnotateOverSrc(src_->status()));\n    }\n  }\n}\n\ntemplate <typename Src>\nvoid Lz4Reader<Src>::SetReadAllHintImpl(bool read_all_hint) {\n  Lz4ReaderBase::SetReadAllHintImpl(read_all_hint);\n  if (src_.IsOwning()) src_->SetReadAllHint(read_all_hint);\n}\n\ntemplate <typename Src>\nvoid Lz4Reader<Src>::VerifyEndImpl() {\n  Lz4ReaderBase::VerifyEndImpl();\n  if (src_.IsOwning() && ABSL_PREDICT_TRUE(ok())) src_->VerifyEnd();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_LZ4_LZ4_READER_H_\n"
  },
  {
    "path": "riegeli/lz4/lz4_writer.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// Enables the experimental lz4 API:\n//  * `LZ4F_compressBegin_usingCDict()`\n#define LZ4F_STATIC_LINKING_ONLY\n\n#include \"riegeli/lz4/lz4_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"lz4frame.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/lz4/lz4_reader.h\"\n\nnamespace riegeli {\n\nvoid Lz4WriterBase::Initialize(Writer* dest, int compression_level,\n                               int window_log, bool store_content_checksum,\n                               bool store_block_checksum) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of Lz4Writer: null Writer pointer\";\n  if (ABSL_PREDICT_FALSE(!dest->ok())) {\n    FailWithoutAnnotation(AnnotateOverDest(dest->status()));\n    return;\n  }\n  initial_compressed_pos_ = dest->pos();\n\n  const LZ4F_CDict* compression_dictionary = nullptr;\n  if (!dictionary_.empty()) {\n    compression_dictionary = dictionary_.PrepareCompressionDictionary();\n    if (ABSL_PREDICT_FALSE(compression_dictionary == nullptr)) {\n      Fail(absl::InternalError(\"LZ4F_createCDict() failed\"));\n      return;\n    }\n  }\n\n  {\n    LZ4F_errorCode_t result = 0;\n    compressor_ =\n        RecyclingPool<LZ4F_cctx, LZ4F_cctxDeleter>::global(\n            recycling_pool_options_)\n            .Get([&result] {\n              LZ4F_cctx* compressor = nullptr;\n              result = LZ4F_createCompressionContext(&compressor, LZ4F_VERSION);\n              return std::unique_ptr<LZ4F_cctx, LZ4F_cctxDeleter>(compressor);\n            });\n    if (ABSL_PREDICT_FALSE(LZ4F_isError(result))) {\n      Fail(absl::InternalError(\n          absl::StrCat(\"LZ4F_createCompressionContext() failed: \",\n                       LZ4F_getErrorName(result))));\n      return;\n    }\n  }\n\n  preferences_.compressionLevel = compression_level;\n  preferences_.frameInfo.blockSizeID = window_log < 18   ? LZ4F_max64KB\n                                       : window_log < 20 ? LZ4F_max256KB\n                                       : window_log < 22 ? LZ4F_max1MB\n                                                         : LZ4F_max4MB;\n  preferences_.frameInfo.contentChecksumFlag = store_content_checksum\n                                                   ? LZ4F_contentChecksumEnabled\n                                                   : LZ4F_noContentChecksum;\n  preferences_.frameInfo.contentSize =\n      pledged_size_ != std::nullopt\n          ? IntCast<unsigned long long>(*pledged_size_)\n          : 0;\n  preferences_.frameInfo.dictID = dictionary_.dict_id();\n  preferences_.frameInfo.blockChecksumFlag =\n      store_block_checksum ? LZ4F_blockChecksumEnabled : LZ4F_noBlockChecksum;\n\n  BufferedWriter::SetWriteSizeHintImpl(pledged_size_);\n  if (ABSL_PREDICT_FALSE(!dest->Push(LZ4F_HEADER_SIZE_MAX))) {\n    FailWithoutAnnotation(AnnotateOverDest(dest->status()));\n    return;\n  }\n  const size_t result = LZ4F_compressBegin_usingCDict(\n      compressor_.get(), dest->cursor(), dest->available(),\n      compression_dictionary, &preferences_);\n  if (ABSL_PREDICT_FALSE(LZ4F_isError(result))) {\n    Fail(absl::InternalError(\n        absl::StrCat(\"LZ4F_compressBegin_usingCDict() failed: \",\n                     LZ4F_getErrorName(result))));\n    return;\n  }\n  dest->move_cursor(result);\n}\n\nvoid Lz4WriterBase::Done() {\n  stable_src_ = true;\n  BufferedWriter::Done();\n  if (ABSL_PREDICT_TRUE(ok())) {\n    if (pledged_size_ != std::nullopt &&\n        ABSL_PREDICT_FALSE(start_pos() < *pledged_size_)) {\n      Fail(absl::FailedPreconditionError(\n          absl::StrCat(\"Actual size does not match pledged size: \", start_pos(),\n                       \" < \", *pledged_size_)));\n    } else if (compressor_ != nullptr) {\n      Writer& dest = *DestWriter();\n      DoneCompression(dest);\n    }\n  }\n  compressor_.reset();\n  dictionary_ = Lz4Dictionary();\n  associated_reader_.Reset();\n}\n\ninline bool Lz4WriterBase::DoneCompression(Writer& dest) {\n  if (ABSL_PREDICT_FALSE(!dest.Push(LZ4F_compressBound(0, &preferences_)))) {\n    return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n  const size_t result = LZ4F_compressEnd(compressor_.get(), dest.cursor(),\n                                         dest.available(), nullptr);\n  RIEGELI_ASSERT(!LZ4F_isError(result))\n      << \"LZ4F_compressEnd() failed: \" << LZ4F_getErrorName(result);\n  dest.move_cursor(result);\n  return true;\n}\n\nabsl::Status Lz4WriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    status = dest.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `dest` with the compressed\n  // position. Clarify that the current position is the uncompressed position\n  // instead of delegating to `BufferedWriter::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status Lz4WriterBase::AnnotateOverDest(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nbool Lz4WriterBase::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of BufferedWriter::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedWriter::WriteInternal()\";\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  if (pledged_size_ != std::nullopt) {\n    const Position next_pos = start_pos() + src.size();\n    if (next_pos >= *pledged_size_) {\n      if (ABSL_PREDICT_FALSE(next_pos > *pledged_size_)) {\n        return Fail(absl::FailedPreconditionError(\n            absl::StrCat(\"Actual size does not match pledged size: \", next_pos,\n                         \" > \", *pledged_size_)));\n      }\n      stable_src_ = true;\n    }\n  }\n  RIEGELI_ASSERT_NE(compressor_, nullptr)\n      << \"compressor_ == nullptr when the pledged size was already written, \"\n         \"which was checked above\";\n  size_t block_size;\n  switch (preferences_.frameInfo.blockSizeID) {\n    case LZ4F_max64KB:\n      block_size = size_t{64} << 10;\n      break;\n    case LZ4F_max256KB:\n      block_size = size_t{256} << 10;\n      break;\n    case LZ4F_max1MB:\n      block_size = size_t{1} << 20;\n      break;\n    case LZ4F_max4MB:\n      block_size = size_t{4} << 20;\n      break;\n    default:\n      RIEGELI_ASSUME_UNREACHABLE()\n          << \"Unexpected preferences_.frameInfo.blockSizeID: \"\n          << preferences_.frameInfo.blockSizeID;\n  }\n  LZ4F_compressOptions_t compress_options{};\n  do {\n    size_t src_length = src.size();\n    compress_options.stableSrc = stable_src_ ? 1 : 0;\n    if (!reserve_max_size_) {\n      // Compressing the whole `src` in one step would require asking `dest`\n      // for a potentially large flat buffer. To avoid this, split `src` into\n      // smaller pieces.\n      src_length = UnsignedMin(src_length, block_size - buffered_length_);\n      // To reduce data copying, claim `compress_options.stableSrc` also when\n      // this piece of `src` will not be needed after `WriteInternal()` returns\n      // because at least a full block follows this piece.\n      if (src.size() >= src_length + block_size) compress_options.stableSrc = 1;\n    }\n    if (ABSL_PREDICT_FALSE(\n            !dest.Push(LZ4F_compressBound(src_length, &preferences_)))) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    }\n    const size_t result =\n        LZ4F_compressUpdate(compressor_.get(), dest.cursor(), dest.available(),\n                            src.data(), src_length, &compress_options);\n    if (ABSL_PREDICT_FALSE(LZ4F_isError(result))) {\n      return Fail(absl::InternalError(absl::StrCat(\n          \"LZ4F_compressUpdate() failed: \", LZ4F_getErrorName(result))));\n    }\n    dest.move_cursor(result);\n    move_start_pos(src_length);\n    src.remove_prefix(src_length);\n    buffered_length_ = (buffered_length_ + src_length) & (block_size - 1);\n  } while (!src.empty());\n  if (stable_src_) {\n    // `LZ4F_compressEnd()` must be called while `src` is still valid.\n    if (ABSL_PREDICT_FALSE(!DoneCompression(dest))) return false;\n    compressor_.reset();\n  }\n  return true;\n}\n\nbool Lz4WriterBase::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!BufferedWriter::FlushImpl(flush_type))) return false;\n  if (compressor_ == nullptr) return true;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!dest.Push(LZ4F_compressBound(0, &preferences_)))) {\n    return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n  const size_t result =\n      LZ4F_flush(compressor_.get(), dest.cursor(), dest.available(), nullptr);\n  RIEGELI_ASSERT(!LZ4F_isError(result))\n      << \"LZ4F_flush() failed: \" << LZ4F_getErrorName(result);\n  dest.move_cursor(result);\n  buffered_length_ = 0;\n  return true;\n}\n\nbool Lz4WriterBase::SupportsReadMode() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsReadMode();\n}\n\nReader* Lz4WriterBase::ReadModeBehindBuffer(Position initial_pos) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::ReadModeBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!Lz4WriterBase::FlushImpl(FlushType::kFromObject))) {\n    return nullptr;\n  }\n  Writer& dest = *DestWriter();\n  Reader* const compressed_reader = dest.ReadMode(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    return nullptr;\n  }\n  Lz4Reader<>* const reader = associated_reader_.ResetReader(\n      compressed_reader,\n      Lz4ReaderBase::Options()\n          .set_dictionary(dictionary_)\n          .set_buffer_options(buffer_options())\n          .set_recycling_pool_options(recycling_pool_options_));\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/lz4/lz4_writer.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_LZ4_LZ4_WRITER_H_\n#define RIEGELI_LZ4_LZ4_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"lz4frame.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/lz4/lz4_dictionary.h\"  // IWYU pragma: export\n\nnamespace riegeli {\n\nclass Reader;\ntemplate <typename Src>\nclass Lz4Reader;\n\n// Template parameter independent part of `Lz4Writer`.\nclass Lz4WriterBase : public BufferedWriter {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {\n      RIEGELI_ASSERT_EQ(LZ4F_compressionLevel_max(), kMaxCompressionLevel)\n          << \"Unexpected value of LZ4F_compressionLevel_max()\";\n    }\n\n    // Tunes the tradeoff between compression density and compression speed\n    // (higher = better density but slower).\n    //\n    // `compression_level` must be between `kMinCompressionLevel` (`-65536`)\n    // and `kMaxCompressionLevel` (12). Levels [0..2] are currently equivalent.\n    // Default: `kDefaultCompressionLevel` (0).\n    static constexpr int kMinCompressionLevel = -(64 << 10);\n    static constexpr int kMaxCompressionLevel =\n        12;  // `LZ4F_compressionLevel_max()`\n    static constexpr int kDefaultCompressionLevel = 0;\n    Options& set_compression_level(int compression_level) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(compression_level, kMinCompressionLevel)\n          << \"Failed precondition of \"\n             \"Lz4WriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      RIEGELI_ASSERT_LE(compression_level, kMaxCompressionLevel)\n          << \"Failed precondition of \"\n             \"Lz4WriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      compression_level_ = compression_level;\n      return *this;\n    }\n    Options&& set_compression_level(int compression_level) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_compression_level(compression_level));\n    }\n    int compression_level() const { return compression_level_; }\n\n    // Logarithm of the block size. This tunes the tradeoff between compression\n    // density and memory usage (higher = better density but more memory).\n    //\n    // Lz4 supports four effective values: 16, 18, 20, and 22. Other values are\n    // rounded downwards.\n    //\n    // `window_log` must be between `kMinWindowLog` (0) and `kMaxWindowLog`\n    // (22). Default: `kDefaultWindowLog` (16).\n    static constexpr int kMinWindowLog = 16;\n    static constexpr int kMaxWindowLog = 22;\n    static constexpr int kDefaultWindowLog = 16;\n    Options& set_window_log(int window_log) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(window_log, kMinWindowLog)\n          << \"Failed precondition of Lz4WriterBase::Options::set_window_log(): \"\n             \"window log out of range\";\n      RIEGELI_ASSERT_LE(window_log, kMaxWindowLog)\n          << \"Failed precondition of Lz4WriterBase::Options::set_window_log(): \"\n             \"window log out of range\";\n      window_log_ = window_log;\n      return *this;\n    }\n    Options&& set_window_log(int window_log) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_window_log(window_log));\n    }\n    int window_log() const { return window_log_; }\n\n    // Lz4 dictionary. The same dictionary must be used for decompression.\n    //\n    // Default: `Lz4Dictionary()`.\n    Options& set_dictionary(Lz4Dictionary dictionary) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      dictionary_ = std::move(dictionary);\n      return *this;\n    }\n    Options&& set_dictionary(Lz4Dictionary dictionary) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_dictionary(std::move(dictionary)));\n    }\n    Lz4Dictionary& dictionary() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n    const Lz4Dictionary& dictionary() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n\n    // If `true`, computes checksum of uncompressed data and stores it in the\n    // compressed stream for each frame, i.e. at coarse granularity. This lets\n    // decompression verify the checksum.\n    //\n    // Default: `false`.\n    Options& set_store_content_checksum(bool store_content_checksum) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      store_content_checksum_ = store_content_checksum;\n      return *this;\n    }\n    Options&& set_store_content_checksum(bool store_content_checksum) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_store_content_checksum(store_content_checksum));\n    }\n    bool store_content_checksum() const { return store_content_checksum_; }\n\n    // If `true`, computes checksum of uncompressed data and stores it in the\n    // compressed stream for each block, i.e. at coarse granularity. This lets\n    // decompression verify the checksum.\n    //\n    // Default: `false`.\n    Options& set_store_block_checksum(bool store_block_checksum) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      store_block_checksum_ = store_block_checksum;\n      return *this;\n    }\n    Options&& set_store_block_checksum(bool store_block_checksum) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_store_block_checksum(store_block_checksum));\n    }\n    bool store_block_checksum() const { return store_block_checksum_; }\n\n    // Exact uncompressed size, or `std::nullopt` if unknown. This may improve\n    // compression density and performance, and causes the size to be stored in\n    // the compressed stream header.\n    //\n    // If the pledged size turns out to not match reality, compression fails.\n    //\n    // Default: `std::nullopt`.\n    Options& set_pledged_size(std::optional<Position> pledged_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      pledged_size_ = pledged_size;\n      return *this;\n    }\n    Options&& set_pledged_size(std::optional<Position> pledged_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_pledged_size(pledged_size));\n    }\n    std::optional<Position> pledged_size() const { return pledged_size_; }\n\n    // If `false`, `Lz4Writer` lets the destination choose buffer sizes\n    // (at least the maximum possible compressed size of a block size though).\n    //\n    // If `true`, `Lz4Writer` tries to compress all data in one step:\n    //\n    //  * Flattens uncompressed data if `pledged_size()` is not `std::nullopt`.\n    //\n    //  * Asks the destination for a flat buffer with the maximum possible\n    //    compressed size for each flat piece of uncompressed data.\n    //\n    // This makes compression slightly faster, but increases memory usage.\n    //\n    // Default: `false`.\n    Options& set_reserve_max_size(bool reserve_max_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      reserve_max_size_ = reserve_max_size;\n      return *this;\n    }\n    Options&& set_reserve_max_size(bool reserve_max_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_reserve_max_size(reserve_max_size));\n    }\n    bool reserve_max_size() const { return reserve_max_size_; }\n\n    // Returns effective `BufferOptions` as overridden by other options:\n    // If `reserve_max_size()` is `true` and `pledged_size()` is not\n    // `std::nullopt`, then `pledged_size()` overrides `buffer_size()`.\n    BufferOptions effective_buffer_options() const {\n      BufferOptions options = buffer_options();\n      if (reserve_max_size() && pledged_size() != std::nullopt) {\n        options.set_buffer_size(\n            UnsignedMax(SaturatingIntCast<size_t>(*pledged_size()), size_t{1}));\n      }\n      return options;\n    }\n\n    // Options for a global `RecyclingPool` of compression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // compression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    Options& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    Options&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    int compression_level_ = kDefaultCompressionLevel;\n    int window_log_ = kDefaultWindowLog;\n    Lz4Dictionary dictionary_;\n    bool store_content_checksum_ = false;\n    bool store_block_checksum_ = false;\n    std::optional<Position> pledged_size_;\n    bool reserve_max_size_ = false;\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Returns the compressed `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsReadMode() override;\n\n protected:\n  explicit Lz4WriterBase(Closed) noexcept : BufferedWriter(kClosed) {}\n\n  explicit Lz4WriterBase(BufferOptions buffer_options,\n                         Lz4Dictionary&& dictionary,\n                         std::optional<Position> pledged_size,\n                         bool reserve_max_size,\n                         const RecyclingPoolOptions& recycling_pool_options);\n\n  Lz4WriterBase(Lz4WriterBase&& that) noexcept;\n  Lz4WriterBase& operator=(Lz4WriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, Lz4Dictionary&& dictionary,\n             std::optional<Position> pledged_size, bool reserve_max_size,\n             const RecyclingPoolOptions& recycling_pool_options);\n  void Initialize(Writer* dest, int compression_level, int window_log,\n                  bool store_content_checksum, bool store_block_checksum);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool WriteInternal(absl::string_view src) override;\n  bool FlushImpl(FlushType flush_type);\n  Reader* ReadModeBehindBuffer(Position initial_pos) override;\n\n private:\n  struct LZ4F_cctxDeleter {\n    void operator()(LZ4F_cctx* ptr) const {\n      const LZ4F_errorCode_t result = LZ4F_freeCompressionContext(ptr);\n      RIEGELI_ASSERT(!LZ4F_isError(result))\n          << \"LZ4F_freeCompressionContext() failed: \"\n          << LZ4F_getErrorName(result);\n    }\n  };\n\n  bool DoneCompression(Writer& dest);\n\n  Lz4Dictionary dictionary_;\n  std::optional<Position> pledged_size_;\n  bool reserve_max_size_ = false;\n  Position initial_compressed_pos_ = 0;\n  LZ4F_preferences_t preferences_{};\n  RecyclingPoolOptions recycling_pool_options_;\n  // If `ok()` but `compressor_ == nullptr` then `LZ4F_compressEnd()` was\n  // already called.\n  RecyclingPool<LZ4F_cctx, LZ4F_cctxDeleter>::Handle compressor_;\n  // `stable_src_` becomes `true` when all data remaining to compress are known\n  // to stay under their current addresses.\n  bool stable_src_ = false;\n  // The amount of uncompressed data buffered in `LZ4F_cctx`. This allows to\n  // reduce data copying by aligning source boundaries appropriately.\n  size_t buffered_length_ = 0;\n\n  AssociatedReader<Lz4Reader<Reader*>> associated_reader_;\n};\n\n// A `Writer` which compresses data with Lz4 before passing it to another\n// `Writer`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Writer` must not be accessed until the `Lz4Writer` is closed\n// or no longer used, except that it is allowed to read the destination of the\n// compressed `Writer` immediately after `Flush()`.\ntemplate <typename Dest = Writer*>\nclass Lz4Writer : public Lz4WriterBase {\n public:\n  // Creates a closed `Lz4Writer`.\n  explicit Lz4Writer(Closed) noexcept : Lz4WriterBase(kClosed) {}\n\n  // Will write to the compressed `Writer` provided by `dest`.\n  explicit Lz4Writer(Initializer<Dest> dest, Options options = Options());\n\n  Lz4Writer(Lz4Writer&& that) = default;\n  Lz4Writer& operator=(Lz4Writer&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `Lz4Writer`. This avoids\n  // constructing a temporary `Lz4Writer` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  // The object providing and possibly owning the compressed `Writer`.\n  Dependency<Writer*, Dest> dest_;\n};\n\nexplicit Lz4Writer(Closed) -> Lz4Writer<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit Lz4Writer(Dest&& dest,\n                   Lz4WriterBase::Options options = Lz4WriterBase::Options())\n    -> Lz4Writer<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline Lz4WriterBase::Lz4WriterBase(\n    BufferOptions buffer_options, Lz4Dictionary&& dictionary,\n    std::optional<Position> pledged_size, bool reserve_max_size,\n    const RecyclingPoolOptions& recycling_pool_options)\n    : BufferedWriter(buffer_options),\n      dictionary_(std::move(dictionary)),\n      pledged_size_(pledged_size),\n      reserve_max_size_(reserve_max_size),\n      recycling_pool_options_(recycling_pool_options) {}\n\ninline Lz4WriterBase::Lz4WriterBase(Lz4WriterBase&& that) noexcept\n    : BufferedWriter(static_cast<BufferedWriter&&>(that)),\n      dictionary_(std::move(that.dictionary_)),\n      pledged_size_(that.pledged_size_),\n      reserve_max_size_(that.reserve_max_size_),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      preferences_(that.preferences_),\n      recycling_pool_options_(that.recycling_pool_options_),\n      compressor_(std::move(that.compressor_)),\n      stable_src_(that.stable_src_),\n      buffered_length_(that.buffered_length_),\n      associated_reader_(std::move(that.associated_reader_)) {}\n\ninline Lz4WriterBase& Lz4WriterBase::operator=(Lz4WriterBase&& that) noexcept {\n  BufferedWriter::operator=(static_cast<BufferedWriter&&>(that));\n  dictionary_ = std::move(that.dictionary_);\n  pledged_size_ = that.pledged_size_;\n  reserve_max_size_ = that.reserve_max_size_;\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  preferences_ = that.preferences_;\n  recycling_pool_options_ = that.recycling_pool_options_;\n  compressor_ = std::move(that.compressor_);\n  stable_src_ = that.stable_src_;\n  buffered_length_ = that.buffered_length_;\n  associated_reader_ = std::move(that.associated_reader_);\n  return *this;\n}\n\ninline void Lz4WriterBase::Reset(Closed) {\n  BufferedWriter::Reset(kClosed);\n  pledged_size_ = std::nullopt;\n  reserve_max_size_ = false;\n  initial_compressed_pos_ = 0;\n  preferences_ = {};\n  recycling_pool_options_ = RecyclingPoolOptions();\n  compressor_.reset();\n  // Must be destroyed after `compressor_`.\n  dictionary_ = Lz4Dictionary();\n  stable_src_ = false;\n  buffered_length_ = 0;\n  associated_reader_.Reset();\n}\n\ninline void Lz4WriterBase::Reset(\n    BufferOptions buffer_options, Lz4Dictionary&& dictionary,\n    std::optional<Position> pledged_size, bool reserve_max_size,\n    const RecyclingPoolOptions& recycling_pool_options) {\n  BufferedWriter::Reset(buffer_options);\n  pledged_size_ = pledged_size;\n  reserve_max_size_ = reserve_max_size;\n  initial_compressed_pos_ = 0;\n  recycling_pool_options_ = recycling_pool_options;\n  preferences_ = {};\n  compressor_.reset();\n  // Must be destroyed after `compressor_`.\n  dictionary_ = std::move(dictionary);\n  stable_src_ = false;\n  buffered_length_ = 0;\n  associated_reader_.Reset();\n}\n\ntemplate <typename Dest>\ninline Lz4Writer<Dest>::Lz4Writer(Initializer<Dest> dest, Options options)\n    : Lz4WriterBase(options.effective_buffer_options(),\n                    std::move(options.dictionary()), options.pledged_size(),\n                    options.reserve_max_size(),\n                    options.recycling_pool_options()),\n      dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.compression_level(), options.window_log(),\n             options.store_content_checksum(), options.store_block_checksum());\n}\n\ntemplate <typename Dest>\ninline void Lz4Writer<Dest>::Reset(Closed) {\n  Lz4WriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void Lz4Writer<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  Lz4WriterBase::Reset(options.effective_buffer_options(),\n                       std::move(options.dictionary()), options.pledged_size(),\n                       options.reserve_max_size(),\n                       options.recycling_pool_options());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.compression_level(), options.window_log(),\n             options.store_content_checksum(), options.store_block_checksum());\n}\n\ntemplate <typename Dest>\nvoid Lz4Writer<Dest>::Done() {\n  Lz4WriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <typename Dest>\nbool Lz4Writer<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!Lz4WriterBase::FlushImpl(flush_type))) return false;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Flush(flush_type))) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n  return true;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_LZ4_LZ4_WRITER_H_\n"
  },
  {
    "path": "riegeli/messages/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"parse_message\",\n    srcs = [\"parse_message.cc\"],\n    hdrs = [\"parse_message.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:cord_iterator_span\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:chain_reader\",\n        \"//riegeli/bytes:cord_reader\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/varint:varint_reading\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_protobuf//:protobuf_lite\",\n    ],\n)\n\ncc_library(\n    name = \"serialize_message\",\n    srcs = [\"serialize_message.cc\"],\n    hdrs = [\"serialize_message.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:compact_string\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:string_utils\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:array_writer\",\n        \"//riegeli/bytes:backward_writer\",\n        \"//riegeli/bytes:chain_writer\",\n        \"//riegeli/bytes:cord_writer\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/varint:varint_writing\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/types:span\",\n        \"@com_google_protobuf//:protobuf_lite\",\n    ],\n)\n\ncc_library(\n    name = \"text_parse_message\",\n    srcs = [\"text_parse_message.cc\"],\n    hdrs = [\"text_parse_message.h\"],\n    deps = [\n        \":parse_message\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/bytes:chain_reader\",\n        \"//riegeli/bytes:cord_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:string_reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_protobuf//:protobuf\",\n    ],\n)\n\ncc_library(\n    name = \"text_print_message\",\n    srcs = [\"text_print_message.cc\"],\n    hdrs = [\"text_print_message.h\"],\n    deps = [\n        \":serialize_message\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/bytes:chain_writer\",\n        \"//riegeli/bytes:cord_writer\",\n        \"//riegeli/bytes:string_writer\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_protobuf//:protobuf\",\n    ],\n)\n\ncc_library(\n    name = \"message_wire_format\",\n    hdrs = [\"message_wire_format.h\"],\n)\n\ncc_library(\n    name = \"map_entry_field\",\n    hdrs = [\"map_entry_field.h\"],\n    deps = [\"@com_google_absl//absl/base:nullability\"],\n)\n\ncc_library(\n    name = \"serialized_message_reader_internal\",\n    srcs = [\"serialized_message_reader.cc\"],\n    hdrs = [\"serialized_message_reader_internal.h\"],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \":message_wire_format\",\n        \"//riegeli/base:cord_iterator_span\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"//riegeli/bytes:reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"serialized_message_reader\",\n    srcs = [\"field_handlers.cc\"],\n    hdrs = [\n        \"field_handlers.h\",\n        \"serialized_message_reader.h\",\n    ],\n    deps = [\n        \":message_wire_format\",\n        \":serialized_message_reader_internal\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:cord_iterator_span\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:type_traits\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:chain_reader\",\n        \"//riegeli/bytes:cord_reader\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:string_reader\",\n        \"//riegeli/endian:endian_reading\",\n        \"//riegeli/varint:varint_reading\",\n        \"@com_google_absl//absl/base\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"context_projection\",\n    hdrs = [\"context_projection.h\"],\n    deps = [\n        \":serialized_message_reader_internal\",\n        \"//riegeli/base:cord_iterator_span\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"dynamic_field_handler\",\n    hdrs = [\"dynamic_field_handler.h\"],\n    deps = [\n        \":serialized_message_reader\",\n        \":serialized_message_reader_internal\",\n        \"//riegeli/base:cord_iterator_span\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"field_handler_map\",\n    hdrs = [\"field_handler_map.h\"],\n    deps = [\n        \":serialized_message_reader\",\n        \":serialized_message_reader_internal\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:cord_iterator_span\",\n        \"//riegeli/base:hybrid_direct_map\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/container:flat_hash_map\",\n        \"@com_google_absl//absl/functional:any_invocable\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"field_copier\",\n    hdrs = [\"field_copier.h\"],\n    deps = [\n        \":message_wire_format\",\n        \":serialized_message_reader\",\n        \":serialized_message_writer\",\n        \"//riegeli/base:cord_iterator_span\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"serialized_message_writer\",\n    srcs = [\"serialized_message_writer.cc\"],\n    hdrs = [\"serialized_message_writer.h\"],\n    deps = [\n        \":message_wire_format\",\n        \":serialize_message\",\n        \"//riegeli/base:any\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:constexpr\",\n        \"//riegeli/base:cord_iterator_span\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:cord_writer\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"//riegeli/bytes:null_writer\",\n        \"//riegeli/bytes:read_all\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:stringify\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/endian:endian_writing\",\n        \"//riegeli/varint:varint_writing\",\n        \"@com_google_absl//absl/base\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_protobuf//:protobuf_lite\",\n    ],\n)\n\ncc_library(\n    name = \"serialized_message_backward_writer\",\n    srcs = [\"serialized_message_backward_writer.cc\"],\n    hdrs = [\"serialized_message_backward_writer.h\"],\n    deps = [\n        \":message_wire_format\",\n        \":serialize_message\",\n        \"//riegeli/base:any\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:cord_iterator_span\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:backward_writer\",\n        \"//riegeli/bytes:copy_all\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:stringify\",\n        \"//riegeli/endian:endian_writing\",\n        \"//riegeli/varint:varint_writing\",\n        \"@com_google_absl//absl/base\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_protobuf//:protobuf_lite\",\n    ],\n)\n\ncc_library(\n    name = \"serialized_message_assembler\",\n    srcs = [\"serialized_message_assembler.cc\"],\n    hdrs = [\"serialized_message_assembler.h\"],\n    deps = [\n        \":field_copier\",\n        \":field_handler_map\",\n        \":serialized_message_reader\",\n        \":serialized_message_writer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:cord_iterator_span\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/bytes:chain_writer\",\n        \"//riegeli/bytes:cord_writer\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:string_writer\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/base:nullability\",\n        \"@com_google_absl//absl/container:inlined_vector\",\n        \"@com_google_absl//absl/container:linked_hash_map\",\n        \"@com_google_absl//absl/functional:any_invocable\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/messages/context_projection.h",
    "content": "// Copyright 2026 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_CONTEXT_PROJECTION_H_\n#define RIEGELI_MESSAGES_CONTEXT_PROJECTION_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <string>\n#include <tuple>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/messages/serialized_message_reader_internal.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// The type of `ContextAt<indices...>`.\ntemplate <size_t... indices>\nstruct ContextAtImpl;\n\n// A projection for `ContextProjection()` which selects a subset of context\n// parameters. The other context parameters are ignored.\ntemplate <size_t... indices>\nconstexpr ContextAtImpl<indices...> ContextAt = {};\n\n// The type returned by `ContextProjection()` with a single projection.\ntemplate <auto projection, typename FieldHandler>\nclass ContextProjectionImpl;\n\n// Adapts a field handler to depend on a part of the context. Makes a field\n// handler taking some outer context parameters from a field handler taking\n// inner context parameters.\n//\n// `projection...` parameters are applied sequentially to transform the outer\n// context to the inner context. Each `projection` is an invocable which\n// returns either a reference or a tuple of references. Common projections\n// include `ContextAt<indices...>` to select a subset of context parameters,\n// and `&Context::member` to select a member of the only context parameter.\n\ntemplate <typename FieldHandler>\nconstexpr FieldHandler&& ContextProjection(FieldHandler&& field_handler) {\n  return std::forward<FieldHandler>(field_handler);\n}\n\ntemplate <auto projection, typename FieldHandler>\nconstexpr ContextProjectionImpl<projection, std::decay_t<FieldHandler>>\nContextProjection(FieldHandler&& field_handler) {\n  return ContextProjectionImpl<projection, std::decay_t<FieldHandler>>(\n      std::forward<FieldHandler>(field_handler));\n}\n\ntemplate <auto first_projection, auto second_projection,\n          auto... rest_projections, typename FieldHandler>\nconstexpr auto ContextProjection(FieldHandler&& field_handler) {\n  return ContextProjection<first_projection>(\n      ContextProjection<second_projection, rest_projections...>(\n          std::forward<FieldHandler>(field_handler)));\n}\n\n// Implementation details follow.\n\nnamespace context_projection_internal {\n\ntemplate <typename T>\nstruct DecodeContextResult {};\n\ntemplate <typename Context>\nstruct DecodeContextResult<Context&>\n    : std::type_identity<std::tuple<Context&>> {};\n\ntemplate <typename... Context>\nstruct DecodeContextResult<std::tuple<Context&...>>\n    : std::type_identity<std::tuple<Context&...>> {};\n\ntemplate <auto projection, typename... Context>\nstruct InnerContextTuple\n    : DecodeContextResult<\n          std::invoke_result_t<decltype(projection), Context&...>> {};\n\n}  // namespace context_projection_internal\n\ntemplate <size_t... indices>\nstruct ContextAtImpl {\n  template <typename... Context>\n  std::tuple<std::tuple_element_t<indices, std::tuple<Context&...>>...>\n  operator()(Context&... context) const {\n    const std::tuple<Context&...> context_tuple = {context...};\n    return {std::get<indices>(context_tuple)...};\n  }\n};\n\ntemplate <auto projection, typename FieldHandler>\nclass ContextProjectionImpl {\n private:\n  template <template <typename T, typename... InnerContext> class Predicate,\n            typename InnerContextTuple>\n  struct IsProjectedFieldHandlerImpl;\n\n  template <template <typename T, typename... InnerContext> class Predicate,\n            typename... InnerContext>\n  struct IsProjectedFieldHandlerImpl<Predicate, std::tuple<InnerContext&...>>\n      : Predicate<FieldHandler, InnerContext...> {};\n\n  template <template <typename T, typename... InnerContext> class Predicate,\n            typename... Context>\n  struct IsProjectedFieldHandler\n      : IsProjectedFieldHandlerImpl<\n            Predicate, typename context_projection_internal::InnerContextTuple<\n                           projection, Context...>::type> {};\n\n public:\n  static constexpr int kFieldNumber = FieldHandler::kFieldNumber;\n\n  template <typename FieldHandlerInitializer,\n            std::enable_if_t<\n                std::is_convertible_v<FieldHandlerInitializer&&, FieldHandler>,\n                int> = 0>\n  explicit constexpr ContextProjectionImpl(\n      FieldHandlerInitializer&& field_handler)\n      : field_handler_(std::forward<FieldHandlerInitializer>(field_handler)) {}\n\n  ContextProjectionImpl(const ContextProjectionImpl& that) = default;\n  ContextProjectionImpl& operator=(const ContextProjectionImpl& that) = default;\n\n  ContextProjectionImpl(ContextProjectionImpl&& that) = default;\n  ContextProjectionImpl& operator=(ContextProjectionImpl&& that) = default;\n\n  template <\n      typename... Context,\n      std::enable_if_t<\n          IsProjectedFieldHandler<\n              serialized_message_reader_internal::IsStaticFieldHandlerForVarint,\n              Context...>::value,\n          int> = 0>\n  absl::Status HandleVarint(uint64_t repr, Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.HandleVarint(repr, inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <typename DependentFieldHandler = FieldHandler,\n            std::enable_if_t<serialized_message_reader_internal::\n                                 IsDynamicFieldHandlerForVarintSomeContext<\n                                     DependentFieldHandler>::value,\n                             int> = 0>\n  auto AcceptVarint(int field_number) const {\n    return field_handler_.AcceptVarint(field_number);\n  }\n\n  template <typename Accepted, typename... Context,\n            std::enable_if_t<\n                IsProjectedFieldHandler<serialized_message_reader_internal::\n                                            IsDynamicFieldHandlerForVarint,\n                                        Context...>::value,\n                int> = 0>\n  absl::Status DynamicHandleVarint(Accepted&& accepted, uint64_t repr,\n                                   Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.DynamicHandleVarint(\n              std::forward<Accepted>(accepted), repr, inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <typename... Context,\n            std::enable_if_t<\n                IsProjectedFieldHandler<serialized_message_reader_internal::\n                                            IsStaticFieldHandlerForFixed32,\n                                        Context...>::value,\n                int> = 0>\n  absl::Status HandleFixed32(uint32_t repr, Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.HandleFixed32(repr, inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <typename DependentFieldHandler = FieldHandler,\n            std::enable_if_t<serialized_message_reader_internal::\n                                 IsDynamicFieldHandlerForFixed32SomeContext<\n                                     DependentFieldHandler>::value,\n                             int> = 0>\n  auto AcceptFixed32(int field_number) const {\n    return field_handler_.AcceptFixed32(field_number);\n  }\n\n  template <typename Accepted, typename... Context,\n            std::enable_if_t<\n                IsProjectedFieldHandler<serialized_message_reader_internal::\n                                            IsDynamicFieldHandlerForFixed32,\n                                        Context...>::value,\n                int> = 0>\n  absl::Status DynamicHandleFixed32(Accepted&& accepted, uint32_t repr,\n                                    Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.DynamicHandleFixed32(\n              std::forward<Accepted>(accepted), repr, inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <typename... Context,\n            std::enable_if_t<\n                IsProjectedFieldHandler<serialized_message_reader_internal::\n                                            IsStaticFieldHandlerForFixed64,\n                                        Context...>::value,\n                int> = 0>\n  absl::Status HandleFixed64(uint64_t repr, Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.HandleFixed64(repr, inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <typename DependentFieldHandler = FieldHandler,\n            std::enable_if_t<serialized_message_reader_internal::\n                                 IsDynamicFieldHandlerForFixed64SomeContext<\n                                     DependentFieldHandler>::value,\n                             int> = 0>\n  auto AcceptFixed64(int field_number) const {\n    return field_handler_.AcceptFixed64(field_number);\n  }\n\n  template <typename Accepted, typename... Context,\n            std::enable_if_t<\n                IsProjectedFieldHandler<serialized_message_reader_internal::\n                                            IsDynamicFieldHandlerForFixed64,\n                                        Context...>::value,\n                int> = 0>\n  absl::Status DynamicHandleFixed64(Accepted&& accepted, uint64_t repr,\n                                    Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.DynamicHandleFixed64(\n              std::forward<Accepted>(accepted), repr, inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <\n      typename... Context,\n      std::enable_if_t<IsProjectedFieldHandler<\n                           serialized_message_reader_internal::\n                               IsStaticFieldHandlerForLengthDelimitedFromReader,\n                           Context...>::value,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromReader(ReaderSpan<> repr,\n                                               Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.HandleLengthDelimitedFromReader(\n              std::move(repr), inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <\n      typename... Context,\n      std::enable_if_t<IsProjectedFieldHandler<\n                           serialized_message_reader_internal::\n                               IsStaticFieldHandlerForLengthDelimitedFromCord,\n                           Context...>::value,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromCord(CordIteratorSpan repr,\n                                             std::string& scratch,\n                                             Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.HandleLengthDelimitedFromCord(\n              std::move(repr), scratch, inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <\n      typename... Context,\n      std::enable_if_t<IsProjectedFieldHandler<\n                           serialized_message_reader_internal::\n                               IsStaticFieldHandlerForLengthDelimitedFromString,\n                           Context...>::value,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromString(absl::string_view repr,\n                                               Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.HandleLengthDelimitedFromString(\n              repr, inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <\n      typename DependentFieldHandler = FieldHandler,\n      std::enable_if_t<serialized_message_reader_internal::\n                           IsDynamicFieldHandlerForLengthDelimitedSomeContext<\n                               DependentFieldHandler>::value,\n                       int> = 0>\n  auto AcceptLengthDelimited(int field_number) const {\n    return field_handler_.AcceptLengthDelimited(field_number);\n  }\n\n  template <typename Accepted, typename... Context,\n            std::enable_if_t<\n                IsProjectedFieldHandler<\n                    serialized_message_reader_internal::\n                        IsDynamicFieldHandlerForLengthDelimitedFromReader,\n                    Context...>::value,\n                int> = 0>\n  absl::Status DynamicHandleLengthDelimitedFromReader(\n      Accepted&& accepted, ReaderSpan<> repr, Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.DynamicHandleLengthDelimitedFromReader(\n              std::forward<Accepted>(accepted), std::move(repr),\n              inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <\n      typename Accepted, typename... Context,\n      std::enable_if_t<IsProjectedFieldHandler<\n                           serialized_message_reader_internal::\n                               IsDynamicFieldHandlerForLengthDelimitedFromCord,\n                           Context...>::value,\n                       int> = 0>\n  absl::Status DynamicHandleLengthDelimitedFromCord(Accepted&& accepted,\n                                                    CordIteratorSpan repr,\n                                                    std::string& scratch,\n                                                    Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.DynamicHandleLengthDelimitedFromCord(\n              std::forward<Accepted>(accepted), std::move(repr), scratch,\n              inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <typename Accepted, typename... Context,\n            std::enable_if_t<\n                IsProjectedFieldHandler<\n                    serialized_message_reader_internal::\n                        IsDynamicFieldHandlerForLengthDelimitedFromString,\n                    Context...>::value,\n                int> = 0>\n  absl::Status DynamicHandleLengthDelimitedFromString(\n      Accepted&& accepted, absl::string_view repr, Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.DynamicHandleLengthDelimitedFromString(\n              std::forward<Accepted>(accepted), repr, inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <typename... Context,\n            std::enable_if_t<\n                IsProjectedFieldHandler<serialized_message_reader_internal::\n                                            IsStaticFieldHandlerForStartGroup,\n                                        Context...>::value,\n                int> = 0>\n  absl::Status HandleStartGroup(Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.HandleStartGroup(inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <typename DependentFieldHandler = FieldHandler,\n            std::enable_if_t<serialized_message_reader_internal::\n                                 IsDynamicFieldHandlerForStartGroupSomeContext<\n                                     DependentFieldHandler>::value,\n                             int> = 0>\n  auto AcceptStartGroup(int field_number) const {\n    return field_handler_.AcceptStartGroup(field_number);\n  }\n\n  template <typename Accepted, typename... Context,\n            std::enable_if_t<\n                IsProjectedFieldHandler<serialized_message_reader_internal::\n                                            IsDynamicFieldHandlerForStartGroup,\n                                        Context...>::value,\n                int> = 0>\n  absl::Status DynamicHandleStartGroup(Accepted&& accepted,\n                                       Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.DynamicHandleStartGroup(\n              std::forward<Accepted>(accepted), inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <typename... Context,\n            std::enable_if_t<\n                IsProjectedFieldHandler<serialized_message_reader_internal::\n                                            IsStaticFieldHandlerForEndGroup,\n                                        Context...>::value,\n                int> = 0>\n  absl::Status HandleEndGroup(Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.HandleEndGroup(inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n  template <typename DependentFieldHandler = FieldHandler,\n            std::enable_if_t<serialized_message_reader_internal::\n                                 IsDynamicFieldHandlerForEndGroupSomeContext<\n                                     DependentFieldHandler>::value,\n                             int> = 0>\n  auto AcceptEndGroup(int field_number) const {\n    return field_handler_.AcceptEndGroup(field_number);\n  }\n\n  template <typename Accepted, typename... Context,\n            std::enable_if_t<\n                IsProjectedFieldHandler<serialized_message_reader_internal::\n                                            IsDynamicFieldHandlerForEndGroup,\n                                        Context...>::value,\n                int> = 0>\n  absl::Status DynamicHandleEndGroup(Accepted&& accepted,\n                                     Context&... context) const {\n    return std::apply(\n        [&](auto&... inner_context) {\n          return field_handler_.DynamicHandleEndGroup(\n              std::forward<Accepted>(accepted), inner_context...);\n        },\n        InnerContext(context...));\n  }\n\n private:\n  template <typename... Context>\n  static\n      typename context_projection_internal::InnerContextTuple<projection,\n                                                              Context...>::type\n      InnerContext(Context&... context) {\n    // If `projection` returns a reference, this returns a tuple with a single\n    // element. If `projection` returns a tuple of references, this returns the\n    // tuple while the braces are redundant.\n    return {std::invoke(projection, context...)};\n  }\n\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS FieldHandler field_handler_;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_CONTEXT_PROJECTION_H_\n"
  },
  {
    "path": "riegeli/messages/dynamic_field_handler.h",
    "content": "// Copyright 2026 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_DYNAMIC_FIELD_HANDLER_H_\n#define RIEGELI_MESSAGES_DYNAMIC_FIELD_HANDLER_H_\n\n#include <cstdint>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/messages/serialized_message_reader.h\"\n#include \"riegeli/messages/serialized_message_reader_internal.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\ntemplate <typename FieldHandler>\nclass DynamicFieldHandlerType;\n\n// A field handler for `SerializedMessageReader` for a single field, with the\n// field number specified at runtime.\n//\n// It is created from an unbound field handler.\n//\n// `DynamicFieldHandler` is similar to a `FieldHandlerMap` with a single\n// registered field handler, but more efficient and without type erasure.\ntemplate <typename BaseFieldHandler>\nconstexpr DynamicFieldHandlerType<std::decay_t<BaseFieldHandler>>\nDynamicFieldHandler(int field_number, BaseFieldHandler&& field_handler) {\n  return DynamicFieldHandlerType<std::decay_t<BaseFieldHandler>>(\n      field_number, std::forward<BaseFieldHandler>(field_handler));\n}\n\n// Implementation details follow.\ntemplate <typename BaseFieldHandler>\nclass DynamicFieldHandlerType {\n private:\n  struct Accepted {};\n\n  class MaybeAccepted {\n   public:\n    explicit MaybeAccepted(bool accepted) : accepted_(accepted) {}\n\n    MaybeAccepted(const MaybeAccepted& that) = default;\n\n    explicit operator bool() const { return accepted_; }\n\n    Accepted operator*() const { return Accepted(); }\n\n   private:\n    bool accepted_;\n  };\n\n public:\n  template <\n      typename BaseFieldHandlerInitializer,\n      std::enable_if_t<std::is_convertible_v<BaseFieldHandlerInitializer&&,\n                                             BaseFieldHandler>,\n                       int> = 0>\n  explicit constexpr DynamicFieldHandlerType(\n      int field_number, BaseFieldHandlerInitializer&& base_field_handler)\n      : field_number_(field_number),\n        base_field_handler_(\n            std::forward<BaseFieldHandlerInitializer>(base_field_handler)) {}\n\n  // Implement the field handler protocol.\n\n  static constexpr int kFieldNumber = kDynamicFieldNumber;\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  MaybeAccepted AcceptVarint(int field_number) const {\n    return MaybeAccepted(field_number == field_number_);\n  }\n\n  template <\n      typename... Context,\n      std::enable_if_t<\n          std::conjunction_v<\n              serialized_message_reader_internal::IsUnboundFieldHandler<\n                  BaseFieldHandler, Context...>,\n              serialized_message_reader_internal::IsStaticFieldHandlerForVarint<\n                  BaseFieldHandler, Context...>>,\n          int> = 0>\n  absl::Status DynamicHandleVarint(Accepted, uint64_t repr,\n                                   Context&... context) const {\n    return base_field_handler_.HandleVarint(repr, context...);\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  MaybeAccepted AcceptFixed32(int field_number) const {\n    return MaybeAccepted(field_number == field_number_);\n  }\n\n  template <\n      typename... Context,\n      std::enable_if_t<\n          std::conjunction_v<\n              serialized_message_reader_internal::IsUnboundFieldHandler<\n                  BaseFieldHandler, Context...>,\n              serialized_message_reader_internal::\n                  IsStaticFieldHandlerForFixed32<BaseFieldHandler, Context...>>,\n          int> = 0>\n  absl::Status DynamicHandleFixed32(Accepted, uint32_t repr,\n                                    Context&... context) const {\n    return base_field_handler_.HandleFixed32(repr, context...);\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  MaybeAccepted AcceptFixed64(int field_number) const {\n    return MaybeAccepted(field_number == field_number_);\n  }\n\n  template <\n      typename... Context,\n      std::enable_if_t<\n          std::conjunction_v<\n              serialized_message_reader_internal::IsUnboundFieldHandler<\n                  BaseFieldHandler, Context...>,\n              serialized_message_reader_internal::\n                  IsStaticFieldHandlerForFixed64<BaseFieldHandler, Context...>>,\n          int> = 0>\n  absl::Status DynamicHandleFixed64(Accepted, uint64_t repr,\n                                    Context&... context) const {\n    return base_field_handler_.HandleFixed64(repr, context...);\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE MaybeAccepted\n  AcceptLengthDelimited(int field_number) const {\n    return MaybeAccepted(field_number == field_number_);\n  }\n\n  template <typename... Context,\n            std::enable_if_t<\n                std::conjunction_v<\n                    serialized_message_reader_internal::IsUnboundFieldHandler<\n                        BaseFieldHandler, Context...>,\n                    serialized_message_reader_internal::\n                        IsStaticFieldHandlerForLengthDelimitedFromReader<\n                            BaseFieldHandler, Context...>>,\n                int> = 0>\n  absl::Status DynamicHandleLengthDelimitedFromReader(\n      Accepted, ReaderSpan<> repr, Context&... context) const {\n    return base_field_handler_.HandleLengthDelimitedFromReader(std::move(repr),\n                                                               context...);\n  }\n\n  template <typename... Context,\n            std::enable_if_t<\n                std::conjunction_v<\n                    serialized_message_reader_internal::IsUnboundFieldHandler<\n                        BaseFieldHandler, Context...>,\n                    serialized_message_reader_internal::\n                        IsStaticFieldHandlerForLengthDelimitedFromCord<\n                            BaseFieldHandler, Context...>>,\n                int> = 0>\n  absl::Status DynamicHandleLengthDelimitedFromCord(Accepted,\n                                                    CordIteratorSpan repr,\n                                                    std::string& scratch,\n                                                    Context&... context) const {\n    return base_field_handler_.HandleLengthDelimitedFromCord(\n        std::move(repr), scratch, context...);\n  }\n\n  template <typename... Context,\n            std::enable_if_t<\n                std::conjunction_v<\n                    serialized_message_reader_internal::IsUnboundFieldHandler<\n                        BaseFieldHandler, Context...>,\n                    serialized_message_reader_internal::\n                        IsStaticFieldHandlerForLengthDelimitedFromString<\n                            BaseFieldHandler, Context...>>,\n                int> = 0>\n  absl::Status DynamicHandleLengthDelimitedFromString(\n      Accepted, absl::string_view repr, Context&... context) const {\n    return base_field_handler_.HandleLengthDelimitedFromString(repr,\n                                                               context...);\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  MaybeAccepted AcceptStartGroup(int field_number) const {\n    return MaybeAccepted(field_number == field_number_);\n  }\n\n  template <typename... Context,\n            std::enable_if_t<\n                std::conjunction_v<\n                    serialized_message_reader_internal::IsUnboundFieldHandler<\n                        BaseFieldHandler, Context...>,\n                    serialized_message_reader_internal::\n                        IsStaticFieldHandlerForStartGroup<BaseFieldHandler,\n                                                          Context...>>,\n                int> = 0>\n  absl::Status DynamicHandleStartGroup(Accepted, Context&... context) const {\n    return base_field_handler_.HandleStartGroup(context...);\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  MaybeAccepted AcceptEndGroup(int field_number) const {\n    return MaybeAccepted(field_number == field_number_);\n  }\n\n  template <typename... Context,\n            std::enable_if_t<\n                std::conjunction_v<\n                    serialized_message_reader_internal::IsUnboundFieldHandler<\n                        BaseFieldHandler, Context...>,\n                    serialized_message_reader_internal::\n                        IsStaticFieldHandlerForEndGroup<BaseFieldHandler,\n                                                        Context...>>,\n                int> = 0>\n  absl::Status DynamicHandleEndGroup(Accepted, Context&... context) const {\n    return base_field_handler_.HandleEndGroup(context...);\n  }\n\n private:\n  int field_number_;\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS BaseFieldHandler base_field_handler_;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_DYNAMIC_FIELD_HANDLER_H_\n"
  },
  {
    "path": "riegeli/messages/field_copier.h",
    "content": "// Copyright 2026 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_FIELD_COPIER_H_\n#define RIEGELI_MESSAGES_FIELD_COPIER_H_\n\n#include <stdint.h>\n\n#include <optional>\n#include <string>\n#include <tuple>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/messages/message_wire_format.h\"\n#include \"riegeli/messages/serialized_message_reader.h\"\n#include \"riegeli/messages/serialized_message_writer.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// The type returned by `FieldCopier()`.\ntemplate <int field_number, WireTypeSet wire_types = AllWireTypes>\nclass FieldCopierType;\n\n// The type returned by `DynamicFieldCopier()`.\ntemplate <typename Accept, WireTypeSet wire_types = AllWireTypes>\nclass DynamicFieldCopierType;\n\n// The type of the `accept` function used by `AnyFieldCopier()`.\nstruct AcceptAnyField;\n\n// The type returned by `AnyFieldCopier()`.\nusing AnyFieldCopierType = DynamicFieldCopierType<AcceptAnyField>;\n\n// The type returned by `UnboundFieldCopier()`.\ntemplate <WireTypeSet wire_types = AllWireTypes>\nclass UnboundFieldCopierType;\n\n// A field handler for `SerializedMessageReader` which copies the given field to\n// a `SerializedMessageWriter`.\n//\n// As an optimization, `wire_types` constrains the set of wire types to handle.\n// This yields smaller and faster code.\n//\n// `Context...` types must contain exactly one occurrence of\n// `SerializedMessageWriter`. Use `ContextProjection()` to select the\n// `SerializedMessageWriter` if this is not the case.\ntemplate <int field_number, WireTypeSet wire_types = AllWireTypes>\nconstexpr FieldCopierType<field_number, wire_types> FieldCopier() {\n  return FieldCopierType<field_number, wire_types>();\n}\n\n// A field handler for `SerializedMessageReader` which copies the given field to\n// a `SerializedMessageWriter`, with the predicate over field numbers specified\n// at runtime, possibly mapping them to other field numbers.\n//\n// `accept` is an invocable taking the field number as `int` and returning\n// `std::optional<int>`. If it returns a value other than `std::nullopt`,\n// the field is copied, and the returned value is used as the new field number.\n//\n// `Context...` types must contain exactly one occurrence of\n// `SerializedMessageWriter`. Use `ContextProjection()` to select the\n// `SerializedMessageWriter` if this is not the case.\ntemplate <WireTypeSet wire_types = AllWireTypes, typename Accept>\nconstexpr DynamicFieldCopierType<std::decay_t<Accept>, wire_types>\nDynamicFieldCopier(Accept&& accept) {\n  return DynamicFieldCopierType<std::decay_t<Accept>, wire_types>(\n      std::forward<Accept>(accept));\n}\n\n// A field handler for `SerializedMessageReader` which copies any field to a\n// `SerializedMessageWriter`.\n//\n// It is meant to be used as the last field handler, so that remaining fields\n// not handled by previous field handlers will be copied unchanged.\n//\n// `Context...` types must contain exactly one occurrence of\n// `SerializedMessageWriter`. Use `ContextProjection()` to select the\n// `SerializedMessageWriter` if this is not the case.\nconstexpr AnyFieldCopierType AnyFieldCopier();\n\n// An unbound field handler for a `DynamicFieldHandler` or `FieldHandlerMap`\n// which copies the current field to a `SerializedMessageWriter`, with the\n// source and destination field numbers specified at runtime.\n//\n// As an optimization, `wire_types` constrains the set of wire types to handle.\n// This yields smaller and faster code.\n//\n// `Context...` types must contain exactly one occurrence of\n// `SerializedMessageWriter`. Use `ContextProjection()` to select the\n// `SerializedMessageWriter` if this is not the case.\ntemplate <WireTypeSet wire_types = AllWireTypes>\nconstexpr UnboundFieldCopierType<wire_types> UnboundFieldCopier(\n    int field_number) {\n  return UnboundFieldCopierType<wire_types>(field_number);\n}\n\n// Implementation details follow.\n\ntemplate <int field_number, WireTypeSet wire_types>\nclass FieldCopierType {\n public:\n  static constexpr int kFieldNumber = field_number;\n\n  constexpr FieldCopierType() = default;\n\n  FieldCopierType(const FieldCopierType& that) = default;\n  FieldCopierType& operator=(const FieldCopierType& that) = default;\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kVarint) ==\n                                 WireTypeSet::kVarint,\n                             int> = 0>\n  absl::Status HandleVarint(uint64_t repr, Context&... context) const {\n    return message_writer(context...).WriteUInt64(field_number, repr);\n  }\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kFixed32) ==\n                                 WireTypeSet::kFixed32,\n                             int> = 0>\n  absl::Status HandleFixed32(uint32_t repr, Context&... context) const {\n    return message_writer(context...).WriteFixed32(field_number, repr);\n  }\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kFixed64) ==\n                                 WireTypeSet::kFixed64,\n                             int> = 0>\n  absl::Status HandleFixed64(uint64_t repr, Context&... context) const {\n    return message_writer(context...).WriteFixed64(field_number, repr);\n  }\n\n  template <\n      typename... Context, WireTypeSet dependent_wire_types = wire_types,\n      std::enable_if_t<(dependent_wire_types & WireTypeSet::kLengthDelimited) ==\n                           WireTypeSet::kLengthDelimited,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromReader(ReaderSpan<> repr,\n                                               Context&... context) const {\n    return message_writer(context...)\n        .WriteString(field_number, std::move(repr));\n  }\n\n  template <\n      typename... Context, WireTypeSet dependent_wire_types = wire_types,\n      std::enable_if_t<(dependent_wire_types & WireTypeSet::kLengthDelimited) ==\n                           WireTypeSet::kLengthDelimited,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromCord(\n      CordIteratorSpan repr, ABSL_ATTRIBUTE_UNUSED std::string& scratch,\n      Context&... context) const {\n    return message_writer(context...)\n        .WriteString(field_number, std::move(repr));\n  }\n\n  template <\n      typename... Context, WireTypeSet dependent_wire_types = wire_types,\n      std::enable_if_t<(dependent_wire_types & WireTypeSet::kLengthDelimited) ==\n                           WireTypeSet::kLengthDelimited,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromString(absl::string_view repr,\n                                               Context&... context) const {\n    return message_writer(context...).WriteString(field_number, repr);\n  }\n\n  template <\n      typename... Context, WireTypeSet dependent_wire_types = wire_types,\n      std::enable_if_t<(dependent_wire_types & WireTypeSet::kStartGroup) ==\n                           WireTypeSet::kStartGroup,\n                       int> = 0>\n  absl::Status HandleStartGroup(Context&... context) const {\n    return message_writer(context...).OpenGroup(field_number);\n  }\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kEndGroup) ==\n                                 WireTypeSet::kEndGroup,\n                             int> = 0>\n  absl::Status HandleEndGroup(Context&... context) const {\n    return message_writer(context...).CloseGroup(field_number);\n  }\n\n private:\n  template <typename... Context>\n  static SerializedMessageWriter& message_writer(Context&... context) {\n    return std::get<SerializedMessageWriter&>(\n        std::tuple<Context&...>(context...));\n  }\n};\n\ntemplate <typename Accept, WireTypeSet wire_types>\nclass DynamicFieldCopierType {\n public:\n  static constexpr int kFieldNumber = kDynamicFieldNumber;\n\n  template <typename AcceptInitializer,\n            std::enable_if_t<std::is_convertible_v<AcceptInitializer&&, Accept>,\n                             int> = 0>\n  explicit constexpr DynamicFieldCopierType(AcceptInitializer&& accept)\n      : accept_(std::forward<AcceptInitializer>(accept)) {}\n\n  DynamicFieldCopierType() = default;\n\n  DynamicFieldCopierType(const DynamicFieldCopierType& that) = default;\n  DynamicFieldCopierType& operator=(const DynamicFieldCopierType& that) =\n      default;\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kVarint) ==\n                                 WireTypeSet::kVarint,\n                             int> = 0>\n  std::optional<int> AcceptVarint(int field_number) const {\n    return accept_(field_number);\n  }\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kVarint) ==\n                                 WireTypeSet::kVarint,\n                             int> = 0>\n  absl::Status DynamicHandleVarint(int field_number, uint64_t repr,\n                                   Context&... context) const {\n    return message_writer(context...).WriteUInt64(field_number, repr);\n  }\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kFixed32) ==\n                                 WireTypeSet::kFixed32,\n                             int> = 0>\n  std::optional<int> AcceptFixed32(int field_number) const {\n    return accept_(field_number);\n  }\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kFixed32) ==\n                                 WireTypeSet::kFixed32,\n                             int> = 0>\n  absl::Status DynamicHandleFixed32(int field_number, uint32_t repr,\n                                    Context&... context) const {\n    return message_writer(context...).WriteFixed32(field_number, repr);\n  }\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kFixed64) ==\n                                 WireTypeSet::kFixed64,\n                             int> = 0>\n  std::optional<int> AcceptFixed64(int field_number) const {\n    return accept_(field_number);\n  }\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kFixed64) ==\n                                 WireTypeSet::kFixed64,\n                             int> = 0>\n  absl::Status DynamicHandleFixed64(int field_number, uint64_t repr,\n                                    Context&... context) const {\n    return message_writer(context...).WriteFixed64(field_number, repr);\n  }\n\n  template <typename... Context>\n  std::optional<int> AcceptLengthDelimited(int field_number) const {\n    return accept_(field_number);\n  }\n\n  template <\n      typename... Context, WireTypeSet dependent_wire_types = wire_types,\n      std::enable_if_t<(dependent_wire_types & WireTypeSet::kLengthDelimited) ==\n                           WireTypeSet::kLengthDelimited,\n                       int> = 0>\n  absl::Status DynamicHandleLengthDelimitedFromReader(\n      int field_number, ReaderSpan<> repr, Context&... context) const {\n    return message_writer(context...)\n        .WriteString(field_number, std::move(repr));\n  }\n\n  template <\n      typename... Context, WireTypeSet dependent_wire_types = wire_types,\n      std::enable_if_t<(dependent_wire_types & WireTypeSet::kLengthDelimited) ==\n                           WireTypeSet::kLengthDelimited,\n                       int> = 0>\n  absl::Status DynamicHandleLengthDelimitedFromCord(\n      int field_number, CordIteratorSpan repr,\n      ABSL_ATTRIBUTE_UNUSED std::string& scratch, Context&... context) const {\n    return message_writer(context...)\n        .WriteString(field_number, std::move(repr));\n  }\n\n  template <\n      typename... Context, WireTypeSet dependent_wire_types = wire_types,\n      std::enable_if_t<(dependent_wire_types & WireTypeSet::kLengthDelimited) ==\n                           WireTypeSet::kLengthDelimited,\n                       int> = 0>\n  absl::Status DynamicHandleLengthDelimitedFromString(\n      int field_number, absl::string_view repr, Context&... context) const {\n    return message_writer(context...).WriteString(field_number, repr);\n  }\n\n  template <\n      typename... Context, WireTypeSet dependent_wire_types = wire_types,\n      std::enable_if_t<(dependent_wire_types & WireTypeSet::kStartGroup) ==\n                           WireTypeSet::kStartGroup,\n                       int> = 0>\n  std::optional<int> AcceptStartGroup(int field_number) const {\n    return accept_(field_number);\n  }\n\n  template <\n      typename... Context, WireTypeSet dependent_wire_types = wire_types,\n      std::enable_if_t<(dependent_wire_types & WireTypeSet::kStartGroup) ==\n                           WireTypeSet::kStartGroup,\n                       int> = 0>\n  absl::Status DynamicHandleStartGroup(int field_number,\n                                       Context&... context) const {\n    return message_writer(context...).OpenGroup(field_number);\n  }\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kEndGroup) ==\n                                 WireTypeSet::kEndGroup,\n                             int> = 0>\n  std::optional<int> AcceptEndGroup(int field_number) const {\n    return accept_(field_number);\n  }\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kEndGroup) ==\n                                 WireTypeSet::kEndGroup,\n                             int> = 0>\n  absl::Status DynamicHandleEndGroup(int field_number,\n                                     Context&... context) const {\n    return message_writer(context...).CloseGroup(field_number);\n  }\n\n private:\n  template <typename... Context>\n  static SerializedMessageWriter& message_writer(Context&... context) {\n    return std::get<SerializedMessageWriter&>(\n        std::tuple<Context&...>(context...));\n  }\n\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS Accept accept_;\n};\n\nstruct AcceptAnyField {\n  std::optional<int> operator()(int field_number) const { return field_number; }\n};\n\nconstexpr AnyFieldCopierType AnyFieldCopier() {\n  return AnyFieldCopierType(AcceptAnyField());\n}\n\ntemplate <WireTypeSet wire_types>\nclass UnboundFieldCopierType {\n public:\n  static constexpr int kFieldNumber = kUnboundFieldNumber;\n\n  constexpr UnboundFieldCopierType(int field_number)\n      : field_number_(field_number) {}\n\n  UnboundFieldCopierType(const UnboundFieldCopierType& that) = default;\n  UnboundFieldCopierType& operator=(const UnboundFieldCopierType& that) =\n      default;\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kVarint) ==\n                                 WireTypeSet::kVarint,\n                             int> = 0>\n  absl::Status HandleVarint(uint64_t repr, Context&... context) const {\n    return message_writer(context...).WriteUInt64(field_number_, repr);\n  }\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kFixed32) ==\n                                 WireTypeSet::kFixed32,\n                             int> = 0>\n  absl::Status HandleFixed32(uint32_t repr, Context&... context) const {\n    return message_writer(context...).WriteFixed32(field_number_, repr);\n  }\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kFixed64) ==\n                                 WireTypeSet::kFixed64,\n                             int> = 0>\n  absl::Status HandleFixed64(uint64_t repr, Context&... context) const {\n    return message_writer(context...).WriteFixed64(field_number_, repr);\n  }\n\n  template <\n      typename... Context, WireTypeSet dependent_wire_types = wire_types,\n      std::enable_if_t<(dependent_wire_types & WireTypeSet::kLengthDelimited) ==\n                           WireTypeSet::kLengthDelimited,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromReader(ReaderSpan<> repr,\n                                               Context&... context) const {\n    return message_writer(context...)\n        .WriteString(field_number_, std::move(repr));\n  }\n\n  template <\n      typename... Context, WireTypeSet dependent_wire_types = wire_types,\n      std::enable_if_t<(dependent_wire_types & WireTypeSet::kLengthDelimited) ==\n                           WireTypeSet::kLengthDelimited,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromCord(\n      CordIteratorSpan repr, ABSL_ATTRIBUTE_UNUSED std::string& scratch,\n      Context&... context) const {\n    return message_writer(context...)\n        .WriteString(field_number_, std::move(repr));\n  }\n\n  template <\n      typename... Context, WireTypeSet dependent_wire_types = wire_types,\n      std::enable_if_t<(dependent_wire_types & WireTypeSet::kLengthDelimited) ==\n                           WireTypeSet::kLengthDelimited,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromString(absl::string_view repr,\n                                               Context&... context) const {\n    return message_writer(context...).WriteString(field_number_, repr);\n  }\n\n  template <\n      typename... Context, WireTypeSet dependent_wire_types = wire_types,\n      std::enable_if_t<(dependent_wire_types & WireTypeSet::kStartGroup) ==\n                           WireTypeSet::kStartGroup,\n                       int> = 0>\n  absl::Status HandleStartGroup(Context&... context) const {\n    return message_writer(context...).OpenGroup(field_number_);\n  }\n\n  template <typename... Context, WireTypeSet dependent_wire_types = wire_types,\n            std::enable_if_t<(dependent_wire_types & WireTypeSet::kEndGroup) ==\n                                 WireTypeSet::kEndGroup,\n                             int> = 0>\n  absl::Status HandleEndGroup(Context&... context) const {\n    return message_writer(context...).CloseGroup(field_number_);\n  }\n\n private:\n  template <typename... Context>\n  static SerializedMessageWriter& message_writer(Context&... context) {\n    return std::get<SerializedMessageWriter&>(\n        std::tuple<Context&...>(context...));\n  }\n\n  int field_number_;\n};\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_FIELD_COPIER_H_\n"
  },
  {
    "path": "riegeli/messages/field_handler_map.h",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_FIELD_HANDLER_MAP_H_\n#define RIEGELI_MESSAGES_FIELD_HANDLER_MAP_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/container/flat_hash_map.h\"\n#include \"absl/functional/any_invocable.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/base/hybrid_direct_map.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/messages/serialized_message_reader.h\"\n#include \"riegeli/messages/serialized_message_reader_internal.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\ntemplate <typename... Context>\nclass FieldHandlerMapBuilder;\n\n// A map of field handlers for `SerializedMessageReader` registered at runtime.\n//\n// A `FieldHandlerMap` is itself a dynamic field handler, used with\n// `SerializedMessageReader` with compatible `Context...`.\n//\n// `FieldHandlerMap` is created from `FieldHandlerMap::Builder`, with which\n// field handlers are registered.\n//\n// Registered field handlers must be unbound, not yet associated with a field\n// number. The field number is specified during registration.\n//\n// Field handlers for length-delimited fields must be directly applicable to a\n// string source. This makes `FieldHandlerMap` itself directly applicable to a\n// string or `Cord` source.\n//\n// `DynamicFieldHandler` is similar to a `FieldHandlerMap` with a single\n// registered field handler, but more efficient and without type erasure.\ntemplate <typename... Context>\nclass FieldHandlerMap {\n private:\n  template <typename... Value>\n  using FieldAction =\n      absl::AnyInvocable<absl::Status(Value..., Context&...) const>;\n\n  struct LengthDelimitedActions;\n\n public:\n  // Prepares a `FieldHandlerMap`.\n  using Builder = FieldHandlerMapBuilder<Context...>;\n\n  // Creates an empty `FieldHandlerMap`. Designed for `Reset()`.\n  FieldHandlerMap() = default;\n\n  // Builds a `FieldHandlerMap`.\n  explicit FieldHandlerMap(Builder&& builder);\n\n  FieldHandlerMap(FieldHandlerMap&& that) = default;\n  FieldHandlerMap& operator=(FieldHandlerMap&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `FieldHandlerMap`.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Builder&& builder);\n\n  // Implement the field handler protocol.\n\n  static constexpr int kFieldNumber = kDynamicFieldNumber;\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  const FieldAction<uint64_t>* absl_nullable AcceptVarint(\n      int field_number) const {\n    return varint_handlers_.FindOrNull(field_number);\n  }\n\n  absl::Status DynamicHandleVarint(const FieldAction<uint64_t>& handler,\n                                   uint64_t repr, Context&... context) const {\n    return handler(repr, context...);\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  const FieldAction<uint32_t>* absl_nullable AcceptFixed32(\n      int field_number) const {\n    return fixed32_handlers_.FindOrNull(field_number);\n  }\n\n  absl::Status DynamicHandleFixed32(const FieldAction<uint32_t>& handler,\n                                    uint32_t repr, Context&... context) const {\n    return handler(repr, context...);\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  const FieldAction<uint64_t>* absl_nullable AcceptFixed64(\n      int field_number) const {\n    return fixed64_handlers_.FindOrNull(field_number);\n  }\n\n  absl::Status DynamicHandleFixed64(const FieldAction<uint64_t>& handler,\n                                    uint64_t repr, Context&... context) const {\n    return handler(repr, context...);\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  const LengthDelimitedActions* absl_nullable AcceptLengthDelimited(\n      int field_number) const {\n    return length_delimited_handlers_.FindOrNull(field_number);\n  }\n\n  absl::Status DynamicHandleLengthDelimitedFromReader(\n      const LengthDelimitedActions& handler, ReaderSpan<> repr,\n      Context&... context) const {\n    return handler.action_from_reader(std::move(repr), context...);\n  }\n\n  absl::Status DynamicHandleLengthDelimitedFromCord(\n      const LengthDelimitedActions& handler, CordIteratorSpan repr,\n      std::string& scratch, Context&... context) const {\n    return handler.action_from_cord(std::move(repr), scratch, context...);\n  }\n\n  absl::Status DynamicHandleLengthDelimitedFromString(\n      const LengthDelimitedActions& handler, absl::string_view repr,\n      Context&... context) const {\n    return handler.action_from_string(repr, context...);\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  const FieldAction<>* absl_nullable AcceptStartGroup(int field_number) const {\n    return start_group_handlers_.FindOrNull(field_number);\n  }\n\n  absl::Status DynamicHandleStartGroup(const FieldAction<>& handler,\n                                       Context&... context) const {\n    return handler(context...);\n  }\n\n  ABSL_ATTRIBUTE_ALWAYS_INLINE\n  const FieldAction<>* absl_nullable AcceptEndGroup(int field_number) const {\n    return end_group_handlers_.FindOrNull(field_number);\n  }\n\n  absl::Status DynamicHandleEndGroup(const FieldAction<>& handler,\n                                     Context&... context) const {\n    return handler(context...);\n  }\n\n private:\n  // For `FieldAction` and `LengthDelimitedActions`.\n  friend class FieldHandlerMapBuilder<Context...>;\n\n  template <typename Value>\n  using FieldMap = HybridDirectMap<int, Value, HybridDirectTraits<int, 1>>;\n\n  FieldMap<FieldAction<uint64_t>> varint_handlers_;\n  FieldMap<FieldAction<uint32_t>> fixed32_handlers_;\n  FieldMap<FieldAction<uint64_t>> fixed64_handlers_;\n  FieldMap<LengthDelimitedActions> length_delimited_handlers_;\n  FieldMap<FieldAction<>> start_group_handlers_;\n  FieldMap<FieldAction<>> end_group_handlers_;\n};\n\ntemplate <typename... Context>\nexplicit FieldHandlerMap(FieldHandlerMapBuilder<Context...>&& builder)\n    -> FieldHandlerMap<Context...>;\n\ntemplate <typename... Context>\nclass FieldHandlerMapBuilder {\n public:\n  // Creates an empty `Builder`.\n  FieldHandlerMapBuilder() = default;\n\n  FieldHandlerMapBuilder(FieldHandlerMapBuilder&& that) = default;\n  FieldHandlerMapBuilder& operator=(FieldHandlerMapBuilder&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `Builder`.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n\n  // Registers a field handler for a field with the given `field_number`.\n  // The field handler must be unbound, not yet associated with a field number.\n  // A field handler for a length-delimited field must be directly applicable\n  // to a string source.\n  //\n  // Returns `true` if successful, or `false` if `field_number` was already\n  // registered.\n  template <typename FieldHandler,\n            std::enable_if_t<IsUnboundFieldHandlerFromString<\n                                 TargetT<FieldHandler>, Context...>::value,\n                             int> = 0>\n  bool RegisterField(int field_number, const FieldHandler& field_handler);\n\n private:\n  friend class FieldHandlerMap<Context...>;  // For member variables.\n\n  template <typename... Value>\n  using FieldAction =\n      typename FieldHandlerMap<Context...>::template FieldAction<Value...>;\n  using LengthDelimitedActions =\n      typename FieldHandlerMap<Context...>::LengthDelimitedActions;\n\n  absl::flat_hash_map<int, FieldAction<uint64_t>> varint_handlers_;\n  absl::flat_hash_map<int, FieldAction<uint32_t>> fixed32_handlers_;\n  absl::flat_hash_map<int, FieldAction<uint64_t>> fixed64_handlers_;\n  absl::flat_hash_map<int, LengthDelimitedActions> length_delimited_handlers_;\n  absl::flat_hash_map<int, FieldAction<>> start_group_handlers_;\n  absl::flat_hash_map<int, FieldAction<>> end_group_handlers_;\n};\n\n// Implementation details follow.\n\ntemplate <typename... Context>\nvoid FieldHandlerMapBuilder<Context...>::Reset() {\n  varint_handlers_.clear();\n  fixed32_handlers_.clear();\n  fixed64_handlers_.clear();\n  length_delimited_handlers_.clear();\n  start_group_handlers_.clear();\n  end_group_handlers_.clear();\n}\n\ntemplate <typename... Context>\ntemplate <typename FieldHandler,\n          std::enable_if_t<IsUnboundFieldHandlerFromString<\n                               TargetT<FieldHandler>, Context...>::value,\n                           int>>\nbool FieldHandlerMapBuilder<Context...>::RegisterField(\n    int field_number, const FieldHandler& field_handler) {\n  bool all_registered = true;\n  if constexpr (serialized_message_reader_internal::\n                    IsStaticFieldHandlerForVarint<TargetT<FieldHandler>,\n                                                  Context...>::value) {\n    if (ABSL_PREDICT_FALSE(\n            !varint_handlers_\n                 .try_emplace(\n                     field_number,\n                     [field_handler = TargetT<FieldHandler>(field_handler)](\n                         uint64_t repr, Context&... context) {\n                       return field_handler.HandleVarint(repr, context...);\n                     })\n                 .second)) {\n      all_registered = false;\n    }\n  }\n  if constexpr (serialized_message_reader_internal::\n                    IsStaticFieldHandlerForFixed32<TargetT<FieldHandler>,\n                                                   Context...>::value) {\n    if (ABSL_PREDICT_FALSE(\n            !fixed32_handlers_\n                 .try_emplace(\n                     field_number,\n                     [field_handler = TargetT<FieldHandler>(field_handler)](\n                         uint32_t repr, Context&... context) {\n                       return field_handler.HandleFixed32(repr, context...);\n                     })\n                 .second)) {\n      all_registered = false;\n    }\n  }\n  if constexpr (serialized_message_reader_internal::\n                    IsStaticFieldHandlerForFixed64<TargetT<FieldHandler>,\n                                                   Context...>::value) {\n    if (ABSL_PREDICT_FALSE(\n            !fixed64_handlers_\n                 .try_emplace(\n                     field_number,\n                     [field_handler = TargetT<FieldHandler>(field_handler)](\n                         uint64_t repr, Context&... context) {\n                       return field_handler.HandleFixed64(repr, context...);\n                     })\n                 .second)) {\n      all_registered = false;\n    }\n  }\n  if constexpr (serialized_message_reader_internal::\n                    IsStaticFieldHandlerForLengthDelimitedFromString<\n                        TargetT<FieldHandler>, Context...>::value) {\n    if (ABSL_PREDICT_FALSE(\n            !length_delimited_handlers_\n                 .try_emplace(\n                     field_number,\n                     [field_handler = TargetT<FieldHandler>(field_handler)](\n                         ReaderSpan<> repr, Context&... context) {\n                       if constexpr (\n                           serialized_message_reader_internal::\n                               IsStaticFieldHandlerForLengthDelimitedFromReader<\n                                   TargetT<FieldHandler>, Context...>::value) {\n                         return field_handler.HandleLengthDelimitedFromReader(\n                             std::move(repr), context...);\n                       } else {\n                         absl::string_view value;\n                         if (ABSL_PREDICT_FALSE(!repr.reader().Read(\n                                 IntCast<size_t>(repr.length()), value))) {\n                           return serialized_message_reader_internal::\n                               ReadLengthDelimitedValueError(repr.reader());\n                         }\n                         return field_handler.HandleLengthDelimitedFromString(\n                             value, context...);\n                       }\n                     },\n                     [field_handler = TargetT<FieldHandler>(field_handler)](\n                         CordIteratorSpan repr, std::string& scratch,\n                         Context&... context) {\n                       if constexpr (\n                           serialized_message_reader_internal::\n                               IsStaticFieldHandlerForLengthDelimitedFromCord<\n                                   TargetT<FieldHandler>, Context...>::value) {\n                         return field_handler.HandleLengthDelimitedFromCord(\n                             std::move(repr), scratch, context...);\n                       } else {\n                         return field_handler.HandleLengthDelimitedFromString(\n                             std::move(repr).ToStringView(scratch), context...);\n                       }\n                     },\n                     [field_handler = TargetT<FieldHandler>(field_handler)](\n                         absl::string_view repr, Context&... context) {\n                       return field_handler.HandleLengthDelimitedFromString(\n                           repr, context...);\n                     })\n                 .second)) {\n      all_registered = false;\n    }\n  }\n  if constexpr (serialized_message_reader_internal::\n                    IsStaticFieldHandlerForStartGroup<TargetT<FieldHandler>,\n                                                      Context...>::value) {\n    if (ABSL_PREDICT_FALSE(\n            !start_group_handlers_\n                 .try_emplace(field_number,\n                              [field_handler = TargetT<FieldHandler>(\n                                   field_handler)](Context&... context) {\n                                return field_handler.HandleStartGroup(\n                                    context...);\n                              })\n                 .second)) {\n      all_registered = false;\n    }\n  }\n  if constexpr (serialized_message_reader_internal::\n                    IsStaticFieldHandlerForEndGroup<TargetT<FieldHandler>,\n                                                    Context...>::value) {\n    if (ABSL_PREDICT_FALSE(\n            !end_group_handlers_\n                 .try_emplace(field_number,\n                              [field_handler = TargetT<FieldHandler>(\n                                   field_handler)](Context&... context) {\n                                return field_handler.HandleEndGroup(context...);\n                              })\n                 .second)) {\n      all_registered = false;\n    }\n  }\n  return all_registered;\n}\n\ntemplate <typename... Context>\nstruct FieldHandlerMap<Context...>::LengthDelimitedActions {\n#if !__cpp_aggregate_paren_init\n  LengthDelimitedActions() = default;\n\n  template <\n      typename ActionFromReader, typename ActionFromCord,\n      typename ActionFromString,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::is_convertible<ActionFromReader&&,\n                                  FieldAction<ReaderSpan<>>>,\n              std::is_convertible<ActionFromCord&&,\n                                  FieldAction<CordIteratorSpan, std::string&>>,\n              std::is_convertible<ActionFromString&&,\n                                  FieldAction<absl::string_view>>>,\n          int> = 0>\n  explicit LengthDelimitedActions(ActionFromReader&& action_from_reader,\n                                  ActionFromCord&& action_from_cord,\n                                  ActionFromString&& action_from_string)\n      : action_from_reader(std::forward<ActionFromReader>(action_from_reader)),\n        action_from_cord(std::forward<ActionFromCord>(action_from_cord)),\n        action_from_string(std::forward<ActionFromString>(action_from_string)) {\n  }\n\n  LengthDelimitedActions(LengthDelimitedActions&& that) = default;\n  LengthDelimitedActions& operator=(LengthDelimitedActions&& that) = default;\n#endif\n\n  FieldAction<ReaderSpan<>> action_from_reader;\n  FieldAction<CordIteratorSpan, std::string&> action_from_cord;\n  FieldAction<absl::string_view> action_from_string;\n};\n\ntemplate <typename... Context>\nFieldHandlerMap<Context...>::FieldHandlerMap(Builder&& builder)\n    : varint_handlers_(std::move(builder.varint_handlers_)),\n      fixed32_handlers_(std::move(builder.fixed32_handlers_)),\n      fixed64_handlers_(std::move(builder.fixed64_handlers_)),\n      length_delimited_handlers_(std::move(builder.length_delimited_handlers_)),\n      start_group_handlers_(std::move(builder.start_group_handlers_)),\n      end_group_handlers_(std::move(builder.end_group_handlers_)) {}\n\ntemplate <typename... Context>\nvoid FieldHandlerMap<Context...>::Reset() {\n  varint_handlers_.Reset();\n  fixed32_handlers_.Reset();\n  fixed64_handlers_.Reset();\n  length_delimited_handlers_.Reset();\n  start_group_handlers_.Reset();\n  end_group_handlers_.Reset();\n}\n\ntemplate <typename... Context>\nvoid FieldHandlerMap<Context...>::Reset(Builder&& builder) {\n  varint_handlers_.Reset(std::move(builder.varint_handlers_));\n  fixed32_handlers_.Reset(std::move(builder.fixed32_handlers_));\n  fixed64_handlers_.Reset(std::move(builder.fixed64_handlers_));\n  length_delimited_handlers_.Reset(\n      std::move(builder.length_delimited_handlers_));\n  start_group_handlers_.Reset(std::move(builder.start_group_handlers_));\n  end_group_handlers_.Reset(std::move(builder.end_group_handlers_));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_FIELD_HANDLER_MAP_H_\n"
  },
  {
    "path": "riegeli/messages/field_handlers.cc",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/messages/field_handlers.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <utility>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/bytes/reader.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli::field_handlers_internal {\n\nabsl::Status AnnotateByReader(absl::Status status, Reader& reader) {\n  if (absl::IsCancelled(status)) {\n    return status;\n  } else {\n    return reader.StatusOrAnnotate(std::move(status));\n  }\n}\n\ntemplate <>\nabsl::Status VarintOverflowError<int32_t, field_handlers::VarintKind::kPlain>(\n    uint64_t repr) {\n  return absl::InvalidArgumentError(\n      absl::StrCat(\"int32 field overflow: \", repr));\n}\n\ntemplate <>\nabsl::Status VarintOverflowError<uint32_t, field_handlers::VarintKind::kPlain>(\n    uint64_t repr) {\n  return absl::InvalidArgumentError(\n      absl::StrCat(\"uint32 field overflow: \", repr));\n}\n\ntemplate <>\nabsl::Status VarintOverflowError<int32_t, field_handlers::VarintKind::kSigned>(\n    uint64_t repr) {\n  return absl::InvalidArgumentError(\n      absl::StrCat(\"sint32 field overflow: \", repr));\n}\n\ntemplate <>\nabsl::Status VarintOverflowError<bool, field_handlers::VarintKind::kPlain>(\n    uint64_t repr) {\n  return absl::InvalidArgumentError(\n      absl::StrCat(\"bool field overflow: \", repr));\n}\n\nabsl::Status EnumOverflowError(uint64_t repr) {\n  return absl::InvalidArgumentError(\n      absl::StrCat(\"enum field overflow: \", repr));\n}\n\ntemplate <>\nabsl::Status VarintOverflowError<int32_t, field_handlers::VarintKind::kPlain>(\n    Reader& src, uint64_t repr) {\n  return src.StatusOrAnnotate(\n      VarintOverflowError<int32_t, field_handlers::VarintKind::kPlain>(repr));\n}\n\ntemplate <>\nabsl::Status VarintOverflowError<uint32_t, field_handlers::VarintKind::kPlain>(\n    Reader& src, uint64_t repr) {\n  return src.StatusOrAnnotate(\n      VarintOverflowError<uint32_t, field_handlers::VarintKind::kPlain>(repr));\n}\n\ntemplate <>\nabsl::Status VarintOverflowError<int32_t, field_handlers::VarintKind::kSigned>(\n    Reader& src, uint64_t repr) {\n  return src.StatusOrAnnotate(\n      VarintOverflowError<int32_t, field_handlers::VarintKind::kSigned>(repr));\n}\n\nabsl::Status EnumOverflowError(Reader& src, uint64_t repr) {\n  return src.StatusOrAnnotate(EnumOverflowError(repr));\n}\n\ntemplate <>\nabsl::Status VarintOverflowError<bool, field_handlers::VarintKind::kPlain>(\n    Reader& src, uint64_t repr) {\n  return src.StatusOrAnnotate(\n      VarintOverflowError<bool, field_handlers::VarintKind::kPlain>(repr));\n}\n\nabsl::Status ReadPackedVarintError() {\n  return absl::InvalidArgumentError(\n      \"Could not read a varint element of a packed repeated field\");\n}\n\nabsl::Status ReadPackedVarintError(Reader& src) {\n  return src.StatusOrAnnotate(ReadPackedVarintError());\n}\n\ntemplate <>\nabsl::Status ReadPackedFixedError<sizeof(uint32_t)>() {\n  return absl::InvalidArgumentError(\n      \"Could not read a fixed32 element of a packed repeated field\");\n}\n\ntemplate <>\nabsl::Status ReadPackedFixedError<sizeof(uint64_t)>() {\n  return absl::InvalidArgumentError(\n      \"Could not read a fixed64 element of a packed repeated field\");\n}\n\ntemplate <>\nabsl::Status ReadPackedFixedError<sizeof(uint32_t)>(Reader& src) {\n  return src.StatusOrAnnotate(ReadPackedFixedError<sizeof(uint32_t)>());\n}\n\ntemplate <>\nabsl::Status ReadPackedFixedError<sizeof(uint64_t)>(Reader& src) {\n  return src.StatusOrAnnotate(ReadPackedFixedError<sizeof(uint64_t)>());\n}\n\n}  // namespace riegeli::field_handlers_internal\n"
  },
  {
    "path": "riegeli/messages/field_handlers.h",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_FIELD_HANDLERS_H_\n#define RIEGELI_MESSAGES_FIELD_HANDLERS_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/casts.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/endian/endian_reading.h\"\n#include \"riegeli/messages/serialized_message_reader.h\"\n#include \"riegeli/messages/serialized_message_reader_internal.h\"\n#include \"riegeli/varint/varint_reading.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nnamespace field_handlers {\n\n// The kind of a varint field, assuming that its C++ type is known.\nenum class VarintKind {\n  kPlain,   // `int32`, `int64`, `uint32`, `uint64`, `bool`\n  kSigned,  // `sint32`, `sint64`\n  kEnum,    // `enum`\n};\n\n// Class types of common field handlers. Usually it is enough to use `auto`\n// instead of spelling these types.\ntemplate <typename Value, VarintKind kind, int field_number, typename Action>\nclass OnOptionalVarintType;\ntemplate <typename Value, VarintKind kind, int field_number, typename Action>\nclass OnRepeatedVarintType;\ntemplate <typename Value, int field_number, typename Action>\nclass OnOptionalFixedType;\ntemplate <typename Value, int field_number, typename Action>\nclass OnRepeatedFixedType;\ntemplate <typename BaseFieldHandler, typename PackedFieldHandler>\nclass OnPackedType;\ntemplate <int field_number, typename Action>\nclass OnLengthDelimitedType;\ntemplate <int field_number, typename Action>\nclass BeforeGroupType;\ntemplate <int field_number, typename Action>\nclass AfterGroupType;\n\n// Common field handlers for `SerializedMessageReader`, `DynamicFieldHandler`,\n// and `FieldHandlerMap`.\n//\n// For a `MessageType` with generated code, the field number of a field named\n// `foo_bar` can be obtained as `MessageType::kFooBarFieldNumber`.\n//\n// For numeric fields, `Optional` and `Repeated` variants of field handlers are\n// provided. An `Optional` variant is intended for singular fields, and it\n// will be called also for elements of a non-packed repeated field (which have\n// the same wire representation). A `Repeated` variant is intended for repeated\n// fields (packed or not), and it will be called also for singular fields\n// (which have the same wire representation as a non-packed repeated field).\n//\n// For varint fields, in contrast to native proto parsing, 64-bit values which\n// overflow the provided 32-bit type are reported as errors instead of being\n// silently truncated.\n//\n// Two kinds of field handlers are provided by these functions:\n//\n//  * Static, which handle a single field number known at compile time.\n//    The `field_number` template parameter must be specified and positive.\n//\n//  * Unbound, which are meant to be registered in a `FieldHandlerMap` with a\n//    field number specified during registration. The `field_number` template\n//    parameter must not be specified.\n\n// Field handler of a singular `int32` field. The value is provided as\n// `int32_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnOptionalVarintType<int32_t, VarintKind::kPlain, field_number,\n                               std::decay_t<Action>>\nOnOptionalInt32(Action&& action) {\n  return OnOptionalVarintType<int32_t, VarintKind::kPlain, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated `int32` field. The value is\n// provided as `int32_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnRepeatedVarintType<int32_t, VarintKind::kPlain, field_number,\n                               std::decay_t<Action>>\nOnRepeatedInt32(Action&& action) {\n  return OnRepeatedVarintType<int32_t, VarintKind::kPlain, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of a singular `int64` field. The value is provided as\n// `int64_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnOptionalVarintType<int64_t, VarintKind::kPlain, field_number,\n                               std::decay_t<Action>>\nOnOptionalInt64(Action&& action) {\n  return OnOptionalVarintType<int64_t, VarintKind::kPlain, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated `int64` field. The value is\n// provided as `int64_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnRepeatedVarintType<int64_t, VarintKind::kPlain, field_number,\n                               std::decay_t<Action>>\nOnRepeatedInt64(Action&& action) {\n  return OnRepeatedVarintType<int64_t, VarintKind::kPlain, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of a singular `uint32` field. The value is provided as\n// `uint32_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnOptionalVarintType<uint32_t, VarintKind::kPlain, field_number,\n                               std::decay_t<Action>>\nOnOptionalUInt32(Action&& action) {\n  return OnOptionalVarintType<uint32_t, VarintKind::kPlain, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated `uint32` field. The value is\n// provided as `uint32_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnRepeatedVarintType<uint32_t, VarintKind::kPlain, field_number,\n                               std::decay_t<Action>>\nOnRepeatedUInt32(Action&& action) {\n  return OnRepeatedVarintType<uint32_t, VarintKind::kPlain, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of a singular `uint64` field. The value is provided as\n// `uint64_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnOptionalVarintType<uint64_t, VarintKind::kPlain, field_number,\n                               std::decay_t<Action>>\nOnOptionalUInt64(Action&& action) {\n  return OnOptionalVarintType<uint64_t, VarintKind::kPlain, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated `uint64` field. The value is\n// provided as `uint64_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnRepeatedVarintType<uint64_t, VarintKind::kPlain, field_number,\n                               std::decay_t<Action>>\nOnRepeatedUInt64(Action&& action) {\n  return OnRepeatedVarintType<uint64_t, VarintKind::kPlain, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of a singular `sint32` field. The value is provided as\n// `int32_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnOptionalVarintType<int32_t, VarintKind::kSigned, field_number,\n                               std::decay_t<Action>>\nOnOptionalSInt32(Action&& action) {\n  return OnOptionalVarintType<int32_t, VarintKind::kSigned, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated `sint32` field. The value is\n// provided as `int32_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnRepeatedVarintType<int32_t, VarintKind::kSigned, field_number,\n                               std::decay_t<Action>>\nOnRepeatedSInt32(Action&& action) {\n  return OnRepeatedVarintType<int32_t, VarintKind::kSigned, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of a singular `sint64` field. The value is provided as\n// `int64_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnOptionalVarintType<int64_t, VarintKind::kSigned, field_number,\n                               std::decay_t<Action>>\nOnOptionalSInt64(Action&& action) {\n  return OnOptionalVarintType<int64_t, VarintKind::kSigned, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated `sint64` field. The value is\n// provided as `int64_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnRepeatedVarintType<int64_t, VarintKind::kSigned, field_number,\n                               std::decay_t<Action>>\nOnRepeatedSInt64(Action&& action) {\n  return OnRepeatedVarintType<int64_t, VarintKind::kSigned, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of a singular `bool` field. The value is provided as `bool`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnOptionalVarintType<bool, VarintKind::kPlain, field_number,\n                               std::decay_t<Action>>\nOnOptionalBool(Action&& action) {\n  return OnOptionalVarintType<bool, VarintKind::kPlain, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated `bool` field. The value is provided\n// as `bool`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnRepeatedVarintType<bool, VarintKind::kPlain, field_number,\n                               std::decay_t<Action>>\nOnRepeatedBool(Action&& action) {\n  return OnRepeatedVarintType<bool, VarintKind::kPlain, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of a singular `fixed32` field. The value is provided as\n// `uint32_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnOptionalFixedType<uint32_t, field_number, std::decay_t<Action>>\nOnOptionalFixed32(Action&& action) {\n  return OnOptionalFixedType<uint32_t, field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated `fixed32` field. The value is\n// provided as `uint32_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnRepeatedFixedType<uint32_t, field_number, std::decay_t<Action>>\nOnRepeatedFixed32(Action&& action) {\n  return OnRepeatedFixedType<uint32_t, field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of a singular `fixed64` field. The value is provided as\n// `uint64_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnOptionalFixedType<uint64_t, field_number, std::decay_t<Action>>\nOnOptionalFixed64(Action&& action) {\n  return OnOptionalFixedType<uint64_t, field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated `fixed64` field. The value is\n// provided as `uint64_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnRepeatedFixedType<uint64_t, field_number, std::decay_t<Action>>\nOnRepeatedFixed64(Action&& action) {\n  return OnRepeatedFixedType<uint64_t, field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of a singular `sfixed32` field. The value is provided as\n// `int32_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnOptionalFixedType<int32_t, field_number, std::decay_t<Action>>\nOnOptionalSFixed32(Action&& action) {\n  return OnOptionalFixedType<int32_t, field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated `sfixed32` field. The value is\n// provided as `int32_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnRepeatedFixedType<int32_t, field_number, std::decay_t<Action>>\nOnRepeatedSFixed32(Action&& action) {\n  return OnRepeatedFixedType<int32_t, field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of a singular `sfixed64` field. The value is provided as\n// `int64_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnOptionalFixedType<int64_t, field_number, std::decay_t<Action>>\nOnOptionalSFixed64(Action&& action) {\n  return OnOptionalFixedType<int64_t, field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated `sfixed64` field. The value is\n// provided as `int64_t`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnRepeatedFixedType<int64_t, field_number, std::decay_t<Action>>\nOnRepeatedSFixed64(Action&& action) {\n  return OnRepeatedFixedType<int64_t, field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of a singular `float` field. The value is provided as `float`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnOptionalFixedType<float, field_number, std::decay_t<Action>>\nOnOptionalFloat(Action&& action) {\n  return OnOptionalFixedType<float, field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated `float` field. The value is\n// provided as `float`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnRepeatedFixedType<float, field_number, std::decay_t<Action>>\nOnRepeatedFloat(Action&& action) {\n  return OnRepeatedFixedType<float, field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of a singular `double` field. The value is provided as\n// `double`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnOptionalFixedType<double, field_number, std::decay_t<Action>>\nOnOptionalDouble(Action&& action) {\n  return OnOptionalFixedType<double, field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated `double` field. The value is\n// provided as `double`.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnRepeatedFixedType<double, field_number, std::decay_t<Action>>\nOnRepeatedDouble(Action&& action) {\n  return OnRepeatedFixedType<double, field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of a singular enum field. The value is provided as an enum type\n// (C++ or proto enum) or an integral type.\ntemplate <\n    typename EnumType, int field_number = kUnboundFieldNumber, typename Action,\n    std::enable_if_t<\n        std::disjunction_v<std::is_enum<EnumType>, std::is_integral<EnumType>>,\n        int> = 0>\nconstexpr OnOptionalVarintType<EnumType, VarintKind::kEnum, field_number,\n                               std::decay_t<Action>>\nOnOptionalEnum(Action&& action) {\n  return OnOptionalVarintType<EnumType, VarintKind::kEnum, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler of an element of a repeated enum field. The value is provided\n// as an enum type (C++ or proto enum) or an integral type.\ntemplate <\n    typename EnumType, int field_number = kUnboundFieldNumber, typename Action,\n    std::enable_if_t<\n        std::disjunction_v<std::is_enum<EnumType>, std::is_integral<EnumType>>,\n        int> = 0>\nconstexpr OnRepeatedVarintType<EnumType, VarintKind::kEnum, field_number,\n                               std::decay_t<Action>>\nOnRepeatedEnum(Action&& action) {\n  return OnRepeatedVarintType<EnumType, VarintKind::kEnum, field_number,\n                              std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler with a dedicated implementation for a packed repeated field.\n//\n// Uses `BaseFieldHandler` for scalar wire types, and `PackedFieldHandler` for\n// length-delimited wire type.\n//\n// Regular `OnRepeated...()` field handlers already support packed repeated\n// fields, but they call the base action repeatedly for each element, which\n// can be less efficient than a dedicated implementation.\ntemplate <typename BaseFieldHandler, typename PackedFieldHandler>\nconstexpr OnPackedType<std::decay_t<BaseFieldHandler>,\n                       std::decay_t<PackedFieldHandler>>\nOnPacked(BaseFieldHandler&& base_field_handler,\n         PackedFieldHandler&& packed_action) {\n  return OnPackedType<std::decay_t<BaseFieldHandler>,\n                      std::decay_t<PackedFieldHandler>>(\n      std::forward<BaseFieldHandler>(base_field_handler),\n      std::forward<PackedFieldHandler>(packed_action));\n}\n\n// Field handler of a singular or an element of a repeated `string`, `bytes`,\n// or submessage field.\n//\n// For a `Reader` source, the value is provided as `ReaderSpan<>`,\n// `absl::string_view`, `std::string&&`, `Chain&&`, or `absl::Cord&&`,\n// depending on what is accepted.\n//\n// For a `Cord` source, the value is provided as `CordIteratorSpan`,\n// `absl::string_view`, `std::string&&`, `Chain&&`, or `absl::Cord&&`,\n// depending on what is accepted.\n//\n// For a string source, the value is provided as `absl::string_view`,\n// `std::string&&`, `Chain&&`, or `absl::Cord&&`, depending on what is accepted.\n//\n// For a string source, if the action accepts `absl::string_view`, then the\n// value is guaranteed to be a substring of the original string. This guarantee\n// is absent for a `Reader` or `Cord` source.\n//\n// If the action accepts the first candidate for each source type, i.e.\n// `ReaderSpan<>`, `CordIteratorSpan`, and `absl::string_view`, then the action\n// parameter can be declared as `auto`. This is convenient if the implementation\n// can treat these types uniformly, e.g. when the value is passed to\n// `riegeli::ParseMessage()`, `SerializedMessageReader::ReadMessage()`, or\n// `SerializedMessageWriter::WriteString()`.\n//\n// Alternatively, the action can use `absl::Overload{}` to provide variants with\n// separate implementations for these parameter types.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr OnLengthDelimitedType<field_number, std::decay_t<Action>>\nOnLengthDelimited(Action&& action) {\n  return OnLengthDelimitedType<field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler called before the given group.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr BeforeGroupType<field_number, std::decay_t<Action>> BeforeGroup(\n    Action&& action) {\n  return BeforeGroupType<field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n// Field handler called after the given group.\ntemplate <int field_number = kUnboundFieldNumber, typename Action>\nconstexpr AfterGroupType<field_number, std::decay_t<Action>> AfterGroup(\n    Action&& action) {\n  return AfterGroupType<field_number, std::decay_t<Action>>(\n      std::forward<Action>(action));\n}\n\n}  // namespace field_handlers\n\n// Implementation details follow.\n\nnamespace field_handlers_internal {\n\nABSL_ATTRIBUTE_COLD absl::Status AnnotateByReader(absl::Status status,\n                                                  Reader& reader);\n\nABSL_ATTRIBUTE_COLD absl::Status EnumOverflowError(uint64_t repr);\n\ntemplate <typename Value, field_handlers::VarintKind kind>\nabsl::Status VarintOverflowError(uint64_t repr) {\n  RIEGELI_ASSERT(kind == field_handlers::VarintKind::kEnum)\n      << \"Remaining VarintOverflowError() instantiations should be for enums\";\n  return EnumOverflowError(repr);\n}\ntemplate <>\nABSL_ATTRIBUTE_COLD absl::Status\nVarintOverflowError<int32_t, field_handlers::VarintKind::kPlain>(uint64_t repr);\ntemplate <>\nABSL_ATTRIBUTE_COLD absl::Status\nVarintOverflowError<uint32_t, field_handlers::VarintKind::kPlain>(\n    uint64_t repr);\ntemplate <>\nABSL_ATTRIBUTE_COLD absl::Status\nVarintOverflowError<int32_t, field_handlers::VarintKind::kSigned>(\n    uint64_t repr);\ntemplate <>\nABSL_ATTRIBUTE_COLD absl::Status\nVarintOverflowError<bool, field_handlers::VarintKind::kPlain>(uint64_t repr);\n\nABSL_ATTRIBUTE_COLD absl::Status EnumOverflowError(Reader& src, uint64_t repr);\n\ntemplate <typename Value, field_handlers::VarintKind kind>\nabsl::Status VarintOverflowError(Reader& src, uint64_t repr) {\n  RIEGELI_ASSERT(kind == field_handlers::VarintKind::kEnum)\n      << \"Remaining VarintOverflowError() instantiations should be for enums\";\n  return EnumOverflowError(src, repr);\n}\ntemplate <>\nABSL_ATTRIBUTE_COLD absl::Status\nVarintOverflowError<int32_t, field_handlers::VarintKind::kPlain>(Reader& src,\n                                                                 uint64_t repr);\ntemplate <>\nABSL_ATTRIBUTE_COLD absl::Status\nVarintOverflowError<uint32_t, field_handlers::VarintKind::kPlain>(\n    Reader& src, uint64_t repr);\ntemplate <>\nABSL_ATTRIBUTE_COLD absl::Status\nVarintOverflowError<int32_t, field_handlers::VarintKind::kSigned>(\n    Reader& src, uint64_t repr);\ntemplate <>\nABSL_ATTRIBUTE_COLD absl::Status\nVarintOverflowError<bool, field_handlers::VarintKind::kPlain>(Reader& src,\n                                                              uint64_t repr);\n\nABSL_ATTRIBUTE_COLD absl::Status ReadPackedVarintError();\n\nABSL_ATTRIBUTE_COLD absl::Status ReadPackedVarintError(Reader& src);\n\ntemplate <size_t size>\nabsl::Status ReadPackedFixedError();\ntemplate <>\nABSL_ATTRIBUTE_COLD absl::Status ReadPackedFixedError<sizeof(uint32_t)>();\ntemplate <>\nABSL_ATTRIBUTE_COLD absl::Status ReadPackedFixedError<sizeof(uint64_t)>();\n\ntemplate <size_t size>\nabsl::Status ReadPackedFixedError(Reader& src);\ntemplate <>\nABSL_ATTRIBUTE_COLD absl::Status ReadPackedFixedError<sizeof(uint32_t)>(\n    Reader& src);\ntemplate <>\nABSL_ATTRIBUTE_COLD absl::Status ReadPackedFixedError<sizeof(uint64_t)>(\n    Reader& src);\n\ntemplate <typename Value, field_handlers::VarintKind kind>\ninline bool VarintIsValid(uint64_t repr) {\n  if constexpr (std::disjunction_v<std::is_same<Value, uint64_t>,\n                                   std::is_same<Value, int64_t>>) {\n    return true;\n  } else if constexpr (kind == field_handlers::VarintKind::kSigned) {\n    return uint64_t{static_cast<std::make_unsigned_t<Value>>(repr)} == repr;\n  } else if constexpr (std::is_enum_v<Value>) {\n    static_assert(kind == field_handlers::VarintKind::kEnum);\n    return static_cast<uint64_t>(\n               static_cast<std::underlying_type_t<Value>>(repr)) == repr;\n  } else {\n    return static_cast<uint64_t>(static_cast<Value>(repr)) == repr;\n  }\n}\n\ntemplate <typename Value, field_handlers::VarintKind kind>\ninline Value DecodeVarint(uint64_t repr) {\n  if constexpr (kind == field_handlers::VarintKind::kSigned) {\n    return static_cast<Value>(DecodeVarintSigned64(repr));\n  } else if constexpr (std::is_enum_v<Value>) {\n    static_assert(kind == field_handlers::VarintKind::kEnum);\n    // Casting an out of range value to an enum has undefined behavior.\n    // Casting such a value to an integral type wraps around.\n    return static_cast<Value>(static_cast<std::underlying_type_t<Value>>(repr));\n  } else {\n    return static_cast<Value>(repr);\n  }\n}\n\n}  // namespace field_handlers_internal\n\nnamespace field_handlers {\n\ntemplate <typename Value, VarintKind kind, int field_number, typename Action>\nclass OnOptionalVarintType {\n public:\n  static constexpr int kFieldNumber = field_number;\n\n  template <typename ActionInitializer,\n            std::enable_if_t<std::is_convertible_v<ActionInitializer&&, Action>,\n                             int> = 0>\n  explicit constexpr OnOptionalVarintType(ActionInitializer&& action)\n      : action_(std::forward<ActionInitializer>(action)) {}\n\n  template <\n      typename... Context,\n      std::enable_if_t<std::is_invocable_v<const Action&, Value, Context&...>,\n                       int> = 0>\n  absl::Status HandleVarint(uint64_t repr, Context&... context) const {\n    if (ABSL_PREDICT_FALSE(\n            (!field_handlers_internal::VarintIsValid<Value, kind>(repr)))) {\n      return field_handlers_internal::VarintOverflowError<Value, kind>(repr);\n    }\n    return action_(field_handlers_internal::DecodeVarint<Value, kind>(repr),\n                   context...);\n  }\n\n protected:\n  const Action& action() const { return action_; }\n\n private:\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS Action action_;\n};\n\ntemplate <typename Value, VarintKind kind, int field_number, typename Action>\nclass OnRepeatedVarintType\n    : public OnOptionalVarintType<Value, kind, field_number, Action> {\n public:\n  using OnRepeatedVarintType::OnOptionalVarintType::OnOptionalVarintType;\n\n  template <\n      typename... Context,\n      std::enable_if_t<std::is_invocable_v<const Action&, Value, Context&...>,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromReader(ReaderSpan<> repr,\n                                               Context&... context) const;\n\n  template <\n      typename... Context,\n      std::enable_if_t<std::is_invocable_v<const Action&, Value, Context&...>,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromCord(CordIteratorSpan repr,\n                                             std::string& scratch,\n                                             Context&... context) const;\n\n  template <\n      typename... Context,\n      std::enable_if_t<std::is_invocable_v<const Action&, Value, Context&...>,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromString(absl::string_view repr,\n                                               Context&... context) const;\n};\n\ntemplate <typename Value, int field_number, typename Action>\nclass OnOptionalFixedType {\n public:\n  static constexpr int kFieldNumber = field_number;\n\n  template <typename ActionInitializer,\n            std::enable_if_t<std::is_convertible_v<ActionInitializer&&, Action>,\n                             int> = 0>\n  explicit constexpr OnOptionalFixedType(ActionInitializer&& action)\n      : action_(std::forward<ActionInitializer>(action)) {}\n\n  template <typename... Context,\n            std::enable_if_t<\n                std::conjunction_v<\n                    std::bool_constant<sizeof(Value) == sizeof(uint32_t)>,\n                    std::is_invocable<const Action&, Value, Context&...>>,\n                int> = 0>\n  absl::Status HandleFixed32(uint32_t repr, Context&... context) const {\n    return action_(absl::bit_cast<Value>(repr), context...);\n  }\n\n  template <typename... Context,\n            std::enable_if_t<\n                std::conjunction_v<\n                    std::bool_constant<sizeof(Value) == sizeof(uint64_t)>,\n                    std::is_invocable<const Action&, Value, Context&...>>,\n                int> = 0>\n  absl::Status HandleFixed64(uint64_t repr, Context&... context) const {\n    return action_(absl::bit_cast<Value>(repr), context...);\n  }\n\n protected:\n  const Action& action() const { return action_; }\n\n private:\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS Action action_;\n};\n\ntemplate <typename Value, int field_number, typename Action>\nclass OnRepeatedFixedType\n    : public OnOptionalFixedType<Value, field_number, Action> {\n public:\n  using OnRepeatedFixedType::OnOptionalFixedType::OnOptionalFixedType;\n\n  template <\n      typename... Context,\n      std::enable_if_t<std::is_invocable_v<const Action&, Value, Context&...>,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromReader(ReaderSpan<> repr,\n                                               Context&... context) const;\n\n  template <\n      typename... Context,\n      std::enable_if_t<std::is_invocable_v<const Action&, Value, Context&...>,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromCord(CordIteratorSpan repr,\n                                             std::string& scratch,\n                                             Context&... context) const;\n\n  template <\n      typename... Context,\n      std::enable_if_t<std::is_invocable_v<const Action&, Value, Context&...>,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromString(absl::string_view repr,\n                                               Context&... context) const;\n\n private:\n  template <typename... Context>\n  absl::Status HandleLengthDelimitedFromStringInternal(\n      absl::string_view repr, Context&... context) const {\n    const char* cursor = repr.data();\n    const char* const limit = repr.data() + repr.size();\n    while (cursor < limit) {\n      const Value element = ReadLittleEndian<Value>(cursor);\n      if (absl::Status status = this->action()(element, context...);\n          ABSL_PREDICT_FALSE(!status.ok())) {\n        return status;\n      }\n      cursor += sizeof(Value);\n    }\n    return absl::OkStatus();\n  }\n};\n\ntemplate <typename BaseFieldHandler, typename PackedFieldHandler>\nclass OnPackedType {\n private:\n  template <typename... Context>\n  struct ExpectedFieldHandlers\n      : std::conjunction<\n            std::disjunction<\n                serialized_message_reader_internal::\n                    IsStaticFieldHandlerForVarint<BaseFieldHandler, Context...>,\n                serialized_message_reader_internal::\n                    IsStaticFieldHandlerForFixed32<BaseFieldHandler,\n                                                   Context...>,\n                serialized_message_reader_internal::\n                    IsStaticFieldHandlerForFixed64<BaseFieldHandler,\n                                                   Context...>>,\n            serialized_message_reader_internal::\n                IsStaticFieldHandlerForLengthDelimited<PackedFieldHandler,\n                                                       Context...>> {};\n\n public:\n  static constexpr int kFieldNumber = BaseFieldHandler::kFieldNumber;\n\n  template <\n      typename BaseFieldHandlerInitializer,\n      typename PackedFieldHandlerInitializer,\n      std::enable_if_t<std::conjunction_v<\n                           std::is_convertible<BaseFieldHandlerInitializer&&,\n                                               BaseFieldHandler>,\n                           std::is_convertible<PackedFieldHandlerInitializer&&,\n                                               PackedFieldHandler>>,\n                       int> = 0>\n  explicit constexpr OnPackedType(\n      BaseFieldHandlerInitializer&& base_field_handler,\n      PackedFieldHandlerInitializer&& packed_action)\n      : base_field_handler_(\n            std::forward<BaseFieldHandlerInitializer>(base_field_handler)),\n        packed_field_handler_(\n            std::forward<PackedFieldHandlerInitializer>(packed_action)) {}\n\n  template <\n      typename... Context,\n      std::enable_if_t<\n          std::conjunction_v<\n              ExpectedFieldHandlers<Context...>,\n              serialized_message_reader_internal::IsStaticFieldHandlerForVarint<\n                  BaseFieldHandler, Context...>>,\n          int> = 0>\n  absl::Status HandleVarint(uint64_t repr, Context&... context) const {\n    return base_field_handler_.HandleVarint(repr, context...);\n  }\n\n  template <typename... Context,\n            std::enable_if_t<\n                std::conjunction_v<ExpectedFieldHandlers<Context...>,\n                                   serialized_message_reader_internal::\n                                       IsStaticFieldHandlerForFixed32<\n                                           BaseFieldHandler, Context...>>,\n                int> = 0>\n  absl::Status HandleFixed32(uint32_t repr, Context&... context) const {\n    return base_field_handler_.HandleFixed32(repr, context...);\n  }\n\n  template <typename... Context,\n            std::enable_if_t<\n                std::conjunction_v<ExpectedFieldHandlers<Context...>,\n                                   serialized_message_reader_internal::\n                                       IsStaticFieldHandlerForFixed64<\n                                           BaseFieldHandler, Context...>>,\n                int> = 0>\n  absl::Status HandleFixed64(uint64_t repr, Context&... context) const {\n    return base_field_handler_.HandleFixed64(repr, context...);\n  }\n\n  template <\n      typename... Context,\n      std::enable_if_t<std::conjunction_v<\n                           ExpectedFieldHandlers<Context...>,\n                           serialized_message_reader_internal::\n                               IsStaticFieldHandlerForLengthDelimitedFromReader<\n                                   PackedFieldHandler, Context...>>,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromReader(ReaderSpan<> repr,\n                                               Context&... context) const {\n    return packed_field_handler_.HandleLengthDelimitedFromReader(\n        std::move(repr), context...);\n  }\n\n  template <\n      typename... Context,\n      std::enable_if_t<\n          std::conjunction_v<ExpectedFieldHandlers<Context...>,\n                             serialized_message_reader_internal::\n                                 IsStaticFieldHandlerForLengthDelimitedFromCord<\n                                     PackedFieldHandler, Context...>>,\n          int> = 0>\n  absl::Status HandleLengthDelimitedFromCord(CordIteratorSpan repr,\n                                             std::string& scratch,\n                                             Context&... context) const {\n    return packed_field_handler_.HandleLengthDelimitedFromCord(\n        std::move(repr), scratch, context...);\n  }\n\n  template <\n      typename... Context,\n      std::enable_if_t<std::conjunction_v<\n                           ExpectedFieldHandlers<Context...>,\n                           serialized_message_reader_internal::\n                               IsStaticFieldHandlerForLengthDelimitedFromString<\n                                   PackedFieldHandler, Context...>>,\n                       int> = 0>\n  absl::Status HandleLengthDelimitedFromString(absl::string_view repr,\n                                               Context&... context) const {\n    return packed_field_handler_.HandleLengthDelimitedFromString(repr,\n                                                                 context...);\n  }\n\n private:\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS BaseFieldHandler base_field_handler_;\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS PackedFieldHandler packed_field_handler_;\n};\n\ntemplate <int field_number, typename Action>\nclass OnLengthDelimitedType {\n public:\n  static constexpr int kFieldNumber = field_number;\n\n  template <typename ActionInitializer,\n            std::enable_if_t<std::is_convertible_v<ActionInitializer&&, Action>,\n                             int> = 0>\n  explicit constexpr OnLengthDelimitedType(ActionInitializer&& action)\n      : action_(std::forward<ActionInitializer>(action)) {}\n\n  template <\n      typename... Context,\n      std::enable_if_t<\n          std::disjunction_v<\n              std::is_invocable<const Action&, ReaderSpan<>, Context&...>,\n              std::is_invocable<const Action&, absl::string_view, Context&...>,\n              std::is_invocable<const Action&, std::string&&, Context&...>,\n              std::is_invocable<const Action&, Chain&&, Context&...>,\n              std::is_invocable<const Action&, absl::Cord&&, Context&...>>,\n          int> = 0>\n  absl::Status HandleLengthDelimitedFromReader(ReaderSpan<> repr,\n                                               Context&... context) const {\n    if constexpr (std::is_invocable_v<const Action&, ReaderSpan<>,\n                                      Context&...>) {\n      return SkipLengthDelimitedFromReader(\n          repr, [&] { return action_(std::move(repr), context...); });\n    } else if constexpr (std::is_invocable_v<const Action&, absl::string_view,\n                                             Context&...>) {\n      return HandleString<absl::string_view>(std::move(repr), context...);\n    } else if constexpr (std::is_invocable_v<const Action&, std::string&&,\n                                             Context&...>) {\n      return HandleString<std::string>(std::move(repr), context...);\n    } else if constexpr (std::is_invocable_v<const Action&, Chain&&,\n                                             Context&...>) {\n      return HandleString<Chain>(std::move(repr), context...);\n    } else if constexpr (std::is_invocable_v<const Action&, absl::Cord&&,\n                                             Context&...>) {\n      return HandleString<absl::Cord>(std::move(repr), context...);\n    } else {\n      static_assert(false, \"No string-like type accepted\");\n    }\n  }\n\n  template <\n      typename... Context,\n      std::enable_if_t<\n          std::disjunction_v<\n              std::is_invocable<const Action&, CordIteratorSpan, Context&...>,\n              std::is_invocable<const Action&, absl::string_view, Context&...>,\n              std::is_invocable<const Action&, std::string&&, Context&...>,\n              std::is_invocable<const Action&, Chain, Context&...>,\n              std::is_invocable<const Action&, absl::Cord, Context&...>>,\n          int> = 0>\n  absl::Status HandleLengthDelimitedFromCord(CordIteratorSpan repr,\n                                             std::string& scratch,\n                                             Context&... context) const {\n    if constexpr (std::is_invocable_v<const Action&, CordIteratorSpan,\n                                      Context&...>) {\n      return SkipLengthDelimitedFromCord(\n          repr, [&] { return action_(std::move(repr), context...); });\n    } else if constexpr (std::is_invocable_v<const Action&, absl::string_view,\n                                             Context&...>) {\n      return action_(std::move(repr).ToStringView(scratch), context...);\n    } else if constexpr (std::is_invocable_v<const Action&, std::string&&,\n                                             Context&...>) {\n      std::move(repr).ToString(scratch);\n      return action_(std::move(scratch), context...);\n    } else if constexpr (std::is_invocable_v<const Action&, Chain,\n                                             Context&...>) {\n      return action_(Chain(std::move(repr).ToCord()), context...);\n    } else if constexpr (std::is_invocable_v<const Action&, absl::Cord,\n                                             Context&...>) {\n      return action_(std::move(repr).ToCord(), context...);\n    } else {\n      static_assert(false, \"No string-like type accepted\");\n    }\n  }\n\n  template <\n      typename... Context,\n      std::enable_if_t<\n          std::disjunction_v<\n              std::is_invocable<const Action&, absl::string_view, Context&...>,\n              std::is_invocable<const Action&, std::string&&, Context&...>,\n              std::is_invocable<const Action&, Chain&&, Context&...>,\n              std::is_invocable<const Action&, absl::Cord&&, Context&...>>,\n          int> = 0>\n  absl::Status HandleLengthDelimitedFromString(absl::string_view repr,\n                                               Context&... context) const {\n    if constexpr (std::is_invocable_v<const Action&, absl::string_view,\n                                      Context&...>) {\n      return action_(repr, context...);\n    } else if constexpr (std::is_invocable_v<const Action&, std::string&&,\n                                             Context&...>) {\n      return action_(std::string(repr), context...);\n    } else if constexpr (std::is_invocable_v<const Action&, Chain&&,\n                                             Context&...>) {\n      return action_(Chain(repr), context...);\n    } else if constexpr (std::is_invocable_v<const Action&, absl::Cord&&,\n                                             Context&...>) {\n      return action_(absl::Cord(repr), context...);\n    } else {\n      static_assert(false, \"No string-like type accepted\");\n    }\n  }\n\n private:\n  template <typename StringType, typename... Context>\n  absl::Status HandleString(ReaderSpan<> repr, Context&... context) const {\n    StringType value;\n    if (ABSL_PREDICT_FALSE(\n            !repr.reader().Read(IntCast<size_t>(repr.length()), value))) {\n      return serialized_message_reader_internal::ReadLengthDelimitedValueError(\n          repr.reader());\n    }\n    absl::Status status = action_(std::move(value), context...);\n    // Comparison against `absl::CancelledError()` is a fast path of\n    // `absl::IsCancelled()`.\n    if (ABSL_PREDICT_FALSE(!status.ok() && status != absl::CancelledError())) {\n      status = field_handlers_internal::AnnotateByReader(std::move(status),\n                                                         repr.reader());\n    }\n    return status;\n  }\n\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS Action action_;\n};\n\ntemplate <int field_number, typename Action>\nclass BeforeGroupType {\n public:\n  static constexpr int kFieldNumber = field_number;\n\n  template <typename ActionInitializer,\n            std::enable_if_t<std::is_convertible_v<ActionInitializer&&, Action>,\n                             int> = 0>\n  explicit constexpr BeforeGroupType(ActionInitializer&& action)\n      : action_(std::forward<ActionInitializer>(action)) {}\n\n  template <typename... Context,\n            std::enable_if_t<std::is_invocable_v<const Action&, Context&...>,\n                             int> = 0>\n  absl::Status HandleStartGroup(Context&... context) const {\n    return action_(context...);\n  }\n\n private:\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS Action action_;\n};\n\ntemplate <int field_number, typename Action>\nclass AfterGroupType {\n public:\n  static constexpr int kFieldNumber = field_number;\n\n  template <typename ActionInitializer,\n            std::enable_if_t<std::is_convertible_v<ActionInitializer&&, Action>,\n                             int> = 0>\n  explicit constexpr AfterGroupType(ActionInitializer&& action)\n      : action_(std::forward<ActionInitializer>(action)) {}\n\n  template <typename... Context,\n            std::enable_if_t<std::is_invocable_v<const Action&, Context&...>,\n                             int> = 0>\n  absl::Status HandleEndGroup(Context&... context) const {\n    return action_(context...);\n  }\n\n private:\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS Action action_;\n};\n\ntemplate <typename Value, VarintKind kind, int field_number, typename Action>\ntemplate <typename... Context,\n          std::enable_if_t<\n              std::is_invocable_v<const Action&, Value, Context&...>, int>>\nabsl::Status OnRepeatedVarintType<Value, kind, field_number, Action>::\n    HandleLengthDelimitedFromReader(ReaderSpan<> repr,\n                                    Context&... context) const {\n  if (repr.reader().Pull(1, IntCast<size_t>(repr.length())) &&\n      repr.reader().available() >= IntCast<size_t>(repr.length())) {\n    const absl::string_view value(repr.reader().cursor(),\n                                  IntCast<size_t>(repr.length()));\n    repr.reader().move_cursor(IntCast<size_t>(repr.length()));\n    absl::Status status = HandleLengthDelimitedFromString(value, context...);\n    // Comparison against `absl::CancelledError()` is a fast path of\n    // `absl::IsCancelled()`.\n    if (ABSL_PREDICT_FALSE(!status.ok() && status != absl::CancelledError())) {\n      status = field_handlers_internal::AnnotateByReader(std::move(status),\n                                                         repr.reader());\n    }\n    return status;\n  }\n  ScopedLimiter scoped_limiter(repr);\n  uint64_t element;\n  while (ReadVarint64(repr.reader(), element)) {\n    if (ABSL_PREDICT_FALSE(\n            (!field_handlers_internal::VarintIsValid<Value, kind>(element)))) {\n      return field_handlers_internal::VarintOverflowError<Value, kind>(\n          repr.reader(), element);\n    }\n    if (absl::Status status = this->action()(\n            field_handlers_internal::DecodeVarint<Value, kind>(element),\n            context...);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      // Comparison against `absl::CancelledError()` is a fast path of\n      // `absl::IsCancelled()`.\n      if (ABSL_PREDICT_FALSE(status != absl::CancelledError())) {\n        status = field_handlers_internal::AnnotateByReader(std::move(status),\n                                                           repr.reader());\n      }\n      return status;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(repr.reader().pos() < repr.reader().max_pos())) {\n    return field_handlers_internal::ReadPackedVarintError(repr.reader());\n  }\n  return absl::OkStatus();\n}\n\ntemplate <typename Value, VarintKind kind, int field_number, typename Action>\ntemplate <typename... Context,\n          std::enable_if_t<\n              std::is_invocable_v<const Action&, Value, Context&...>, int>>\nabsl::Status OnRepeatedVarintType<Value, kind, field_number, Action>::\n    HandleLengthDelimitedFromCord(CordIteratorSpan repr,\n                                  ABSL_ATTRIBUTE_UNUSED std::string& scratch,\n                                  Context&... context) const {\n  if (const absl::string_view chunk =\n          absl::Cord::ChunkRemaining(repr.iterator());\n      chunk.size() >= IntCast<size_t>(repr.length())) {\n    const absl::string_view value =\n        chunk.substr(0, IntCast<size_t>(repr.length()));\n    absl::Cord::AdvanceAndRead(&repr.iterator(),\n                               IntCast<size_t>(repr.length()));\n    return HandleLengthDelimitedFromString(value, context...);\n  }\n  const size_t limit =\n      CordIteratorSpan::Remaining(repr.iterator()) - repr.length();\n  uint64_t element;\n  while (ReadVarint64(repr.iterator(),\n                      CordIteratorSpan::Remaining(repr.iterator()) - limit,\n                      element)) {\n    if (ABSL_PREDICT_FALSE(\n            (!field_handlers_internal::VarintIsValid<Value, kind>(element)))) {\n      return field_handlers_internal::VarintOverflowError<Value, kind>(element);\n    }\n    if (absl::Status status = this->action()(\n            field_handlers_internal::DecodeVarint<Value, kind>(element),\n            context...);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(CordIteratorSpan::Remaining(repr.iterator()) >\n                         limit)) {\n    return field_handlers_internal::ReadPackedVarintError();\n  }\n  return absl::OkStatus();\n}\n\ntemplate <typename Value, VarintKind kind, int field_number, typename Action>\ntemplate <typename... Context,\n          std::enable_if_t<\n              std::is_invocable_v<const Action&, Value, Context&...>, int>>\nabsl::Status OnRepeatedVarintType<Value, kind, field_number, Action>::\n    HandleLengthDelimitedFromString(absl::string_view repr,\n                                    Context&... context) const {\n  const char* cursor = repr.data();\n  const char* const limit = repr.data() + repr.size();\n  uint64_t element;\n  while (const size_t length =\n             ReadVarint64(cursor, PtrDistance(cursor, limit), element)) {\n    cursor += length;\n    if (ABSL_PREDICT_FALSE(\n            (!field_handlers_internal::VarintIsValid<Value, kind>(element)))) {\n      return field_handlers_internal::VarintOverflowError<Value, kind>(element);\n    }\n    if (absl::Status status = this->action()(\n            field_handlers_internal::DecodeVarint<Value, kind>(element),\n            context...);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(cursor < limit)) {\n    return field_handlers_internal::ReadPackedVarintError();\n  }\n  return absl::OkStatus();\n}\n\ntemplate <typename Value, int field_number, typename Action>\ntemplate <typename... Context,\n          std::enable_if_t<\n              std::is_invocable_v<const Action&, Value, Context&...>, int>>\nabsl::Status OnRepeatedFixedType<Value, field_number, Action>::\n    HandleLengthDelimitedFromReader(ReaderSpan<> repr,\n                                    Context&... context) const {\n  if (ABSL_PREDICT_FALSE(repr.length() % sizeof(Value) > 0)) {\n    return field_handlers_internal::ReadPackedFixedError<sizeof(Value)>(\n        repr.reader());\n  }\n  if (repr.reader().Pull(1, IntCast<size_t>(repr.length())) &&\n      repr.reader().available() >= IntCast<size_t>(repr.length())) {\n    const absl::string_view value(repr.reader().cursor(),\n                                  IntCast<size_t>(repr.length()));\n    repr.reader().move_cursor(IntCast<size_t>(repr.length()));\n    absl::Status status =\n        HandleLengthDelimitedFromStringInternal(value, context...);\n    // Comparison against `absl::CancelledError()` is a fast path of\n    // `absl::IsCancelled()`.\n    if (ABSL_PREDICT_FALSE(!status.ok() && status != absl::CancelledError())) {\n      status = field_handlers_internal::AnnotateByReader(std::move(status),\n                                                         repr.reader());\n    }\n    return status;\n  }\n  Position length = repr.length();\n  while (length > 0) {\n    Value element;\n    if (ABSL_PREDICT_FALSE(!ReadLittleEndian<Value>(repr.reader(), element))) {\n      return field_handlers_internal::ReadPackedFixedError<sizeof(Value)>(\n          repr.reader());\n    }\n    if (absl::Status status = this->action()(element, context...);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      // Comparison against `absl::CancelledError()` is a fast path of\n      // `absl::IsCancelled()`.\n      if (ABSL_PREDICT_FALSE(status != absl::CancelledError())) {\n        status = field_handlers_internal::AnnotateByReader(std::move(status),\n                                                           repr.reader());\n      }\n      return status;\n    }\n    length -= sizeof(Value);\n  }\n  return absl::OkStatus();\n}\n\ntemplate <typename Value, int field_number, typename Action>\ntemplate <typename... Context,\n          std::enable_if_t<\n              std::is_invocable_v<const Action&, Value, Context&...>, int>>\nabsl::Status\nOnRepeatedFixedType<Value, field_number, Action>::HandleLengthDelimitedFromCord(\n    CordIteratorSpan repr, ABSL_ATTRIBUTE_UNUSED std::string& scratch,\n    Context&... context) const {\n  if (ABSL_PREDICT_FALSE(repr.length() % sizeof(Value) > 0)) {\n    return field_handlers_internal::ReadPackedFixedError<sizeof(Value)>();\n  }\n  if (const absl::string_view chunk =\n          absl::Cord::ChunkRemaining(repr.iterator());\n      chunk.size() >= IntCast<size_t>(repr.length())) {\n    const absl::string_view value =\n        chunk.substr(0, IntCast<size_t>(repr.length()));\n    absl::Cord::AdvanceAndRead(&repr.iterator(),\n                               IntCast<size_t>(repr.length()));\n    return HandleLengthDelimitedFromStringInternal(value, context...);\n  }\n  Position length = repr.length();\n  while (length > 0) {\n    char buffer[sizeof(Value)];\n    CordIteratorSpan::Read(repr.iterator(), sizeof(Value), buffer);\n    const Value element = ReadLittleEndian<Value>(buffer);\n    if (absl::Status status = this->action()(element, context...);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n    length -= sizeof(Value);\n  }\n  return absl::OkStatus();\n}\n\ntemplate <typename Value, int field_number, typename Action>\ntemplate <typename... Context,\n          std::enable_if_t<\n              std::is_invocable_v<const Action&, Value, Context&...>, int>>\nabsl::Status OnRepeatedFixedType<Value, field_number, Action>::\n    HandleLengthDelimitedFromString(absl::string_view repr,\n                                    Context&... context) const {\n  if (ABSL_PREDICT_FALSE(repr.size() % sizeof(Value) > 0)) {\n    return field_handlers_internal::ReadPackedFixedError<sizeof(Value)>();\n  }\n  return HandleLengthDelimitedFromStringInternal(repr, context...);\n}\n\n}  // namespace field_handlers\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_FIELD_HANDLERS_H_\n"
  },
  {
    "path": "riegeli/messages/map_entry_field.h",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_MAP_ENTRY_FIELD_H_\n#define RIEGELI_MESSAGES_MAP_ENTRY_FIELD_H_\n\n#include \"absl/base/nullability.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// Specifies a field of synthetic map entry message.\nenum MapEntryField { kMapEntryKey = 1, kMapEntryValue = 2 };\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_MAP_ENTRY_FIELD_H_\n"
  },
  {
    "path": "riegeli/messages/message_wire_format.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_MESSAGE_WIRE_FORMAT_H_\n#define RIEGELI_MESSAGES_MESSAGE_WIRE_FORMAT_H_\n\n#include <stdint.h>\n\nnamespace riegeli {\n\n// Low level functions for writing and reading serialized proto messages\n// directly.\n//\n// They mostly correspond to selected members of\n// `google::protobuf::internal::WireFormatLite`.\n\n// The part of a field tag which denotes the representation of the field value\n// which follows the tag.\nenum class WireType : uint32_t {\n  kVarint = 0,\n  kFixed32 = 5,\n  kFixed64 = 1,\n  kLengthDelimited = 2,\n  kStartGroup = 3,\n  kEndGroup = 4,\n  kInvalid6 = 6,\n  kInvalid7 = 7,\n};\n\n// Composes/decomposes a field tag.\nconstexpr uint32_t MakeTag(int field_number, WireType wire_type);\nconstexpr WireType GetTagWireType(uint32_t tag);\nconstexpr int GetTagFieldNumber(uint32_t tag);\n\n// Represents a set of wire types.\n\nenum class WireTypeSet : uint32_t {\n  kVarint = 1 << static_cast<uint32_t>(WireType::kVarint),\n  kFixed32 = 1 << static_cast<uint32_t>(WireType::kFixed32),\n  kFixed64 = 1 << static_cast<uint32_t>(WireType::kFixed64),\n  kLengthDelimited = 1 << static_cast<uint32_t>(WireType::kLengthDelimited),\n  kStartGroup = 1 << static_cast<uint32_t>(WireType::kStartGroup),\n  kEndGroup = 1 << static_cast<uint32_t>(WireType::kEndGroup),\n  kInvalid6 = 1 << static_cast<uint32_t>(WireType::kInvalid6),\n  kInvalid7 = 1 << static_cast<uint32_t>(WireType::kInvalid7),\n};\n\nconstexpr WireTypeSet operator|(WireTypeSet a, WireTypeSet b) {\n  return WireTypeSet{static_cast<uint32_t>(a) | static_cast<uint32_t>(b)};\n}\n\nconstexpr WireTypeSet operator^(WireTypeSet a, WireTypeSet b) {\n  return WireTypeSet{static_cast<uint32_t>(a) ^ static_cast<uint32_t>(b)};\n}\n\nconstexpr WireTypeSet operator&(WireTypeSet a, WireTypeSet b) {\n  return WireTypeSet{static_cast<uint32_t>(a) & static_cast<uint32_t>(b)};\n}\n\ntemplate <WireType... wire_types>\nconstexpr WireTypeSet WireTypeSetOf() {\n  return (WireTypeSet{0} | ... |\n          WireTypeSet{uint32_t{1} << static_cast<uint32_t>(wire_types)});\n}\n\nconstexpr WireTypeSet NoWireTypes = WireTypeSetOf<>();\n\nconstexpr WireTypeSet AllWireTypes =\n    WireTypeSetOf<WireType::kVarint, WireType::kFixed32, WireType::kFixed64,\n                  WireType::kLengthDelimited, WireType::kStartGroup,\n                  WireType::kEndGroup, WireType::kInvalid6,\n                  WireType::kInvalid7>();\n\nconstexpr WireTypeSet operator~(WireTypeSet a) { return AllWireTypes ^ a; }\n\n// Implementation details follow.\n\nconstexpr uint32_t MakeTag(int field_number, WireType wire_type) {\n  return (static_cast<uint32_t>(field_number) << 3) |\n         static_cast<uint32_t>(wire_type);\n}\n\nconstexpr WireType GetTagWireType(uint32_t tag) {\n  return static_cast<WireType>(tag & 7);\n}\n\nconstexpr int GetTagFieldNumber(uint32_t tag) {\n  return static_cast<int>(tag >> 3);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_MESSAGE_WIRE_FORMAT_H_\n"
  },
  {
    "path": "riegeli/messages/parse_message.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/messages/parse_message.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/protobuf/io/coded_stream.h\"\n#include \"google/protobuf/io/zero_copy_stream_impl_lite.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/cord_reader.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/varint/varint_reading.h\"\n\nnamespace riegeli {\n\nnamespace {\n\nABSL_ATTRIBUTE_COLD inline absl::Status ParseError(\n    google::protobuf::MessageLite& dest) {\n  return absl::InvalidArgumentError(\n      absl::StrCat(\"Failed to parse message of type \", dest.GetTypeName()));\n}\n\nABSL_ATTRIBUTE_COLD inline absl::Status ParseError(\n    Reader& src, google::protobuf::MessageLite& dest) {\n  return src.AnnotateStatus(ParseError(dest));\n}\n\ninline absl::Status CheckInitialized(google::protobuf::MessageLite& dest,\n                                     ParseMessageOptions options) {\n  if (!options.partial() && ABSL_PREDICT_FALSE(!dest.IsInitialized())) {\n    return absl::InvalidArgumentError(\n        absl::StrCat(\"Failed to parse message of type \", dest.GetTypeName(),\n                     \" because it is missing required fields: \",\n                     dest.InitializationErrorString()));\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status CheckInitialized(Reader& src,\n                                     google::protobuf::MessageLite& dest,\n                                     ParseMessageOptions options) {\n  if (!options.partial() && ABSL_PREDICT_FALSE(!dest.IsInitialized())) {\n    return src.AnnotateStatus(absl::InvalidArgumentError(\n        absl::StrCat(\"Failed to parse message of type \", dest.GetTypeName(),\n                     \" because it is missing required fields: \",\n                     dest.InitializationErrorString())));\n  }\n  return absl::OkStatus();\n}\n\ntemplate <typename ReaderType>\nabsl::Status ParseMessageFromReaderSpanImpl(ReaderSpan<ReaderType> src,\n                                            google::protobuf::MessageLite& dest,\n                                            ParseMessageOptions options) {\n  if (!options.merge() &&\n      options.recursion_limit() ==\n          google::protobuf::io::CodedInputStream::GetDefaultRecursionLimit() &&\n      src.length() <= kMaxBytesToCopy) {\n    src.reader().Pull();\n    if (src.reader().available() >= src.length()) {\n      // The data are flat. `ParsePartialFromArray()` is faster than\n      // `ParsePartialFromZeroCopyStream()`.\n      const bool parse_ok = dest.ParsePartialFromArray(\n          src.reader().cursor(), IntCast<int>(src.length()));\n      src.reader().move_cursor(IntCast<size_t>(src.length()));\n      if (ABSL_PREDICT_FALSE(!parse_ok)) return ParseError(src.reader(), dest);\n      return CheckInitialized(src.reader(), dest, options);\n    }\n  }\n  ScopedLimiterOrLimitingReader scoped_limiter(src);\n  ReaderInputStream input_stream(&scoped_limiter.reader());\n  bool parse_ok;\n  if (!options.merge() &&\n      options.recursion_limit() ==\n          google::protobuf::io::CodedInputStream::GetDefaultRecursionLimit()) {\n    parse_ok = dest.ParsePartialFromZeroCopyStream(&input_stream);\n  } else {\n    if (!options.merge()) dest.Clear();\n    google::protobuf::io::CodedInputStream coded_stream(&input_stream);\n    coded_stream.SetRecursionLimit(options.recursion_limit());\n    parse_ok = dest.MergePartialFromCodedStream(&coded_stream) &&\n               coded_stream.ConsumedEntireMessage();\n  }\n  if (ABSL_PREDICT_FALSE(!scoped_limiter.reader().ok())) {\n    return scoped_limiter.reader().status();\n  }\n  if (ABSL_PREDICT_FALSE(!parse_ok)) {\n    return ParseError(scoped_limiter.reader(), dest);\n  }\n  const absl::Status status =\n      CheckInitialized(scoped_limiter.reader(), dest, options);\n  RIEGELI_EVAL_ASSERT(scoped_limiter.Close())\n      << \"LimitingReader with !fail_if_longer() \"\n         \"has no reason to fail only in Close(): \"\n      << scoped_limiter.reader().status();\n  return status;\n}\n\ntemplate <typename Src>\nabsl::Status ParseLengthPrefixedMessageImpl(Src& src,\n                                            google::protobuf::MessageLite& dest,\n                                            ParseMessageOptions options) {\n  uint32_t length;\n  if (ABSL_PREDICT_FALSE(!ReadVarint32(src, length) ||\n                         length >\n                             uint32_t{std::numeric_limits<int32_t>::max()})) {\n    return src.StatusOrAnnotate(\n        absl::InvalidArgumentError(\"Failed to parse message length\"));\n  }\n  return parse_message_internal::ParseMessageImpl(\n      ReaderSpan(&src, Position{length}), dest, options);\n}\n\n}  // namespace\n\nnamespace parse_message_internal {\n\nabsl::Status ParseMessageImpl(Reader& src, google::protobuf::MessageLite& dest,\n                              ParseMessageOptions options) {\n  if (!options.merge() &&\n      options.recursion_limit() ==\n          google::protobuf::io::CodedInputStream::GetDefaultRecursionLimit() &&\n      src.SupportsSize()) {\n    const std::optional<Position> size = src.Size();\n    if (ABSL_PREDICT_FALSE(size == std::nullopt)) return src.status();\n    src.Pull();\n    if (src.limit_pos() == *size && src.available() <= kMaxBytesToCopy) {\n      // The data are flat. `ParsePartialFromArray()` is faster than\n      // `ParsePartialFromZeroCopyStream()`.\n      const bool parse_ok = dest.ParsePartialFromArray(\n          src.cursor(), IntCast<int>(src.available()));\n      src.move_cursor(src.available());\n      if (ABSL_PREDICT_FALSE(!parse_ok)) return ParseError(src, dest);\n      return CheckInitialized(src, dest, options);\n    }\n  }\n  ReaderInputStream input_stream(&src);\n  bool parse_ok;\n  if (!options.merge() &&\n      options.recursion_limit() ==\n          google::protobuf::io::CodedInputStream::GetDefaultRecursionLimit()) {\n    parse_ok = dest.ParsePartialFromZeroCopyStream(&input_stream);\n  } else {\n    if (!options.merge()) dest.Clear();\n    google::protobuf::io::CodedInputStream coded_stream(&input_stream);\n    coded_stream.SetRecursionLimit(options.recursion_limit());\n    parse_ok = dest.MergePartialFromCodedStream(&coded_stream) &&\n               coded_stream.ConsumedEntireMessage();\n  }\n  if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n  if (ABSL_PREDICT_FALSE(!parse_ok)) return ParseError(src, dest);\n  return CheckInitialized(src, dest, options);\n}\n\nabsl::Status ParseMessageImpl(ReaderSpan<Reader> src,\n                              google::protobuf::MessageLite& dest,\n                              ParseMessageOptions options) {\n  return ParseMessageFromReaderSpanImpl(std::move(src), dest, options);\n}\n\nabsl::Status ParseMessageImpl(ReaderSpan<> src,\n                              google::protobuf::MessageLite& dest,\n                              ParseMessageOptions options) {\n  return ParseMessageFromReaderSpanImpl(std::move(src), dest, options);\n}\n\n}  // namespace parse_message_internal\n\nabsl::Status ParseLengthPrefixedMessage(Reader& src,\n                                        google::protobuf::MessageLite& dest,\n                                        ParseMessageOptions options) {\n  return ParseLengthPrefixedMessageImpl(src, dest, options);\n}\n\nabsl::Status ParseLengthPrefixedMessage(LimitingReaderBase& src,\n                                        google::protobuf::MessageLite& dest,\n                                        ParseMessageOptions options) {\n  return ParseLengthPrefixedMessageImpl(src, dest, options);\n}\n\nabsl::Status ParseMessage(BytesRef src, google::protobuf::MessageLite& dest,\n                          ParseMessageOptions options) {\n  bool parse_ok;\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         unsigned{std::numeric_limits<int>::max()})) {\n    parse_ok = false;\n  } else if (!options.merge() && options.recursion_limit() ==\n                                     google::protobuf::io::CodedInputStream::\n                                         GetDefaultRecursionLimit()) {\n    parse_ok = dest.ParsePartialFromArray(src.data(), IntCast<int>(src.size()));\n  } else {\n    if (!options.merge()) dest.Clear();\n    google::protobuf::io::ArrayInputStream input_stream(\n        src.data(), IntCast<int>(src.size()));\n    google::protobuf::io::CodedInputStream coded_stream(&input_stream);\n    coded_stream.SetRecursionLimit(options.recursion_limit());\n    parse_ok = dest.MergePartialFromCodedStream(&coded_stream) &&\n               coded_stream.ConsumedEntireMessage();\n  }\n  if (ABSL_PREDICT_FALSE(!parse_ok)) return ParseError(dest);\n  return CheckInitialized(dest, options);\n}\n\nabsl::Status ParseMessage(const Chain& src, google::protobuf::MessageLite& dest,\n                          ParseMessageOptions options) {\n  if (!options.merge() &&\n      options.recursion_limit() ==\n          google::protobuf::io::CodedInputStream::GetDefaultRecursionLimit() &&\n      src.size() <= kMaxBytesToCopy) {\n    if (const std::optional<absl::string_view> flat = src.TryFlat();\n        flat != std::nullopt) {\n      // The data are flat. `ParsePartialFromArray()` is faster than\n      // `ParsePartialFromZeroCopyStream()`.\n      if (ABSL_PREDICT_FALSE(!dest.ParsePartialFromArray(\n              flat->data(), IntCast<int>(flat->size())))) {\n        return ParseError(dest);\n      }\n      return CheckInitialized(dest, options);\n    }\n  }\n  ChainReader reader(&src);\n  ReaderInputStream input_stream(&reader);\n  bool parse_ok;\n  if (!options.merge() &&\n      options.recursion_limit() ==\n          google::protobuf::io::CodedInputStream::GetDefaultRecursionLimit()) {\n    parse_ok = dest.ParsePartialFromZeroCopyStream(&input_stream);\n  } else {\n    if (!options.merge()) dest.Clear();\n    google::protobuf::io::CodedInputStream coded_stream(&input_stream);\n    coded_stream.SetRecursionLimit(options.recursion_limit());\n    parse_ok = dest.MergePartialFromCodedStream(&coded_stream) &&\n               coded_stream.ConsumedEntireMessage();\n  }\n  RIEGELI_ASSERT_OK(reader) << \"ChainReader has no reason to fail\";\n  if (ABSL_PREDICT_FALSE(!parse_ok)) return ParseError(dest);\n  return CheckInitialized(dest, options);\n}\n\nabsl::Status ParseMessage(const absl::Cord& src,\n                          google::protobuf::MessageLite& dest,\n                          ParseMessageOptions options) {\n  if (!options.merge() &&\n      options.recursion_limit() ==\n          google::protobuf::io::CodedInputStream::GetDefaultRecursionLimit() &&\n      src.size() <= kMaxBytesToCopy) {\n    if (const std::optional<absl::string_view> flat = src.TryFlat();\n        flat != std::nullopt) {\n      // The data are flat. `ParsePartialFromArray()` is faster than\n      // `ParsePartialFromZeroCopyStream()`.\n      if (ABSL_PREDICT_FALSE(!dest.ParsePartialFromArray(\n              flat->data(), IntCast<int>(flat->size())))) {\n        return ParseError(dest);\n      }\n      return CheckInitialized(dest, options);\n    }\n  }\n  CordReader reader(&src);\n  ReaderInputStream input_stream(&reader);\n  bool parse_ok;\n  if (!options.merge() &&\n      options.recursion_limit() ==\n          google::protobuf::io::CodedInputStream::GetDefaultRecursionLimit()) {\n    parse_ok = dest.ParsePartialFromZeroCopyStream(&input_stream);\n  } else {\n    if (!options.merge()) dest.Clear();\n    google::protobuf::io::CodedInputStream coded_stream(&input_stream);\n    coded_stream.SetRecursionLimit(options.recursion_limit());\n    parse_ok = dest.MergePartialFromCodedStream(&coded_stream) &&\n               coded_stream.ConsumedEntireMessage();\n  }\n  RIEGELI_ASSERT_OK(reader) << \"CordReader has no reason to fail\";\n  if (ABSL_PREDICT_FALSE(!parse_ok)) return ParseError(dest);\n  return CheckInitialized(dest, options);\n}\n\nabsl::Status ParseMessage(CordIteratorSpan src,\n                          google::protobuf::MessageLite& dest,\n                          ParseMessageOptions options) {\n  if (!options.merge() &&\n      options.recursion_limit() ==\n          google::protobuf::io::CodedInputStream::GetDefaultRecursionLimit() &&\n      src.length() <= kMaxBytesToCopy) {\n    if (const std::optional<absl::string_view> flat = src.TryFlat();\n        flat != std::nullopt) {\n      absl::Cord::Advance(&src.iterator(), flat->size());\n      // The data are flat. `ParsePartialFromArray()` is faster than\n      // `ParsePartialFromZeroCopyStream()`.\n      if (ABSL_PREDICT_FALSE(!dest.ParsePartialFromArray(\n              flat->data(), IntCast<int>(flat->size())))) {\n        return ParseError(dest);\n      }\n      return CheckInitialized(dest, options);\n    }\n  }\n  CordReader reader(std::move(src));\n  ReaderInputStream input_stream(&reader);\n  bool parse_ok;\n  if (!options.merge() &&\n      options.recursion_limit() ==\n          google::protobuf::io::CodedInputStream::GetDefaultRecursionLimit()) {\n    parse_ok = dest.ParsePartialFromZeroCopyStream(&input_stream);\n  } else {\n    if (!options.merge()) dest.Clear();\n    google::protobuf::io::CodedInputStream coded_stream(&input_stream);\n    coded_stream.SetRecursionLimit(options.recursion_limit());\n    parse_ok = dest.MergePartialFromCodedStream(&coded_stream) &&\n               coded_stream.ConsumedEntireMessage();\n  }\n  RIEGELI_ASSERT_OK(reader) << \"CordReader has no reason to fail\";\n  if (ABSL_PREDICT_FALSE(!parse_ok)) return ParseError(dest);\n  return CheckInitialized(dest, options);\n}\n\nbool ReaderInputStream::Next(const void** data, int* size) {\n  RIEGELI_ASSERT_NE(src_, nullptr)\n      << \"Failed precondition of ReaderInputStream::Next(): \"\n         \"ReaderInputStream not initialized\";\n  if (ABSL_PREDICT_FALSE(src_->pos() >=\n                         Position{std::numeric_limits<int64_t>::max()})) {\n    return false;\n  }\n  const Position max_length =\n      Position{std::numeric_limits<int64_t>::max()} - src_->pos();\n  if (ABSL_PREDICT_FALSE(!src_->Pull())) return false;\n  *data = src_->cursor();\n  *size = SaturatingIntCast<int>(UnsignedMin(src_->available(), max_length));\n  src_->move_cursor(IntCast<size_t>(*size));\n  return true;\n}\n\nvoid ReaderInputStream::BackUp(int length) {\n  RIEGELI_ASSERT_NE(src_, nullptr)\n      << \"Failed precondition of ReaderInputStream::BackUp(): \"\n         \"ReaderInputStream not initialized\";\n  RIEGELI_ASSERT_GE(length, 0)\n      << \"Failed precondition of ZeroCopyInputStream::BackUp(): \"\n         \"negative length\";\n  RIEGELI_ASSERT_LE(IntCast<size_t>(length), src_->start_to_cursor())\n      << \"Failed precondition of ZeroCopyInputStream::BackUp(): \"\n         \"length larger than the amount of buffered data\";\n  src_->set_cursor(src_->cursor() - length);\n}\n\nbool ReaderInputStream::Skip(int length) {\n  RIEGELI_ASSERT_NE(src_, nullptr)\n      << \"Failed precondition of ReaderInputStream::Skip(): \"\n         \"ReaderInputStream not initialized\";\n  RIEGELI_ASSERT_GE(length, 0)\n      << \"Failed precondition of ZeroCopyInputStream::Skip(): negative length\";\n  const Position max_length =\n      SaturatingSub(Position{std::numeric_limits<int64_t>::max()}, src_->pos());\n  const size_t length_to_skip =\n      UnsignedMin(IntCast<size_t>(length), max_length);\n  return src_->Skip(length_to_skip) &&\n         length_to_skip == IntCast<size_t>(length);\n}\n\nint64_t ReaderInputStream::ByteCount() const {\n  RIEGELI_ASSERT_NE(src_, nullptr)\n      << \"Failed precondition of ReaderInputStream::ByteCount(): \"\n         \"ReaderInputStream not initialized\";\n  return SaturatingIntCast<int64_t>(src_->pos());\n}\n\nbool ReaderInputStream::ReadCord(absl::Cord* cord, int length) {\n  RIEGELI_ASSERT_NE(src_, nullptr)\n      << \"Failed precondition of ReaderInputStream::ReadCord(): \"\n         \"ReaderInputStream not initialized\";\n  RIEGELI_ASSERT_GE(length, 0)\n      << \"Failed precondition of ZeroCopyInputStream::ReadCord(): \"\n         \"negative length\";\n  const Position max_length =\n      SaturatingSub(Position{std::numeric_limits<int64_t>::max()}, src_->pos());\n  const size_t length_to_read =\n      UnsignedMin(IntCast<size_t>(length), max_length);\n  return src_->ReadAndAppend(length_to_read, *cord) &&\n         length_to_read == IntCast<size_t>(length);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/messages/parse_message.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_PARSE_MESSAGE_H_\n#define RIEGELI_MESSAGES_PARSE_MESSAGE_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"google/protobuf/io/coded_stream.h\"\n#include \"google/protobuf/io/zero_copy_stream.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nclass ParseMessageOptions {\n public:\n  ParseMessageOptions() noexcept {}\n\n  // If `false`, replaces existing contents of the destination, clearing it\n  // first.\n  //\n  // If `true`, merges to existing contents of the destination.\n  //\n  // Default: `false`.\n  ParseMessageOptions& set_merge(bool merge) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    merge_ = merge;\n    return *this;\n  }\n  ParseMessageOptions&& set_merge(bool merge) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_merge(merge));\n  }\n  bool merge() const { return merge_; }\n\n  // If `false`, missing required fields cause a failure.\n  //\n  // If `true`, missing required fields result in a partial parsed message,\n  // not having these fields.\n  //\n  // Default: `false`.\n  ParseMessageOptions& set_partial(bool partial) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    partial_ = partial;\n    return *this;\n  }\n  ParseMessageOptions&& set_partial(bool partial) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_partial(partial));\n  }\n  bool partial() const { return partial_; }\n\n  // Maximum depth of submessages allowed.\n  //\n  // `recursion_limit` must be non-negative.\n  // Default:\n  // `google::protobuf::io::CodedInputStream::GetDefaultRecursionLimit()`\n  // (usually 100).\n  ParseMessageOptions& set_recursion_limit(int recursion_limit) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT_GE(recursion_limit, 0)\n        << \"Failed precondition of ParseMessageOptions::set_recursion_limit(): \"\n           \"recursion limit out of range\";\n    recursion_limit_ = recursion_limit;\n    return *this;\n  }\n  ParseMessageOptions&& set_recursion_limit(int recursion_limit) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_recursion_limit(recursion_limit));\n  }\n  int recursion_limit() const { return recursion_limit_; }\n\n private:\n  bool merge_ = false;\n  bool partial_ = false;\n  int recursion_limit_ =\n      google::protobuf::io::CodedInputStream::GetDefaultRecursionLimit();\n};\n\n// Reads a message in binary format from the given `Reader`. If successful, the\n// entire input will be consumed.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `Reader`. `Src` must support\n// `DependencyRef<Reader*, Src>`, e.g. `Reader&` (not owned),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `AnyRef<Reader*>` (maybe owned).\n//\n// Returns status:\n//  * `status.ok()`  - success (`dest` is filled)\n//  * `!status.ok()` - failure (`dest` is unspecified)\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status ParseMessage(Src&& src, google::protobuf::MessageLite& dest,\n                          ParseMessageOptions options = ParseMessageOptions());\n\n// Optimized implementation for `ReaderSpan`.\ntemplate <typename ReaderType>\nabsl::Status ParseMessage(ReaderSpan<ReaderType> src,\n                          google::protobuf::MessageLite& dest,\n                          ParseMessageOptions options = ParseMessageOptions());\n\n// Reads a message length as varint32 (at most 2G), then a message in binary\n// format with that length from the given `Reader`.\n//\n// Returns status:\n//  * `status.ok()`  - success (`dest` is filled)\n//  * `!status.ok()` - failure (`dest` is unspecified)\nabsl::Status ParseLengthPrefixedMessage(\n    Reader& src, google::protobuf::MessageLite& dest,\n    ParseMessageOptions options = ParseMessageOptions());\nabsl::Status ParseLengthPrefixedMessage(\n    LimitingReaderBase& src, google::protobuf::MessageLite& dest,\n    ParseMessageOptions options = ParseMessageOptions());\n\n// Reads a message in binary format from `src`.\n//\n// Returns status:\n//  * `status.ok()`  - success (`dest` is filled)\n//  * `!status.ok()` - failure (`dest` is unspecified)\nabsl::Status ParseMessage(BytesRef src, google::protobuf::MessageLite& dest,\n                          ParseMessageOptions options = ParseMessageOptions());\nabsl::Status ParseMessage(const Chain& src, google::protobuf::MessageLite& dest,\n                          ParseMessageOptions options = ParseMessageOptions());\nabsl::Status ParseMessage(const absl::Cord& src,\n                          google::protobuf::MessageLite& dest,\n                          ParseMessageOptions options = ParseMessageOptions());\nabsl::Status ParseMessage(CordIteratorSpan src,\n                          google::protobuf::MessageLite& dest,\n                          ParseMessageOptions options = ParseMessageOptions());\n\n// Adapts a `Reader` to a `google::protobuf::io::ZeroCopyInputStream`.\nclass ReaderInputStream : public google::protobuf::io::ZeroCopyInputStream {\n public:\n  // Creates a dummy `ReaderInputStream`. Use `Reset(src)` to initialize it to\n  // usable state.\n  ReaderInputStream() = default;\n\n  // Will read from `*src`.\n  explicit ReaderInputStream(Reader* src ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : src_(RIEGELI_EVAL_ASSERT_NOTNULL(src)) {}\n\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset() { src_ = nullptr; }\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Reader* src) {\n    src_ = RIEGELI_EVAL_ASSERT_NOTNULL(src);\n  }\n\n  bool Next(const void** data, int* size) override;\n  void BackUp(int length) override;\n  bool Skip(int length) override;\n  int64_t ByteCount() const override;\n  bool ReadCord(absl::Cord* cord, int length) override;\n\n private:\n  Reader* src_ = nullptr;\n};\n\n// Implementation details follow.\n\nnamespace parse_message_internal {\n\nabsl::Status ParseMessageImpl(Reader& src, google::protobuf::MessageLite& dest,\n                              ParseMessageOptions options);\n\nabsl::Status ParseMessageImpl(ReaderSpan<Reader> src,\n                              google::protobuf::MessageLite& dest,\n                              ParseMessageOptions options);\n\nabsl::Status ParseMessageImpl(ReaderSpan<> src,\n                              google::protobuf::MessageLite& dest,\n                              ParseMessageOptions options);\n\n}  // namespace parse_message_internal\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status ParseMessage(Src&& src, google::protobuf::MessageLite& dest,\n                                 ParseMessageOptions options) {\n  DependencyRef<Reader*, Src> src_dep(std::forward<Src>(src));\n  if (src_dep.IsOwning()) src_dep->SetReadAllHint(true);\n  absl::Status status =\n      parse_message_internal::ParseMessageImpl(*src_dep, dest, options);\n  if (src_dep.IsOwning()) {\n    if (ABSL_PREDICT_TRUE(status.ok())) src_dep->VerifyEnd();\n    if (ABSL_PREDICT_FALSE(!src_dep->Close())) status.Update(src_dep->status());\n  }\n  return status;\n}\n\ntemplate <typename ReaderType>\nabsl::Status ParseMessage(ReaderSpan<ReaderType> src,\n                          google::protobuf::MessageLite& dest,\n                          ParseMessageOptions options) {\n  return parse_message_internal::ParseMessageImpl(std::move(src), dest,\n                                                  options);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_PARSE_MESSAGE_H_\n"
  },
  {
    "path": "riegeli/messages/serialize_message.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/messages/serialize_message.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/cord_buffer.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/types/span.h\"\n#include \"google/protobuf/io/coded_stream.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/compact_string.h\"\n#include \"riegeli/base/string_utils.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/array_writer.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/chain_writer.h\"\n#include \"riegeli/bytes/cord_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/varint/varint_writing.h\"\n\nnamespace riegeli {\n\nnamespace {\n\nABSL_ATTRIBUTE_COLD inline absl::Status FailSizeOverflow(\n    const google::protobuf::MessageLite& src, size_t size) {\n  return absl::ResourceExhaustedError(\n      absl::StrCat(\"Failed to serialize message of type \", src.GetTypeName(),\n                   \" because its size must be smaller than 2GiB: \", size));\n}\n\nABSL_ATTRIBUTE_COLD inline absl::Status FailSizeOverflow(\n    const google::protobuf::MessageLite& src, Writer& dest, size_t size) {\n  return dest.AnnotateStatus(FailSizeOverflow(src, size));\n}\n\nABSL_ATTRIBUTE_COLD inline absl::Status FailSizeOverflow(\n    const google::protobuf::MessageLite& src, BackwardWriter& dest,\n    size_t size) {\n  return dest.AnnotateStatus(FailSizeOverflow(src, size));\n}\n\ninline absl::Status SerializeMessageUsingStream(\n    const google::protobuf::MessageLite& src, Writer& dest,\n    std::optional<bool> deterministic, size_t size) {\n  WriterOutputStream output_stream(&dest);\n  google::protobuf::io::CodedOutputStream coded_stream(&output_stream);\n  if (deterministic != std::nullopt) {\n    coded_stream.SetSerializationDeterministic(*deterministic);\n  }\n  src.SerializeWithCachedSizes(&coded_stream);\n  RIEGELI_ASSERT_EQ(src.ByteSizeLong(), size)\n      << src.GetTypeName() << \" was modified concurrently during serialization\";\n  // Flush `coded_stream` before checking `dest.ok()`.\n  coded_stream.Trim();\n  if (ABSL_PREDICT_FALSE(!dest.ok())) return dest.status();\n  RIEGELI_ASSERT(!coded_stream.HadError())\n      << \"Failed to serialize message of type \" << src.GetTypeName()\n      << \": SerializeWithCachedSizes() failed for an unknown reason\";\n  RIEGELI_ASSERT_EQ(IntCast<size_t>(coded_stream.ByteCount()), size)\n      << \"Byte size calculation and serialization were inconsistent. \"\n         \"This may indicate a bug in protocol buffers \"\n         \"or it may be caused by concurrent modification of \"\n      << src.GetTypeName();\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializeMessageHavingSize(\n    const google::protobuf::MessageLite& src, Writer& dest,\n    std::optional<bool> deterministic, size_t size) {\n  if (size <= kMaxBytesToCopy && deterministic == std::nullopt) {\n    // The data are small, so making a flat output is harmless.\n    // `SerializeWithCachedSizesToArray()` is faster than\n    // `SerializeWithCachedSizes()`.\n    //\n    // Avoid `nullptr` violation in `SerializeWithCachedSizesToArray()`.\n    if (size == 0) return absl::OkStatus();\n    if (ABSL_PREDICT_FALSE(!dest.Push(size))) return dest.status();\n    char* const cursor =\n        reinterpret_cast<char*>(src.SerializeWithCachedSizesToArray(\n            reinterpret_cast<uint8_t*>(dest.cursor())));\n    RIEGELI_ASSERT_EQ(src.ByteSizeLong(), size)\n        << src.GetTypeName()\n        << \" was modified concurrently during serialization\";\n    RIEGELI_ASSERT_EQ(PtrDistance(dest.cursor(), cursor), size)\n        << \"Byte size calculation and serialization were inconsistent. \"\n           \"This may indicate a bug in protocol buffers \"\n           \"or it may be caused by concurrent modification of \"\n        << src.GetTypeName();\n    dest.set_cursor(cursor);\n    return absl::OkStatus();\n  }\n  return SerializeMessageUsingStream(src, dest, deterministic, size);\n}\n\ninline absl::Status SerializeMessageHavingSize(\n    const google::protobuf::MessageLite& src, BackwardWriter& dest,\n    std::optional<bool> deterministic, size_t size) {\n  if (size <= kMaxBytesToCopy && deterministic == std::nullopt) {\n    // The data are small, so making a flat output is harmless.\n    // `SerializeWithCachedSizesToArray()` is faster than\n    // `SerializeWithCachedSizes()`.\n    //\n    // Avoid `nullptr` violation in `SerializeWithCachedSizesToArray()`.\n    if (size == 0) return absl::OkStatus();\n    if (ABSL_PREDICT_FALSE(!dest.Push(size))) return dest.status();\n    dest.move_cursor(size);\n    char* const cursor =\n        reinterpret_cast<char*>(src.SerializeWithCachedSizesToArray(\n            reinterpret_cast<uint8_t*>(dest.cursor())));\n    RIEGELI_ASSERT_EQ(src.ByteSizeLong(), size)\n        << src.GetTypeName()\n        << \" was modified concurrently during serialization\";\n    RIEGELI_ASSERT_EQ(PtrDistance(dest.cursor(), cursor), size)\n        << \"Byte size calculation and serialization were inconsistent. \"\n           \"This may indicate a bug in protocol buffers \"\n           \"or it may be caused by concurrent modification of \"\n        << src.GetTypeName();\n    return absl::OkStatus();\n  }\n  riegeli::CordWriter cord_writer;\n  if (absl::Status status =\n          SerializeMessageUsingStream(src, cord_writer, deterministic, size);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return status;\n  }\n  if (ABSL_PREDICT_FALSE(!cord_writer.Close())) return cord_writer.status();\n  if (ABSL_PREDICT_FALSE(!dest.Write(std::move(cord_writer.dest())))) {\n    return dest.status();\n  }\n  return absl::OkStatus();\n}\n\n}  // namespace\n\nnamespace serialize_message_internal {\n\nabsl::Status SerializeMessageImpl(const google::protobuf::MessageLite& src,\n                                  Writer& dest, SerializeMessageOptions options,\n                                  bool set_write_hint) {\n  RIEGELI_ASSERT(options.partial() || src.IsInitialized())\n      << \"Failed to serialize message of type \" << src.GetTypeName()\n      << \" because it is missing required fields: \"\n      << src.InitializationErrorString();\n  const size_t size = options.GetByteSize(src);\n  if (ABSL_PREDICT_FALSE(size >\n                         uint32_t{std::numeric_limits<int32_t>::max()})) {\n    return FailSizeOverflow(src, dest, size);\n  }\n  if (set_write_hint) dest.SetWriteSizeHint(size);\n  return SerializeMessageHavingSize(src, dest, options.deterministic(), size);\n}\n\nabsl::Status SerializeMessageImpl(const google::protobuf::MessageLite& src,\n                                  BackwardWriter& dest,\n                                  SerializeMessageOptions options,\n                                  bool set_write_hint) {\n  RIEGELI_ASSERT(options.partial() || src.IsInitialized())\n      << \"Failed to serialize message of type \" << src.GetTypeName()\n      << \" because it is missing required fields: \"\n      << src.InitializationErrorString();\n  const size_t size = options.GetByteSize(src);\n  if (ABSL_PREDICT_FALSE(size >\n                         uint32_t{std::numeric_limits<int32_t>::max()})) {\n    return FailSizeOverflow(src, dest, size);\n  }\n  if (set_write_hint) dest.SetWriteSizeHint(size);\n  return SerializeMessageHavingSize(src, dest, options.deterministic(), size);\n}\n\n}  // namespace serialize_message_internal\n\nabsl::Status SerializeLengthPrefixedMessage(\n    const google::protobuf::MessageLite& src, Writer& dest,\n    SerializeMessageOptions options) {\n  RIEGELI_ASSERT(options.partial() || src.IsInitialized())\n      << \"Failed to serialize message of type \" << src.GetTypeName()\n      << \" because it is missing required fields: \"\n      << src.InitializationErrorString();\n  const size_t size = options.GetByteSize(src);\n  if (ABSL_PREDICT_FALSE(size >\n                         uint32_t{std::numeric_limits<int32_t>::max()})) {\n    return FailSizeOverflow(src, dest, size);\n  }\n  if (ABSL_PREDICT_FALSE(!WriteVarint32(IntCast<uint32_t>(size), dest))) {\n    return dest.status();\n  }\n  return SerializeMessageHavingSize(src, dest, options.deterministic(), size);\n}\n\nabsl::Status SerializeMessage(const google::protobuf::MessageLite& src,\n                              std::string& dest,\n                              SerializeMessageOptions options) {\n  RIEGELI_ASSERT(options.partial() || src.IsInitialized())\n      << \"Failed to serialize message of type \" << src.GetTypeName()\n      << \" because it is missing required fields: \"\n      << src.InitializationErrorString();\n  const size_t size = options.GetByteSize(src);\n  if (ABSL_PREDICT_FALSE(size >\n                         uint32_t{std::numeric_limits<int32_t>::max()})) {\n    return FailSizeOverflow(src, size);\n  }\n  dest.clear();\n  // Avoid `nullptr` violation in `SerializeWithCachedSizesToArray()`.\n  if (size == 0) return absl::OkStatus();\n  absl::Status status;\n  riegeli::StringResizeAndOverwriteAmortized(\n      dest, size, [&](char* data, size_t size) {\n        if (options.deterministic() == std::nullopt) {\n          // Creating a string, which is necessarily flat.\n          // `SerializeWithCachedSizesToArray()` is faster than\n          // `SerializeWithCachedSizes()`.\n          char* const cursor =\n              reinterpret_cast<char*>(src.SerializeWithCachedSizesToArray(\n                  reinterpret_cast<uint8_t*>(data)));\n          RIEGELI_ASSERT_EQ(src.ByteSizeLong(), size)\n              << src.GetTypeName()\n              << \" was modified concurrently during serialization\";\n          RIEGELI_ASSERT_EQ(PtrDistance(data, cursor), size)\n              << \"Byte size calculation and serialization were inconsistent. \"\n                 \"This may indicate a bug in protocol buffers \"\n                 \"or it may be caused by concurrent modification of \"\n              << src.GetTypeName();\n          return size;\n        }\n        riegeli::ArrayWriter writer(data, size);\n        status = SerializeMessageUsingStream(src, writer,\n                                             options.deterministic(), size);\n        RIEGELI_EVAL_ASSERT(writer.Close())\n            << \"ArrayWriter has no reason to fail \"\n               \"if the size does not overflow: \"\n            << writer.status();\n        return writer.written().size();\n      });\n  return status;\n}\n\nabsl::Status SerializeMessage(const google::protobuf::MessageLite& src,\n                              CompactString& dest,\n                              SerializeMessageOptions options) {\n  RIEGELI_ASSERT(options.partial() || src.IsInitialized())\n      << \"Failed to serialize message of type \" << src.GetTypeName()\n      << \" because it is missing required fields: \"\n      << src.InitializationErrorString();\n  const size_t size = options.GetByteSize(src);\n  if (ABSL_PREDICT_FALSE(size >\n                         uint32_t{std::numeric_limits<int32_t>::max()})) {\n    return FailSizeOverflow(src, size);\n  }\n  char* const data = dest.resize(size, 0);\n  RIEGELI_ASSERT(data != nullptr) << \"CompactString data are never nullptr\";\n  if (options.deterministic() == std::nullopt) {\n    // Creating a string, which is necessarily flat.\n    // `SerializeWithCachedSizesToArray()` is faster than\n    // `SerializeWithCachedSizes()`.\n    char* const cursor = reinterpret_cast<char*>(\n        src.SerializeWithCachedSizesToArray(reinterpret_cast<uint8_t*>(data)));\n    RIEGELI_ASSERT_EQ(src.ByteSizeLong(), size)\n        << src.GetTypeName()\n        << \" was modified concurrently during serialization\";\n    RIEGELI_ASSERT_EQ(PtrDistance(data, cursor), size)\n        << \"Byte size calculation and serialization were inconsistent. \"\n           \"This may indicate a bug in protocol buffers \"\n           \"or it may be caused by concurrent modification of \"\n        << src.GetTypeName();\n    return absl::OkStatus();\n  }\n  riegeli::ArrayWriter writer(data, size);\n  const absl::Status status =\n      SerializeMessageUsingStream(src, writer, options.deterministic(), size);\n  RIEGELI_EVAL_ASSERT(writer.Close()) << \"ArrayWriter has no reason to fail \"\n                                         \"if the size does not overflow: \"\n                                      << writer.status();\n  return status;\n}\n\nabsl::Status SerializeMessage(const google::protobuf::MessageLite& src,\n                              Chain& dest, SerializeMessageOptions options) {\n  RIEGELI_ASSERT(options.partial() || src.IsInitialized())\n      << \"Failed to serialize message of type \" << src.GetTypeName()\n      << \" because it is missing required fields: \"\n      << src.InitializationErrorString();\n  const size_t size = options.GetByteSize(src);\n  if (ABSL_PREDICT_FALSE(size >\n                         uint32_t{std::numeric_limits<int32_t>::max()})) {\n    return FailSizeOverflow(src, size);\n  }\n  if (size <= kMaxBytesToCopy && options.deterministic() == std::nullopt) {\n    // The data are small, so making a flat output is harmless.\n    // `SerializeWithCachedSizesToArray()` is faster than\n    // `SerializeWithCachedSizes()`.\n    dest.Clear();\n    // Avoid `nullptr` violation in `SerializeWithCachedSizesToArray()`.\n    if (size == 0) return absl::OkStatus();\n    const absl::Span<char> buffer =\n        dest.AppendFixedBuffer(size, Chain::Options().set_size_hint(size));\n    char* const cursor =\n        reinterpret_cast<char*>(src.SerializeWithCachedSizesToArray(\n            reinterpret_cast<uint8_t*>(buffer.data())));\n    RIEGELI_ASSERT_EQ(src.ByteSizeLong(), size)\n        << src.GetTypeName()\n        << \" was modified concurrently during serialization\";\n    RIEGELI_ASSERT_EQ(PtrDistance(buffer.data(), cursor), size)\n        << \"Byte size calculation and serialization were inconsistent. \"\n           \"This may indicate a bug in protocol buffers \"\n           \"or it may be caused by concurrent modification of \"\n        << src.GetTypeName();\n    return absl::OkStatus();\n  }\n  riegeli::ChainWriter writer(&dest);\n  writer.SetWriteSizeHint(size);\n  const absl::Status status =\n      SerializeMessageUsingStream(src, writer, options.deterministic(), size);\n  RIEGELI_EVAL_ASSERT(writer.Close())\n      << \"ChainWriter has no reason to fail: \" << writer.status();\n  return status;\n}\n\nabsl::Status SerializeMessage(const google::protobuf::MessageLite& src,\n                              absl::Cord& dest,\n                              SerializeMessageOptions options) {\n  RIEGELI_ASSERT(options.partial() || src.IsInitialized())\n      << \"Failed to serialize message of type \" << src.GetTypeName()\n      << \" because it is missing required fields: \"\n      << src.InitializationErrorString();\n  const size_t size = options.GetByteSize(src);\n  if (ABSL_PREDICT_FALSE(size >\n                         uint32_t{std::numeric_limits<int32_t>::max()})) {\n    return FailSizeOverflow(src, size);\n  }\n  if (size <= kMaxBytesToCopy && options.deterministic() == std::nullopt) {\n    // The data are small, so making a flat output is harmless.\n    // `SerializeWithCachedSizesToArray()` is faster than\n    // `SerializeWithCachedSizes()`.\n    absl::CordBuffer buffer = dest.GetAppendBuffer(0, 0);\n    dest.Clear();\n    // Avoid `nullptr` violation in `SerializeWithCachedSizesToArray()`.\n    if (size == 0) return absl::OkStatus();\n    if (buffer.capacity() < size) {\n      static_assert(kMaxBytesToCopy <= absl::CordBuffer::kDefaultLimit,\n                    \"Guarantees that buffer.capacity() will be at least size\");\n      buffer = absl::CordBuffer::CreateWithDefaultLimit(size);\n    }\n    buffer.SetLength(size);\n    char* const cursor =\n        reinterpret_cast<char*>(src.SerializeWithCachedSizesToArray(\n            reinterpret_cast<uint8_t*>(buffer.data())));\n    RIEGELI_ASSERT_EQ(src.ByteSizeLong(), size)\n        << src.GetTypeName()\n        << \" was modified concurrently during serialization\";\n    RIEGELI_ASSERT_EQ(PtrDistance(buffer.data(), cursor), size)\n        << \"Byte size calculation and serialization were inconsistent. \"\n           \"This may indicate a bug in protocol buffers \"\n           \"or it may be caused by concurrent modification of \"\n        << src.GetTypeName();\n    dest.Append(std::move(buffer));\n    return absl::OkStatus();\n  }\n  riegeli::CordWriter writer(&dest);\n  writer.SetWriteSizeHint(size);\n  const absl::Status status =\n      SerializeMessageUsingStream(src, writer, options.deterministic(), size);\n  RIEGELI_EVAL_ASSERT(writer.Close())\n      << \"CordWriter has no reason to fail: \" << writer.status();\n  return status;\n}\n\nbool WriterOutputStream::Next(void** data, int* size) {\n  RIEGELI_ASSERT_NE(dest_, nullptr)\n      << \"Failed precondition of WriterOutputStream::Next(): \"\n         \"WriterOutputStream not initialized\";\n  if (ABSL_PREDICT_FALSE(dest_->pos() >=\n                         Position{std::numeric_limits<int64_t>::max()})) {\n    return false;\n  }\n  const Position max_length =\n      Position{std::numeric_limits<int64_t>::max()} - dest_->pos();\n  if (ABSL_PREDICT_FALSE(!dest_->Push())) return false;\n  *data = dest_->cursor();\n  *size = SaturatingIntCast<int>(UnsignedMin(dest_->available(), max_length));\n  dest_->move_cursor(IntCast<size_t>(*size));\n  return true;\n}\n\nvoid WriterOutputStream::BackUp(int length) {\n  RIEGELI_ASSERT_NE(dest_, nullptr)\n      << \"Failed precondition of WriterOutputStream::BackUp(): \"\n         \"WriterOutputStream not initialized\";\n  RIEGELI_ASSERT_GE(length, 0)\n      << \"Failed precondition of ZeroCopyOutputStream::BackUp(): \"\n         \"negative length\";\n  RIEGELI_ASSERT_LE(IntCast<size_t>(length), dest_->start_to_cursor())\n      << \"Failed precondition of ZeroCopyOutputStream::BackUp(): \"\n         \"length larger than the amount of buffered data\";\n  dest_->set_cursor(dest_->cursor() - length);\n}\n\nint64_t WriterOutputStream::ByteCount() const {\n  RIEGELI_ASSERT_NE(dest_, nullptr)\n      << \"Failed precondition of WriterOutputStream::ByteCount(): \"\n         \"WriterOutputStream not initialized\";\n  return SaturatingIntCast<int64_t>(dest_->pos());\n}\n\nbool WriterOutputStream::WriteCord(const absl::Cord& src) {\n  RIEGELI_ASSERT_NE(dest_, nullptr)\n      << \"Failed precondition of WriterOutputStream::WriteCord(): \"\n         \"WriterOutputStream not initialized\";\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         Position{std::numeric_limits<int64_t>::max()} -\n                             dest_->pos())) {\n    return false;\n  }\n  return dest_->Write(src);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/messages/serialize_message.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_SERIALIZE_MESSAGE_H_\n#define RIEGELI_MESSAGES_SERIALIZE_MESSAGE_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"google/protobuf/io/coded_stream.h\"\n#include \"google/protobuf/io/zero_copy_stream.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/compact_string.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nclass SerializeMessageOptions {\n public:\n  SerializeMessageOptions() noexcept {}\n\n  // If `false`, all required fields must be set. This is verified in debug\n  // mode.\n  //\n  // If `true`, missing required fields result in a partial serialized message,\n  // not having these fields.\n  //\n  // Default: `false`.\n  SerializeMessageOptions& set_partial(bool partial) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    partial_ = partial;\n    return *this;\n  }\n  SerializeMessageOptions&& set_partial(bool partial) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_partial(partial));\n  }\n  bool partial() const { return partial_; }\n\n  // If `false`, a deterministic result is not guaranteed but serialization can\n  // be faster.\n  //\n  // If `true`, a deterministic result is guaranteed (as long as the schema\n  // does not change in inappropriate ways and there are no unknown fields)\n  // but serialization can be slower.\n  //\n  // `std::nullopt` is equivalent to\n  // `google::protobuf::io::CodedOutputStream::IsDefaultSerializationDeterministic()`.\n  //\n  // This matches\n  // `google::protobuf::io::CodedOutputStream::SetSerializationDeterministic()`.\n  //\n  // Default: `std::nullopt`.\n  SerializeMessageOptions& set_deterministic(\n      std::optional<bool> deterministic) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    deterministic_ = deterministic;\n    return *this;\n  }\n  SerializeMessageOptions&& set_deterministic(\n      std::optional<bool> deterministic) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_deterministic(deterministic));\n  }\n  std::optional<bool> deterministic() const { return deterministic_; }\n\n  // If `true`, promises that `ByteSizeLong()` has been called on the message\n  // being serialized after its last modification, and that its result does not\n  // exceed `std::numeric_limits<int32_t>::max()`.\n  //\n  // This makes serialization faster by allowing to use `GetCachedSize()`\n  // instead of `ByteSizeLong()`.\n  //\n  // Default: `false`.\n  SerializeMessageOptions& set_has_cached_size(bool has_cached_size) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    has_cached_size_ = has_cached_size;\n    return *this;\n  }\n  SerializeMessageOptions&& set_has_cached_size(bool has_cached_size) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_has_cached_size(has_cached_size));\n  }\n  bool has_cached_size() const { return has_cached_size_; }\n\n  // Returns `message.ByteSizeLong()` faster.\n  //\n  // Requires that these `SerializeMessageOptions` will be used only for\n  // serializing this `message`, which will not be modified in the meantime.\n  //\n  // This consults and updates `has_cached_size()` in these\n  // `SerializeMessageOptions`.\n  size_t GetByteSize(const google::protobuf::MessageLite& message);\n\n private:\n  bool partial_ = false;\n  std::optional<bool> deterministic_;\n  bool has_cached_size_ = false;\n};\n\n// Writes the message in binary format to the given `Writer`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `Writer`. `Dest` must support\n// `DependencyRef<Writer*, Dest>`, e.g. `Writer&` (not owned),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `AnyRef<Writer*>` (maybe owned).\n//\n// Returns status:\n//  * `status.ok()`  - success (`dest` is written to)\n//  * `!status.ok()` - failure (`dest` is unspecified)\ntemplate <typename Dest,\n          std::enable_if_t<TargetRefSupportsDependency<Writer*, Dest>::value,\n                           int> = 0>\nabsl::Status SerializeMessage(\n    const google::protobuf::MessageLite& src, Dest&& dest,\n    SerializeMessageOptions options = SerializeMessageOptions());\n\n// Writes the message in binary format to the given `BackwardWriter`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `BackwardWriter`. `Dest` must support\n// `DependencyRef<BackwardWriter*, Dest>`, e.g. `BackwardWriter&` (not owned),\n// `ChainBackwardWriter<>` (owned), `std::unique_ptr<BackwardWriter>` (owned),\n// `AnyRef<BackwardWriter*>` (maybe owned).\n//\n// Returns status:\n//  * `status.ok()`  - success (`dest` is written to)\n//  * `!status.ok()` - failure (`dest` is unspecified)\ntemplate <\n    typename Dest,\n    std::enable_if_t<TargetRefSupportsDependency<BackwardWriter*, Dest>::value,\n                     int> = 0>\nabsl::Status SerializeMessage(\n    const google::protobuf::MessageLite& src, Dest&& dest,\n    SerializeMessageOptions options = SerializeMessageOptions());\n\n// Writes the message length as varint32, then the message in binary format to\n// the given `Writer`.\n//\n// Returns status:\n//  * `status.ok()`  - success (`dest` is written to)\n//  * `!status.ok()` - failure (`dest` is unspecified)\nabsl::Status SerializeLengthPrefixedMessage(\n    const google::protobuf::MessageLite& src, Writer& dest,\n    SerializeMessageOptions options = SerializeMessageOptions());\n\n// Writes the message in binary format to `dest`, clearing any existing data in\n// `dest`.\n//\n// Returns status:\n//  * `status.ok()`  - success (`dest` is filled)\n//  * `!status.ok()` - failure (`dest` is unspecified)\nabsl::Status SerializeMessage(\n    const google::protobuf::MessageLite& src, std::string& dest,\n    SerializeMessageOptions options = SerializeMessageOptions());\nabsl::Status SerializeMessage(\n    const google::protobuf::MessageLite& src, CompactString& dest,\n    SerializeMessageOptions options = SerializeMessageOptions());\nabsl::Status SerializeMessage(\n    const google::protobuf::MessageLite& src, Chain& dest,\n    SerializeMessageOptions options = SerializeMessageOptions());\nabsl::Status SerializeMessage(\n    const google::protobuf::MessageLite& src, absl::Cord& dest,\n    SerializeMessageOptions options = SerializeMessageOptions());\n\n// Adapts a `Writer` to a `google::protobuf::io::ZeroCopyOutputStream`.\nclass WriterOutputStream : public google::protobuf::io::ZeroCopyOutputStream {\n public:\n  // Creates a dummy `WriterOutputStream`. Use `Reset(dest)` to initialize it to\n  // usable state.\n  WriterOutputStream() = default;\n\n  // Will write to `*dest`.\n  explicit WriterOutputStream(Writer* dest ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : dest_(RIEGELI_EVAL_ASSERT_NOTNULL(dest)) {}\n\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset() { dest_ = nullptr; }\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Writer* dest) {\n    dest_ = RIEGELI_EVAL_ASSERT_NOTNULL(dest);\n  }\n\n  bool Next(void** data, int* size) override;\n  void BackUp(int length) override;\n  int64_t ByteCount() const override;\n  bool WriteCord(const absl::Cord& src) override;\n\n private:\n  Writer* dest_ = nullptr;\n};\n\n// Implementation details follow.\n\ninline size_t SerializeMessageOptions::GetByteSize(\n    const google::protobuf::MessageLite& message) {\n  if (has_cached_size()) return IntCast<size_t>(message.GetCachedSize());\n  const size_t size = message.ByteSizeLong();\n  if (ABSL_PREDICT_TRUE(size <=\n                        uint32_t{std::numeric_limits<int32_t>::max()})) {\n    set_has_cached_size(true);\n  }\n  return size;\n}\n\nnamespace serialize_message_internal {\n\nabsl::Status SerializeMessageImpl(const google::protobuf::MessageLite& src,\n                                  Writer& dest, SerializeMessageOptions options,\n                                  bool set_write_hint);\n\nabsl::Status SerializeMessageImpl(const google::protobuf::MessageLite& src,\n                                  BackwardWriter& dest,\n                                  SerializeMessageOptions options,\n                                  bool set_write_hint);\n\n}  // namespace serialize_message_internal\n\ntemplate <\n    typename Dest,\n    std::enable_if_t<TargetRefSupportsDependency<Writer*, Dest>::value, int>>\ninline absl::Status SerializeMessage(const google::protobuf::MessageLite& src,\n                                     Dest&& dest,\n                                     SerializeMessageOptions options) {\n  DependencyRef<Writer*, Dest> dest_dep(std::forward<Dest>(dest));\n  absl::Status status = serialize_message_internal::SerializeMessageImpl(\n      src, *dest_dep, options, dest_dep.IsOwning());\n  if (dest_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_dep->Close())) {\n      status.Update(dest_dep->status());\n    }\n  }\n  return status;\n}\n\ntemplate <typename Dest,\n          std::enable_if_t<\n              TargetRefSupportsDependency<BackwardWriter*, Dest>::value, int>>\ninline absl::Status SerializeMessage(const google::protobuf::MessageLite& src,\n                                     Dest&& dest,\n                                     SerializeMessageOptions options) {\n  DependencyRef<BackwardWriter*, Dest> dest_dep(std::forward<Dest>(dest));\n  absl::Status status = serialize_message_internal::SerializeMessageImpl(\n      src, *dest_dep, options, dest_dep.IsOwning());\n  if (dest_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_dep->Close())) {\n      status.Update(dest_dep->status());\n    }\n  }\n  return status;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_SERIALIZE_MESSAGE_H_\n"
  },
  {
    "path": "riegeli/messages/serialized_message_assembler.cc",
    "content": "// Copyright 2026 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/messages/serialized_message_assembler.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <string>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/container/inlined_vector.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/messages/field_handler_map.h\"\n#include \"riegeli/messages/serialized_message_reader.h\"\n#include \"riegeli/messages/serialized_message_writer.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nSerializedMessageAssembler::Builder::Builder(Builder&& that) noexcept = default;\n\nSerializedMessageAssembler::Builder&\nSerializedMessageAssembler::Builder::operator=(Builder&& that) noexcept =\n    default;\n\nSerializedMessageAssembler::Builder::~Builder() = default;\n\nvoid SerializedMessageAssembler::Builder::Reset() {\n  next_parent_for_add_ = {/*raw=*/0};\n  next_field_for_remove_ = {/*raw=*/0};\n  RIEGELI_ASSERT_EQ(root_builder_.field_for_remove.raw,\n                    FieldForRemove::kUnregistered);\n  root_builder_.parent_for_add = ParentForAdd();\n  root_builder_.children.clear();\n}\n\nSerializedMessageAssembler::ParentForAdd\nSerializedMessageAssembler::Builder::RegisterParentForAdd(\n    ParentPath parent_path) {\n  RegisteredFieldBuilder& parent_builder = RegisterParentInternal(parent_path);\n  if (parent_builder.parent_for_add.raw == ParentForAdd::kUnregistered) {\n    parent_builder.parent_for_add.raw = next_parent_for_add_.raw++;\n  }\n  return parent_builder.parent_for_add;\n}\n\nSerializedMessageAssembler::FieldForRemove\nSerializedMessageAssembler::Builder::RegisterFieldForRemove(\n    ParentPath parent_path, int field_number) {\n  RegisteredFieldBuilder& parent_builder = RegisterParentInternal(parent_path);\n  RegisteredFieldBuilder& field_builder =\n      parent_builder.children.try_emplace(field_number).first->second;\n  if (field_builder.field_for_remove.raw == FieldForRemove::kUnregistered) {\n    field_builder.field_for_remove.raw = next_field_for_remove_.raw++;\n  }\n  return field_builder.field_for_remove;\n}\n\nSerializedMessageAssembler::RegisteredFieldBuilder&\nSerializedMessageAssembler::Builder::RegisterParentInternal(\n    ParentPath parent_path) {\n  RegisteredFieldBuilder* parent_builder = &root_builder_;\n  for (const int field_number : parent_path) {\n    parent_builder =\n        &parent_builder->children.try_emplace(field_number).first->second;\n  }\n  return *parent_builder;\n}\n\nSerializedMessageAssembler::Session::Session(Session&& that) noexcept\n    : fields_to_add_(std::move(that.fields_to_add_)),\n      fields_to_remove_(std::exchange(that.fields_to_remove_, {})) {}\n\nSerializedMessageAssembler::Session&\nSerializedMessageAssembler::Session::operator=(Session&& that) noexcept {\n  fields_to_add_ = std::move(that.fields_to_add_);\n  fields_to_remove_ = std::exchange(that.fields_to_remove_, {});\n  return *this;\n}\n\nSerializedMessageAssembler::Session::~Session() = default;\n\nvoid SerializedMessageAssembler::Session::Reset() {\n  for (FieldValues& field_values : fields_to_add_) {\n    field_values.clear();\n  }\n  fields_to_remove_.clear();\n}\nvoid SerializedMessageAssembler::Session::Reset(\n    const SerializedMessageAssembler& assembler) {\n  Reset();\n  fields_to_add_.resize(assembler.num_fields_for_add());\n  fields_to_remove_.resize(assembler.num_fields_for_remove());\n}\n\nSerializedMessageAssembler::SerializedMessageAssembler(\n    SerializedMessageAssembler&& that) noexcept = default;\n\nSerializedMessageAssembler& SerializedMessageAssembler::operator=(\n    SerializedMessageAssembler&& that) noexcept = default;\n\nSerializedMessageAssembler::~SerializedMessageAssembler() = default;\n\nSerializedMessageAssembler::SerializedMessageAssembler(Builder&& builder)\n    : num_fields_for_add_(builder.next_parent_for_add_),\n      num_fields_for_remove_(builder.next_field_for_remove_),\n      root_(Build(\n          /*field_number=*/0, std::move(builder.root_builder_))) {}\n\nvoid SerializedMessageAssembler::Reset() {\n  num_fields_for_add_ = {/*raw=*/0};\n  num_fields_for_remove_ = {/*raw=*/0};\n  RIEGELI_ASSERT_EQ(root_.field_number, 0);\n  root_.parent_for_add = ParentForAdd();\n  root_.submessages.clear();\n  root_.handlers.Reset();\n}\n\nvoid SerializedMessageAssembler::Reset(Builder&& builder) {\n  num_fields_for_add_ = builder.next_parent_for_add_;\n  num_fields_for_remove_ = builder.next_field_for_remove_;\n  RIEGELI_ASSERT_EQ(root_.field_number, 0);\n  root_.submessages.clear();\n  Build(\n      /*field_number=*/0, std::move(builder.root_builder_), root_);\n}\n\nstruct SerializedMessageAssembler::LeafHandler {\n  static constexpr int kFieldNumber = kUnboundFieldNumber;\n\n  absl::Status HandleVarint(\n      uint64_t repr,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<FieldValues> fields_to_add,\n      absl::Span<const bool> fields_to_remove,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<bool> submessages_rewritten,\n      SerializedMessageWriter& message_writer) const {\n    if (field_for_remove.raw < fields_to_remove.size() &&\n        fields_to_remove[field_for_remove.raw]) {\n      return absl::OkStatus();\n    }\n    return message_writer.WriteUInt64(field_number, repr);\n  }\n\n  absl::Status HandleFixed32(\n      uint32_t repr,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<FieldValues> fields_to_add,\n      absl::Span<const bool> fields_to_remove,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<bool> submessages_rewritten,\n      SerializedMessageWriter& message_writer) const {\n    if (field_for_remove.raw < fields_to_remove.size() &&\n        fields_to_remove[field_for_remove.raw]) {\n      return absl::OkStatus();\n    }\n    return message_writer.WriteFixed32(field_number, repr);\n  }\n\n  absl::Status HandleFixed64(\n      uint64_t repr,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<FieldValues> fields_to_add,\n      absl::Span<const bool> fields_to_remove,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<bool> submessages_rewritten,\n      SerializedMessageWriter& message_writer) const {\n    if (field_for_remove.raw < fields_to_remove.size() &&\n        fields_to_remove[field_for_remove.raw]) {\n      return absl::OkStatus();\n    }\n    return message_writer.WriteFixed64(field_number, repr);\n  }\n\n  absl::Status HandleLengthDelimitedFromReader(\n      ReaderSpan<> repr,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<FieldValues> fields_to_add,\n      absl::Span<const bool> fields_to_remove,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<bool> submessages_rewritten,\n      SerializedMessageWriter& message_writer) const {\n    if (field_for_remove.raw < fields_to_remove.size() &&\n        fields_to_remove[field_for_remove.raw]) {\n      return SkipLengthDelimitedFromReader(std::move(repr));\n    }\n    return message_writer.WriteString(field_number, std::move(repr));\n  }\n\n  absl::Status HandleLengthDelimitedFromCord(\n      CordIteratorSpan repr, ABSL_ATTRIBUTE_UNUSED std::string& scratch,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<FieldValues> fields_to_add,\n      absl::Span<const bool> fields_to_remove,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<bool> submessages_rewritten,\n      SerializedMessageWriter& message_writer) const {\n    if (field_for_remove.raw < fields_to_remove.size() &&\n        fields_to_remove[field_for_remove.raw]) {\n      return SkipLengthDelimitedFromCord(std::move(repr));\n    }\n    return message_writer.WriteString(field_number, std::move(repr));\n  }\n\n  absl::Status HandleLengthDelimitedFromString(\n      absl::string_view repr,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<FieldValues> fields_to_add,\n      absl::Span<const bool> fields_to_remove,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<bool> submessages_rewritten,\n      SerializedMessageWriter& message_writer) const {\n    if (field_for_remove.raw < fields_to_remove.size() &&\n        fields_to_remove[field_for_remove.raw]) {\n      return absl::OkStatus();\n    }\n    return message_writer.WriteString(field_number, repr);\n  }\n\n  absl::Status HandleBeginGroup(\n      ABSL_ATTRIBUTE_UNUSED absl::Span<FieldValues> fields_to_add,\n      absl::Span<const bool> fields_to_remove,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<bool> submessages_rewritten,\n      SerializedMessageWriter& message_writer) const {\n    if (field_for_remove.raw < fields_to_remove.size() &&\n        fields_to_remove[field_for_remove.raw]) {\n      return absl::OkStatus();\n    }\n    return message_writer.OpenGroup(field_number);\n  }\n\n  absl::Status HandleEndGroup(\n      ABSL_ATTRIBUTE_UNUSED absl::Span<FieldValues> fields_to_add,\n      absl::Span<const bool> fields_to_remove,\n      ABSL_ATTRIBUTE_UNUSED absl::Span<bool> submessages_rewritten,\n      SerializedMessageWriter& message_writer) const {\n    if (field_for_remove.raw < fields_to_remove.size() &&\n        fields_to_remove[field_for_remove.raw]) {\n      return absl::OkStatus();\n    }\n    return message_writer.CloseGroup(field_number);\n  }\n\n  int field_number;\n  FieldForRemove field_for_remove;\n};\n\nstruct SerializedMessageAssembler::SubmessageHandler {\n  static constexpr int kFieldNumber = kUnboundFieldNumber;\n\n  absl::Status HandleLengthDelimitedFromReader(\n      ReaderSpan<> repr, absl::Span<FieldValues> fields_to_add,\n      absl::Span<const bool> fields_to_remove,\n      absl::Span<bool> submessages_rewritten,\n      SerializedMessageWriter& message_writer) const {\n    if (field_for_remove.raw < fields_to_remove.size() &&\n        fields_to_remove[field_for_remove.raw]) {\n      return SkipLengthDelimitedFromReader(std::move(repr));\n    }\n    submessages_rewritten[submessage_index_in_parent] = true;\n    message_writer.OpenLengthDelimited();\n    if (absl::Status status =\n            RewriteFields(fields_to_add, fields_to_remove, *message,\n                          std::move(repr), message_writer);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n    return message_writer.CloseLengthDelimited(message->field_number);\n  }\n\n  absl::Status HandleLengthDelimitedFromCord(\n      CordIteratorSpan repr, ABSL_ATTRIBUTE_UNUSED std::string& scratch,\n      absl::Span<FieldValues> fields_to_add,\n      absl::Span<const bool> fields_to_remove,\n      absl::Span<bool> submessages_rewritten,\n      SerializedMessageWriter& message_writer) const {\n    if (field_for_remove.raw < fields_to_remove.size() &&\n        fields_to_remove[field_for_remove.raw]) {\n      return SkipLengthDelimitedFromCord(std::move(repr));\n    }\n    submessages_rewritten[submessage_index_in_parent] = true;\n    message_writer.OpenLengthDelimited();\n    if (absl::Status status =\n            RewriteFields(fields_to_add, fields_to_remove, *message,\n                          std::move(repr), message_writer);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n    return message_writer.CloseLengthDelimited(message->field_number);\n  }\n\n  absl::Status HandleLengthDelimitedFromString(\n      absl::string_view repr, absl::Span<FieldValues> fields_to_add,\n      absl::Span<const bool> fields_to_remove,\n      absl::Span<bool> submessages_rewritten,\n      SerializedMessageWriter& message_writer) const {\n    if (field_for_remove.raw < fields_to_remove.size() &&\n        fields_to_remove[field_for_remove.raw]) {\n      return absl::OkStatus();\n    }\n    submessages_rewritten[submessage_index_in_parent] = true;\n    message_writer.OpenLengthDelimited();\n    if (absl::Status status = RewriteFields(fields_to_add, fields_to_remove,\n                                            *message, repr, message_writer);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n    return message_writer.CloseLengthDelimited(message->field_number);\n  }\n\n  SubmessageIndex submessage_index_in_parent;\n  FieldForRemove field_for_remove;\n  const RegisteredMessage* message;\n};\n\nvoid SerializedMessageAssembler::Build(int field_number,\n                                       RegisteredFieldBuilder&& message_builder,\n                                       RegisteredMessage& message) {\n  RIEGELI_ASSERT(message.submessages.empty())\n      << \"Failed precondition of SerializedMessageAssembler::Build(): \"\n         \"message not empty\";\n  message.field_number = field_number;\n  message.parent_for_add = message_builder.parent_for_add;\n  size_t num_submessages = 0;\n  RewriteHandlers::Builder fields_builder;\n  for (auto& entry : message_builder.children) {\n    if (entry.second.parent_for_add.raw != ParentForAdd::kUnregistered ||\n        !entry.second.children.empty()) {\n      ++num_submessages;\n    }\n  }\n  message.submessages.resize(num_submessages);\n  size_t submessage_index = 0;\n  for (auto& entry : message_builder.children) {\n    if (entry.second.parent_for_add.raw != ParentForAdd::kUnregistered ||\n        !entry.second.children.empty()) {\n      RegisteredMessage* const submessage =\n          &message.submessages[submessage_index];\n      fields_builder.RegisterField(\n          entry.first,\n          SubmessageHandler{IntCast<SubmessageIndex>(submessage_index),\n                            entry.second.field_for_remove, submessage});\n      Build(entry.first, std::move(entry.second), *submessage);\n      ++submessage_index;\n    } else {\n      fields_builder.RegisterField(\n          entry.first, LeafHandler{entry.first, entry.second.field_for_remove});\n    }\n  }\n  RIEGELI_ASSERT_EQ(submessage_index, num_submessages)\n      << \"The whole submessages array should have been filled\";\n  riegeli::Reset(message.handlers, std::move(fields_builder));\n}\n\ntemplate absl::Status SerializedMessageAssembler::RewriteFields(\n    absl::Span<FieldValues> fields_to_add,\n    absl::Span<const bool> fields_to_remove, const RegisteredMessage& message,\n    ReaderSpan<>&& src, SerializedMessageWriter& message_writer);\ntemplate absl::Status SerializedMessageAssembler::RewriteFields(\n    absl::Span<FieldValues> fields_to_add,\n    absl::Span<const bool> fields_to_remove, const RegisteredMessage& message,\n    CordIteratorSpan&& src, SerializedMessageWriter& message_writer);\ntemplate absl::Status SerializedMessageAssembler::RewriteFields(\n    absl::Span<FieldValues> fields_to_add,\n    absl::Span<const bool> fields_to_remove, const RegisteredMessage& message,\n    absl::string_view&& src, SerializedMessageWriter& message_writer);\n\nabsl::Status SerializedMessageAssembler::WriteNewFields(\n    absl::Span<FieldValues> fields_to_add, const RegisteredMessage& message,\n    absl::Span<const bool> submessages_rewritten,\n    SerializedMessageWriter& message_writer) {\n  const absl::Span<const RegisteredMessage> submessages = message.submessages;\n  RIEGELI_ASSERT_EQ(submessages.size(), submessages_rewritten.size());\n\n  // Create submessages for new fields and write these fields.\n  for (size_t submessage_index = 0; submessage_index < submessages.size();\n       ++submessage_index) {\n    if (submessages_rewritten[submessage_index]) continue;\n    const RegisteredMessage& submessage = submessages[submessage_index];\n    message_writer.OpenLengthDelimited();\n    if (absl::Status status = WriteNewFields(\n            fields_to_add, submessage,\n            SubmessageSet(submessage.submessages.size()), message_writer);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n    if (absl::Status status = message_writer.CloseOptionalLengthDelimited(\n            submessage.field_number);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n  }\n\n  // Write new fields directly to this message.\n  if (message.parent_for_add.raw < fields_to_add.size()) {\n    FieldValues& field_values = fields_to_add[message.parent_for_add.raw];\n    for (FieldValue& field_value : field_values) {\n      if (absl::Status status = std::move(field_value)(message_writer);\n          ABSL_PREDICT_FALSE(!status.ok())) {\n        return status;\n      }\n    }\n    field_values.clear();\n  }\n\n  return absl::OkStatus();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/messages/serialized_message_assembler.h",
    "content": "// Copyright 2026 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_SERIALIZED_MESSAGE_ASSEMBLER_H_\n#define RIEGELI_MESSAGES_SERIALIZED_MESSAGE_ASSEMBLER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <functional>\n#include <limits>\n#include <string>\n#include <type_traits>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/container/inlined_vector.h\"\n#include \"absl/container/linked_hash_map.h\"\n#include \"absl/functional/any_invocable.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/bytes/chain_writer.h\"\n#include \"riegeli/bytes/cord_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/string_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/messages/field_copier.h\"\n#include \"riegeli/messages/field_handler_map.h\"\n#include \"riegeli/messages/serialized_message_reader.h\"\n#include \"riegeli/messages/serialized_message_writer.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `SerializedMessageAssembler` writes a serialized proto message based on a\n// serialized base message and added and/or removed fields.\n//\n// Parents of added fields are created implicitly if they are not present in the\n// base message.\n//\n// A `SerializedMessageAssembler` object maintains a structure of fields to be\n// added and/or removed, which can be efficiently reused for multiple messages.\n//\n// Usage:\n// ```\n//   // Register message structure.\n//   riegeli::SerializedMessageAssembler::Builder message_assembler_builder;\n//   for (const auto& field_name : fields_to_add) {\n//     field_name.parent_for_add =\n//         message_assembler_builder.RegisterParentForAdd(\n//             field_name.parent_path);\n//   }\n//   for (const auto& field_name : fields_to_remove) {\n//     field_name.parent_for_remove =\n//         message_assembler_builder.RegisterFieldForRemove(\n//             field_name.parent_path, field_name.field_number);\n//   }\n//   SerializedMessageAssembler message_assembler(\n//       std::move(message_assembler_builder));\n//\n//   // Assemble messages.\n//   for (auto& message : messages) {\n//     riegeli::SerializedMessageAssembler::Session session(message_assembler);\n//     for (const auto& field : message.fields_to_add) {\n//       session.AddField(\n//           field.parent_for_add,\n//           [field_number = field.field_number, value = field.value](\n//               riegeli::SerializedMessageWriter& message_writer) mutable {\n//             // Example for a string field:\n//             return message_writer.WriteString(field_number,\n//                                               std::move(value));\n//           });\n//     }\n//     for (const auto& field : message.fields_to_remove) {\n//       session.RemoveField(field.parent_for_remove);\n//     }\n//     std::move(session).WriteMessage(message_assembler, base_message,\n//                                     message.serialized);\n//   }\n// ```\n//\n// Alternatively, the stages of registration and assembly can be interleaved.\n// This is reasonable for assembling a single message. This would be inefficient\n// for multiple messages.\n// ```\n//   riegeli::SerializedMessageAssembler::Builder message_assembler_builder;\n//   riegeli::SerializedMessageAssembler::Session session;\n//   for (const auto& field : fields_to_add) {\n//     session.AddField(\n//         message_assembler_builder.RegisterParentForAdd(field.parent_path),\n//         [field_number = field.field_number, value = field.value](\n//             riegeli::SerializedMessageWriter& message_writer) mutable {\n//           // Example for a string field:\n//           return message_writer.WriteString(field_number, std::move(value));\n//         });\n//   }\n//   for (const auto& field : fields_to_remove) {\n//     session.RemoveField(message_assembler_builder.RegisterFieldForRemove(\n//         field.parent_path, field.field_number));\n//   }\n//   SerializedMessageAssembler message_assembler(\n//       std::move(message_assembler_builder));\n//   std::move(session).WriteMessage(message_assembler, base_message,\n//                                   serialized_message);\n// ```\nclass SerializedMessageAssembler {\n public:\n  // A sequence of field numbers identifying a length-delimited field.\n  //\n  // Each element of a `ParentPath` should correspond to a singular submessage\n  // field contained in the previous message, except that the last element may\n  // correspond to a singular `string` or `bytes` field, or to a packed repeated\n  // field.\n  //\n  // An empty `ParentPath` indicates the root message.\n  using ParentPath = absl::Span<const int>;\n\n  // Identifies the parent of fields to be added, binding\n  // `Builder::RegisterParentForAdd()` with `Session::AddField()`.\n  // Usually treated as an opaque type.\n  struct ParentForAdd {\n    static constexpr uint32_t kUnregistered =\n        std::numeric_limits<uint32_t>::max();\n    uint32_t raw = kUnregistered;\n  };\n\n  // Identifies a field to be removed, binding\n  // `Builder::RegisterFieldForRemove()` with `Session::RemoveField()`.\n  // Usually treated as an opaque type.\n  struct FieldForRemove {\n    static constexpr uint32_t kUnregistered =\n        std::numeric_limits<uint32_t>::max();\n    uint32_t raw = kUnregistered;\n  };\n\n  // Prepares a `SerializedMessageAssembler`.\n  class Builder;\n\n  // Maintains field values during writing a single message.\n  class Session;\n\n  // Creates an empty `SerializedMessageAssembler`. Designed for `Reset()`.\n  SerializedMessageAssembler() = default;\n\n  // Builds a `SerializedMessageAssembler`.\n  explicit SerializedMessageAssembler(Builder&& builder);\n\n  SerializedMessageAssembler(SerializedMessageAssembler&& that) noexcept;\n  SerializedMessageAssembler& operator=(\n      SerializedMessageAssembler&& that) noexcept;\n\n  ~SerializedMessageAssembler();\n\n  // Makes `*this` equivalent to a newly constructed\n  // `SerializedMessageAssembler`.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Builder&& builder);\n\n private:\n  // A value to be written to a `SerializedMessageWriter`. Its type varies,\n  // so it is represented as an invocable.\n  using FieldValue = absl::AnyInvocable<absl::Status(\n      SerializedMessageWriter& message_writer) &&>;\n  // Values to be written to a `SerializedMessageWriter` under one parent.\n  using FieldValues = std::vector<FieldValue>;\n\n  // Which submessages of a given message are present in the base message,\n  // in the order corresponding to `RegisteredMessage::submessages`.\n  using SubmessageSet = absl::InlinedVector<bool, 8>;\n  // An index into `SubmessageSet`.\n  using SubmessageIndex = uint32_t;\n\n  // Field handlers for reading the base message.\n  using RewriteHandlers = FieldHandlerMap<\n      // Remaining field values to write, indexed by `ParentForAdd`.\n      const absl::Span<FieldValues>,\n      // Fields to remove, indexed by `FieldForRemove`.\n      const absl::Span<const bool>,\n      // Which submessages of this message have been rewritten, in the order\n      // corresponding to `RegisteredMessage::submessages`. Submessages absent\n      // in the base message will be created at the end of the message.\n      const absl::Span<bool>,\n      // The destination message writer.\n      SerializedMessageWriter>;\n\n  // During registration, maintains information about a field or root.\n  struct RegisteredFieldBuilder {\n    ParentForAdd parent_for_add;\n    FieldForRemove field_for_remove;  // Not used in root.\n    absl::linked_hash_map<int, RegisteredFieldBuilder> children;\n  };\n\n  // During assembly, maintains information about a message, including root.\n  struct RegisteredMessage {\n    RegisteredMessage() noexcept : field_number(0) {}\n    explicit RegisteredMessage(int field_number) : field_number(field_number) {}\n\n    int field_number;  // Not used in root.\n    ParentForAdd parent_for_add;\n    // In registration order. Must not be reallocated after initialization.\n    std::vector<RegisteredMessage> submessages;\n    // `handlers` may refer to elements of `submessages`.\n    RewriteHandlers handlers;\n  };\n\n  struct LeafHandler;\n  struct SubmessageHandler;\n\n  static RegisteredMessage Build(int field_number,\n                                 RegisteredFieldBuilder&& message_builder) {\n    RegisteredMessage message;\n    Build(field_number, std::move(message_builder), message);\n    return message;\n  }\n\n  static void Build(int field_number, RegisteredFieldBuilder&& message_builder,\n                    RegisteredMessage& message);\n\n  // Copies fields, rewriting submessages present in the base message.\n  template <typename Src>\n  static absl::Status RewriteFields(absl::Span<FieldValues> fields_to_add,\n                                    absl::Span<const bool> fields_to_remove,\n                                    const RegisteredMessage& message, Src&& src,\n                                    SerializedMessageWriter& message_writer);\n\n  // Writes fields from submessages which were not present in the base message,\n  // and fields added directly to the message. Creates required parent\n  // submessages if they turn out to be non-empty.\n  static absl::Status WriteNewFields(\n      absl::Span<FieldValues> fields_to_add, const RegisteredMessage& message,\n      absl::Span<const bool> submessages_rewritten,\n      SerializedMessageWriter& message_writer);\n\n  size_t num_fields_for_add() const { return size_t{num_fields_for_add_.raw}; }\n  size_t num_fields_for_remove() const {\n    return size_t{num_fields_for_remove_.raw};\n  }\n\n  ParentForAdd num_fields_for_add_ = {/*raw=*/0};\n  FieldForRemove num_fields_for_remove_ = {/*raw=*/0};\n  RegisteredMessage root_;\n};\n\nclass SerializedMessageAssembler::Builder {\n public:\n  // Creates an empty `SerializedMessageAssembler::Builder`.\n  Builder() = default;\n\n  Builder(Builder&& that) noexcept;\n  Builder& operator=(Builder&& that) noexcept;\n\n  ~Builder();\n\n  // Makes `*this` equivalent to a newly constructed\n  // `SerializedMessageAssembler::Builder`.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n\n  // Registers interest in writing fields under this parent.\n  //\n  // If the parent message is present in the base message (and not removed),\n  // and/or is used for multiple fields, then its contents are merged. If it is\n  // present multiple times in the base message, then contents are merged to its\n  // first occurrence.\n  //\n  // Returns a `ParentForAdd` to be passed to `Session::AddField()`.\n  //\n  // Can be called multiple times with the same `parent_path`, returning the\n  // same `ParentForAdd` each time.\n  ParentForAdd RegisterParentForAdd(ParentPath parent_path);\n\n  // Registers interest in removing this field from the base message.\n  //\n  // Returns a `FieldForRemove` to be passed to `Session::RemoveField()`.\n  //\n  // Can be called multiple times with the same `parent_path` and\n  // `field_number`, returning the same `FieldForRemove` each time.\n  FieldForRemove RegisterFieldForRemove(ParentPath parent_path,\n                                        int field_number);\n\n private:\n  friend class SerializedMessageAssembler;  // For member variables.\n\n  RegisteredFieldBuilder& RegisterParentInternal(ParentPath parent_path);\n\n  ParentForAdd next_parent_for_add_ = {/*raw=*/0};\n  FieldForRemove next_field_for_remove_ = {/*raw=*/0};\n  RegisteredFieldBuilder root_builder_;\n};\n\nclass SerializedMessageAssembler::Session {\n private:\n  template <typename Src>\n  struct IsSource\n      : std::disjunction<TargetRefSupportsDependency<Reader*, Src>,\n                         std::is_convertible<Src&&, BytesRef>,\n                         std::is_convertible<Src&&, Chain>,\n                         std::is_convertible<Src&&, const absl::Cord&>,\n                         std::is_convertible<Src&&, CordIteratorSpan>> {};\n\n  template <typename Dest>\n  struct IsDestination\n      : std::disjunction<TargetRefSupportsDependency<Writer*, Dest>,\n                         std::is_convertible<Dest&&, std::string&>,\n                         std::is_convertible<Dest&&, Chain&>,\n                         std::is_convertible<Dest&&, absl::Cord&>> {};\n\n public:\n  // The constructor to use when the `SerializedMessageAssembler` does not have\n  // registered fields yet. In this case there is no benefit in passing the\n  // `SerializedMessageAssembler` to the constructor.\n  Session() = default;\n\n  // The constructor to use when the `SerializedMessageAssembler` has fields\n  // already registered. In this case this constructor is more efficient than\n  // the default constructor.\n  explicit Session(const SerializedMessageAssembler& assembler)\n      : fields_to_add_(assembler.num_fields_for_add()),\n        fields_to_remove_(assembler.num_fields_for_remove()) {}\n\n  // The source `Session` is left empty.\n  Session(Session&& that) noexcept;\n  Session& operator=(Session&& that) noexcept;\n\n  ~Session();\n\n  // Makes `*this` equivalent to a newly constructed `Session`.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset();\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      const SerializedMessageAssembler& assembler);\n\n  // Adds the value of a field identified by `parent_for_add`, which was\n  // returned by `Builder::RegisterParentForAdd()`.\n  //\n  // `action` will be called exactly once by a successful `WriteMessage()`,\n  // with a `SerializedMessageWriter&` parameter. It should write the field\n  // there and return `absl::Status`.\n  //\n  // If the parent message is present in the base message (and not removed),\n  // and/or `AddField()` is called multiple times with the same\n  // `parent_for_add`, then the contents of the parent message are merged. If it\n  // is present multiple times in the base message, then contents are merged to\n  // its first occurrence.\n  //\n  // If multiple fields are written to a parent message, then they are written\n  // separately rather than merged.\n  //\n  // The consequences of having multiple occurrences of a field follow the proto\n  // semantics of merging:\n  //\n  //  * A singular non-message field overrides the previous value.\n  //  * A singular message field is merged with the previous value.\n  //  * A repeated field has a new element added.\n  //\n  // In the case of a singular field or a packed repeated field, this makes the\n  // serialized representation non-canonical or with redundant overridden data.\n  // In these cases it is better to avoid having multiple occurrences. Even\n  // though regular proto parsing merges these occurrences, this can be\n  // confusing for non-standard parsers, and the serialized representation is\n  // larger than necessary.\n  //\n  // It follows that there are two ways to write a length-delimited field:\n  //\n  //  1. Specify its parent in `RegisterParentForAdd()`.\n  //     Write the whole field in `AddField()` action.\n  //\n  //  2. Specify the field itself in `RegisterParentForAdd()`.\n  //     Write its contents only in `AddField()` action.\n  //\n  // If the field is not present in the base message (or removed), and\n  // `AddField()` is called once, then both ways have the same effect.\n  // The first way is more efficient, because the length of field contents\n  // is usually known in advance, in which case the contents can be written\n  // directly instead of gathering them separately to compute their length\n  // when they are complete.\n  //\n  // Otherwise, the first way writes a separate field, which may undergo\n  // implicit merging during parsing, while the second way merges it in the\n  // serialized message.\n  template <\n      typename Action,\n      std::enable_if_t<std::is_invocable_v<Action&&, SerializedMessageWriter&>,\n                       int> = 0>\n  void AddField(ParentForAdd parent_for_add, Action&& action);\n\n  // Removes from the base message the field identified by `parent_for_remove`,\n  // which was returned by `Builder::RegisterFieldForRemove()`.\n  //\n  // If the field occurs multiple times, all its occurrences are removed.\n  //\n  // If `RemoveField()` is called multiple times with the same\n  // `parent_for_remove`, the effect is the same as a single call.\n  //\n  // This does not apply to fields added with `AddField()`, only to fields from\n  // the base message.\n  void RemoveField(FieldForRemove parent_for_remove);\n\n  // Writes a serialized message based on the serialized message in `base_src`\n  // (empty by default), adding fields specified by `Session::AddField()`,\n  // and removing fields specified by `Session::RemoveField()`.\n  //\n  // The `Session` is left empty.\n  template <\n      typename BaseSrc, typename Dest,\n      std::enable_if_t<\n          std::conjunction_v<IsSource<BaseSrc>, IsDestination<Dest>>, int> = 0>\n  absl::Status WriteMessage(const SerializedMessageAssembler& message_assembler,\n                            BaseSrc&& base_src, Dest&& dest) &&;\n  template <typename Dest,\n            std::enable_if_t<IsDestination<Dest>::value, int> = 0>\n  absl::Status WriteMessage(const SerializedMessageAssembler& message_assembler,\n                            Dest&& dest) &&;\n\n private:\n  template <\n      typename Dest,\n      std::enable_if_t<TargetRefSupportsDependency<Writer*, Dest>::value, int>>\n  static Dest&& MakeWriter(Dest&& dest) {\n    return std::forward<Dest>(dest);\n  }\n  static StringWriter<> MakeWriter(std::string& dest) {\n    return StringWriter(&dest);\n  }\n  static ChainWriter<> MakeWriter(Chain& dest) { return ChainWriter(&dest); }\n  static CordWriter<> MakeWriter(absl::Cord& dest) { return CordWriter(&dest); }\n\n  template <typename BaseSrc, typename Dest>\n  absl::Status WriteMessageInternal(\n      const SerializedMessageAssembler& message_assembler, BaseSrc&& base_src,\n      Dest&& dest) &&;\n  template <typename Dest>\n  absl::Status WriteMessageInternal(\n      const SerializedMessageAssembler& message_assembler, Dest&& dest) &&;\n\n  // Indexed by `ParentForAdd`.\n  std::vector<FieldValues> fields_to_add_;\n  // Indexed by `FieldForRemove`.\n  absl::InlinedVector<bool, 8> fields_to_remove_;\n};\n\n// Implementation details follow.\n\ntemplate <typename Action,\n          std::enable_if_t<\n              std::is_invocable_v<Action&&, SerializedMessageWriter&>, int>>\ninline void SerializedMessageAssembler::Session::AddField(\n    ParentForAdd parent_for_add, Action&& action) {\n  if (ABSL_PREDICT_FALSE(parent_for_add.raw >= fields_to_add_.size())) {\n    fields_to_add_.resize(parent_for_add.raw + 1);\n  }\n  fields_to_add_[parent_for_add.raw].emplace_back(std::forward<Action>(action));\n}\n\ninline void SerializedMessageAssembler::Session::RemoveField(\n    FieldForRemove parent_for_remove) {\n  if (ABSL_PREDICT_FALSE(parent_for_remove.raw >= fields_to_remove_.size())) {\n    fields_to_remove_.resize(parent_for_remove.raw + 1);\n  }\n  fields_to_remove_[parent_for_remove.raw] = true;\n}\n\ntemplate <typename BaseSrc, typename Dest,\n          std::enable_if_t<\n              std::conjunction_v<\n                  SerializedMessageAssembler::Session::IsSource<BaseSrc>,\n                  SerializedMessageAssembler::Session::IsDestination<Dest>>,\n              int>>\nabsl::Status SerializedMessageAssembler::Session::WriteMessage(\n    const SerializedMessageAssembler& message_assembler, BaseSrc&& base_src,\n    Dest&& dest) && {\n  return std::move(*this).WriteMessageInternal(\n      message_assembler, std::forward<BaseSrc>(base_src),\n      MakeWriter(std::forward<Dest>(dest)));\n}\n\ntemplate <\n    typename Dest,\n    std::enable_if_t<\n        SerializedMessageAssembler::Session::IsDestination<Dest>::value, int>>\nabsl::Status SerializedMessageAssembler::Session::WriteMessage(\n    const SerializedMessageAssembler& message_assembler, Dest&& dest) && {\n  return std::move(*this).WriteMessageInternal(\n      message_assembler, MakeWriter(std::forward<Dest>(dest)));\n}\n\ntemplate <typename BaseSrc, typename Dest>\ninline absl::Status SerializedMessageAssembler::Session::WriteMessageInternal(\n    const SerializedMessageAssembler& message_assembler, BaseSrc&& base_src,\n    Dest&& dest) && {\n  DependencyRef<Writer*, Dest> dest_dep(std::forward<Dest>(dest));\n  SerializedMessageWriter message_writer(dest_dep.get());\n\n  absl::Status status = RewriteFields(\n      absl::MakeSpan(fields_to_add_), absl::MakeSpan(fields_to_remove_),\n      message_assembler.root_, std::forward<BaseSrc>(base_src), message_writer);\n\n  // Each element of `fields_to_add_` is already cleared.\n  fields_to_remove_.clear();\n  if (dest_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_dep->Close())) {\n      status.Update(dest_dep->status());\n    }\n  }\n  return status;\n}\n\ntemplate <typename Dest>\ninline absl::Status SerializedMessageAssembler::Session::WriteMessageInternal(\n    const SerializedMessageAssembler& message_assembler, Dest&& dest) && {\n  DependencyRef<Writer*, Dest> dest_dep(std::forward<Dest>(dest));\n  SerializedMessageWriter message_writer(dest_dep.get());\n\n  absl::Status status =\n      WriteNewFields(absl::MakeSpan(fields_to_add_), message_assembler.root_,\n                     SubmessageSet(message_assembler.root_.submessages.size()),\n                     message_writer);\n\n  // Each element of `fields_to_add_` is already cleared.\n  fields_to_remove_.clear();\n  if (dest_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_dep->Close())) {\n      status.Update(dest_dep->status());\n    }\n  }\n  return status;\n}\n\ntemplate <typename Src>\nabsl::Status SerializedMessageAssembler::RewriteFields(\n    absl::Span<FieldValues> fields_to_add,\n    absl::Span<const bool> fields_to_remove, const RegisteredMessage& message,\n    Src&& src, SerializedMessageWriter& message_writer) {\n  SubmessageSet submessages_rewritten(message.submessages.size());\n\n  // Rewrite fields which are present in the base message.\n  if (absl::Status status =\n          SerializedMessageReader<\n              const absl::Span<FieldValues>, const absl::Span<const bool>,\n              const absl::Span<bool>, SerializedMessageWriter>(\n              std::cref(message.handlers), AnyFieldCopier())\n              .ReadMessage(\n                  std::forward<Src>(src), fields_to_add, fields_to_remove,\n                  absl::MakeSpan(submessages_rewritten), message_writer);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return status;\n  }\n\n  // Write new fields which were not present in the base message.\n  return WriteNewFields(fields_to_add, message, submessages_rewritten,\n                        message_writer);\n}\n\nextern template absl::Status SerializedMessageAssembler::RewriteFields(\n    absl::Span<FieldValues> fields_to_add,\n    absl::Span<const bool> fields_to_remove, const RegisteredMessage& message,\n    ReaderSpan<>&& src, SerializedMessageWriter& message_writer);\nextern template absl::Status SerializedMessageAssembler::RewriteFields(\n    absl::Span<FieldValues> fields_to_add,\n    absl::Span<const bool> fields_to_remove, const RegisteredMessage& message,\n    CordIteratorSpan&& src, SerializedMessageWriter& message_writer);\nextern template absl::Status SerializedMessageAssembler::RewriteFields(\n    absl::Span<FieldValues> fields_to_add,\n    absl::Span<const bool> fields_to_remove, const RegisteredMessage& message,\n    absl::string_view&& src, SerializedMessageWriter& message_writer);\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_SERIALIZED_MESSAGE_ASSEMBLER_H_\n"
  },
  {
    "path": "riegeli/messages/serialized_message_backward_writer.cc",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/messages/serialized_message_backward_writer.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <utility>\n\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"riegeli/base/any.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/copy_all.h\"\n#include \"riegeli/bytes/reader.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nabsl::Status SerializedMessageBackwardWriter::LengthOverflowError(\n    Position length) {\n  return absl::ResourceExhaustedError(\n      absl::StrCat(\"Failed to write length-delimited field \"\n                   \"because its size must be smaller than 2GiB: \",\n                   length));\n}\n\nabsl::Status SerializedMessageBackwardWriter::WriteStringFailed(\n    Reader& src, BackwardWriter& dest) {\n  return !dest.ok()\n             ? dest.status()\n             : src.StatusOrAnnotate(absl::InvalidArgumentError(\n                   \"Could not read contents for a length-delimited field\"));\n}\n\nabsl::Status SerializedMessageBackwardWriter::WriteString(int field_number,\n                                                          AnyRef<Reader*> src) {\n  if (src.IsOwning()) src->SetReadAllHint(true);\n  const Position pos_after = writer().pos();\n  if (absl::Status status = CopyAll(std::move(src), writer());\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return status;\n  }\n  RIEGELI_ASSERT_GE(writer().pos(), pos_after)\n      << \"CopyAll() decreased dest.pos()\";\n  return WriteLengthUnchecked(field_number, writer().pos() - pos_after);\n}\n\nabsl::Status SerializedMessageBackwardWriter::WriteString(\n    int field_number, CordIteratorSpan src) {\n  if (src.length() <= kMaxBytesToCopy) {\n    if (ABSL_PREDICT_FALSE(!writer().Push(src.length()))) {\n      return writer().status();\n    }\n    writer().move_cursor(src.length());\n    CordIteratorSpan::Read(src.iterator(), src.length(), writer().cursor());\n    return WriteLengthUnchecked(field_number, src.length());\n  }\n  return WriteString(field_number, std::move(src).ToCord());\n}\n\nvoid SerializedMessageBackwardWriter::OpenLengthDelimited() {\n  submessages_.push_back(writer().pos());\n}\n\nabsl::Status SerializedMessageBackwardWriter::CloseLengthDelimited(\n    int field_number) {\n  RIEGELI_ASSERT(!submessages_.empty())\n      << \"Failed precondition of \"\n         \"SerializedMessageBackwardWriter::CloseLengthDelimited(): \"\n         \"no matching OpenLengthDelimited() call\";\n  RIEGELI_ASSERT_GE(writer().pos(), submessages_.back())\n      << \"Failed precondition of \"\n         \"SerializedMessageBackwardWriter::CloseLengthDelimited(): \"\n         \"writer().pos() decreased since OpenLengthDelimited()\";\n  const Position length = writer().pos() - submessages_.back();\n  submessages_.pop_back();\n  return WriteLengthUnchecked(field_number, length);\n}\n\nabsl::Status SerializedMessageBackwardWriter::CloseOptionalLengthDelimited(\n    int field_number) {\n  RIEGELI_ASSERT(!submessages_.empty())\n      << \"Failed precondition of \"\n         \"SerializedMessageBackwardWriter::CloseOptionalLengthDelimited(): \"\n         \"no matching OpenLengthDelimited() call\";\n  RIEGELI_ASSERT_GE(writer().pos(), submessages_.back())\n      << \"Failed precondition of \"\n         \"SerializedMessageBackwardWriter::CloseOptionalLengthDelimited(): \"\n         \"writer().pos() decreased since OpenLengthDelimited()\";\n  const Position length = writer().pos() - submessages_.back();\n  submessages_.pop_back();\n  if (length > 0) return WriteLengthUnchecked(field_number, length);\n  return absl::OkStatus();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/messages/serialized_message_backward_writer.h",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_SERIALIZED_MESSAGE_BACKWARD_WRITER_H_\n#define RIEGELI_MESSAGES_SERIALIZED_MESSAGE_BACKWARD_WRITER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <type_traits>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/casts.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/any.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/stringify.h\"\n#include \"riegeli/endian/endian_writing.h\"\n#include \"riegeli/messages/message_wire_format.h\"\n#include \"riegeli/messages/serialize_message.h\"\n#include \"riegeli/varint/varint_writing.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `SerializedMessageBackwardWriter` builds a serialized proto message,\n// specifying contents of particular fields, instead of traversing an in-memory\n// message object like in `SerializeMessage()`. Building proceeds back to front,\n// using `BackwardWriter`.\n//\n// Use cases:\n//\n//  * Processing a subset of fields without the overhead of materializing the\n//    message object, i.e. without processing fields contained in submessages\n//    which can be processed as a whole, and without keeping the whole parsed\n//    message in memory.\n//\n//  * Processing a message in a way known at runtime, possibly with the schema\n//    known at runtime, possibly partially known.\n//\n//  * Processing messages with so many elements of toplevel repeated fields that\n//    the total message size exceeds 2GiB. This is not a great idea in itself,\n//    because such messages cannot be processed using native proto parsing and\n//    serialization.\n//\n// `SerializedMessageBackwardWriter` is more efficient than\n// `SerializedMessageWriter` in the case of nested messages, because their\n// contents can be written directly to the original `BackwardWriter`, with the\n// length known and written after building the contents.\n//\n// Building elements of a repeated field is done in the opposite order than in\n// `SerializedMessageWriter`, and if a non-repeated field is written multiple\n// times then they are overridden or merged in the opposite order. Otherwise the\n// field order does not influence how the message is parsed.\nclass SerializedMessageBackwardWriter {\n public:\n  // An empty object. It can be associated with a particular message by\n  // `set_dest()` or assignment.\n  //\n  // An empty `SerializedMessageBackwardWriter` is not usable directly.\n  SerializedMessageBackwardWriter() = default;\n\n  // Will write to `*dest`, which is not owned and must outlive usages of this\n  // object.\n  explicit SerializedMessageBackwardWriter(\n      BackwardWriter* absl_nullable dest ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : dest_(dest) {}\n\n  SerializedMessageBackwardWriter(\n      SerializedMessageBackwardWriter&& that) noexcept;\n  SerializedMessageBackwardWriter& operator=(\n      SerializedMessageBackwardWriter&& that) noexcept;\n\n  // Returns the original `BackwardWriter` of the root message.\n  BackwardWriter* absl_nullable dest() const { return dest_; }\n\n  // Changes the `BackwardWriter` of the root message.\n  //\n  // This can be called even during building, even when submessages are open,\n  // but the position must be the same. It particular this must be called when\n  // the original `BackwardWriter` has been moved.\n  void set_dest(BackwardWriter* absl_nullable dest);\n\n  // Returns the `BackwardWriter` of the current message or length-delimited\n  // field being built.\n  //\n  // `writer()` is the same as `*dest()`. Functions are separate for consistency\n  // with `SerializedMessageWriter`.\n  //\n  // This can be used to write parts of the message directly, apart from\n  // `Write...()` functions which write whole fields.\n  //\n  // Building elements of a repeated field and writing parts of a single field\n  // to `writer()` is done in the opposite order than in\n  // `SerializedMessageWriter`.\n  BackwardWriter& writer() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT(dest_ != nullptr)\n        << \"Failed precondition of SerializedMessageBackwardWriter::writer(): \"\n           \"dest() not set\";\n    return *dest_;\n  }\n\n  // Writes the field tag and the numeric or enum field value.\n  absl::Status WriteInt32(int field_number, int32_t value);\n  absl::Status WriteInt64(int field_number, int64_t value);\n  absl::Status WriteUInt32(int field_number, uint32_t value);\n  absl::Status WriteUInt64(int field_number, uint64_t value);\n  absl::Status WriteSInt32(int field_number, int32_t value);\n  absl::Status WriteSInt64(int field_number, int64_t value);\n  absl::Status WriteBool(int field_number, bool value);\n  absl::Status WriteFixed32(int field_number, uint32_t value);\n  absl::Status WriteFixed64(int field_number, uint64_t value);\n  absl::Status WriteSFixed32(int field_number, int32_t value);\n  absl::Status WriteSFixed64(int field_number, int64_t value);\n  absl::Status WriteFloat(int field_number, float value);\n  absl::Status WriteDouble(int field_number, double value);\n  template <typename EnumType,\n            std::enable_if_t<std::disjunction_v<std::is_enum<EnumType>,\n                                                std::is_integral<EnumType>>,\n                             int> = 0>\n  absl::Status WriteEnum(int field_number, EnumType value);\n\n  // Writes the field tag and the `string`, `bytes`, or submessage field value.\n  //\n  // Message objects are excluded here because they are stringified in the text\n  // format, which is rarely intended as a field value. A separate overload\n  // below serializes a message object in the binary format.\n  template <typename... Values\n#if !__cpp_concepts\n            ,\n            std::enable_if_t<\n                std::conjunction_v<\n                    IsStringifiable<Values...>,\n                    std::negation<std::is_convertible<\n                        Values&&, const google::protobuf::MessageLite&>>...>,\n                int> = 0\n#endif\n            >\n#if __cpp_concepts\n  // For conjunctions, `requires` gives better error messages than\n  // `std::enable_if_t`, indicating the relevant argument.\n    requires((IsStringifiable<Values>::value && ...) &&\n             (!std::is_convertible_v<Values &&,\n                                     const google::protobuf::MessageLite&> &&\n              ...))\n#endif\n  absl::Status WriteString(int field_number, Values&&... values);\n  absl::Status WriteString(int field_number, AnyRef<Reader*> src);\n  template <typename ReaderType>\n  absl::Status WriteString(int field_number, ReaderSpan<ReaderType> src);\n  absl::Status WriteString(int field_number, CordIteratorSpan src);\n\n  // Writes the field tag of a length-delimited field and serializes a message\n  // as the field value.\n  absl::Status WriteString(int field_number,\n                           const google::protobuf::MessageLite& message,\n                           SerializeMessageOptions options = {});\n\n  // Writes an element of a packed repeated field.\n  //\n  // The field must have been opened with `OpenLengthDelimited()` or\n  // `WriteLengthUnchecked()`.\n  absl::Status WritePackedInt32(int32_t value);\n  absl::Status WritePackedInt64(int64_t value);\n  absl::Status WritePackedUInt32(uint32_t value);\n  absl::Status WritePackedUInt64(uint64_t value);\n  absl::Status WritePackedSInt32(int32_t value);\n  absl::Status WritePackedSInt64(int64_t value);\n  absl::Status WritePackedBool(bool value);\n  absl::Status WritePackedFixed32(uint32_t value);\n  absl::Status WritePackedFixed64(uint64_t value);\n  absl::Status WritePackedSFixed32(int32_t value);\n  absl::Status WritePackedSFixed64(int64_t value);\n  absl::Status WritePackedFloat(float value);\n  absl::Status WritePackedDouble(double value);\n  template <typename EnumType,\n            std::enable_if_t<std::disjunction_v<std::is_enum<EnumType>,\n                                                std::is_integral<EnumType>>,\n                             int> = 0>\n  absl::Status WritePackedEnum(EnumType value);\n\n  // Begins accumulating contents of a length-delimited field.\n  //\n  // `writer().pos()` is remembered, and field contents written to `writer()`\n  // are written directly to `dest()`.\n  void OpenLengthDelimited();\n\n  // Ends accumulating contents of a length-delimited field, and writes the\n  // field tag and length to the parent message.\n  //\n  // Each `OpenLengthDelimited()` call must be matched with a\n  // `CloseLengthDelimited()` or `CloseOptionalLengthDelimited()` call, unless\n  // the `SerializedMessageBackwardWriter` and its `dest()` are no longer used.\n  absl::Status CloseLengthDelimited(int field_number);\n\n  // Like `CloseLengthDelimited()`, but does not write the field tag and length\n  // if its contents turn out to be empty.\n  absl::Status CloseOptionalLengthDelimited(int field_number);\n\n  // Writes the field tag and the length of a length-delimited field.\n  //\n  // The value must have been written beforehand to `writer()`, with exactly\n  // `length` bytes, unless the `SerializedMessageBackwardWriter` and its\n  // `dest()` are no longer used.\n  //\n  // Fails if `length` exceeds 2GiB.\n  //\n  // `WriteLengthUnchecked()` is slightly more efficient than\n  // `OpenLengthDelimited()` or `NewLengthDelimited()` with\n  // `CloseLengthDelimited()`, but harder to use: the length must be specified\n  // explicitly, and its correctness is not checked.\n  //\n  // Writing the contents and calling `WriteLengthUnchecked()` is done in the\n  // opposite order than in `SerializedMessageWriter`.\n  absl::Status WriteLengthUnchecked(int field_number, Position length);\n\n  // Writes a group delimiter.\n  //\n  // Each `OpenGroup()` must be matched with a `CloseGroup()` call, unless the\n  // `SerializedMessageBackwardWriter` and its `dest()` are no longer used.\n  absl::Status OpenGroup(int field_number);\n  absl::Status CloseGroup(int field_number);\n\n private:\n  ABSL_ATTRIBUTE_COLD static absl::Status LengthOverflowError(Position length);\n  ABSL_ATTRIBUTE_COLD static absl::Status WriteStringFailed(\n      Reader& src, BackwardWriter& dest);\n\n  BackwardWriter* absl_nullable dest_ = nullptr;\n  std::vector<Position> submessages_;\n\n  // Invariant: if `!submessages_.empty()` then `dest_ != nullptr`\n};\n\n// Implementation details follow.\n\ninline SerializedMessageBackwardWriter::SerializedMessageBackwardWriter(\n    SerializedMessageBackwardWriter&& that) noexcept\n    : dest_(that.dest_), submessages_(std::exchange(that.submessages_, {})) {}\n\ninline SerializedMessageBackwardWriter&\nSerializedMessageBackwardWriter::operator=(\n    SerializedMessageBackwardWriter&& that) noexcept {\n  dest_ = that.dest_;\n  submessages_ = std::exchange(that.submessages_, {});\n  return *this;\n}\n\ninline void SerializedMessageBackwardWriter::set_dest(\n    BackwardWriter* absl_nullable dest) {\n  if (!submessages_.empty()) {\n    RIEGELI_ASSERT(dest != nullptr)\n        << \"Failed precondition of \"\n           \"SerializedMessageBackwardWriter::set_dest(): \"\n           \"null BackwardWriter pointer while writing a submessage\";\n    RIEGELI_ASSERT(dest_ != nullptr)\n        << \"Failed invariant of SerializedMessageBackwardWriter: \"\n           \"dest_ is null while writing a submessage\";\n    RIEGELI_ASSERT_EQ(dest->pos(), dest_->pos())\n        << \"Failed precondition of \"\n           \"SerializedMessageBackwardWriter::set_dest(): \"\n           \"pos() changes while writing a submessage\";\n  }\n  dest_ = dest;\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteInt32(\n    int field_number, int32_t value) {\n  return WriteUInt64(field_number, static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteInt64(\n    int field_number, int64_t value) {\n  return WriteUInt64(field_number, static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteUInt32(\n    int field_number, uint32_t value) {\n  const uint32_t tag = MakeTag(field_number, WireType::kVarint);\n  const size_t length = LengthVarint32(tag) + LengthVarint32(value);\n  if (ABSL_PREDICT_FALSE(!writer().Push(length))) return writer().status();\n  writer().move_cursor(length);\n  char* const ptr = WriteVarint32(tag, writer().cursor());\n  WriteVarint32(value, ptr);\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteUInt64(\n    int field_number, uint64_t value) {\n  const uint32_t tag = MakeTag(field_number, WireType::kVarint);\n  const size_t length = LengthVarint32(tag) + LengthVarint64(value);\n  if (ABSL_PREDICT_FALSE(!writer().Push(length))) return writer().status();\n  writer().move_cursor(length);\n  char* const ptr = WriteVarint32(tag, writer().cursor());\n  WriteVarint64(value, ptr);\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteSInt32(\n    int field_number, int32_t value) {\n  return WriteUInt32(field_number, EncodeVarintSigned32(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteSInt64(\n    int field_number, int64_t value) {\n  return WriteUInt64(field_number, EncodeVarintSigned64(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteBool(int field_number,\n                                                               bool value) {\n  return WriteUInt32(field_number, value ? 1 : 0);\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteFixed32(\n    int field_number, uint32_t value) {\n  const uint32_t tag = MakeTag(field_number, WireType::kFixed32);\n  const size_t length = LengthVarint32(tag) + sizeof(uint32_t);\n  if (ABSL_PREDICT_FALSE(!writer().Push(length))) return writer().status();\n  writer().move_cursor(length);\n  char* const ptr = WriteVarint32(tag, writer().cursor());\n  WriteLittleEndian<uint32_t>(value, ptr);\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteFixed64(\n    int field_number, uint64_t value) {\n  const uint32_t tag = MakeTag(field_number, WireType::kFixed64);\n  const size_t length = LengthVarint32(tag) + sizeof(uint64_t);\n  if (ABSL_PREDICT_FALSE(!writer().Push(length))) return writer().status();\n  writer().move_cursor(length);\n  char* const ptr = WriteVarint32(tag, writer().cursor());\n  WriteLittleEndian<uint64_t>(value, ptr);\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteSFixed32(\n    int field_number, int32_t value) {\n  return WriteFixed32(field_number, static_cast<uint32_t>(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteSFixed64(\n    int field_number, int64_t value) {\n  return WriteFixed64(field_number, static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteFloat(\n    int field_number, float value) {\n  return WriteFixed32(field_number, absl::bit_cast<uint32_t>(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteDouble(\n    int field_number, double value) {\n  return WriteFixed64(field_number, absl::bit_cast<uint64_t>(value));\n}\n\ntemplate <typename EnumType,\n          std::enable_if_t<std::disjunction_v<std::is_enum<EnumType>,\n                                              std::is_integral<EnumType>>,\n                           int>>\ninline absl::Status SerializedMessageBackwardWriter::WriteEnum(int field_number,\n                                                               EnumType value) {\n  return WriteUInt64(field_number, static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WritePackedInt32(\n    int32_t value) {\n  return WritePackedUInt64(static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WritePackedInt64(\n    int64_t value) {\n  return WritePackedUInt64(static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WritePackedUInt32(\n    uint32_t value) {\n  if (ABSL_PREDICT_FALSE(!WriteVarint32(value, writer()))) {\n    return writer().status();\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WritePackedUInt64(\n    uint64_t value) {\n  if (ABSL_PREDICT_FALSE(!WriteVarint64(value, writer()))) {\n    return writer().status();\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WritePackedSInt32(\n    int32_t value) {\n  return WritePackedUInt32(EncodeVarintSigned32(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WritePackedSInt64(\n    int64_t value) {\n  return WritePackedUInt64(EncodeVarintSigned64(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WritePackedBool(\n    bool value) {\n  return WritePackedUInt32(value ? 1 : 0);\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WritePackedFixed32(\n    uint32_t value) {\n  if (ABSL_PREDICT_FALSE(!WriteLittleEndian<uint32_t>(value, writer()))) {\n    return writer().status();\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WritePackedFixed64(\n    uint64_t value) {\n  if (ABSL_PREDICT_FALSE(!WriteLittleEndian<uint64_t>(value, writer()))) {\n    return writer().status();\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WritePackedSFixed32(\n    int32_t value) {\n  return WritePackedFixed32(static_cast<uint32_t>(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WritePackedSFixed64(\n    int64_t value) {\n  return WritePackedFixed64(static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WritePackedFloat(\n    float value) {\n  return WritePackedFixed32(absl::bit_cast<uint32_t>(value));\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WritePackedDouble(\n    double value) {\n  return WritePackedFixed64(absl::bit_cast<uint64_t>(value));\n}\n\ntemplate <typename EnumType,\n          std::enable_if_t<std::disjunction_v<std::is_enum<EnumType>,\n                                              std::is_integral<EnumType>>,\n                           int>>\ninline absl::Status SerializedMessageBackwardWriter::WritePackedEnum(\n    EnumType value) {\n  return WritePackedUInt64(static_cast<uint64_t>(value));\n}\n\ntemplate <typename... Values\n#if !__cpp_concepts\n          ,\n          std::enable_if_t<\n              std::conjunction_v<\n                  IsStringifiable<Values...>,\n                  std::negation<std::is_convertible<\n                      Values&&, const google::protobuf::MessageLite&>>...>,\n              int>\n#endif\n          >\n#if __cpp_concepts\n  requires((IsStringifiable<Values>::value && ...) &&\n           (!std::is_convertible_v<Values &&,\n                                   const google::protobuf::MessageLite&> &&\n            ...))\n#endif\ninline absl::Status SerializedMessageBackwardWriter::WriteString(\n    int field_number, Values&&... values) {\n  const Position pos_before = writer().pos();\n  if (ABSL_PREDICT_FALSE(!writer().Write(std::forward<Values>(values)...))) {\n    return writer().status();\n  }\n  RIEGELI_ASSERT_GE(writer().pos(), pos_before)\n      << \"BackwardWriter::Write() decreased pos()\";\n  return WriteLengthUnchecked(field_number, writer().pos() - pos_before);\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteString(\n    int field_number, const google::protobuf::MessageLite& message,\n    SerializeMessageOptions options) {\n  const size_t length = options.GetByteSize(message);\n  if (absl::Status status =\n          riegeli::SerializeMessage(message, writer(), options);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return status;\n  }\n  return WriteLengthUnchecked(field_number, length);\n}\n\ntemplate <typename ReaderType>\nabsl::Status SerializedMessageBackwardWriter::WriteString(\n    int field_number, ReaderSpan<ReaderType> src) {\n  if (ABSL_PREDICT_FALSE(src.length() >\n                         uint32_t{std::numeric_limits<int32_t>::max()})) {\n    return LengthOverflowError(src.length());\n  }\n  if (ABSL_PREDICT_FALSE(\n          !src.reader().Copy(IntCast<size_t>(src.length()), writer()))) {\n    return WriteStringFailed(src.reader(), writer());\n  }\n  return WriteLengthUnchecked(field_number, src.length());\n}\n\ninline absl::Status SerializedMessageBackwardWriter::WriteLengthUnchecked(\n    int field_number, Position length) {\n  if (ABSL_PREDICT_FALSE(length >\n                         uint32_t{std::numeric_limits<int32_t>::max()})) {\n    return LengthOverflowError(length);\n  }\n  const uint32_t tag = MakeTag(field_number, WireType::kLengthDelimited);\n  const size_t header_length =\n      LengthVarint32(tag) + LengthVarint32(IntCast<uint32_t>(length));\n  if (ABSL_PREDICT_FALSE(!writer().Push(header_length))) {\n    return writer().status();\n  }\n  writer().move_cursor(header_length);\n  char* const ptr = WriteVarint32(tag, writer().cursor());\n  WriteVarint32(IntCast<uint32_t>(length), ptr);\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageBackwardWriter::OpenGroup(\n    int field_number) {\n  if (ABSL_PREDICT_FALSE(!WriteVarint32(\n          MakeTag(field_number, WireType::kEndGroup), writer()))) {\n    return writer().status();\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageBackwardWriter::CloseGroup(\n    int field_number) {\n  if (ABSL_PREDICT_FALSE(!WriteVarint32(\n          MakeTag(field_number, WireType::kStartGroup), writer()))) {\n    return writer().status();\n  }\n  return absl::OkStatus();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_SERIALIZED_MESSAGE_BACKWARD_WRITER_H_\n"
  },
  {
    "path": "riegeli/messages/serialized_message_internal.h",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_SERIALIZED_MESSAGE_INTERNAL_\n#define RIEGELI_MESSAGES_SERIALIZED_MESSAGE_INTERNAL_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <functional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/type_erased_ref.h\"\n\nnamespace riegeli {\n\nclass LimitingReaderBase;\nclass SerializedMessageWriter;\n\nnamespace serialized_message_internal {\n\ntemplate <typename Enable, typename Context, typename Action, typename... Args>\nstruct IsInvocableWithContextImpl : std::false_type {};\n\ntemplate <typename Context, typename Action, typename... Args>\nstruct IsInvocableWithContextImpl<std::enable_if_t<!std::is_void_v<Context>>,\n                                  Context, Action, Args...>\n    : std::is_invocable_r<absl::Status, Action, Args..., Context&> {};\n\ntemplate <typename Context, typename Action, typename... Args>\nstruct IsInvocableWithContext\n    : IsInvocableWithContextImpl<void, Context, Action, Args...> {};\n\ntemplate <typename Action, typename... Args>\nstruct IsInvocableWithoutContext\n    : std::is_invocable_r<absl::Status, Action, Args...> {};\n\ntemplate <typename Context, typename Action, typename... Args>\nstruct IsAction\n    : std::disjunction<IsInvocableWithContext<Context, Action, Args...>,\n                       IsInvocableWithoutContext<Action, Args...>> {};\n\ntemplate <typename Context, typename Action, typename... Args,\n          std::enable_if_t<\n              IsInvocableWithContext<Context, Action, Args...>::value, int> = 0>\ninline absl::Status InvokeAction(TypeErasedRef context, Action&& action,\n                                 Args&&... args) {\n  return std::invoke(std::forward<Action>(action), std::forward<Args>(args)...,\n                     context.Cast<Context&>());\n}\n\ntemplate <typename Context, typename Action, typename... Args,\n          std::enable_if_t<IsInvocableWithoutContext<Action, Args...>::value,\n                           int> = 0>\ninline absl::Status InvokeAction(ABSL_ATTRIBUTE_UNUSED TypeErasedRef context,\n                                 Action&& action, Args&&... args) {\n  return std::invoke(std::forward<Action>(action), std::forward<Args>(args)...);\n}\n\ntemplate <typename Context, typename Action, typename... Args>\nstruct IsActionWithSrc\n    : IsAction<Context, Action, Args..., LimitingReaderBase&> {};\n\ntemplate <typename Context, typename Action, typename... Args>\nstruct IsActionWithoutSrc : IsAction<Context, Action, Args...> {};\n\ntemplate <typename Context, typename Action, typename... Args>\nstruct IsActionWithOptionalSrc\n    : std::disjunction<IsActionWithSrc<Context, Action, Args...>,\n                       IsActionWithoutSrc<Context, Action, Args...>> {};\n\ntemplate <\n    typename Context, typename Action, typename... Args,\n    std::enable_if_t<IsActionWithSrc<Context, Action, Args...>::value, int> = 0>\ninline absl::Status InvokeActionWithSrc(LimitingReaderBase& src,\n                                        TypeErasedRef context, Action&& action,\n                                        Args&&... args) {\n  return InvokeAction<Context>(context, std::forward<Action>(action),\n                               std::forward<Args>(args)..., src);\n}\n\ntemplate <typename Context, typename Action, typename... Args,\n          std::enable_if_t<IsActionWithoutSrc<Context, Action, Args...>::value,\n                           int> = 0>\ninline absl::Status InvokeActionWithSrc(\n    ABSL_ATTRIBUTE_UNUSED LimitingReaderBase& src, TypeErasedRef context,\n    Action&& action, Args&&... args) {\n  return InvokeAction<Context>(context, std::forward<Action>(action),\n                               std::forward<Args>(args)...);\n}\n\ntemplate <typename Context, typename Action, typename... Args>\nstruct IsActionWithDest\n    : IsAction<Context, Action, Args..., SerializedMessageWriter&> {};\n\ntemplate <typename Context, typename Action, typename... Args>\nstruct IsActionWithoutDest : IsAction<Context, Action, Args...> {};\n\ntemplate <typename Context, typename Action, typename... Args>\nstruct IsActionWithOptionalDest\n    : std::disjunction<IsActionWithDest<Context, Action, Args...>,\n                       IsActionWithoutDest<Context, Action, Args...>> {};\n\ntemplate <typename Context, typename Action, typename... Args,\n          std::enable_if_t<IsActionWithDest<Context, Action, Args...>::value,\n                           int> = 0>\ninline absl::Status InvokeActionWithDest(SerializedMessageWriter& dest,\n                                         TypeErasedRef context, Action&& action,\n                                         Args&&... args) {\n  return InvokeAction<Context>(context, std::forward<Action>(action),\n                               std::forward<Args>(args)..., dest);\n}\n\ntemplate <typename Context, typename Action, typename... Args,\n          std::enable_if_t<IsActionWithoutDest<Context, Action, Args...>::value,\n                           int> = 0>\ninline absl::Status InvokeActionWithDest(\n    ABSL_ATTRIBUTE_UNUSED SerializedMessageWriter& dest, TypeErasedRef context,\n    Action&& action, Args&&... args) {\n  return InvokeAction<Context>(context, std::forward<Action>(action),\n                               std::forward<Args>(args)...);\n}\n\ntemplate <typename Context, typename Action, typename... Args>\nstruct IsActionWithRequiredSrcAndOptionalDest\n    : IsActionWithOptionalDest<Context, Action, Args..., LimitingReaderBase&> {\n};\n\ntemplate <typename Context, typename Action, typename... Args>\nstruct IsActionWithOptionalSrcAndDest\n    : std::disjunction<\n          IsActionWithRequiredSrcAndOptionalDest<Context, Action, Args...>,\n          IsActionWithOptionalDest<Context, Action, Args...>> {};\n\ntemplate <typename Context, typename Action, typename... Args,\n          std::enable_if_t<IsActionWithRequiredSrcAndOptionalDest<\n                               Context, Action, Args...>::value,\n                           int> = 0>\ninline absl::Status InvokeActionWithSrcAndDest(LimitingReaderBase& src,\n                                               SerializedMessageWriter& dest,\n                                               TypeErasedRef context,\n                                               Action&& action,\n                                               Args&&... args) {\n  return InvokeActionWithDest<Context>(dest, context,\n                                       std::forward<Action>(action),\n                                       std::forward<Args>(args)..., src);\n}\n\ntemplate <\n    typename Context, typename Action, typename... Args,\n    std::enable_if_t<IsActionWithOptionalDest<Context, Action, Args...>::value,\n                     int> = 0>\ninline absl::Status InvokeActionWithSrcAndDest(\n    ABSL_ATTRIBUTE_UNUSED LimitingReaderBase& src,\n    SerializedMessageWriter& dest, TypeErasedRef context, Action&& action,\n    Args&&... args) {\n  return InvokeActionWithDest<Context>(\n      dest, context, std::forward<Action>(action), std::forward<Args>(args)...);\n}\n\n}  // namespace serialized_message_internal\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_SERIALIZED_MESSAGE_INTERNAL_\n"
  },
  {
    "path": "riegeli/messages/serialized_message_reader.cc",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/messages/message_wire_format.h\"\n#include \"riegeli/messages/serialized_message_reader_internal.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli::serialized_message_reader_internal {\n\nnamespace {\n\nABSL_ATTRIBUTE_COLD absl::Status ReadVarintError() {\n  return absl::InvalidArgumentError(\"Could not read a varint field\");\n}\n\nABSL_ATTRIBUTE_COLD absl::Status ReadFixed32Error() {\n  return absl::InvalidArgumentError(\"Could not read a fixed32 field\");\n}\n\nABSL_ATTRIBUTE_COLD absl::Status ReadFixed64Error() {\n  return absl::InvalidArgumentError(\"Could not read a fixed64 field\");\n}\n\nABSL_ATTRIBUTE_COLD absl::Status NotEnoughError(uint32_t expected_length,\n                                                Position available) {\n  return absl::InvalidArgumentError(\n      absl::StrCat(\"Not enough data: expected at least \", expected_length,\n                   \" more, will have at most \", available, \" more\"));\n}\n\nABSL_ATTRIBUTE_COLD\nabsl::Status ReadLengthDelimitedLengthError() {\n  return absl::InvalidArgumentError(\n      \"Could not read a length-delimited field length\");\n}\n\n}  // namespace\n\nabsl::Status AnnotateWithFieldNumberSlow(absl::Status status,\n                                         int field_number) {\n  if (absl::IsCancelled(status)) return status;\n  return riegeli::Annotate(status,\n                           absl::StrCat(\"field number: \", field_number));\n}\n\nabsl::Status AnnotateWithSourceAndFieldNumberSlow(absl::Status status,\n                                                  Reader& src,\n                                                  int field_number) {\n  if (absl::IsCancelled(status)) return status;\n  return AnnotateWithFieldNumberSlow(src.StatusOrAnnotate(std::move(status)),\n                                     field_number);\n}\n\nabsl::Status ReadTagError() {\n  return absl::InvalidArgumentError(\"Could not read field tag\");\n}\n\nabsl::Status ReadTagError(Reader& src) {\n  return src.StatusOrAnnotate(ReadTagError());\n}\n\nabsl::Status ReadVarintError(int field_number) {\n  return AnnotateWithFieldNumberSlow(ReadVarintError(), field_number);\n}\n\nabsl::Status ReadVarintError(Reader& src, int field_number) {\n  return AnnotateWithSourceAndFieldNumberSlow(ReadVarintError(), src,\n                                              field_number);\n}\n\nabsl::Status ReadFixed32Error(int field_number) {\n  return AnnotateWithFieldNumberSlow(ReadFixed32Error(), field_number);\n}\n\nabsl::Status ReadFixed32Error(Reader& src, int field_number) {\n  return AnnotateWithSourceAndFieldNumberSlow(ReadFixed32Error(), src,\n                                              field_number);\n}\n\nabsl::Status ReadFixed64Error(int field_number) {\n  return AnnotateWithFieldNumberSlow(ReadFixed64Error(), field_number);\n}\n\nabsl::Status ReadFixed64Error(Reader& src, int field_number) {\n  return AnnotateWithSourceAndFieldNumberSlow(ReadFixed64Error(), src,\n                                              field_number);\n}\n\nabsl::Status NotEnoughError(int field_number, uint32_t expected_length,\n                            size_t available) {\n  return AnnotateWithFieldNumberSlow(NotEnoughError(expected_length, available),\n                                     field_number);\n}\n\nabsl::Status NotEnoughError(LimitingReaderBase& src, int field_number,\n                            uint32_t expected_length) {\n  return AnnotateWithSourceAndFieldNumberSlow(\n      NotEnoughError(expected_length, src.max_length()), src, field_number);\n}\n\nabsl::Status ReadLengthDelimitedLengthError(int field_number) {\n  return AnnotateWithFieldNumberSlow(ReadLengthDelimitedLengthError(),\n                                     field_number);\n}\n\nabsl::Status ReadLengthDelimitedLengthError(Reader& src, int field_number) {\n  return AnnotateWithSourceAndFieldNumberSlow(ReadLengthDelimitedLengthError(),\n                                              src, field_number);\n}\n\nabsl::Status ReadLengthDelimitedValueError(Reader& src) {\n  return src.StatusOrAnnotate(\n      absl::InvalidArgumentError(\"Could not read a length-delimited field\"));\n}\n\nabsl::Status ReadLengthDelimitedValueError(Reader& src, int field_number) {\n  return AnnotateWithFieldNumberSlow(ReadLengthDelimitedValueError(src),\n                                     field_number);\n}\n\nabsl::Status InvalidWireTypeError(uint32_t tag) {\n  return absl::InvalidArgumentError(absl::StrCat(\n      \"Invalid wire type: \", static_cast<int>(GetTagWireType(tag))));\n}\n\nabsl::Status InvalidWireTypeError(Reader& src, uint32_t tag) {\n  return src.StatusOrAnnotate(InvalidWireTypeError(tag));\n}\n\n}  // namespace riegeli::serialized_message_reader_internal\n"
  },
  {
    "path": "riegeli/messages/serialized_message_reader.h",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_SERIALIZED_MESSAGE_READER_H_\n#define RIEGELI_MESSAGES_SERIALIZED_MESSAGE_READER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <optional>\n#include <string>\n#include <tuple>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/cord_reader.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/string_reader.h\"\n#include \"riegeli/endian/endian_reading.h\"\n#include \"riegeli/messages/message_wire_format.h\"\n#include \"riegeli/messages/serialized_message_reader_internal.h\"\n#include \"riegeli/varint/varint_reading.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// Overview of `SerializedMessageReader`\n// --------------------------------------\n//\n// `SerializedMessageReader` reads a serialized proto message, performing\n// specified actions on encountering particular fields, instead of filling an\n// in-memory message object like in `ParseMessage()`.\n//\n// Use cases:\n//\n//  * Processing a subset of fields without the overhead of materializing the\n//    message object, i.e. without processing the remaining fields, without\n//    processing fields contained in submessages which can be processed as a\n//    whole, and without keeping the whole parsed message in memory.\n//\n//  * Processing a message in a way known at runtime, possibly with the schema\n//    known at runtime, possibly partially known.\n//\n//  * Processing messages with so many elements of toplevel repeated fields that\n//    the total message size exceeds 2GiB. This is not a great idea in itself,\n//    because such messages cannot be processed using native proto parsing and\n//    serialization.\n//\n// Fields to be handled are specified by field handlers. For each field present\n// in the serialized message, the first handler which accepts the field is\n// invoked. If no handler accepts the field, the field is skipped.\n//\n// The serialized message is read from a `Reader`, `Cord`, or string. Other\n// sources can be expressed as a `Reader` or string.\n\n// Field handlers\n// --------------\n//\n// Field handlers handle fields with particular field numbers and wire types\n// (varint, fixed32, fixed64, length-delimited, start-group, or end-group)\n// by invoking specific actions.\n//\n// There are three categories of field handlers:\n//\n//  * Static: the field number and wire type are statically known.\n//\n//  * Dynamic: they can handle a variety of field numbers and wire types,\n//    determined at runtime.\n//\n//  * Unbound: the field number is unspecified, while the wire type is\n//    statically known.\n//\n// Static and dynamic field handlers can be used directly with\n// `SerializedMessageReader`. Unbound field handlers are meant to be registered\n// with `DynamicFieldHandler` or `FieldHandlerMap`, with field numbers known\n// at runtime.\n//\n// A family of functions returning static and unbound field handlers is defined\n// in namespace `riegeli::field_handlers`.\n//\n// The primary dynamic field handlers are `DynamicFieldHandler`,\n// `FieldHandlerMap`, `DynamicFieldCopier`, and `AnyFieldCopier`.\n//\n// All field handlers stored in a single `SerializedMessageReader` are usually\n// conceptually associated with a single message type.\n//\n// If a field handler returns a failed `absl::Status`, `ReadMessage()` is\n// cancelled and propagates the status, annotated by the `Reader` and/or with\n// the field number. Annotations are skipped for `absl::CancelledError()` to\n// make it more efficient to cancel a handler when cancellation is likely.\n//\n// A field handler can also be expressed as a reference to a const-qualified\n// proper field handler, to avoid `SerializedMessageReader` taking the\n// ownership. Use `std::cref()` in a `SerializedMessageReader()` call.\n\n// Context types\n// -------------\n//\n// It is recommended to make field handler actions stateless, i.e. independent\n// from any state specific to the message object being read. This makes field\n// handlers and `SerializedMessageReader` themselves stateless. Such a\n// `SerializedMessageReader` can usually be stored in a `static constexpr`\n// variable and reused for multiple messages.\n//\n// To facilitate this, the `SerializedMessageReader`, its field handlers, and\n// their actions are parameterized by a sequence of `Context` types. Actions of\n// field handlers receive additional `Context&...` parameters. Reading a\n// serialized message provides `Context&...` arguments.\n\n// Field handler protocol\n// ----------------------\n//\n// Users of defined field handlers do not need to be concerned with this.\n// This is relevant for writing custom field handlers.\n//\n// A field handler of a length-delimited field is applicable to a `Reader`\n// source, and possibly also directly applicable to a `Cord` and/or string\n// source. Applicability to a string source implies applicability to a `Reader`\n// and `Cord` source, by reading a `string_view` from the `Reader` or `Cord`\n// first.\n//\n// If `SerializedMessageReader::ReadMessage()` is called with a `Cord` or\n// string, and some field handlers are not directly applicable to that source,\n// then the source is wrapped in an appropriate `Reader`. This is done on the\n// level of the whole `ReadMessage()`, instead of creating a `Reader` for each\n// affected field, so that the cost of creating a `Reader` is paid once.\n//\n// Field handlers have a `static constexpr int kFieldNumber` member variable:\n//  * For static field handlers: a positive field number.\n//  * For dynamic field handlers: `kDynamicFieldNumber`.\n//  * For unbound field handlers: `kUnboundFieldNumber`.\n//\n// Static and unbound field handlers provide at least one of the following\n// member functions, with parameters followed by `Context&...`:\n// ```\n//   absl::Status HandleVarint(uint64_t repr) const;\n//\n//   absl::Status HandleFixed32(uint32_t repr) const;\n//\n//   absl::Status HandleFixed64(uint64_t repr) const;\n//\n//   // Directly applicable to a `Reader` source.\n//   //\n//   // `HandleLengthDelimitedFromReader()` must read to the end of the\n//   // `ReaderSpan<>` or fail. `SkipLengthDelimitedFromReader()` can be used\n//   // to ensure this property.\n//   absl::Status HandleLengthDelimitedFromReader(ReaderSpan<> repr) const;\n//\n//   // Directly applicable to a `Cord` source.\n//   //\n//   // May use `scratch` for storage for temporary data.\n//   //\n//   // `HandleLengthDelimitedFromCord()` must read to the end of the\n//   // `CordIteratorSpan` or fail. `SkipLengthDelimitedFromCord()` can be used\n//   // to ensure this property.\n//   absl::Status HandleLengthDelimitedFromCord(CordIteratorSpan repr,\n//                                              std::string& scratch) const;\n//\n//   // Directly applicable to a string source.\n//   absl::Status HandleLengthDelimitedFromString(absl::string_view repr)\n//       const;\n//\n//   absl::Status HandleStartGroup() const;\n//\n//   absl::Status HandleEndGroup() const;\n// ```\n//\n// For a `Reader` source, `HandleLengthDelimitedFromReader()` or\n// `HandleLengthDelimitedFromString()` is used, depending on what is defined.\n//\n// For a `Cord` source, `HandleLengthDelimitedFromCord()` or\n// `HandleLengthDelimitedFromString()` is used, depending on what is defined.\n// If neither is defined but `HandleLengthDelimitedFromReader()` is defined,\n// then the source is wrapped in a `CordReader`.\n//\n// For a string source, `HandleLengthDelimitedFromString()` is used. If it is\n// not defined but `HandleLengthDelimitedFromReader()` is defined, then the\n// source is wrapped in a `StringReader`.\n//\n// For a string source, if `HandleLengthDelimitedFromString()` is used, then the\n// `absl::string_view` is guaranteed to be a substring of the original string.\n// This guarantee is absent for a `Reader` or `Cord` source.\n//\n// Dynamic field handlers provide at least one of the following pairs of\n// member functions corresponding to some wire type `X` as above, with `T...`\n// as above, and with `DynamicHandleX()` parameters followed by `Context&...`\n// ```\n//   MaybeAccepted AcceptX(int field_number) const;`\n//\n//   absl::Status DynamicHandleX(Accepted accepted, T... repr) const;\n// ```\n//\n// For length-delimited fields, a single `AcceptLengthDelimited()` function\n// corresponds to `DynamicHandleLengthDelimitedFromReader()`,\n// `DynamicHandleLengthDelimitedFromCord()`, and/or\n// `DynamicHandleLengthDelimitedFromString()`.\n//\n// `MaybeAccepted` is some type explicitly convertible to `bool`, with\n// `operator*` returning some `Accepted` type. `MaybeAccepted` can be e.g.\n// `std::optional<Accepted>` or `Accepted*`. If `AcceptX()` returns a value\n// explicitly convertible to `true`, then the field is accepted, and the\n// corresponding `HandleX()` function is called with the result of `operator*`\n// as the first argument.\n\n// In `FieldHandler::kFieldNumber`, marks a dynamic field handler.\ninline constexpr int kDynamicFieldNumber =\n    serialized_message_reader_internal::kDynamicFieldNumber;\n\n// In `FieldHandler::kFieldNumber`, marks an unbound field handler.\ninline constexpr int kUnboundFieldNumber =\n    serialized_message_reader_internal::kUnboundFieldNumber;\n\n// `IsFieldHandler<T, Context...>::value` is `true` if `T` is a valid argument\n// type for `SerializedMessageReader<Context...>`.\ntemplate <typename T, typename... Context>\nstruct IsFieldHandler;\n\n// `IsUnboundFieldHandlerFromString<T, Context...>::value` is `true` if `T`\n// is a valid argument type for `DynamicFieldHandler()` or\n// `FieldHandlerMap::Builder::RegisterField()`.\n//\n// A length-delimited field handler must be directly applicable to a string\n// source.\ntemplate <typename T, typename... Context>\nstruct IsUnboundFieldHandlerFromString;\n\n// Like `IsUnboundFieldHandlerFromString`, but does not require the field\n// handler to be directly applicable to a string source.\ntemplate <typename T, typename... Context>\nstruct IsUnboundFieldHandler;\n\n// `IsUnboundFieldHandler<T, Context...>::value` is `true` if `T` is a valid\n// argument type for `FieldHandlerMap::Builder::RegisterParent()`, assuming that\n// `Context...` begins with `const FieldHandlerMap&`.\n//\n// The field handler must be directly applicable to a string source.\ntemplate <typename T, typename... Context>\nstruct IsUnboundFieldHandlerForLengthDelimitedFromString;\n\n// Like `IsUnboundFieldHandlerForLengthDelimitedFromString`, but does not\n// require the field handler to be directly applicable to a string source.\ntemplate <typename T, typename... Context>\nstruct IsUnboundFieldHandlerForLengthDelimited;\n\n// For technical reasons related to template argument deduction,\n// `SerializedMessageReader` is not a class template but a function template.\n// Its return type is called `SerializedMessageReaderType`.\n//\n// The type is usually spelled `const auto`, preferably `static constexpr auto`.\n//\n// `FieldHandlers` is a `std::tuple` of field handlers.\ntemplate <typename FieldHandlers, typename... Context>\nclass SerializedMessageReaderType;\n\ntemplate <typename... FieldHandlers, typename... Context>\nclass SerializedMessageReaderType<std::tuple<FieldHandlers...>, Context...> {\n public:\n  // Creates a `SerializedMessageReader` with default-initialized field\n  // handlers. Designed for `Reset()`.\n  SerializedMessageReaderType() = default;\n\n  // Constructs a `SerializedMessageReader` from field handlers.\n  template <\n      typename... FieldHandlerInitializers,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::bool_constant<(sizeof...(FieldHandlerInitializers) > 0)>,\n              NotSameRef<SerializedMessageReaderType,\n                         FieldHandlerInitializers&&...>,\n              std::is_convertible<FieldHandlerInitializers&&,\n                                  FieldHandlers>...>,\n          int> = 0>\n  explicit constexpr SerializedMessageReaderType(\n      FieldHandlerInitializers&&... field_handlers)\n      : field_handlers_(\n            std::forward<FieldHandlerInitializers>(field_handlers)...) {}\n\n  SerializedMessageReaderType(const SerializedMessageReaderType&) = default;\n  SerializedMessageReaderType& operator=(const SerializedMessageReaderType&) =\n      default;\n\n  SerializedMessageReaderType(SerializedMessageReaderType&& that) = default;\n  SerializedMessageReaderType& operator=(SerializedMessageReaderType&& that) =\n      default;\n\n  // Makes `*this` equivalent to a newly constructed `SerializedMessageReader`.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset() {\n    ResetImpl(std::index_sequence_for<FieldHandlers...>());\n  }\n  template <\n      typename... FieldHandlerInitializers,\n      std::enable_if_t<\n          std::conjunction_v<\n              std::bool_constant<(sizeof...(FieldHandlerInitializers) > 0)>,\n              SupportsReset<FieldHandlers, FieldHandlerInitializers&&>...>,\n          int> = 0>\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(\n      FieldHandlerInitializers&&... field_handlers) {\n    ResetImpl(std::index_sequence_for<FieldHandlers...>(),\n              std::forward<FieldHandlerInitializers>(field_handlers)...);\n  }\n\n  // Reads a serialized message, letting field handlers process the fields.\n  //\n  // Reading from a string is more efficient than from other sources if all\n  // field handlers are applicable to a string source. Otherwise, the source is\n  // wrapped in an appropriate `Reader` if needed.\n  //\n  // For a `Reader` source, if any field handler handles length-delimited\n  // fields, then the root `Reader` will be wrapped in a `LimitingReader`,\n  // unless it is already statically known to be a `LimitingReaderBase`.\n  // This allows field handlers to use `ScopedLimiter` for reading the value\n  // of a length-delimited field.\n  //\n  // If `Reader::SupportsSize()` and it was wrapped in a `LimitingReader`,\n  // then the `LimitingReader` is initially limited to the whole message.\n  // This helps parsing untrusted data: if the size of the message is bounded,\n  // then claimed lengths of length-delimited fields are bounded as well,\n  // and thus it is safe to e.g. pass such a length to `Reader::Read()`.\n  template <typename Src,\n            std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value,\n                             int> = 0>\n  absl::Status ReadMessage(Src&& src, Context&... context) const;\n  absl::Status ReadMessage(BytesRef src, Context&... context) const;\n  absl::Status ReadMessage(const Chain& src, Context&... context) const;\n  absl::Status ReadMessage(const absl::Cord& src, Context&... context) const;\n  absl::Status ReadMessage(CordIteratorSpan src, Context&... context) const;\n\n private:\n  template <size_t... indices>\n  void ResetImpl(std::index_sequence<indices...>) {\n    (riegeli::Reset(std::get<indices>(field_handlers_)), ...);\n  }\n  template <\n      size_t... indices, typename... FieldHandlerInitializers,\n      std::enable_if_t<(sizeof...(FieldHandlerInitializers) > 0), int> = 0>\n  void ResetImpl(std::index_sequence<indices...>,\n                 FieldHandlerInitializers&&... field_handlers) {\n    (riegeli::Reset(std::get<indices>(field_handlers_),\n                    std::forward<FieldHandlerInitializers>(field_handlers)...),\n     ...);\n  }\n\n  template <typename ReaderType>\n  absl::Status ReadMessageFromReader(ReaderType& src,\n                                     Context&... context) const;\n  absl::Status ReadMessageFromCord(absl::Cord::CharIterator& src,\n                                   size_t available, Context&... context) const;\n  absl::Status ReadMessageFromString(absl::string_view src,\n                                     Context&... context) const;\n\n  template <size_t index = 0>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool HandleVarintField(\n      int field_number, uint64_t value, absl::Status& status,\n      Context&... context) const {\n    if constexpr (index < sizeof...(FieldHandlers)) {\n      return (serialized_message_reader_internal::ReadVarintField(\n                  field_number, value, status, std::get<index>(field_handlers_),\n                  context...) ||\n              HandleVarintField<index + 1>(field_number, value, status,\n                                           context...));\n    } else {\n      return false;\n    }\n  }\n\n  template <size_t index = 0>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool HandleFixed32Field(\n      int field_number, uint32_t value, absl::Status& status,\n      Context&... context) const {\n    if constexpr (index < sizeof...(FieldHandlers)) {\n      return (serialized_message_reader_internal::ReadFixed32Field(\n                  field_number, value, status, std::get<index>(field_handlers_),\n                  context...) ||\n              HandleFixed32Field<index + 1>(field_number, value, status,\n                                            context...));\n    } else {\n      return false;\n    }\n  }\n\n  template <size_t index = 0>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool HandleFixed64Field(\n      int field_number, uint64_t value, absl::Status& status,\n      Context&... context) const {\n    if constexpr (index < sizeof...(FieldHandlers)) {\n      return (serialized_message_reader_internal::ReadFixed64Field(\n                  field_number, value, status, std::get<index>(field_handlers_),\n                  context...) ||\n              HandleFixed64Field<index + 1>(field_number, value, status,\n                                            context...));\n    } else {\n      return false;\n    }\n  }\n\n  template <size_t index = 0>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool HandleLengthDelimitedFieldFromReader(\n      int field_number, LimitingReaderBase& src, size_t length,\n      absl::Status& status, Context&... context) const {\n    if constexpr (index < sizeof...(FieldHandlers)) {\n      return (serialized_message_reader_internal::\n                  ReadLengthDelimitedFieldFromReader(\n                      field_number, src, length, status,\n                      std::get<index>(field_handlers_), context...) ||\n              HandleLengthDelimitedFieldFromReader<index + 1>(\n                  field_number, src, length, status, context...));\n    } else {\n      return false;\n    }\n  }\n\n  template <size_t index = 0>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool HandleLengthDelimitedFieldFromCord(\n      int field_number, absl::Cord::CharIterator& src, size_t length,\n      std::string& scratch, absl::Status& status, Context&... context) const {\n    if constexpr (index < sizeof...(FieldHandlers)) {\n      return (\n          serialized_message_reader_internal::ReadLengthDelimitedFieldFromCord(\n              field_number, src, length, scratch, status,\n              std::get<index>(field_handlers_), context...) ||\n          HandleLengthDelimitedFieldFromCord<index + 1>(\n              field_number, src, length, scratch, status, context...));\n    } else {\n      return false;\n    }\n  }\n\n  template <size_t index = 0>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool HandleLengthDelimitedFieldFromString(\n      int field_number, const char* cursor, size_t length, absl::Status& status,\n      Context&... context) const {\n    if constexpr (index < sizeof...(FieldHandlers)) {\n      return (serialized_message_reader_internal::\n                  ReadLengthDelimitedFieldFromString(\n                      field_number, cursor, length, status,\n                      std::get<index>(field_handlers_), context...) ||\n              HandleLengthDelimitedFieldFromString<index + 1>(\n                  field_number, cursor, length, status, context...));\n    } else {\n      return false;\n    }\n  }\n\n  template <size_t index = 0>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool HandleStartGroupField(\n      int field_number, absl::Status& status, Context&... context) const {\n    if constexpr (index < sizeof...(FieldHandlers)) {\n      return (\n          serialized_message_reader_internal::ReadStartGroupField(\n              field_number, status, std::get<index>(field_handlers_),\n              context...) ||\n          HandleStartGroupField<index + 1>(field_number, status, context...));\n    } else {\n      return false;\n    }\n  }\n\n  template <size_t index = 0>\n  ABSL_ATTRIBUTE_ALWAYS_INLINE bool HandleEndGroupField(\n      int field_number, absl::Status& status, Context&... context) const {\n    if constexpr (index < sizeof...(FieldHandlers)) {\n      return (serialized_message_reader_internal::ReadEndGroupField(\n                  field_number, status, std::get<index>(field_handlers_),\n                  context...) ||\n              HandleEndGroupField<index + 1>(field_number, status, context...));\n    } else {\n      return false;\n    }\n  }\n\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS std::tuple<FieldHandlers...> field_handlers_;\n};\n\n// Returns a `SerializedMessageReaderType` which reads serialized messages using\n// the given field handlers.\n//\n// Typical usage:\n// ```\n//   static constexpr auto message_reader =\n//       riegeli::SerializedMessageReader<Context...>(\n//           field_handlers...);\n//   absl::Status status = message_reader.ReadMessage(src, context...);\n// ```\n//\n// In the `message_reader` name, it can be helpful to replace `message` with the\n// actual message type being read.\n//\n// `Context` types must be specified explicitly for `SerializedMessageReader`.\n// Field handlers and their actions must accept compatible `Context&...`\n// parameters.\ntemplate <\n    typename... Context, typename... FieldHandlerInitializers\n#if !__cpp_concepts\n    ,\n    std::enable_if_t<std::conjunction_v<IsFieldHandler<\n                         TargetT<FieldHandlerInitializers>, Context...>...>,\n                     int> = 0\n#endif\n    >\n#if __cpp_concepts\n// For conjunctions, `requires` gives better error messages than\n// `std::enable_if_t`, indicating the relevant argument.\n  requires(\n      IsFieldHandler<TargetT<FieldHandlerInitializers>, Context...>::value &&\n      ...)\n#endif\nconstexpr SerializedMessageReaderType<\n    std::tuple<TargetT<FieldHandlerInitializers>...>, Context...>\nSerializedMessageReader(FieldHandlerInitializers&&... field_handlers) {\n  return SerializedMessageReaderType<\n      std::tuple<TargetT<FieldHandlerInitializers>...>, Context...>(\n      std::forward<FieldHandlerInitializers>(field_handlers)...);\n}\n\n// In the field handler protocol, `HandleLengthDelimitedFromReader()` must read\n// to the end of the `ReaderSpan<>` or fail.\n//\n// `SkipLengthDelimitedFromReader()` can be used to ensure this property.\n// With an action, the part of the field not read by the action is skipped.\n// Without an action, the whole field is skipped.\n\ntemplate <typename Action,\n          std::enable_if_t<std::is_invocable_v<Action>, int> = 0>\nabsl::Status SkipLengthDelimitedFromReader(const ReaderSpan<>& value,\n                                           Action&& action);\n\nabsl::Status SkipLengthDelimitedFromReader(ReaderSpan<> value);\n\n// In the field handler protocol, `HandleLengthDelimitedFromCord()` must read\n// to the end of the `CordIteratorSpan` or fail.\n//\n// `SkipLengthDelimitedFromCord()` can be used to ensure this property.\n// With an action, the part of the field not read by the action is skipped.\n// Without an action, the whole field is skipped.\n\ntemplate <typename Action,\n          std::enable_if_t<std::is_invocable_v<Action>, int> = 0>\nabsl::Status SkipLengthDelimitedFromCord(const CordIteratorSpan& value,\n                                         Action&& action);\n\nabsl::Status SkipLengthDelimitedFromCord(CordIteratorSpan value);\n\n// Implementation details follow.\n\ntemplate <typename T, typename... Context>\nstruct IsFieldHandler\n    : std::disjunction<serialized_message_reader_internal::IsStaticFieldHandler<\n                           T, Context...>,\n                       serialized_message_reader_internal::\n                           IsDynamicFieldHandler<T, Context...>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsUnboundFieldHandlerFromString\n    : serialized_message_reader_internal::IsUnboundFieldHandlerFromString<\n          T, Context...> {};\n\ntemplate <typename T, typename... Context>\nstruct IsUnboundFieldHandler\n    : serialized_message_reader_internal::IsUnboundFieldHandler<T, Context...> {\n};\n\ntemplate <typename T, typename... Context>\nstruct IsUnboundFieldHandlerForLengthDelimitedFromString\n    : std::conjunction<\n          serialized_message_reader_internal::\n              IsFieldHandlerWithUnboundFieldNumber<T>,\n          serialized_message_reader_internal::\n              IsStaticFieldHandlerForLengthDelimitedFromString<T, Context...>> {\n};\n\ntemplate <typename T, typename... Context>\nstruct IsUnboundFieldHandlerForLengthDelimited\n    : std::conjunction<\n          serialized_message_reader_internal::\n              IsFieldHandlerWithUnboundFieldNumber<T>,\n          serialized_message_reader_internal::\n              IsStaticFieldHandlerForLengthDelimited<T, Context...>> {};\n\ntemplate <typename... FieldHandlers, typename... Context>\ntemplate <typename ReaderType>\nabsl::Status SerializedMessageReaderType<\n    std::tuple<FieldHandlers...>,\n    Context...>::ReadMessageFromReader(ReaderType& src,\n                                       Context&... context) const {\n  uint32_t tag;\n  while (ReadVarint32(src, tag)) {\n    const int field_number = GetTagFieldNumber(tag);\n    switch (GetTagWireType(tag)) {\n      case WireType::kVarint: {\n        if constexpr (std::disjunction_v<\n                          serialized_message_reader_internal::\n                              IsFieldHandlerForVarint<FieldHandlers,\n                                                      Context...>...>) {\n          uint64_t value;\n          if (ABSL_PREDICT_FALSE(!ReadVarint64(src, value))) {\n            return serialized_message_reader_internal::ReadVarintError(\n                src, field_number);\n          }\n          absl::Status status;\n          if (HandleVarintField(field_number, value, status, context...)) {\n            if (ABSL_PREDICT_FALSE(!status.ok())) {\n              return serialized_message_reader_internal::\n                  AnnotateWithSourceAndFieldNumber(std::move(status), src,\n                                                   field_number);\n            }\n          }\n        } else {\n          // The value is not needed. Use more efficient `SkipVarint64()`\n          // instead of `ReadVarint64()`.\n          if (ABSL_PREDICT_FALSE(!SkipVarint64(src))) {\n            return serialized_message_reader_internal::ReadVarintError(\n                src, field_number);\n          }\n        }\n        continue;\n      }\n      case WireType::kFixed32: {\n        uint32_t value;\n        if (ABSL_PREDICT_FALSE(!ReadLittleEndian<uint32_t>(src, value))) {\n          return serialized_message_reader_internal::ReadFixed32Error(\n              src, field_number);\n        }\n        absl::Status status;\n        if (HandleFixed32Field(field_number, value, status, context...)) {\n          if (ABSL_PREDICT_FALSE(!status.ok())) {\n            return serialized_message_reader_internal::\n                AnnotateWithSourceAndFieldNumber(std::move(status), src,\n                                                 field_number);\n          }\n        }\n        continue;\n      }\n      case WireType::kFixed64: {\n        uint64_t value;\n        if (ABSL_PREDICT_FALSE(!ReadLittleEndian<uint64_t>(src, value))) {\n          return serialized_message_reader_internal::ReadFixed64Error(\n              src, field_number);\n        }\n        absl::Status status;\n        if (HandleFixed64Field(field_number, value, status, context...)) {\n          if (ABSL_PREDICT_FALSE(!status.ok())) {\n            return serialized_message_reader_internal::\n                AnnotateWithSourceAndFieldNumber(std::move(status), src,\n                                                 field_number);\n          }\n        }\n        continue;\n      }\n      case WireType::kLengthDelimited: {\n        uint32_t length;\n        if (ABSL_PREDICT_FALSE(\n                !ReadVarint32(src, length) ||\n                length > uint32_t{std::numeric_limits<int32_t>::max()})) {\n          return serialized_message_reader_internal::\n              ReadLengthDelimitedLengthError(src, field_number);\n        }\n        if constexpr (std::disjunction_v<\n                          serialized_message_reader_internal::\n                              IsFieldHandlerForLengthDelimited<\n                                  FieldHandlers, Context...>...>) {\n          static_assert(\n              std::is_same_v<ReaderType, LimitingReaderBase>,\n              \"If there are any field handlers for length-delimited fields, \"\n              \"ReadInternal() must be called with LimitingReaderBase\");\n          if (ABSL_PREDICT_FALSE(length > src.max_length())) {\n            return serialized_message_reader_internal::NotEnoughError(\n                src, field_number, length);\n          }\n          const Position end_pos = src.pos() + size_t{length};\n          absl::Status status;\n          if (HandleLengthDelimitedFieldFromReader(\n                  field_number, src, size_t{length}, status, context...)) {\n            if (ABSL_PREDICT_FALSE(!status.ok())) {\n              return serialized_message_reader_internal::\n                  AnnotateWithFieldNumber(std::move(status), field_number);\n            }\n            RIEGELI_ASSERT_EQ(src.pos(), end_pos)\n                << \"A field handler of a length-delimited field \"\n                   \"must read to the end of the ReaderSpan<> or fail; \"\n                   \"the value of field \"\n                << field_number << \" at position \" << (end_pos - size_t{length})\n                << \" has length \" << size_t{length} << \" but length \"\n                << static_cast<int64_t>(src.pos() - (end_pos - size_t{length}))\n                << \" has been read; \"\n                   \"consider using SkipLengthDelimitedFromReader()\";\n            continue;\n          }\n        }\n        if (ABSL_PREDICT_FALSE(!src.Skip(size_t{length}))) {\n          return serialized_message_reader_internal::\n              ReadLengthDelimitedValueError(src, field_number);\n        }\n        continue;\n      }\n      case WireType::kStartGroup: {\n        absl::Status status;\n        if (HandleStartGroupField(field_number, status, context...)) {\n          if (ABSL_PREDICT_FALSE(!status.ok())) {\n            return serialized_message_reader_internal::\n                AnnotateWithSourceAndFieldNumber(std::move(status), src,\n                                                 field_number);\n          }\n        }\n        continue;\n      }\n      case WireType::kEndGroup: {\n        absl::Status status;\n        if (HandleEndGroupField(field_number, status, context...)) {\n          if (ABSL_PREDICT_FALSE(!status.ok())) {\n            return serialized_message_reader_internal::\n                AnnotateWithSourceAndFieldNumber(std::move(status), src,\n                                                 field_number);\n          }\n        }\n        continue;\n      }\n      case WireType::kInvalid6:\n      case WireType::kInvalid7:\n        return serialized_message_reader_internal::InvalidWireTypeError(tag);\n    }\n    RIEGELI_ASSUME_UNREACHABLE()\n        << \"Impossible wire type: \" << static_cast<int>(GetTagWireType(tag));\n  }\n  if (ABSL_PREDICT_FALSE(src.available() > 0)) {\n    return serialized_message_reader_internal::ReadTagError(src);\n  }\n  return absl::OkStatus();\n}\n\ntemplate <typename... FieldHandlers, typename... Context>\ninline absl::Status SerializedMessageReaderType<\n    std::tuple<FieldHandlers...>,\n    Context...>::ReadMessageFromCord(absl::Cord::CharIterator& src,\n                                     size_t available,\n                                     Context&... context) const {\n  const size_t limit = CordIteratorSpan::Remaining(src) - available;\n  std::string scratch;\n  uint32_t tag;\n  while (ReadVarint32(src, CordIteratorSpan::Remaining(src) - limit, tag)) {\n    const int field_number = GetTagFieldNumber(tag);\n    switch (GetTagWireType(tag)) {\n      case WireType::kVarint: {\n        if constexpr (std::disjunction_v<\n                          serialized_message_reader_internal::\n                              IsFieldHandlerForVarint<FieldHandlers,\n                                                      Context...>...>) {\n          uint64_t value;\n          if (ABSL_PREDICT_FALSE(!ReadVarint64(\n                  src, CordIteratorSpan::Remaining(src) - limit, value))) {\n            return serialized_message_reader_internal::ReadVarintError(\n                field_number);\n          }\n          absl::Status status;\n          if (HandleVarintField(field_number, value, status, context...)) {\n            if (ABSL_PREDICT_FALSE(!status.ok())) {\n              return serialized_message_reader_internal::\n                  AnnotateWithFieldNumber(std::move(status), field_number);\n            }\n          }\n        } else {\n          // The value is not needed. Use more efficient `SkipVarint64()`\n          // instead of `ReadVarint64()`.\n          if (ABSL_PREDICT_FALSE(!SkipVarint64(\n                  src, CordIteratorSpan::Remaining(src) - limit))) {\n            return serialized_message_reader_internal::ReadVarintError(\n                field_number);\n          }\n        }\n        continue;\n      }\n      case WireType::kFixed32: {\n        if (ABSL_PREDICT_FALSE(CordIteratorSpan::Remaining(src) - limit <\n                               sizeof(uint32_t))) {\n          return serialized_message_reader_internal::ReadFixed32Error(\n              field_number);\n        }\n        if constexpr (std::disjunction_v<\n                          serialized_message_reader_internal::\n                              IsFieldHandlerForFixed32<FieldHandlers,\n                                                       Context...>...>) {\n          char buffer[sizeof(uint32_t)];\n          CordIteratorSpan::Read(src, sizeof(uint32_t), buffer);\n          const uint32_t value = ReadLittleEndian<uint32_t>(buffer);\n          absl::Status status;\n          if (HandleFixed32Field(field_number, value, status, context...)) {\n            if (ABSL_PREDICT_FALSE(!status.ok())) {\n              return serialized_message_reader_internal::\n                  AnnotateWithFieldNumber(std::move(status), field_number);\n            }\n          }\n        } else {\n          // The value is not needed. Use more efficient `Cord::Advance()`\n          // instead of `CordIteratorSpan::Read()`.\n          absl::Cord::Advance(&src, sizeof(uint32_t));\n        }\n        continue;\n      }\n      case WireType::kFixed64: {\n        if (ABSL_PREDICT_FALSE(CordIteratorSpan::Remaining(src) - limit <\n                               sizeof(uint64_t))) {\n          return serialized_message_reader_internal::ReadFixed64Error(\n              field_number);\n        }\n        if constexpr (std::disjunction_v<\n                          serialized_message_reader_internal::\n                              IsFieldHandlerForFixed64<FieldHandlers,\n                                                       Context...>...>) {\n          char buffer[sizeof(uint64_t)];\n          CordIteratorSpan::Read(src, sizeof(uint64_t), buffer);\n          const uint64_t value = ReadLittleEndian<uint64_t>(buffer);\n          absl::Status status;\n          if (HandleFixed64Field(field_number, value, status, context...)) {\n            if (ABSL_PREDICT_FALSE(!status.ok())) {\n              return serialized_message_reader_internal::\n                  AnnotateWithFieldNumber(std::move(status), field_number);\n            }\n          }\n        } else {\n          // The value is not needed. Use more efficient `Cord::Advance()`\n          // instead of `CordIteratorSpan::Read()`.\n          absl::Cord::Advance(&src, sizeof(uint64_t));\n        }\n        continue;\n      }\n      case WireType::kLengthDelimited: {\n        uint32_t length;\n        if (ABSL_PREDICT_FALSE(\n                !ReadVarint32(src, CordIteratorSpan::Remaining(src) - limit,\n                              length) ||\n                length > uint32_t{std::numeric_limits<int32_t>::max()})) {\n          return serialized_message_reader_internal::\n              ReadLengthDelimitedLengthError(field_number);\n        }\n        const size_t available_for_value =\n            CordIteratorSpan::Remaining(src) - limit;\n        if (ABSL_PREDICT_FALSE(length > available_for_value)) {\n          return serialized_message_reader_internal::NotEnoughError(\n              field_number, length, available_for_value);\n        }\n        if constexpr (std::disjunction_v<\n                          serialized_message_reader_internal::\n                              IsFieldHandlerForLengthDelimited<\n                                  FieldHandlers, Context...>...>) {\n          absl::Status status;\n          if (HandleLengthDelimitedFieldFromCord(field_number, src,\n                                                 size_t{length}, scratch,\n                                                 status, context...)) {\n            if (ABSL_PREDICT_FALSE(!status.ok())) {\n              return serialized_message_reader_internal::\n                  AnnotateWithFieldNumber(std::move(status), field_number);\n            }\n            RIEGELI_ASSERT_EQ(CordIteratorSpan::Remaining(src),\n                              available_for_value + limit - size_t{length})\n                << \"A field handler of a length-delimited field \"\n                   \"must read to the end of the CordIteratorSpan or fail; \"\n                   \"the value of field \"\n                << field_number << \" has length \" << size_t{length}\n                << \" but length \"\n                << (available_for_value + limit -\n                    CordIteratorSpan::Remaining(src))\n                << \" has been read\";\n            continue;\n          }\n        }\n        absl::Cord::Advance(&src, size_t{length});\n        continue;\n      }\n      case WireType::kStartGroup: {\n        absl::Status status;\n        if (HandleStartGroupField(field_number, status, context...)) {\n          if (ABSL_PREDICT_FALSE(!status.ok())) {\n            return serialized_message_reader_internal::AnnotateWithFieldNumber(\n                std::move(status), field_number);\n          }\n        }\n        continue;\n      }\n      case WireType::kEndGroup: {\n        absl::Status status;\n        if (HandleEndGroupField(field_number, status, context...)) {\n          if (ABSL_PREDICT_FALSE(!status.ok())) {\n            return serialized_message_reader_internal::AnnotateWithFieldNumber(\n                std::move(status), field_number);\n          }\n        }\n        continue;\n      }\n      case WireType::kInvalid6:\n      case WireType::kInvalid7:\n        return serialized_message_reader_internal::InvalidWireTypeError(tag);\n    }\n    RIEGELI_ASSUME_UNREACHABLE()\n        << \"Impossible wire type: \" << static_cast<int>(GetTagWireType(tag));\n  }\n  if (ABSL_PREDICT_FALSE(CordIteratorSpan::Remaining(src) > limit)) {\n    return serialized_message_reader_internal::ReadTagError();\n  }\n  return absl::OkStatus();\n}\n\ntemplate <typename... FieldHandlers, typename... Context>\ninline absl::Status SerializedMessageReaderType<\n    std::tuple<FieldHandlers...>,\n    Context...>::ReadMessageFromString(absl::string_view src,\n                                       Context&... context) const {\n  const char* absl_nullable cursor = src.data();\n  const char* const absl_nullable limit = src.data() + src.size();\n  uint32_t tag;\n  while (const size_t tag_length =\n             ReadVarint32(cursor, PtrDistance(cursor, limit), tag)) {\n    RIEGELI_ASSERT(cursor != nullptr);\n    RIEGELI_ASSERT(limit != nullptr);\n    cursor += tag_length;\n    const int field_number = GetTagFieldNumber(tag);\n    switch (GetTagWireType(tag)) {\n      case WireType::kVarint: {\n        if constexpr (std::disjunction_v<\n                          serialized_message_reader_internal::\n                              IsFieldHandlerForVarint<FieldHandlers,\n                                                      Context...>...>) {\n          uint64_t value;\n          const size_t length_of_value =\n              ReadVarint64(cursor, PtrDistance(cursor, limit), value);\n          if (ABSL_PREDICT_FALSE(length_of_value == 0)) {\n            return serialized_message_reader_internal::ReadVarintError(\n                field_number);\n          }\n          cursor += length_of_value;\n          absl::Status status;\n          if (HandleVarintField(field_number, value, status, context...)) {\n            if (ABSL_PREDICT_FALSE(!status.ok())) {\n              return serialized_message_reader_internal::\n                  AnnotateWithFieldNumber(std::move(status), field_number);\n            }\n          }\n        } else {\n          // The value is not needed. Use more efficient `SkipVarint64()`\n          // instead of `ReadVarint64()`.\n          const size_t length_of_value =\n              SkipVarint64(cursor, PtrDistance(cursor, limit));\n          if (ABSL_PREDICT_FALSE(length_of_value == 0)) {\n            return serialized_message_reader_internal::ReadVarintError(\n                field_number);\n          }\n          cursor += length_of_value;\n        }\n        continue;\n      }\n      case WireType::kFixed32: {\n        if (ABSL_PREDICT_FALSE(PtrDistance(cursor, limit) < sizeof(uint32_t))) {\n          return serialized_message_reader_internal::ReadFixed32Error(\n              field_number);\n        }\n        const uint32_t value = ReadLittleEndian<uint32_t>(cursor);\n        cursor += sizeof(uint32_t);\n        absl::Status status;\n        if (HandleFixed32Field(field_number, value, status, context...)) {\n          if (ABSL_PREDICT_FALSE(!status.ok())) {\n            return serialized_message_reader_internal::AnnotateWithFieldNumber(\n                std::move(status), field_number);\n          }\n        }\n        continue;\n      }\n      case WireType::kFixed64: {\n        if (ABSL_PREDICT_FALSE(PtrDistance(cursor, limit) < sizeof(uint64_t))) {\n          return serialized_message_reader_internal::ReadFixed64Error(\n              field_number);\n        }\n        const uint64_t value = ReadLittleEndian<uint64_t>(cursor);\n        cursor += sizeof(uint64_t);\n        absl::Status status;\n        if (HandleFixed64Field(field_number, value, status, context...)) {\n          if (ABSL_PREDICT_FALSE(!status.ok())) {\n            return serialized_message_reader_internal::AnnotateWithFieldNumber(\n                std::move(status), field_number);\n          }\n        }\n        continue;\n      }\n      case WireType::kLengthDelimited: {\n        uint32_t length;\n        const size_t length_of_length =\n            ReadVarint32(cursor, PtrDistance(cursor, limit), length);\n        if (ABSL_PREDICT_FALSE(\n                length_of_length == 0 ||\n                length > uint32_t{std::numeric_limits<int32_t>::max()})) {\n          return serialized_message_reader_internal::\n              ReadLengthDelimitedLengthError(field_number);\n        }\n        cursor += length_of_length;\n        const size_t available_for_value = PtrDistance(cursor, limit);\n        if (ABSL_PREDICT_FALSE(length > available_for_value)) {\n          return serialized_message_reader_internal::NotEnoughError(\n              field_number, length, available_for_value);\n        }\n        absl::Status status;\n        if (HandleLengthDelimitedFieldFromString(\n                field_number, cursor, size_t{length}, status, context...)) {\n          if (ABSL_PREDICT_FALSE(!status.ok())) {\n            return serialized_message_reader_internal::AnnotateWithFieldNumber(\n                std::move(status), field_number);\n          }\n        }\n        cursor += size_t{length};\n        continue;\n      }\n      case WireType::kStartGroup: {\n        absl::Status status;\n        if (HandleStartGroupField(field_number, status, context...)) {\n          if (ABSL_PREDICT_FALSE(!status.ok())) {\n            return serialized_message_reader_internal::AnnotateWithFieldNumber(\n                std::move(status), field_number);\n          }\n        }\n        continue;\n      }\n      case WireType::kEndGroup: {\n        absl::Status status;\n        if (HandleEndGroupField(field_number, status, context...)) {\n          if (ABSL_PREDICT_FALSE(!status.ok())) {\n            return serialized_message_reader_internal::AnnotateWithFieldNumber(\n                std::move(status), field_number);\n          }\n        }\n        continue;\n      }\n      case WireType::kInvalid6:\n      case WireType::kInvalid7:\n        return serialized_message_reader_internal::InvalidWireTypeError(tag);\n    }\n    RIEGELI_ASSUME_UNREACHABLE()\n        << \"Impossible wire type: \" << static_cast<int>(GetTagWireType(tag));\n  }\n  if (ABSL_PREDICT_FALSE(cursor < limit)) {\n    return serialized_message_reader_internal::ReadTagError();\n  }\n  return absl::OkStatus();\n}\n\ntemplate <typename... FieldHandlers, typename... Context>\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\nabsl::Status SerializedMessageReaderType<\n    std::tuple<FieldHandlers...>, Context...>::ReadMessage(Src&& src,\n                                                           Context&... context)\n    const {\n  DependencyRef<Reader*, Src> src_dep(std::forward<Src>(src));\n  if (src_dep.IsOwning()) src_dep->SetReadAllHint(true);\n\n  absl::Status status;\n  if constexpr (std::disjunction_v<serialized_message_reader_internal::\n                                       IsFieldHandlerForLengthDelimited<\n                                           FieldHandlers, Context...>...>) {\n    if constexpr (std::is_convertible_v<\n                      typename DependencyRef<Reader*, Src>::Subhandle,\n                      LimitingReaderBase*>) {\n      status = ReadMessageFromReader<LimitingReaderBase>(*src_dep, context...);\n    } else {\n      LimitingReaderBase::Options options;\n      if (src_dep->SupportsSize()) {\n        const std::optional<Position> size = src_dep->Size();\n        if (ABSL_PREDICT_TRUE(size != std::nullopt)) options.set_max_pos(*size);\n      }\n      LimitingReader<> limiting_reader(src_dep.get(), options);\n      status = ReadMessageFromReader<LimitingReaderBase>(limiting_reader,\n                                                         context...);\n      if (ABSL_PREDICT_FALSE(!limiting_reader.Close())) {\n        status.Update(limiting_reader.status());\n      }\n    }\n  } else {\n    status = ReadMessageFromReader<Reader>(*src_dep, context...);\n  }\n\n  if (src_dep.IsOwning()) {\n    if (ABSL_PREDICT_TRUE(status.ok())) src_dep->VerifyEnd();\n    if (ABSL_PREDICT_FALSE(!src_dep->Close())) status.Update(src_dep->status());\n  }\n  return status;\n}\n\ntemplate <typename... FieldHandlers, typename... Context>\nabsl::Status SerializedMessageReaderType<\n    std::tuple<FieldHandlers...>, Context...>::ReadMessage(BytesRef src,\n                                                           Context&... context)\n    const {\n  if constexpr (std::conjunction_v<serialized_message_reader_internal::\n                                       IsFieldHandlerFromString<\n                                           FieldHandlers, Context...>...>) {\n    return ReadMessageFromString(src, context...);\n  } else {\n    return ReadMessage(StringReader(src), context...);\n  }\n}\n\ntemplate <typename... FieldHandlers, typename... Context>\nabsl::Status SerializedMessageReaderType<\n    std::tuple<FieldHandlers...>, Context...>::ReadMessage(const Chain& src,\n                                                           Context&... context)\n    const {\n  return ReadMessage(ChainReader(&src), context...);\n}\n\ntemplate <typename... FieldHandlers, typename... Context>\nabsl::Status SerializedMessageReaderType<\n    std::tuple<FieldHandlers...>,\n    Context...>::ReadMessage(const absl::Cord& src, Context&... context) const {\n  if constexpr (std::conjunction_v<std::disjunction<\n                    serialized_message_reader_internal::IsFieldHandlerFromCord<\n                        FieldHandlers, Context...>,\n                    serialized_message_reader_internal::\n                        IsFieldHandlerFromString<FieldHandlers,\n                                                 Context...>>...>) {\n    absl::Cord::CharIterator iter = src.char_begin();\n    return ReadMessageFromCord(iter, CordIteratorSpan::Remaining(iter),\n                               context...);\n  } else {\n    return ReadMessage(CordReader(&src), context...);\n  }\n}\n\ntemplate <typename... FieldHandlers, typename... Context>\nabsl::Status SerializedMessageReaderType<\n    std::tuple<FieldHandlers...>, Context...>::ReadMessage(CordIteratorSpan src,\n                                                           Context&... context)\n    const {\n  if constexpr (std::conjunction_v<std::disjunction<\n                    serialized_message_reader_internal::IsFieldHandlerFromCord<\n                        FieldHandlers, Context...>,\n                    serialized_message_reader_internal::\n                        IsFieldHandlerFromString<FieldHandlers,\n                                                 Context...>>...>) {\n    return ReadMessageFromCord(src.iterator(), src.length(), context...);\n  } else {\n    return ReadMessage(CordReader(std::move(src)), context...);\n  }\n}\n\ntemplate <typename Action, std::enable_if_t<std::is_invocable_v<Action>, int>>\ninline absl::Status SkipLengthDelimitedFromReader(const ReaderSpan<>& value,\n                                                  Action&& action) {\n  LimitingReaderBase& reader = value.reader();\n  const Position end_pos = reader.pos() + value.length();\n  if (absl::Status status = std::forward<Action>(action)();\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return status;\n  }\n  if (ABSL_PREDICT_FALSE(!reader.Seek(end_pos))) {\n    return serialized_message_reader_internal::ReadLengthDelimitedValueError(\n        reader);\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status SkipLengthDelimitedFromReader(ReaderSpan<> value) {\n  if (ABSL_PREDICT_FALSE(!value.reader().Skip(value.length()))) {\n    return serialized_message_reader_internal::ReadLengthDelimitedValueError(\n        value.reader());\n  }\n  return absl::OkStatus();\n}\n\ntemplate <typename Action, std::enable_if_t<std::is_invocable_v<Action>, int>>\ninline absl::Status SkipLengthDelimitedFromCord(const CordIteratorSpan& value,\n                                                Action&& action) {\n  absl::Cord::CharIterator& iterator = value.iterator();\n  const size_t limit = CordIteratorSpan::Remaining(iterator) - value.length();\n  if (absl::Status status = std::forward<Action>(action)();\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return status;\n  }\n  RIEGELI_ASSERT_GE(CordIteratorSpan::Remaining(iterator), limit)\n      << \"Length-delimited field handler action read past field contents\";\n  if (ABSL_PREDICT_FALSE(CordIteratorSpan::Remaining(iterator) != limit)) {\n    absl::Cord::Advance(&iterator,\n                        CordIteratorSpan::Remaining(iterator) - limit);\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status SkipLengthDelimitedFromCord(CordIteratorSpan value) {\n  absl::Cord::Advance(&value.iterator(), value.length());\n  return absl::OkStatus();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_SERIALIZED_MESSAGE_READER_H_\n"
  },
  {
    "path": "riegeli/messages/serialized_message_reader_internal.h",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_SERIALIZED_MESSAGE_READER_INTERNAL_H_\n#define RIEGELI_MESSAGES_SERIALIZED_MESSAGE_READER_INTERNAL_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli::serialized_message_reader_internal {\n\ninline constexpr int kDynamicFieldNumber = -1;\n\ninline constexpr int kUnboundFieldNumber = -2;\n\ntemplate <typename T, typename Enable = void>\nstruct IsFieldHandlerWithStaticFieldNumber : std::false_type {};\n\ntemplate <typename T>\nstruct IsFieldHandlerWithStaticFieldNumber<\n    T, std::enable_if_t<(std::remove_reference_t<T>::kFieldNumber > 0)>>\n    : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct IsFieldHandlerWithDynamicFieldNumber : std::false_type {};\n\ntemplate <typename T>\nstruct IsFieldHandlerWithDynamicFieldNumber<\n    T, std::enable_if_t<std::remove_reference_t<T>::kFieldNumber ==\n                        kDynamicFieldNumber>> : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct IsFieldHandlerWithUnboundFieldNumber : std::false_type {};\n\ntemplate <typename T>\nstruct IsFieldHandlerWithUnboundFieldNumber<\n    T, std::enable_if_t<(std::remove_reference_t<T>::kFieldNumber ==\n                         kUnboundFieldNumber)>> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsStaticFieldHandlerForVarintImpl : std::false_type {};\n\ntemplate <typename T, typename... Context>\nstruct IsStaticFieldHandlerForVarintImpl<\n    T,\n    std::void_t<decltype(std::declval<const T&>().HandleVarint(\n        std::declval<uint64_t>(), std::declval<Context&>()...))>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsStaticFieldHandlerForFixed32Impl : std::false_type {};\n\ntemplate <typename T, typename... Context>\nstruct IsStaticFieldHandlerForFixed32Impl<\n    T,\n    std::void_t<decltype(std::declval<const T&>().HandleFixed32(\n        std::declval<uint32_t>(), std::declval<Context&>()...))>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsStaticFieldHandlerForFixed64Impl : std::false_type {};\n\ntemplate <typename T, typename... Context>\nstruct IsStaticFieldHandlerForFixed64Impl<\n    T,\n    std::void_t<decltype(std::declval<const T&>().HandleFixed64(\n        std::declval<uint64_t>(), std::declval<Context&>()...))>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsStaticFieldHandlerForLengthDelimitedFromReaderImpl : std::false_type {\n};\n\ntemplate <typename T, typename... Context>\nstruct IsStaticFieldHandlerForLengthDelimitedFromReaderImpl<\n    T,\n    std::void_t<\n        decltype(std::declval<const T&>().HandleLengthDelimitedFromReader(\n            std::declval<ReaderSpan<>>(), std::declval<Context&>()...))>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsStaticFieldHandlerForLengthDelimitedFromCordImpl : std::false_type {};\n\ntemplate <typename T, typename... Context>\nstruct IsStaticFieldHandlerForLengthDelimitedFromCordImpl<\n    T,\n    std::void_t<decltype(std::declval<const T&>().HandleLengthDelimitedFromCord(\n        std::declval<CordIteratorSpan>(), std::declval<std::string&>(),\n        std::declval<Context&>()...))>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsStaticFieldHandlerForLengthDelimitedFromStringImpl : std::false_type {\n};\n\ntemplate <typename T, typename... Context>\nstruct IsStaticFieldHandlerForLengthDelimitedFromStringImpl<\n    T,\n    std::void_t<\n        decltype(std::declval<const T&>().HandleLengthDelimitedFromString(\n            std::declval<absl::string_view>(), std::declval<Context&>()...))>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsStaticFieldHandlerForStartGroupImpl : std::false_type {};\n\ntemplate <typename T, typename... Context>\nstruct IsStaticFieldHandlerForStartGroupImpl<\n    T,\n    std::void_t<decltype(std::declval<const T&>().HandleStartGroup(\n        std::declval<Context&>()...))>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsStaticFieldHandlerForEndGroupImpl : std::false_type {};\n\ntemplate <typename T, typename... Context>\nstruct IsStaticFieldHandlerForEndGroupImpl<\n    T,\n    std::void_t<decltype(std::declval<const T&>().HandleEndGroup(\n        std::declval<Context&>()...))>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct IsDynamicFieldHandlerForVarintSomeContext : std::false_type {};\n\ntemplate <typename T>\nstruct IsDynamicFieldHandlerForVarintSomeContext<\n    T, std::enable_if_t<std::is_constructible_v<\n           bool, decltype(std::declval<const T&>().AcceptVarint(\n                     std::declval<int>()))>>> : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct IsDynamicFieldHandlerForFixed32SomeContext : std::false_type {};\n\ntemplate <typename T>\nstruct IsDynamicFieldHandlerForFixed32SomeContext<\n    T, std::enable_if_t<std::is_constructible_v<\n           bool, decltype(std::declval<const T&>().AcceptFixed32(\n                     std::declval<int>()))>>> : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct IsDynamicFieldHandlerForFixed64SomeContext : std::false_type {};\n\ntemplate <typename T>\nstruct IsDynamicFieldHandlerForFixed64SomeContext<\n    T, std::enable_if_t<std::is_constructible_v<\n           bool, decltype(std::declval<const T&>().AcceptFixed64(\n                     std::declval<int>()))>>> : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct IsDynamicFieldHandlerForLengthDelimitedSomeContext : std::false_type {};\n\ntemplate <typename T>\nstruct IsDynamicFieldHandlerForLengthDelimitedSomeContext<\n    T, std::enable_if_t<std::is_constructible_v<\n           bool, decltype(std::declval<const T&>().AcceptLengthDelimited(\n                     std::declval<int>()))>>> : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct IsDynamicFieldHandlerForStartGroupSomeContext : std::false_type {};\n\ntemplate <typename T>\nstruct IsDynamicFieldHandlerForStartGroupSomeContext<\n    T, std::enable_if_t<std::is_constructible_v<\n           bool, decltype(std::declval<const T&>().AcceptStartGroup(\n                     std::declval<int>()))>>> : std::true_type {};\n\ntemplate <typename T, typename Enable = void>\nstruct IsDynamicFieldHandlerForEndGroupSomeContext : std::false_type {};\n\ntemplate <typename T>\nstruct IsDynamicFieldHandlerForEndGroupSomeContext<\n    T, std::enable_if_t<std::is_constructible_v<\n           bool, decltype(std::declval<const T&>().AcceptEndGroup(\n                     std::declval<int>()))>>> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsDynamicFieldHandlerForVarintImpl : std::false_type {};\n\ntemplate <typename T, typename... Context>\nstruct IsDynamicFieldHandlerForVarintImpl<\n    T,\n    std::enable_if_t<\n        IsDynamicFieldHandlerForVarintSomeContext<T>::value,\n        std::void_t<decltype(std::declval<const T&>().DynamicHandleVarint(\n            *std::declval<const T&>().AcceptVarint(std::declval<int>()),\n            std::declval<uint64_t>(), std::declval<Context&>()...))>>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsDynamicFieldHandlerForFixed32Impl : std::false_type {};\n\ntemplate <typename T, typename... Context>\nstruct IsDynamicFieldHandlerForFixed32Impl<\n    T,\n    std::enable_if_t<\n        IsDynamicFieldHandlerForFixed32SomeContext<T>::value,\n        std::void_t<decltype(std::declval<const T&>().DynamicHandleFixed32(\n            *std::declval<const T&>().AcceptFixed32(std::declval<int>()),\n            std::declval<uint32_t>(), std::declval<Context&>()...))>>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsDynamicFieldHandlerForFixed64Impl : std::false_type {};\n\ntemplate <typename T, typename... Context>\nstruct IsDynamicFieldHandlerForFixed64Impl<\n    T,\n    std::enable_if_t<\n        IsDynamicFieldHandlerForFixed64SomeContext<T>::value,\n        std::void_t<decltype(std::declval<const T&>().DynamicHandleFixed64(\n            *std::declval<const T&>().AcceptFixed64(std::declval<int>()),\n            std::declval<uint64_t>(), std::declval<Context&>()...))>>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsDynamicFieldHandlerForLengthDelimitedFromReaderImpl : std::false_type {\n};\n\ntemplate <typename T, typename... Context>\nstruct IsDynamicFieldHandlerForLengthDelimitedFromReaderImpl<\n    T,\n    std::enable_if_t<\n        IsDynamicFieldHandlerForLengthDelimitedSomeContext<T>::value,\n        std::void_t<\n            decltype(std::declval<const T&>()\n                         .DynamicHandleLengthDelimitedFromReader(\n                             *std::declval<const T&>().AcceptLengthDelimited(\n                                 std::declval<int>()),\n                             std::declval<ReaderSpan<>>(),\n                             std::declval<Context&>()...))>>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsDynamicFieldHandlerForLengthDelimitedFromCordImpl : std::false_type {};\n\ntemplate <typename T, typename... Context>\nstruct IsDynamicFieldHandlerForLengthDelimitedFromCordImpl<\n    T,\n    std::enable_if_t<\n        IsDynamicFieldHandlerForLengthDelimitedSomeContext<T>::value,\n        std::void_t<\n            decltype(std::declval<const T&>()\n                         .DynamicHandleLengthDelimitedFromCord(\n                             *std::declval<const T&>().AcceptLengthDelimited(\n                                 std::declval<int>()),\n                             std::declval<CordIteratorSpan>(),\n                             std::declval<std::string&>(),\n                             std::declval<Context&>()...))>>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsDynamicFieldHandlerForLengthDelimitedFromStringImpl : std::false_type {\n};\n\ntemplate <typename T, typename... Context>\nstruct IsDynamicFieldHandlerForLengthDelimitedFromStringImpl<\n    T,\n    std::enable_if_t<\n        IsDynamicFieldHandlerForLengthDelimitedSomeContext<T>::value,\n        std::void_t<\n            decltype(std::declval<const T&>()\n                         .DynamicHandleLengthDelimitedFromString(\n                             *std::declval<const T&>().AcceptLengthDelimited(\n                                 std::declval<int>()),\n                             std::declval<absl::string_view>(),\n                             std::declval<Context&>()...))>>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsDynamicFieldHandlerForStartGroupImpl : std::false_type {};\n\ntemplate <typename T, typename... Context>\nstruct IsDynamicFieldHandlerForStartGroupImpl<\n    T,\n    std::enable_if_t<\n        IsDynamicFieldHandlerForStartGroupSomeContext<T>::value,\n        std::void_t<decltype(std::declval<const T&>().DynamicHandleStartGroup(\n            *std::declval<const T&>().AcceptStartGroup(std::declval<int>()),\n            std::declval<Context&>()...))>>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename Enable, typename... Context>\nstruct IsDynamicFieldHandlerForEndGroupImpl : std::false_type {};\n\ntemplate <typename T, typename... Context>\nstruct IsDynamicFieldHandlerForEndGroupImpl<\n    T,\n    std::enable_if_t<\n        IsDynamicFieldHandlerForEndGroupSomeContext<T>::value,\n        std::void_t<decltype(std::declval<const T&>().DynamicHandleEndGroup(\n            *std::declval<const T&>().AcceptEndGroup(std::declval<int>()),\n            std::declval<Context&>()...))>>,\n    Context...> : std::true_type {};\n\ntemplate <typename T, typename... Context>\nusing IsStaticFieldHandlerForVarint =\n    IsStaticFieldHandlerForVarintImpl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nusing IsStaticFieldHandlerForFixed32 =\n    IsStaticFieldHandlerForFixed32Impl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nusing IsStaticFieldHandlerForFixed64 =\n    IsStaticFieldHandlerForFixed64Impl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nusing IsStaticFieldHandlerForLengthDelimitedFromReader =\n    IsStaticFieldHandlerForLengthDelimitedFromReaderImpl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nusing IsStaticFieldHandlerForLengthDelimitedFromCord =\n    IsStaticFieldHandlerForLengthDelimitedFromCordImpl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nusing IsStaticFieldHandlerForLengthDelimitedFromString =\n    IsStaticFieldHandlerForLengthDelimitedFromStringImpl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nstruct IsStaticFieldHandlerForLengthDelimited\n    : std::disjunction<\n          IsStaticFieldHandlerForLengthDelimitedFromReader<T, Context...>,\n          // `IsStaticFieldHandlerForLengthDelimitedFromCord` alone\n          // is insufficient.\n          IsStaticFieldHandlerForLengthDelimitedFromString<T, Context...>> {};\n\ntemplate <typename T, typename... Context>\nusing IsStaticFieldHandlerForStartGroup =\n    IsStaticFieldHandlerForStartGroupImpl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nusing IsStaticFieldHandlerForEndGroup =\n    IsStaticFieldHandlerForEndGroupImpl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nusing IsDynamicFieldHandlerForVarint =\n    IsDynamicFieldHandlerForVarintImpl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nusing IsDynamicFieldHandlerForFixed32 =\n    IsDynamicFieldHandlerForFixed32Impl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nusing IsDynamicFieldHandlerForFixed64 =\n    IsDynamicFieldHandlerForFixed64Impl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nusing IsDynamicFieldHandlerForLengthDelimitedFromReader =\n    IsDynamicFieldHandlerForLengthDelimitedFromReaderImpl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nusing IsDynamicFieldHandlerForLengthDelimitedFromCord =\n    IsDynamicFieldHandlerForLengthDelimitedFromCordImpl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nusing IsDynamicFieldHandlerForLengthDelimitedFromString =\n    IsDynamicFieldHandlerForLengthDelimitedFromStringImpl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nstruct IsDynamicFieldHandlerForLengthDelimited\n    : std::disjunction<\n          IsDynamicFieldHandlerForLengthDelimitedFromReader<T, Context...>,\n          // `IsDynamicFieldHandlerForLengthDelimitedFromCord` alone\n          // is insufficient.\n          IsDynamicFieldHandlerForLengthDelimitedFromString<T, Context...>> {};\n\ntemplate <typename T, typename... Context>\nusing IsDynamicFieldHandlerForStartGroup =\n    IsDynamicFieldHandlerForStartGroupImpl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nusing IsDynamicFieldHandlerForEndGroup =\n    IsDynamicFieldHandlerForEndGroupImpl<T, void, Context...>;\n\ntemplate <typename T, typename... Context>\nstruct IsFieldHandlerForVarint\n    : std::disjunction<IsStaticFieldHandlerForVarint<T, Context...>,\n                       IsDynamicFieldHandlerForVarint<T, Context...>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsFieldHandlerForFixed32\n    : std::disjunction<IsStaticFieldHandlerForFixed32<T, Context...>,\n                       IsDynamicFieldHandlerForFixed32<T, Context...>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsFieldHandlerForFixed64\n    : std::disjunction<IsStaticFieldHandlerForFixed64<T, Context...>,\n                       IsDynamicFieldHandlerForFixed64<T, Context...>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsFieldHandlerForLengthDelimitedFromReader\n    : std::disjunction<\n          IsStaticFieldHandlerForLengthDelimitedFromReader<T, Context...>,\n          IsDynamicFieldHandlerForLengthDelimitedFromReader<T, Context...>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsFieldHandlerForLengthDelimitedFromCord\n    : std::disjunction<\n          IsStaticFieldHandlerForLengthDelimitedFromCord<T, Context...>,\n          IsDynamicFieldHandlerForLengthDelimitedFromCord<T, Context...>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsFieldHandlerForLengthDelimitedFromString\n    : std::disjunction<\n          IsStaticFieldHandlerForLengthDelimitedFromString<T, Context...>,\n          IsDynamicFieldHandlerForLengthDelimitedFromString<T, Context...>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsFieldHandlerForLengthDelimited\n    : std::disjunction<IsStaticFieldHandlerForLengthDelimited<T, Context...>,\n                       IsDynamicFieldHandlerForLengthDelimited<T, Context...>> {\n};\n\ntemplate <typename T, typename... Context>\nstruct IsFieldHandlerForStartGroup\n    : std::disjunction<IsStaticFieldHandlerForStartGroup<T, Context...>,\n                       IsDynamicFieldHandlerForStartGroup<T, Context...>>,\n      IsDynamicFieldHandlerForStartGroup<T, Context...> {};\n\ntemplate <typename T, typename... Context>\nstruct IsFieldHandlerForEndGroup\n    : std::disjunction<IsStaticFieldHandlerForEndGroup<T, Context...>,\n                       IsDynamicFieldHandlerForEndGroup<T, Context...>>,\n      IsDynamicFieldHandlerForEndGroup<T, Context...> {};\n\ntemplate <typename T, typename... Context>\nstruct IsStaticFieldHandlerFromCord\n    : std::conjunction<\n          IsFieldHandlerWithStaticFieldNumber<T>,\n          std::disjunction<\n              IsStaticFieldHandlerForVarint<T, Context...>,\n              IsStaticFieldHandlerForFixed32<T, Context...>,\n              IsStaticFieldHandlerForFixed64<T, Context...>,\n              IsStaticFieldHandlerForLengthDelimitedFromCord<T, Context...>,\n              IsStaticFieldHandlerForStartGroup<T, Context...>,\n              IsStaticFieldHandlerForEndGroup<T, Context...>>,\n          std::disjunction<\n              IsStaticFieldHandlerForLengthDelimitedFromCord<T, Context...>,\n              std::negation<std::disjunction<\n                  IsStaticFieldHandlerForLengthDelimitedFromReader<T,\n                                                                   Context...>,\n                  IsStaticFieldHandlerForLengthDelimitedFromString<\n                      T, Context...>>>>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsStaticFieldHandlerFromString\n    : std::conjunction<\n          IsFieldHandlerWithStaticFieldNumber<T>,\n          std::disjunction<\n              IsStaticFieldHandlerForVarint<T, Context...>,\n              IsStaticFieldHandlerForFixed32<T, Context...>,\n              IsStaticFieldHandlerForFixed64<T, Context...>,\n              IsStaticFieldHandlerForLengthDelimitedFromString<T, Context...>,\n              IsStaticFieldHandlerForStartGroup<T, Context...>,\n              IsStaticFieldHandlerForEndGroup<T, Context...>>,\n          std::disjunction<\n              IsStaticFieldHandlerForLengthDelimitedFromString<T, Context...>,\n              std::negation<std::disjunction<\n                  IsStaticFieldHandlerForLengthDelimitedFromReader<T,\n                                                                   Context...>,\n                  IsStaticFieldHandlerForLengthDelimitedFromCord<\n                      T, Context...>>>>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsStaticFieldHandler\n    : std::conjunction<\n          IsFieldHandlerWithStaticFieldNumber<T>,\n          std::disjunction<\n              IsStaticFieldHandlerForVarint<T, Context...>,\n              IsStaticFieldHandlerForFixed32<T, Context...>,\n              IsStaticFieldHandlerForFixed64<T, Context...>,\n              IsStaticFieldHandlerForLengthDelimited<T, Context...>,\n              IsStaticFieldHandlerForStartGroup<T, Context...>,\n              IsStaticFieldHandlerForEndGroup<T, Context...>>,\n          std::disjunction<\n              IsStaticFieldHandlerForLengthDelimited<T, Context...>,\n              std::negation<IsStaticFieldHandlerForLengthDelimitedFromCord<\n                  T, Context...>>>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsDynamicFieldHandlerFromCord\n    : std::conjunction<\n          IsFieldHandlerWithDynamicFieldNumber<T>,\n          std::disjunction<\n              IsDynamicFieldHandlerForVarint<T, Context...>,\n              IsDynamicFieldHandlerForFixed32<T, Context...>,\n              IsDynamicFieldHandlerForFixed64<T, Context...>,\n              IsDynamicFieldHandlerForLengthDelimitedFromCord<T, Context...>,\n              IsDynamicFieldHandlerForStartGroup<T, Context...>,\n              IsDynamicFieldHandlerForEndGroup<T, Context...>>,\n          std::disjunction<\n              IsDynamicFieldHandlerForLengthDelimitedFromCord<T, Context...>,\n              std::negation<std::disjunction<\n                  IsDynamicFieldHandlerForLengthDelimitedFromReader<T,\n                                                                    Context...>,\n                  IsDynamicFieldHandlerForLengthDelimitedFromString<\n                      T, Context...>>>>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsDynamicFieldHandlerFromString\n    : std::conjunction<\n          IsFieldHandlerWithDynamicFieldNumber<T>,\n          std::disjunction<\n              IsDynamicFieldHandlerForVarint<T, Context...>,\n              IsDynamicFieldHandlerForFixed32<T, Context...>,\n              IsDynamicFieldHandlerForFixed64<T, Context...>,\n              IsDynamicFieldHandlerForLengthDelimitedFromString<T, Context...>,\n              IsDynamicFieldHandlerForStartGroup<T, Context...>,\n              IsDynamicFieldHandlerForEndGroup<T, Context...>>,\n          std::disjunction<\n              IsDynamicFieldHandlerForLengthDelimitedFromString<T, Context...>,\n              std::negation<IsDynamicFieldHandlerForLengthDelimitedFromReader<\n                  T, Context...>>>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsDynamicFieldHandler\n    : std::conjunction<\n          IsFieldHandlerWithDynamicFieldNumber<T>,\n          std::disjunction<\n              IsDynamicFieldHandlerForVarint<T, Context...>,\n              IsDynamicFieldHandlerForFixed32<T, Context...>,\n              IsDynamicFieldHandlerForFixed64<T, Context...>,\n              IsDynamicFieldHandlerForLengthDelimited<T, Context...>,\n              IsDynamicFieldHandlerForStartGroup<T, Context...>,\n              IsDynamicFieldHandlerForEndGroup<T, Context...>>,\n          std::disjunction<\n              IsDynamicFieldHandlerForLengthDelimited<T, Context...>,\n              std::negation<IsDynamicFieldHandlerForLengthDelimitedFromCord<\n                  T, Context...>>>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsFieldHandlerFromCord\n    : std::disjunction<IsStaticFieldHandlerFromCord<T, Context...>,\n                       IsDynamicFieldHandlerFromCord<T, Context...>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsFieldHandlerFromString\n    : std::disjunction<IsStaticFieldHandlerFromString<T, Context...>,\n                       IsDynamicFieldHandlerFromString<T, Context...>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsUnboundFieldHandlerFromString\n    : std::conjunction<\n          IsFieldHandlerWithUnboundFieldNumber<T>,\n          std::disjunction<\n              IsStaticFieldHandlerForVarint<T, Context...>,\n              IsStaticFieldHandlerForFixed32<T, Context...>,\n              IsStaticFieldHandlerForFixed64<T, Context...>,\n              IsStaticFieldHandlerForLengthDelimitedFromString<T, Context...>,\n              IsStaticFieldHandlerForStartGroup<T, Context...>,\n              IsStaticFieldHandlerForEndGroup<T, Context...>>> {};\n\ntemplate <typename T, typename... Context>\nstruct IsUnboundFieldHandler\n    : std::conjunction<\n          IsFieldHandlerWithUnboundFieldNumber<T>,\n          std::disjunction<\n              IsStaticFieldHandlerForVarint<T, Context...>,\n              IsStaticFieldHandlerForFixed32<T, Context...>,\n              IsStaticFieldHandlerForFixed64<T, Context...>,\n              IsStaticFieldHandlerForLengthDelimited<T, Context...>,\n              IsStaticFieldHandlerForStartGroup<T, Context...>,\n              IsStaticFieldHandlerForEndGroup<T, Context...>>,\n          std::disjunction<\n              IsStaticFieldHandlerForLengthDelimited<T, Context...>,\n              std::negation<IsStaticFieldHandlerForLengthDelimitedFromCord<\n                  T, Context...>>>> {};\n\nABSL_ATTRIBUTE_COLD absl::Status AnnotateWithFieldNumberSlow(\n    absl::Status status, int field_number);\nABSL_ATTRIBUTE_COLD absl::Status AnnotateWithSourceAndFieldNumberSlow(\n    absl::Status status, Reader& src, int field_number);\n\ninline absl::Status AnnotateWithFieldNumber(absl::Status status,\n                                            int field_number) {\n  // Comparison against `absl::CancelledError()` is a fast path of\n  // `absl::IsCancelled()`.\n  if (ABSL_PREDICT_FALSE(status != absl::CancelledError())) {\n    status = AnnotateWithFieldNumberSlow(std::move(status), field_number);\n  }\n  return status;\n}\n\ninline absl::Status AnnotateWithSourceAndFieldNumber(absl::Status status,\n                                                     Reader& src,\n                                                     int field_number) {\n  // Comparison against `absl::CancelledError()` is a fast path of\n  // `absl::IsCancelled()`.\n  if (ABSL_PREDICT_FALSE(status != absl::CancelledError())) {\n    status = AnnotateWithSourceAndFieldNumberSlow(std::move(status), src,\n                                                  field_number);\n  }\n  return status;\n}\n\nABSL_ATTRIBUTE_COLD absl::Status ReadTagError();\nABSL_ATTRIBUTE_COLD absl::Status ReadTagError(Reader& src);\nABSL_ATTRIBUTE_COLD absl::Status ReadVarintError(int field_number);\nABSL_ATTRIBUTE_COLD absl::Status ReadVarintError(Reader& src, int field_number);\nABSL_ATTRIBUTE_COLD absl::Status ReadFixed32Error(int field_number);\nABSL_ATTRIBUTE_COLD absl::Status ReadFixed32Error(Reader& src,\n                                                  int field_number);\nABSL_ATTRIBUTE_COLD absl::Status ReadFixed64Error(int field_number);\nABSL_ATTRIBUTE_COLD absl::Status ReadFixed64Error(Reader& src,\n                                                  int field_number);\nABSL_ATTRIBUTE_COLD absl::Status NotEnoughError(int field_number,\n                                                uint32_t expected_length,\n                                                size_t available);\nABSL_ATTRIBUTE_COLD absl::Status NotEnoughError(LimitingReaderBase& src,\n                                                int field_number,\n                                                uint32_t expected_length);\nABSL_ATTRIBUTE_COLD absl::Status ReadLengthDelimitedLengthError(\n    int field_number);\nABSL_ATTRIBUTE_COLD absl::Status ReadLengthDelimitedLengthError(\n    Reader& src, int field_number);\nABSL_ATTRIBUTE_COLD absl::Status ReadLengthDelimitedValueError(Reader& src);\nABSL_ATTRIBUTE_COLD absl::Status ReadLengthDelimitedValueError(\n    Reader& src, int field_number);\nABSL_ATTRIBUTE_COLD absl::Status InvalidWireTypeError(uint32_t tag);\nABSL_ATTRIBUTE_COLD absl::Status InvalidWireTypeError(Reader& src,\n                                                      uint32_t tag);\n\ntemplate <typename FieldHandler, typename... Context>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool ReadVarintField(\n    int field_number, uint64_t repr, absl::Status& status,\n    const FieldHandler& field_handler, Context&... context) {\n  if constexpr (IsStaticFieldHandlerForVarint<FieldHandler,\n                                              Context...>::value) {\n    if (field_number == FieldHandler::kFieldNumber) {\n      status = field_handler.HandleVarint(repr, context...);\n      return true;\n    }\n  }\n  if constexpr (IsDynamicFieldHandlerForVarint<FieldHandler,\n                                               Context...>::value) {\n    auto maybe_accepted = field_handler.AcceptVarint(field_number);\n    if (maybe_accepted) {\n      status = field_handler.DynamicHandleVarint(*std::move(maybe_accepted),\n                                                 repr, context...);\n      return true;\n    }\n  }\n  return false;\n}\n\ntemplate <typename FieldHandler, typename... Context>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool ReadFixed32Field(\n    int field_number, uint32_t repr, absl::Status& status,\n    const FieldHandler& field_handler, Context&... context) {\n  if constexpr (IsStaticFieldHandlerForFixed32<FieldHandler,\n                                               Context...>::value) {\n    if (field_number == FieldHandler::kFieldNumber) {\n      status = field_handler.HandleFixed32(repr, context...);\n      return true;\n    }\n  }\n  if constexpr (IsDynamicFieldHandlerForFixed32<FieldHandler,\n                                                Context...>::value) {\n    auto maybe_accepted = field_handler.AcceptFixed32(field_number);\n    if (maybe_accepted) {\n      status = field_handler.DynamicHandleFixed32(*std::move(maybe_accepted),\n                                                  repr, context...);\n      return true;\n    }\n  }\n  return false;\n}\n\ntemplate <typename FieldHandler, typename... Context>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool ReadFixed64Field(\n    int field_number, uint64_t repr, absl::Status& status,\n    const FieldHandler& field_handler, Context&... context) {\n  if constexpr (IsStaticFieldHandlerForFixed64<FieldHandler,\n                                               Context...>::value) {\n    if (field_number == FieldHandler::kFieldNumber) {\n      status = field_handler.HandleFixed64(repr, context...);\n      return true;\n    }\n  }\n  if constexpr (IsDynamicFieldHandlerForFixed64<FieldHandler,\n                                                Context...>::value) {\n    auto maybe_accepted = field_handler.AcceptFixed64(field_number);\n    if (maybe_accepted) {\n      status = field_handler.DynamicHandleFixed64(*std::move(maybe_accepted),\n                                                  repr, context...);\n      return true;\n    }\n  }\n  return false;\n}\n\ntemplate <typename FieldHandler, typename... Context>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool ReadLengthDelimitedFieldFromReader(\n    int field_number, LimitingReaderBase& src, size_t length,\n    absl::Status& status, const FieldHandler& field_handler,\n    Context&... context) {\n  if constexpr (IsStaticFieldHandlerForLengthDelimitedFromReader<\n                    FieldHandler, Context...>::value) {\n    if (field_number == FieldHandler::kFieldNumber) {\n      status = field_handler.HandleLengthDelimitedFromReader(\n          ReaderSpan<>(&src, length), context...);\n      return true;\n    }\n  } else if constexpr (IsStaticFieldHandlerForLengthDelimitedFromString<\n                           FieldHandler, Context...>::value) {\n    if (field_number == FieldHandler::kFieldNumber) {\n      absl::string_view value;\n      if (ABSL_PREDICT_FALSE(!src.Read(length, value))) {\n        status = ReadLengthDelimitedValueError(src);\n        return true;\n      }\n      status = field_handler.HandleLengthDelimitedFromString(value, context...);\n      return true;\n    }\n  }\n  if constexpr (IsDynamicFieldHandlerForLengthDelimitedFromReader<\n                    FieldHandler, Context...>::value) {\n    auto maybe_accepted = field_handler.AcceptLengthDelimited(field_number);\n    if (maybe_accepted) {\n      status = field_handler.DynamicHandleLengthDelimitedFromReader(\n          *std::move(maybe_accepted), ReaderSpan<>(&src, length), context...);\n      return true;\n    }\n  } else if constexpr (IsDynamicFieldHandlerForLengthDelimitedFromString<\n                           FieldHandler, Context...>::value) {\n    auto maybe_accepted = field_handler.AcceptLengthDelimited(field_number);\n    if (maybe_accepted) {\n      absl::string_view value;\n      if (ABSL_PREDICT_FALSE(!src.Read(length, value))) {\n        status = ReadLengthDelimitedValueError(src);\n        return true;\n      }\n      status = field_handler.DynamicHandleLengthDelimitedFromString(\n          *std::move(maybe_accepted), value, context...);\n      return true;\n    }\n  }\n  return false;\n}\n\ntemplate <typename FieldHandler, typename... Context>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool ReadLengthDelimitedFieldFromCord(\n    int field_number, absl::Cord::CharIterator& src, size_t length,\n    std::string& scratch, absl::Status& status,\n    const FieldHandler& field_handler, Context&... context) {\n  if constexpr (IsStaticFieldHandlerForLengthDelimitedFromCord<\n                    FieldHandler, Context...>::value) {\n    if (field_number == FieldHandler::kFieldNumber) {\n      status = field_handler.HandleLengthDelimitedFromCord(\n          CordIteratorSpan(&src, length), scratch, context...);\n      return true;\n    }\n  } else if constexpr (IsStaticFieldHandlerForLengthDelimitedFromString<\n                           FieldHandler, Context...>::value) {\n    if (field_number == FieldHandler::kFieldNumber) {\n      status = field_handler.HandleLengthDelimitedFromString(\n          CordIteratorSpan(&src, length).ToStringView(scratch), context...);\n      return true;\n    }\n  }\n  if constexpr (IsDynamicFieldHandlerForLengthDelimitedFromCord<\n                    FieldHandler, Context...>::value) {\n    auto maybe_accepted = field_handler.AcceptLengthDelimited(field_number);\n    if (maybe_accepted) {\n      status = field_handler.DynamicHandleLengthDelimitedFromCord(\n          *std::move(maybe_accepted), CordIteratorSpan(&src, length), scratch,\n          context...);\n      return true;\n    }\n  } else if constexpr (IsDynamicFieldHandlerForLengthDelimitedFromString<\n                           FieldHandler, Context...>::value) {\n    auto maybe_accepted = field_handler.AcceptLengthDelimited(field_number);\n    if (maybe_accepted) {\n      status = field_handler.DynamicHandleLengthDelimitedFromString(\n          *std::move(maybe_accepted),\n          CordIteratorSpan(&src, length).ToStringView(scratch), context...);\n      return true;\n    }\n  }\n  return false;\n}\n\ntemplate <typename FieldHandler, typename... Context>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool ReadLengthDelimitedFieldFromString(\n    int field_number, const char* src, size_t length, absl::Status& status,\n    const FieldHandler& field_handler, Context&... context) {\n  if constexpr (IsStaticFieldHandlerForLengthDelimitedFromString<\n                    FieldHandler, Context...>::value) {\n    if (field_number == FieldHandler::kFieldNumber) {\n      status = field_handler.HandleLengthDelimitedFromString(\n          absl::string_view(src, length), context...);\n      return true;\n    }\n  }\n  if constexpr (IsDynamicFieldHandlerForLengthDelimitedFromString<\n                    FieldHandler, Context...>::value) {\n    auto maybe_accepted = field_handler.AcceptLengthDelimited(field_number);\n    if (maybe_accepted) {\n      status = field_handler.DynamicHandleLengthDelimitedFromString(\n          *std::move(maybe_accepted), absl::string_view(src, length),\n          context...);\n      return true;\n    }\n  }\n  return false;\n}\n\ntemplate <typename FieldHandler, typename... Context>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool ReadStartGroupField(\n    int field_number, absl::Status& status, const FieldHandler& field_handler,\n    Context&... context) {\n  if constexpr (IsStaticFieldHandlerForStartGroup<FieldHandler,\n                                                  Context...>::value) {\n    if (field_number == FieldHandler::kFieldNumber) {\n      status = field_handler.HandleStartGroup(context...);\n      return true;\n    }\n  }\n  if constexpr (IsDynamicFieldHandlerForStartGroup<FieldHandler,\n                                                   Context...>::value) {\n    auto maybe_accepted = field_handler.AcceptStartGroup(field_number);\n    if (maybe_accepted) {\n      status = field_handler.DynamicHandleStartGroup(*std::move(maybe_accepted),\n                                                     context...);\n      return true;\n    }\n  }\n  return false;\n}\n\ntemplate <typename FieldHandler, typename... Context>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool ReadEndGroupField(\n    int field_number, absl::Status& status, const FieldHandler& field_handler,\n    Context&... context) {\n  if constexpr (IsStaticFieldHandlerForEndGroup<FieldHandler,\n                                                Context...>::value) {\n    if (field_number == FieldHandler::kFieldNumber) {\n      status = field_handler.HandleEndGroup(context...);\n      return true;\n    }\n  }\n  if constexpr (IsDynamicFieldHandlerForEndGroup<FieldHandler,\n                                                 Context...>::value) {\n    auto maybe_accepted = field_handler.AcceptEndGroup(field_number);\n    if (maybe_accepted) {\n      status = field_handler.DynamicHandleEndGroup(*std::move(maybe_accepted),\n                                                   context...);\n      return true;\n    }\n  }\n  return false;\n}\n\n}  // namespace riegeli::serialized_message_reader_internal\n\n#endif  // RIEGELI_MESSAGES_SERIALIZED_MESSAGE_READER_INTERNAL_H_\n"
  },
  {
    "path": "riegeli/messages/serialized_message_writer.cc",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/messages/serialized_message_writer.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"riegeli/base/any.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/cord_writer.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/bytes/read_all.h\"\n#include \"riegeli/bytes/reader.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\nabsl::Status SerializedMessageWriter::LengthOverflowError(Position length) {\n  return absl::ResourceExhaustedError(\n      absl::StrCat(\"Failed to write length-delimited field \"\n                   \"because its size must be smaller than 2GiB: \",\n                   length));\n}\n\nabsl::Status SerializedMessageWriter::WriteStringFailed(Reader& src,\n                                                        Writer& dest) {\n  return !dest.ok()\n             ? dest.status()\n             : src.StatusOrAnnotate(absl::InvalidArgumentError(\n                   \"Could not read contents for a length-delimited field\"));\n}\n\nabsl::Status SerializedMessageWriter::WriteString(int field_number,\n                                                  AnyRef<Reader*> src) {\n  if (src.IsOwning()) src->SetReadAllHint(true);\n  if (src->SupportsSize()) {\n    const std::optional<Position> size = src->Size();\n    if (ABSL_PREDICT_FALSE(size == std::nullopt)) return src->status();\n    if (absl::Status status = WriteString(\n            field_number,\n            ReaderSpan(src.get(), SaturatingSub(*size, src->pos())));\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n    if (src.IsOwning()) {\n      if (ABSL_PREDICT_FALSE(!src->Close())) return src->status();\n    }\n    return absl::OkStatus();\n  } else {\n    absl::Cord contents;\n    if (absl::Status status = ReadAll(std::move(src), contents);\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n    return WriteString(field_number, std::move(contents));\n  }\n}\n\nabsl::Status SerializedMessageWriter::WriteString(int field_number,\n                                                  CordIteratorSpan src) {\n  if (src.length() <= kMaxBytesToCopy) {\n    if (absl::Status status = WriteLengthUnchecked(field_number, src.length());\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n    if (ABSL_PREDICT_FALSE(!writer().Push(src.length()))) {\n      return writer().status();\n    }\n    CordIteratorSpan::Read(src.iterator(), src.length(), writer().cursor());\n    writer().move_cursor(src.length());\n    return absl::OkStatus();\n  }\n  return WriteString(field_number, std::move(src).ToCord());\n}\n\nvoid SerializedMessageWriter::OpenLengthDelimited() {\n  writer_ = &submessages_.emplace_back();\n}\n\nSerializedMessageWriter SerializedMessageWriter::NewLengthDelimited()\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  SerializedMessageWriter message(&writer());\n  message.OpenLengthDelimited();\n  return message;\n}\n\nabsl::Status SerializedMessageWriter::CloseLengthDelimited(int field_number) {\n  RIEGELI_ASSERT(!submessages_.empty())\n      << \"Failed precondition of \"\n         \"SerializedMessageWriter::CloseLengthDelimited(): \"\n         \"no matching OpenLengthDelimited() call\";\n  RIEGELI_ASSERT(writer_ == &submessages_.back())\n      << \"Failed invariant of SerializedMessageWriter: \"\n         \"writer() does not point to the most recently open submessage\";\n  CordWriter<absl::Cord>& submessage = submessages_.back();\n  if (ABSL_PREDICT_FALSE(!submessage.Close())) return submessage.status();\n  writer_ = submessages_.size() > 1 ? &submessages_.end()[-2] : dest_;\n  if (absl::Status status =\n          WriteLengthUnchecked(field_number, submessage.dest().size());\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return status;\n  }\n  RIEGELI_ASSERT(writer_ != nullptr)\n      << \"Failed precondition of CloseLengthDelimited(): \"\n         \"set_dest() not called before the last CloseLengthDelimited()\";\n  if (ABSL_PREDICT_FALSE(!writer_->Write(std::move(submessage.dest())))) {\n    return writer_->status();\n  }\n  submessages_.pop_back();\n  return absl::OkStatus();\n}\n\nabsl::Status SerializedMessageWriter::CloseOptionalLengthDelimited(\n    int field_number) {\n  RIEGELI_ASSERT(!submessages_.empty())\n      << \"Failed precondition of \"\n         \"SerializedMessageWriter::CloseOptionalLengthDelimited(): \"\n         \"no matching OpenLengthDelimited() call\";\n  RIEGELI_ASSERT(writer_ == &submessages_.back())\n      << \"Failed invariant of SerializedMessageWriter: \"\n         \"writer() does not point to the most recently open submessage\";\n  CordWriter<absl::Cord>& submessage = submessages_.back();\n  if (ABSL_PREDICT_FALSE(!submessage.Close())) return submessage.status();\n  writer_ = submessages_.size() > 1 ? &submessages_.end()[-2] : dest_;\n  if (!submessage.dest().empty()) {\n    if (absl::Status status =\n            WriteLengthUnchecked(field_number, submessage.dest().size());\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n    RIEGELI_ASSERT(writer_ != nullptr)\n        << \"Failed precondition of CloseOptionalLengthDelimited(): \"\n           \"set_dest() not called before the last \"\n           \"CloseOptionalLengthDelimited()\";\n    if (ABSL_PREDICT_FALSE(!writer_->Write(std::move(submessage.dest())))) {\n      return writer_->status();\n    }\n  }\n  submessages_.pop_back();\n  return absl::OkStatus();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/messages/serialized_message_writer.h",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_SERIALIZED_MESSAGE_WRITER_H_\n#define RIEGELI_MESSAGES_SERIALIZED_MESSAGE_WRITER_H_\n\n#include <stdint.h>\n\n#include <limits>\n#include <type_traits>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/casts.h\"\n#include \"absl/base/nullability.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/any.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/constexpr.h\"\n#include \"riegeli/base/cord_iterator_span.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/cord_writer.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/bytes/null_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/stringify.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/endian/endian_writing.h\"\n#include \"riegeli/messages/message_wire_format.h\"\n#include \"riegeli/messages/serialize_message.h\"\n#include \"riegeli/varint/varint_writing.h\"\n\nABSL_POINTERS_DEFAULT_NONNULL\n\nnamespace riegeli {\n\n// `SerializedMessageWriter` builds a serialized proto message, specifying\n// contents of particular fields, instead of traversing an in-memory message\n// object like in `SerializeMessage()`.\n//\n// Use cases:\n//\n//  * Processing a subset of fields without the overhead of materializing the\n//    message object, i.e. without processing fields contained in submessages\n//    which can be processed as a whole, and without keeping the whole parsed\n//    message in memory.\n//\n//  * Processing a message in a way known at runtime, possibly with the schema\n//    known at runtime, possibly partially known.\n//\n//  * Processing messages with so many elements of toplevel repeated fields that\n//    the total message size exceeds 2GiB. This is not a great idea in itself,\n//    because such messages cannot be processed using native proto parsing and\n//    serialization.\n//\n// `SerializedMessageBackwardWriter` is more efficient than\n// `SerializedMessageWriter` in the case of nested messages, because their\n// contents can be written directly to the original `BackwardWriter`, with the\n// length known and written after building the contents.\nclass SerializedMessageWriter {\n public:\n  // An empty object. It can be associated with a particular message by\n  // `set_dest()` or assignment.\n  //\n  // An empty `SerializedMessageWriter` is not usable directly, except that\n  // submessage contents can be accumulated after `OpenLengthDelimited()`\n  // if `set_dest()` is called before the matching `CloseLengthDelimited()`\n  // or `CloseOptionalLengthDelimited()`.\n  SerializedMessageWriter() = default;\n\n  // Will write to `*dest`, which is not owned and must outlive usages of this\n  // object.\n  explicit SerializedMessageWriter(\n      Writer* absl_nullable dest ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : dest_(dest), writer_(dest) {}\n\n  SerializedMessageWriter(SerializedMessageWriter&& that) noexcept;\n  SerializedMessageWriter& operator=(SerializedMessageWriter&& that) noexcept;\n\n  // Returns the original `Writer` of the root message.\n  Writer* absl_nullable dest() const { return dest_; }\n\n  // Changes the `Writer` of the root message.\n  //\n  // This can be called even during building, even when submessages are open.\n  // It particular this must be called when the original `Writer` has been\n  // moved.\n  void set_dest(Writer* absl_nullable dest);\n\n  // Returns the `Writer` of the current message or length-delimited field being\n  // built. This can be the original `Writer` of the root message, or the\n  // `Writer` of a field.\n  //\n  // This can be used to write parts of the message directly, apart from\n  // `Write...()` functions which write whole fields.\n  Writer& writer() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    RIEGELI_ASSERT(writer_ != nullptr)\n        << \"Failed precondition of SerializedMessageWriter::writer(): \"\n           \"dest() not set while writing the root message\";\n    return *writer_;\n  }\n\n  // Writes the field tag and the numeric or enum field value.\n  absl::Status WriteInt32(int field_number, int32_t value);\n  absl::Status WriteInt64(int field_number, int64_t value);\n  absl::Status WriteUInt32(int field_number, uint32_t value);\n  absl::Status WriteUInt64(int field_number, uint64_t value);\n  absl::Status WriteSInt32(int field_number, int32_t value);\n  absl::Status WriteSInt64(int field_number, int64_t value);\n  absl::Status WriteBool(int field_number, bool value);\n  absl::Status WriteFixed32(int field_number, uint32_t value);\n  absl::Status WriteFixed64(int field_number, uint64_t value);\n  absl::Status WriteSFixed32(int field_number, int32_t value);\n  absl::Status WriteSFixed64(int field_number, int64_t value);\n  absl::Status WriteFloat(int field_number, float value);\n  absl::Status WriteDouble(int field_number, double value);\n  template <typename EnumType,\n            std::enable_if_t<std::disjunction_v<std::is_enum<EnumType>,\n                                                std::is_integral<EnumType>>,\n                             int> = 0>\n  absl::Status WriteEnum(int field_number, EnumType value);\n\n  // Writes the field tag and the `string`, `bytes`, or submessage field value.\n  //\n  // Message objects are excluded here because they are stringified in the text\n  // format, which is rarely intended as a field value. A separate overload\n  // below serializes a message object in the binary format.\n  template <typename... Values\n#if !__cpp_concepts\n            ,\n            std::enable_if_t<\n                std::conjunction_v<\n                    IsStringifiable<Values...>,\n                    std::negation<std::is_convertible<\n                        Values&&, const google::protobuf::MessageLite&>>...>,\n                int> = 0\n#endif\n            >\n#if __cpp_concepts\n  // For conjunctions, `requires` gives better error messages than\n  // `std::enable_if_t`, indicating the relevant argument.\n    requires((IsStringifiable<Values>::value && ...) &&\n             (!std::is_convertible_v<Values &&,\n                                     const google::protobuf::MessageLite&> &&\n              ...))\n#endif\n  absl::Status WriteString(int field_number, Values&&... values);\n  absl::Status WriteString(int field_number, AnyRef<Reader*> src);\n  template <typename ReaderType>\n  absl::Status WriteString(int field_number, ReaderSpan<ReaderType> src);\n  absl::Status WriteString(int field_number, CordIteratorSpan src);\n\n  // Writes the field tag of a length-delimited field and serializes a message\n  // as the field value.\n  absl::Status WriteString(int field_number,\n                           const google::protobuf::MessageLite& message,\n                           SerializeMessageOptions options = {});\n\n  // Writes an element of a packed repeated field.\n  //\n  // The field must have been opened with `OpenLengthDelimited()`,\n  // `NewLengthDelimited()`, or `WriteLengthUnchecked()`.\n  absl::Status WritePackedInt32(int32_t value);\n  absl::Status WritePackedInt64(int64_t value);\n  absl::Status WritePackedUInt32(uint32_t value);\n  absl::Status WritePackedUInt64(uint64_t value);\n  absl::Status WritePackedSInt32(int32_t value);\n  absl::Status WritePackedSInt64(int64_t value);\n  absl::Status WritePackedBool(bool value);\n  absl::Status WritePackedFixed32(uint32_t value);\n  absl::Status WritePackedFixed64(uint64_t value);\n  absl::Status WritePackedSFixed32(int32_t value);\n  absl::Status WritePackedSFixed64(int64_t value);\n  absl::Status WritePackedFloat(float value);\n  absl::Status WritePackedDouble(double value);\n  template <typename EnumType,\n            std::enable_if_t<std::disjunction_v<std::is_enum<EnumType>,\n                                                std::is_integral<EnumType>>,\n                             int> = 0>\n  absl::Status WritePackedEnum(EnumType value);\n\n  // Begins accumulating contents of a length-delimited field.\n  //\n  // Field contents written to `writer()` are accumulated in memory until\n  // `CloseLengthDelimited()` or `CloseOptionalLengthDelimited()` is called.\n  //\n  // If `OpenLengthDelimited()` is used a lot and building the message back to\n  // front is feasible, then `SerializedMessageBackwardWriter` is more\n  // efficient.\n  void OpenLengthDelimited();\n\n  // Returns a new `SerializedMessageWriter` which accumulates contents of a\n  // length-delimited field of this `SerializedMessageWriter`.\n  //\n  // The contents are written to the parent `SerializedMessageWriter` by the\n  // matching `CloseLengthDelimited()` or `CloseOptionalLengthDelimited()` call\n  // on the returned `SerializedMessageWriter`. Multiple such\n  // `SerializedMessageWriter` objects can be active at the same time.\n  SerializedMessageWriter NewLengthDelimited() ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Ends accumulating contents of a length-delimited field, and writes the\n  // field to the parent message.\n  //\n  // Each `OpenLengthDelimited()` or `NewLengthDelimited()` call must be matched\n  // with a `CloseLengthDelimited()` or `CloseOptionalLengthDelimited()` call,\n  // unless the `SerializedMessageWriter` is no longer used.\n  absl::Status CloseLengthDelimited(int field_number);\n\n  // Like `CloseLengthDelimited()`, but does not write the field if its contents\n  // turn out to be empty.\n  absl::Status CloseOptionalLengthDelimited(int field_number);\n\n  // Writes the field tag and the length of a length-delimited field.\n  //\n  // The value must be written afterwards to `writer()`, with exactly `length`\n  // bytes, unless the `SerializedMessageWriter` and its `dest()` are no longer\n  // used.\n  //\n  // Fails if `length` exceeds 2GiB.\n  //\n  // `WriteLengthUnchecked()` is more efficient than `OpenLengthDelimited()`\n  // or `NewLengthDelimited()` with `CloseLengthDelimited()`, but harder to use:\n  // the length must be pledged before writing the contents, and its correctness\n  // is not checked.\n  absl::Status WriteLengthUnchecked(int field_number, Position length);\n\n  // Writes a group delimiter.\n  //\n  // Each `OpenGroup()` must be matched with a `CloseGroup()` call, unless the\n  // `SerializedMessageWriter` and its `dest()` are no longer used.\n  absl::Status OpenGroup(int field_number);\n  absl::Status CloseGroup(int field_number);\n\n  // Returns the length of the numeric or enum field which would be written.\n  //\n  // This is useful for `WriteLengthUnchecked()`.\n  static Position LengthOfInt32(int field_number, int32_t value);\n  static Position LengthOfInt64(int field_number, int64_t value);\n  static Position LengthOfUInt32(int field_number, uint32_t value);\n  static Position LengthOfUInt64(int field_number, uint64_t value);\n  static Position LengthOfSInt32(int field_number, int32_t value);\n  static Position LengthOfSInt64(int field_number, int64_t value);\n  static Position LengthOfBool(int field_number);\n  static Position LengthOfFixed32(int field_number);\n  static Position LengthOfFixed64(int field_number);\n  static Position LengthOfSFixed32(int field_number);\n  static Position LengthOfSFixed64(int field_number);\n  static Position LengthOfFloat(int field_number);\n  static Position LengthOfDouble(int field_number);\n  template <typename EnumType,\n            std::enable_if_t<std::disjunction_v<std::is_enum<EnumType>,\n                                                std::is_integral<EnumType>>,\n                             int> = 0>\n  static Position LengthOfEnum(int field_number, EnumType value);\n\n  // Returns the length of the `string`, bytes, or submessage field which would\n  // be written.\n  //\n  // This is useful for `WriteLengthUnchecked()`.\n  template <typename... Values\n#if !__cpp_concepts\n            ,\n            std::enable_if_t<\n                std::conjunction_v<\n                    IsStringifiable<Values...>,\n                    std::negation<std::is_convertible<\n                        Values&&, const google::protobuf::MessageLite&>>...>,\n                int> = 0\n#endif\n            >\n#if __cpp_concepts\n  // For conjunctions, `requires` gives better error messages than\n  // `std::enable_if_t`, indicating the relevant argument.\n    requires((IsStringifiable<Values>::value && ...) &&\n             (!std::is_convertible_v<Values &&,\n                                     const google::protobuf::MessageLite&> &&\n              ...))\n#endif\n  static Position LengthOfString(int field_number, const Values&... values);\n\n  // Returns the length of a length-delimited field which would be written, for\n  // the given length of the value.\n  //\n  // This is useful for `WriteLengthUnchecked()`.\n  static Position LengthOfLengthDelimited(int field_number, Position length);\n\n  // Like `LengthOfLengthDelimited()`, but does not count the field if its\n  // contents turn out to be empty.\n  static Position LengthOfOptionalLengthDelimited(int field_number,\n                                                  Position length);\n\n  // Returns the length of an element of a packed repeated field which would be\n  // written.\n  //\n  // This is useful for `WriteLengthUnchecked()`.\n  static Position LengthOfPackedInt32(int32_t value);\n  static Position LengthOfPackedInt64(int64_t value);\n  static Position LengthOfPackedUInt32(uint32_t value);\n  static Position LengthOfPackedUInt64(uint64_t value);\n  static Position LengthOfPackedSInt32(int32_t value);\n  static Position LengthOfPackedSInt64(int64_t value);\n  static Position LengthOfPackedBool();\n  static Position LengthOfPackedFixed32();\n  static Position LengthOfPackedFixed64();\n  static Position LengthOfPackedSFixed32();\n  static Position LengthOfPackedSFixed64();\n  static Position LengthOfPackedFloat();\n  static Position LengthOfPackedDouble();\n  template <typename EnumType,\n            std::enable_if_t<std::disjunction_v<std::is_enum<EnumType>,\n                                                std::is_integral<EnumType>>,\n                             int> = 0>\n  static Position LengthOfPackedEnum(EnumType value);\n\n  // Returns the length of both group delimiters which would be written.\n  //\n  // This is useful for `WriteLengthUnchecked()`.\n  static Position LengthOfOpenPlusCloseGroup(int field_number);\n\n private:\n  ABSL_ATTRIBUTE_COLD static absl::Status LengthOverflowError(Position length);\n  ABSL_ATTRIBUTE_COLD static absl::Status WriteStringFailed(Reader& src,\n                                                            Writer& dest);\n\n  Writer* absl_nullable dest_ = nullptr;\n  std::vector<CordWriter<absl::Cord>> submessages_;\n  Writer* absl_nullable writer_ = nullptr;\n\n  // Invariant:\n  //   `writer_ == (submessages_.empty() ? dest_ : &submessages_.back())`\n};\n\n// Implementation details follow.\n\ninline void SerializedMessageWriter::set_dest(Writer* absl_nullable dest) {\n  dest_ = dest;\n  if (submessages_.empty()) writer_ = dest;\n}\n\ninline SerializedMessageWriter::SerializedMessageWriter(\n    SerializedMessageWriter&& that) noexcept\n    : dest_(that.dest_),\n      submessages_(std::exchange(that.submessages_, {})),\n      // This relies on pointer stability when `std::vector` is moved.\n      writer_(std::exchange(that.writer_, that.dest_)) {}\n\ninline SerializedMessageWriter& SerializedMessageWriter::operator=(\n    SerializedMessageWriter&& that) noexcept {\n  dest_ = that.dest_;\n  submessages_ = std::exchange(that.submessages_, {});\n  // This relies on pointer stability when `std::vector` is moved.\n  writer_ = std::exchange(that.writer_, that.dest_);\n  return *this;\n}\n\ninline absl::Status SerializedMessageWriter::WriteInt32(int field_number,\n                                                        int32_t value) {\n  return WriteUInt64(field_number, static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageWriter::WriteInt64(int field_number,\n                                                        int64_t value) {\n  return WriteUInt64(field_number, static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageWriter::WriteUInt32(int field_number,\n                                                         uint32_t value) {\n  const uint32_t tag = MakeTag(field_number, WireType::kVarint);\n  if (ABSL_PREDICT_FALSE(!writer().Push(\n          (RIEGELI_IS_CONSTANT(tag) ||\n                   (RIEGELI_IS_CONSTANT(tag < 0x80) && tag < 0x80)\n               ? LengthVarint32(tag)\n               : kMaxLengthVarint32) +\n          (RIEGELI_IS_CONSTANT(value) ||\n                   (RIEGELI_IS_CONSTANT(value < 0x80) && value < 0x80)\n               ? LengthVarint32(value)\n               : kMaxLengthVarint32)))) {\n    return writer().status();\n  }\n  char* ptr = WriteVarint32(tag, writer().cursor());\n  ptr = WriteVarint32(value, ptr);\n  writer().set_cursor(ptr);\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageWriter::WriteUInt64(int field_number,\n                                                         uint64_t value) {\n  const uint32_t tag = MakeTag(field_number, WireType::kVarint);\n  if (ABSL_PREDICT_FALSE(!writer().Push(\n          (RIEGELI_IS_CONSTANT(tag) ||\n                   (RIEGELI_IS_CONSTANT(tag < 0x80) && tag < 0x80)\n               ? LengthVarint32(tag)\n               : kMaxLengthVarint32) +\n          (RIEGELI_IS_CONSTANT(value) ||\n                   (RIEGELI_IS_CONSTANT(value < 0x80) && value < 0x80)\n               ? LengthVarint64(value)\n               : kMaxLengthVarint64)))) {\n    return writer().status();\n  }\n  char* ptr = WriteVarint32(tag, writer().cursor());\n  ptr = WriteVarint64(value, ptr);\n  writer().set_cursor(ptr);\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageWriter::WriteSInt32(int field_number,\n                                                         int32_t value) {\n  return WriteUInt32(field_number, EncodeVarintSigned32(value));\n}\n\ninline absl::Status SerializedMessageWriter::WriteSInt64(int field_number,\n                                                         int64_t value) {\n  return WriteUInt64(field_number, EncodeVarintSigned64(value));\n}\n\ninline absl::Status SerializedMessageWriter::WriteBool(int field_number,\n                                                       bool value) {\n  return WriteUInt32(field_number, value ? 1 : 0);\n}\n\ninline absl::Status SerializedMessageWriter::WriteFixed32(int field_number,\n                                                          uint32_t value) {\n  const uint32_t tag = MakeTag(field_number, WireType::kFixed32);\n  if (ABSL_PREDICT_FALSE(!writer().Push(\n          (RIEGELI_IS_CONSTANT(tag) ||\n                   (RIEGELI_IS_CONSTANT(tag < 0x80) && tag < 0x80)\n               ? LengthVarint32(tag)\n               : kMaxLengthVarint32) +\n          sizeof(uint32_t)))) {\n    return writer().status();\n  }\n  char* ptr = WriteVarint32(tag, writer().cursor());\n  WriteLittleEndian<uint32_t>(value, ptr);\n  ptr += sizeof(uint32_t);\n  writer().set_cursor(ptr);\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageWriter::WriteFixed64(int field_number,\n                                                          uint64_t value) {\n  const uint32_t tag = MakeTag(field_number, WireType::kFixed64);\n  if (ABSL_PREDICT_FALSE(!writer().Push(\n          (RIEGELI_IS_CONSTANT(tag) ||\n                   (RIEGELI_IS_CONSTANT(tag < 0x80) && tag < 0x80)\n               ? LengthVarint32(tag)\n               : kMaxLengthVarint32) +\n          sizeof(uint64_t)))) {\n    return writer().status();\n  }\n  char* ptr = WriteVarint32(tag, writer().cursor());\n  WriteLittleEndian<uint64_t>(value, ptr);\n  ptr += sizeof(uint64_t);\n  writer().set_cursor(ptr);\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageWriter::WriteSFixed32(int field_number,\n                                                           int32_t value) {\n  return WriteFixed32(field_number, static_cast<uint32_t>(value));\n}\n\ninline absl::Status SerializedMessageWriter::WriteSFixed64(int field_number,\n                                                           int64_t value) {\n  return WriteFixed64(field_number, static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageWriter::WriteFloat(int field_number,\n                                                        float value) {\n  return WriteFixed32(field_number, absl::bit_cast<uint32_t>(value));\n}\n\ninline absl::Status SerializedMessageWriter::WriteDouble(int field_number,\n                                                         double value) {\n  return WriteFixed64(field_number, absl::bit_cast<uint64_t>(value));\n}\n\ntemplate <typename EnumType,\n          std::enable_if_t<std::disjunction_v<std::is_enum<EnumType>,\n                                              std::is_integral<EnumType>>,\n                           int>>\ninline absl::Status SerializedMessageWriter::WriteEnum(int field_number,\n                                                       EnumType value) {\n  return WriteUInt64(field_number, static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageWriter::WritePackedInt32(int32_t value) {\n  return WritePackedUInt64(static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageWriter::WritePackedInt64(int64_t value) {\n  return WritePackedUInt64(static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageWriter::WritePackedUInt32(uint32_t value) {\n  if (ABSL_PREDICT_FALSE(!WriteVarint32(value, writer()))) {\n    return writer().status();\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageWriter::WritePackedUInt64(uint64_t value) {\n  if (ABSL_PREDICT_FALSE(!WriteVarint64(value, writer()))) {\n    return writer().status();\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageWriter::WritePackedSInt32(int32_t value) {\n  return WritePackedUInt32(EncodeVarintSigned32(value));\n}\n\ninline absl::Status SerializedMessageWriter::WritePackedSInt64(int64_t value) {\n  return WritePackedUInt64(EncodeVarintSigned64(value));\n}\n\ninline absl::Status SerializedMessageWriter::WritePackedBool(bool value) {\n  return WritePackedUInt32(value ? 1 : 0);\n}\n\ninline absl::Status SerializedMessageWriter::WritePackedFixed32(\n    uint32_t value) {\n  if (ABSL_PREDICT_FALSE(!WriteLittleEndian<uint32_t>(value, writer()))) {\n    return writer().status();\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageWriter::WritePackedFixed64(\n    uint64_t value) {\n  if (ABSL_PREDICT_FALSE(!WriteLittleEndian<uint64_t>(value, writer()))) {\n    return writer().status();\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageWriter::WritePackedSFixed32(\n    int32_t value) {\n  return WritePackedFixed32(static_cast<uint32_t>(value));\n}\n\ninline absl::Status SerializedMessageWriter::WritePackedSFixed64(\n    int64_t value) {\n  return WritePackedFixed64(static_cast<uint64_t>(value));\n}\n\ninline absl::Status SerializedMessageWriter::WritePackedFloat(float value) {\n  return WritePackedFixed32(absl::bit_cast<uint32_t>(value));\n}\n\ninline absl::Status SerializedMessageWriter::WritePackedDouble(double value) {\n  return WritePackedFixed64(absl::bit_cast<uint64_t>(value));\n}\n\ntemplate <typename EnumType,\n          std::enable_if_t<std::disjunction_v<std::is_enum<EnumType>,\n                                              std::is_integral<EnumType>>,\n                           int>>\ninline absl::Status SerializedMessageWriter::WritePackedEnum(EnumType value) {\n  return WritePackedUInt64(static_cast<uint64_t>(value));\n}\n\ntemplate <typename... Values\n#if !__cpp_concepts\n          ,\n          std::enable_if_t<\n              std::conjunction_v<\n                  IsStringifiable<Values...>,\n                  std::negation<std::is_convertible<\n                      Values&&, const google::protobuf::MessageLite&>>...>,\n              int>\n#endif\n          >\n#if __cpp_concepts\n  requires((IsStringifiable<Values>::value && ...) &&\n           (!std::is_convertible_v<Values &&,\n                                   const google::protobuf::MessageLite&> &&\n            ...))\n#endif\ninline absl::Status SerializedMessageWriter::WriteString(int field_number,\n                                                         Values&&... values) {\n  if constexpr (HasStringifiedSize<Values...>::value) {\n    if (absl::Status status = WriteLengthUnchecked(\n            field_number, riegeli::StringifiedSize(values...));\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n    if (ABSL_PREDICT_FALSE(!writer().Write(std::forward<Values>(values)...))) {\n      return writer().status();\n    }\n  } else {\n    CordWriter cord_writer;\n    if (ABSL_PREDICT_FALSE(\n            !cord_writer.Write(std::forward<Values>(values)...) ||\n            !cord_writer.Close())) {\n      return cord_writer.status();\n    }\n    if (absl::Status status =\n            WriteLengthUnchecked(field_number, cord_writer.dest().size());\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      return status;\n    }\n    if (ABSL_PREDICT_FALSE(!writer().Write(std::move(cord_writer.dest())))) {\n      return writer().status();\n    }\n  }\n  return absl::OkStatus();\n}\n\ntemplate <typename ReaderType>\nabsl::Status SerializedMessageWriter::WriteString(int field_number,\n                                                  ReaderSpan<ReaderType> src) {\n  if (absl::Status status = WriteLengthUnchecked(field_number, src.length());\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return status;\n  }\n  if (ABSL_PREDICT_FALSE(!src.reader().Copy(src.length(), writer()))) {\n    return WriteStringFailed(src.reader(), writer());\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageWriter::WriteString(\n    int field_number, const google::protobuf::MessageLite& message,\n    SerializeMessageOptions options) {\n  if (absl::Status status =\n          WriteLengthUnchecked(field_number, options.GetByteSize(message));\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return status;\n  }\n  return riegeli::SerializeMessage(message, writer(), options);\n}\n\ninline absl::Status SerializedMessageWriter::WriteLengthUnchecked(\n    int field_number, Position length) {\n  if (ABSL_PREDICT_FALSE(length >\n                         uint32_t{std::numeric_limits<int32_t>::max()})) {\n    return LengthOverflowError(length);\n  }\n  const uint32_t tag = MakeTag(field_number, WireType::kLengthDelimited);\n  if (ABSL_PREDICT_FALSE(!writer().Push(\n          (RIEGELI_IS_CONSTANT(tag) ||\n                   (RIEGELI_IS_CONSTANT(tag < 0x80) && tag < 0x80)\n               ? LengthVarint32(tag)\n               : kMaxLengthVarint32) +\n          (RIEGELI_IS_CONSTANT(length) ||\n                   (RIEGELI_IS_CONSTANT(length < 0x80) && length < 0x80)\n               ? LengthVarint32(IntCast<uint32_t>(length))\n               : kMaxLengthVarint32)))) {\n    return writer().status();\n  }\n  char* ptr = WriteVarint32(tag, writer().cursor());\n  ptr = WriteVarint32(IntCast<uint32_t>(length), ptr);\n  writer().set_cursor(ptr);\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageWriter::OpenGroup(int field_number) {\n  if (ABSL_PREDICT_FALSE(!WriteVarint32(\n          MakeTag(field_number, WireType::kStartGroup), writer()))) {\n    return writer().status();\n  }\n  return absl::OkStatus();\n}\n\ninline absl::Status SerializedMessageWriter::CloseGroup(int field_number) {\n  if (ABSL_PREDICT_FALSE(!WriteVarint32(\n          MakeTag(field_number, WireType::kEndGroup), writer()))) {\n    return writer().status();\n  }\n  return absl::OkStatus();\n}\n\ninline Position SerializedMessageWriter::LengthOfInt32(int field_number,\n                                                       int32_t value) {\n  return LengthOfUInt64(field_number, static_cast<uint64_t>(value));\n}\n\ninline Position SerializedMessageWriter::LengthOfInt64(int field_number,\n                                                       int64_t value) {\n  return LengthOfUInt64(field_number, static_cast<uint64_t>(value));\n}\n\ninline Position SerializedMessageWriter::LengthOfUInt32(int field_number,\n                                                        uint32_t value) {\n  return LengthVarint32(MakeTag(field_number, WireType::kVarint)) +\n         LengthVarint32(value);\n}\n\ninline Position SerializedMessageWriter::LengthOfUInt64(int field_number,\n                                                        uint64_t value) {\n  return LengthVarint32(MakeTag(field_number, WireType::kVarint)) +\n         LengthVarint64(value);\n}\n\ninline Position SerializedMessageWriter::LengthOfSInt32(int field_number,\n                                                        int32_t value) {\n  return LengthOfUInt32(field_number, EncodeVarintSigned32(value));\n}\n\ninline Position SerializedMessageWriter::LengthOfSInt64(int field_number,\n                                                        int64_t value) {\n  return LengthOfUInt64(field_number, EncodeVarintSigned64(value));\n}\n\ninline Position SerializedMessageWriter::LengthOfBool(int field_number) {\n  return LengthVarint32(MakeTag(field_number, WireType::kVarint)) + 1;\n}\n\ninline Position SerializedMessageWriter::LengthOfFixed32(int field_number) {\n  return LengthVarint32(MakeTag(field_number, WireType::kFixed32)) +\n         sizeof(uint32_t);\n}\n\ninline Position SerializedMessageWriter::LengthOfFixed64(int field_number) {\n  return LengthVarint32(MakeTag(field_number, WireType::kFixed64)) +\n         sizeof(uint64_t);\n}\n\ninline Position SerializedMessageWriter::LengthOfSFixed32(int field_number) {\n  return LengthOfFixed32(field_number);\n}\n\ninline Position SerializedMessageWriter::LengthOfSFixed64(int field_number) {\n  return LengthOfFixed64(field_number);\n}\n\ninline Position SerializedMessageWriter::LengthOfFloat(int field_number) {\n  return LengthOfFixed32(field_number);\n}\n\ninline Position SerializedMessageWriter::LengthOfDouble(int field_number) {\n  return LengthOfFixed64(field_number);\n}\n\ntemplate <typename EnumType,\n          std::enable_if_t<std::disjunction_v<std::is_enum<EnumType>,\n                                              std::is_integral<EnumType>>,\n                           int>>\ninline Position SerializedMessageWriter::LengthOfEnum(int field_number,\n                                                      EnumType value) {\n  return LengthOfUInt64(field_number, static_cast<uint64_t>(value));\n}\n\ninline Position SerializedMessageWriter::LengthOfPackedInt32(int32_t value) {\n  return LengthOfPackedUInt64(static_cast<uint64_t>(value));\n}\n\ninline Position SerializedMessageWriter::LengthOfPackedInt64(int64_t value) {\n  return LengthOfPackedUInt64(static_cast<uint64_t>(value));\n}\n\ninline Position SerializedMessageWriter::LengthOfPackedUInt32(uint32_t value) {\n  return LengthVarint32(value);\n}\n\ninline Position SerializedMessageWriter::LengthOfPackedUInt64(uint64_t value) {\n  return LengthVarint64(value);\n}\n\ninline Position SerializedMessageWriter::LengthOfPackedSInt32(int32_t value) {\n  return LengthOfPackedUInt32(EncodeVarintSigned32(value));\n}\n\ninline Position SerializedMessageWriter::LengthOfPackedSInt64(int64_t value) {\n  return LengthOfPackedUInt64(EncodeVarintSigned64(value));\n}\n\ninline Position SerializedMessageWriter::LengthOfPackedBool() { return 1; }\n\ninline Position SerializedMessageWriter::LengthOfPackedFixed32() {\n  return sizeof(uint32_t);\n}\n\ninline Position SerializedMessageWriter::LengthOfPackedFixed64() {\n  return sizeof(uint64_t);\n}\n\ninline Position SerializedMessageWriter::LengthOfPackedSFixed32() {\n  return LengthOfPackedFixed32();\n}\n\ninline Position SerializedMessageWriter::LengthOfPackedSFixed64() {\n  return LengthOfPackedFixed64();\n}\n\ninline Position SerializedMessageWriter::LengthOfPackedFloat() {\n  return LengthOfPackedFixed32();\n}\n\ninline Position SerializedMessageWriter::LengthOfPackedDouble() {\n  return LengthOfPackedFixed64();\n}\n\ntemplate <typename EnumType,\n          std::enable_if_t<std::disjunction_v<std::is_enum<EnumType>,\n                                              std::is_integral<EnumType>>,\n                           int>>\ninline Position SerializedMessageWriter::LengthOfPackedEnum(EnumType value) {\n  return LengthOfPackedUInt64(static_cast<uint64_t>(value));\n}\n\ntemplate <typename... Values\n#if !__cpp_concepts\n          ,\n          std::enable_if_t<\n              std::conjunction_v<\n                  IsStringifiable<Values...>,\n                  std::negation<std::is_convertible<\n                      Values&&, const google::protobuf::MessageLite&>>...>,\n              int>\n#endif\n          >\n#if __cpp_concepts\n  requires((IsStringifiable<Values>::value && ...) &&\n           (!std::is_convertible_v<Values &&,\n                                   const google::protobuf::MessageLite&> &&\n            ...))\n#endif\ninline Position SerializedMessageWriter::LengthOfString(\n    int field_number, const Values&... values) {\n  if constexpr (HasStringifiedSize<Values...>::value) {\n    return LengthOfLengthDelimited(field_number,\n                                   riegeli::StringifiedSize(values...));\n  } else {\n    NullWriter null_writer;\n    null_writer.Write(values...);\n    null_writer.Close();\n    return LengthOfLengthDelimited(field_number, null_writer.pos());\n  }\n}\n\ninline Position SerializedMessageWriter::LengthOfLengthDelimited(\n    int field_number, Position length) {\n  return LengthVarint32(MakeTag(field_number, WireType::kLengthDelimited)) +\n         LengthVarint32(IntCast<uint32_t>(length)) + length;\n}\n\ninline Position SerializedMessageWriter::LengthOfOptionalLengthDelimited(\n    int field_number, Position length) {\n  return length == 0 ? 0 : LengthOfLengthDelimited(field_number, length);\n}\n\ninline Position SerializedMessageWriter::LengthOfOpenPlusCloseGroup(\n    int field_number) {\n  return 2 * LengthVarint32(MakeTag(field_number, WireType::kStartGroup));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_SERIALIZED_MESSAGE_WRITER_H_\n"
  },
  {
    "path": "riegeli/messages/text_parse_message.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/messages/text_parse_message.h\"\n\n#include <memory>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/protobuf/io/tokenizer.h\"\n#include \"google/protobuf/message.h\"\n#include \"google/protobuf/text_format.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/cord_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/string_reader.h\"\n#include \"riegeli/messages/parse_message.h\"\n\nnamespace riegeli {\n\nnamespace text_parse_message_internal {\n\nvoid StringErrorCollector::RecordError(\n    int line, google::protobuf::io::ColumnNumber column,\n    absl::string_view message) {\n  if (line >= 0) {\n    absl::StrAppend(&errors_, \"\\nAt \", line + 1, \":\", column + 1, \": \",\n                    message);\n  } else {\n    absl::StrAppend(&errors_, \"\\n\", message);\n  }\n}\n\n}  // namespace text_parse_message_internal\n\nTextParseMessageOptions::TextParseMessageOptions()\n    : error_collector_(std::make_unique<\n                       text_parse_message_internal::StringErrorCollector>()) {\n  parser_.RecordErrorsTo(error_collector_.get());\n}\n\nnamespace text_parse_message_internal {\n\nabsl::Status TextParseMessageImpl(Reader& src, google::protobuf::Message& dest,\n                                  const TextParseMessageOptions& options) {\n  ReaderInputStream input_stream(&src);\n  google::protobuf::TextFormat::Parser parser = options.parser();\n  const bool parse_ok = options.merge() ? parser.Merge(&input_stream, &dest)\n                                        : parser.Parse(&input_stream, &dest);\n  if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n  if (ABSL_PREDICT_FALSE(!parse_ok)) {\n    return src.AnnotateStatus(absl::InvalidArgumentError(\n        absl::StrCat(\"Failed to text-parse message of type \",\n                     dest.GetTypeName(), options.error_collector_->errors())));\n  }\n  return absl::OkStatus();\n}\n\n}  // namespace text_parse_message_internal\n\nabsl::Status TextParseMessage(BytesRef src, google::protobuf::Message& dest,\n                              const TextParseMessageOptions& options) {\n  return TextParseMessage(StringReader(src), dest, options);\n}\n\nabsl::Status TextParseMessage(const Chain& src, google::protobuf::Message& dest,\n                              const TextParseMessageOptions& options) {\n  return TextParseMessage(ChainReader(&src), dest, options);\n}\n\nabsl::Status TextParseMessage(const absl::Cord& src,\n                              google::protobuf::Message& dest,\n                              const TextParseMessageOptions& options) {\n  return TextParseMessage(CordReader(&src), dest, options);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/messages/text_parse_message.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_TEXT_PARSE_MESSAGE_H_\n#define RIEGELI_MESSAGES_TEXT_PARSE_MESSAGE_H_\n\n#include <memory>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/protobuf/io/tokenizer.h\"\n#include \"google/protobuf/message.h\"\n#include \"google/protobuf/text_format.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nclass TextParseMessageOptions;\n\nnamespace text_parse_message_internal {\n\nclass StringErrorCollector : public google::protobuf::io::ErrorCollector {\n public:\n  void RecordError(int line, google::protobuf::io::ColumnNumber column,\n                   absl::string_view message) override;\n\n  absl::string_view errors() const { return errors_; }\n\n private:\n  std::string errors_;\n};\n\nabsl::Status TextParseMessageImpl(Reader& src, google::protobuf::Message& dest,\n                                  const TextParseMessageOptions& options);\n\n}  // namespace text_parse_message_internal\n\nclass TextParseMessageOptions {\n public:\n  TextParseMessageOptions();\n\n  // If `false`, replaces existing contents of the destination, clearing it\n  // first.\n  //\n  // If `true`, merges to existing contents of the destination.\n  //\n  // Default: `false`.\n  TextParseMessageOptions& set_merge(bool merge) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    merge_ = merge;\n    return *this;\n  }\n  TextParseMessageOptions&& set_merge(bool merge) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_merge(merge));\n  }\n  bool merge() const { return merge_; }\n\n  // Other text parsing options.\n  //\n  // The default `ErrorCollector` is set up such that errors are returned as\n  // `absl::InvalidArgumengError()` instead of being logged.\n  google::protobuf::TextFormat::Parser& parser() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return parser_;\n  }\n  const google::protobuf::TextFormat::Parser& parser() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return parser_;\n  }\n\n private:\n  // For `error_collector_`.\n  friend absl::Status text_parse_message_internal::TextParseMessageImpl(\n      Reader& src, google::protobuf::Message& dest,\n      const TextParseMessageOptions& options);\n\n  bool merge_ = false;\n  std::unique_ptr<text_parse_message_internal::StringErrorCollector>\n      error_collector_;\n  google::protobuf::TextFormat::Parser parser_;\n};\n\n// Reads a message in text format from the given `Reader`. If successful, the\n// entire input will be consumed.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `Reader`. `Src` must support\n// `DependencyRef<Reader*, Src>`, e.g.  `Reader&` (not owned),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `AnyRef<Reader*>` (maybe owned).\n//\n// Returns status:\n//  * `status.ok()`  - success (`dest` is filled)\n//  * `!status.ok()` - failure (`dest` is unspecified)\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int> = 0>\nabsl::Status TextParseMessage(\n    Src&& src, google::protobuf::Message& dest,\n    const TextParseMessageOptions& options = TextParseMessageOptions());\n\n// Reads a message in text format from `src`.\n//\n// Returns status:\n//  * `status.ok()`  - success (`dest` is filled)\n//  * `!status.ok()` - failure (`dest` is unspecified)\nabsl::Status TextParseMessage(\n    BytesRef src, google::protobuf::Message& dest,\n    const TextParseMessageOptions& options = TextParseMessageOptions());\nabsl::Status TextParseMessage(\n    const Chain& src, google::protobuf::Message& dest,\n    const TextParseMessageOptions& options = TextParseMessageOptions());\nabsl::Status TextParseMessage(\n    const absl::Cord& src, google::protobuf::Message& dest,\n    const TextParseMessageOptions& options = TextParseMessageOptions());\n\n// Implementation details follow.\n\nnamespace text_parse_message_internal {\n\nabsl::Status TextParseMessageImpl(Reader& src, google::protobuf::Message& dest,\n                                  const TextParseMessageOptions& options);\n\n}  // namespace text_parse_message_internal\n\ntemplate <\n    typename Src,\n    std::enable_if_t<TargetRefSupportsDependency<Reader*, Src>::value, int>>\ninline absl::Status TextParseMessage(Src&& src, google::protobuf::Message& dest,\n                                     const TextParseMessageOptions& options) {\n  DependencyRef<Reader*, Src> src_dep(std::forward<Src>(src));\n  if (src_dep.IsOwning()) src_dep->SetReadAllHint(true);\n  absl::Status status = text_parse_message_internal::TextParseMessageImpl(\n      *src_dep, dest, options);\n  if (src_dep.IsOwning()) {\n    if (ABSL_PREDICT_TRUE(status.ok())) src_dep->VerifyEnd();\n    if (ABSL_PREDICT_FALSE(!src_dep->Close())) status.Update(src_dep->status());\n  }\n  return status;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_TEXT_PARSE_MESSAGE_H_\n"
  },
  {
    "path": "riegeli/messages/text_print_message.cc",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/messages/text_print_message.h\"\n\n#include <string>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"google/protobuf/descriptor.h\"\n#include \"google/protobuf/message.h\"\n#include \"google/protobuf/text_format.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/bytes/chain_writer.h\"\n#include \"riegeli/bytes/cord_writer.h\"\n#include \"riegeli/bytes/string_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/messages/serialize_message.h\"\n\nnamespace riegeli {\n\nnamespace text_print_message_internal {\n\nabsl::Status TextPrintMessageImpl(const google::protobuf::Message& src,\n                                  Writer& dest,\n                                  const TextPrintMessageOptions& options) {\n  RIEGELI_ASSERT(options.partial() || src.IsInitialized())\n      << \"Failed to text-print message of type \" << src.GetTypeName()\n      << \" because it is missing required fields: \"\n      << src.InitializationErrorString();\n  if (options.header()) {\n    const google::protobuf::Descriptor* const descriptor = src.GetDescriptor();\n    dest.Write(\"# proto-file: \", descriptor->file()->name(),\n               \"\\n\"\n               \"# proto-message: \",\n               descriptor->containing_type() != nullptr\n                   ? descriptor->full_name()\n                   : descriptor->name(),\n               \"\\n\\n\");\n  }\n  WriterOutputStream output_stream(&dest);\n  const bool print_ok = options.printer().Print(src, &output_stream);\n  if (ABSL_PREDICT_FALSE(!dest.ok())) return dest.status();\n  RIEGELI_ASSERT(print_ok)\n      << \"Failed to text-print message of type \" << src.GetTypeName()\n      << \": TextFormat::Printer::Print() failed for an unknown reason\";\n  return absl::OkStatus();\n}\n\n}  // namespace text_print_message_internal\n\nabsl::Status TextPrintMessage(const google::protobuf::Message& src,\n                              std::string& dest,\n                              const TextPrintMessageOptions& options) {\n  return TextPrintMessage(src, StringWriter(&dest), options);\n}\n\nabsl::Status TextPrintMessage(const google::protobuf::Message& src, Chain& dest,\n                              const TextPrintMessageOptions& options) {\n  return TextPrintMessage(src, ChainWriter(&dest), options);\n}\n\nabsl::Status TextPrintMessage(const google::protobuf::Message& src,\n                              absl::Cord& dest,\n                              const TextPrintMessageOptions& options) {\n  return TextPrintMessage(src, CordWriter(&dest), options);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/messages/text_print_message.h",
    "content": "// Copyright 2022 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_MESSAGES_TEXT_PRINT_MESSAGE_H_\n#define RIEGELI_MESSAGES_TEXT_PRINT_MESSAGE_H_\n\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"google/protobuf/message.h\"\n#include \"google/protobuf/text_format.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nclass TextPrintMessageOptions {\n public:\n  TextPrintMessageOptions() noexcept {}\n\n  // If `false`, all required fields must be set. This is verified in debug\n  // mode.\n  //\n  // If `true`, missing required fields result in a partial serialized message,\n  // not having these fields.\n  //\n  // Default: `false`.\n  TextPrintMessageOptions& set_partial(bool partial) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    partial_ = partial;\n    return *this;\n  }\n  TextPrintMessageOptions&& set_partial(bool partial) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_partial(partial));\n  }\n  bool partial() const { return partial_; }\n\n  // If `false`, print just the message.\n  //\n  // If `true`, prefix the output with comments informing about the filename and\n  // message name, to aid tools and humans in interpreting the file:\n  //\n  // ```\n  // # proto-file: path/filename.proto\n  // # proto-message: MessageName\n  // ```\n  //\n  // See\n  // https://developers.google.com/protocol-buffers/docs/text-format-spec#header\n  // for details.\n  //\n  // Default: `false`.\n  TextPrintMessageOptions& set_header(bool header) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    header_ = header;\n    return *this;\n  }\n  TextPrintMessageOptions&& set_header(bool header) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_header(header));\n  }\n  bool header() const { return header_; }\n\n  // Other text printing options.\n  google::protobuf::TextFormat::Printer& printer()\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return printer_;\n  }\n  const google::protobuf::TextFormat::Printer& printer() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return printer_;\n  }\n\n private:\n  bool partial_ = false;\n  bool header_ = false;\n  google::protobuf::TextFormat::Printer printer_;\n};\n\n// Writes the message in text format to the given `Writer`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `Writer`. `Dest` must support\n// `DependencyRef<Writer*, Dest>`, e.g. `Writer&` (not owned),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `AnyRef<Writer*>` (maybe owned).\n//\n// Returns status:\n//  * `status.ok()`  - success (`dest` is written to)\n//  * `!status.ok()` - failure (`dest` is unspecified)\ntemplate <typename Dest,\n          std::enable_if_t<TargetRefSupportsDependency<Writer*, Dest>::value,\n                           int> = 0>\nabsl::Status TextPrintMessage(\n    const google::protobuf::Message& src, Dest&& dest,\n    const TextPrintMessageOptions& options = TextPrintMessageOptions());\n\n// Writes the message in text format to `dest`, clearing any existing data in\n// `dest`.\n//\n// Returns status:\n//  * `status.ok()`  - success (`dest` is filled)\n//  * `!status.ok()` - failure (`dest` is unspecified)\nabsl::Status TextPrintMessage(\n    const google::protobuf::Message& src, std::string& dest,\n    const TextPrintMessageOptions& options = TextPrintMessageOptions());\nabsl::Status TextPrintMessage(\n    const google::protobuf::Message& src, Chain& dest,\n    const TextPrintMessageOptions& options = TextPrintMessageOptions());\nabsl::Status TextPrintMessage(\n    const google::protobuf::Message& src, absl::Cord& dest,\n    const TextPrintMessageOptions& options = TextPrintMessageOptions());\n\n// Implementation details follow.\n\nnamespace text_print_message_internal {\n\nabsl::Status TextPrintMessageImpl(const google::protobuf::Message& src,\n                                  Writer& dest,\n                                  const TextPrintMessageOptions& options);\n\n}  // namespace text_print_message_internal\n\ntemplate <\n    typename Dest,\n    std::enable_if_t<TargetRefSupportsDependency<Writer*, Dest>::value, int>>\ninline absl::Status TextPrintMessage(const google::protobuf::Message& src,\n                                     Dest&& dest,\n                                     const TextPrintMessageOptions& options) {\n  DependencyRef<Writer*, Dest> dest_dep(std::forward<Dest>(dest));\n  absl::Status status = text_print_message_internal::TextPrintMessageImpl(\n      src, *dest_dep, options);\n  if (dest_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_dep->Close())) {\n      status.Update(dest_dep->status());\n    }\n  }\n  return status;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_MESSAGES_TEXT_PRINT_MESSAGE_H_\n"
  },
  {
    "path": "riegeli/ordered_varint/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"ordered_varint_reading\",\n    srcs = [\n        \"ordered_varint_internal.h\",\n        \"ordered_varint_reading.cc\",\n    ],\n    hdrs = [\"ordered_varint_reading.h\"],\n    deps = [\n        \"//riegeli/base:assert\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/endian:endian_reading\",\n        \"@com_google_absl//absl/base:core_headers\",\n    ],\n)\n\ncc_library(\n    name = \"ordered_varint_writing\",\n    srcs = [\n        \"ordered_varint_internal.h\",\n        \"ordered_varint_writing.cc\",\n    ],\n    hdrs = [\"ordered_varint_writing.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/endian:endian_writing\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:bits\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/ordered_varint/ordered_varint_internal.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_ORDERED_VARINT_ORDERED_VARINT_INTERNAL_H_\n#define RIEGELI_ORDERED_VARINT_ORDERED_VARINT_INTERNAL_H_\n\n// IWYU pragma: private, include \"riegeli/ordered_varint/ordered_varint_reading.h\"\n// IWYU pragma: private, include \"riegeli/ordered_varint/ordered_varint_writing.h\"\n\n#include <stddef.h>\n\nnamespace riegeli {\n\ninline constexpr size_t kMaxLengthOrderedVarint32 = 5;\ninline constexpr size_t kMaxLengthOrderedVarint64 = 9;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_ORDERED_VARINT_ORDERED_VARINT_INTERNAL_H_\n"
  },
  {
    "path": "riegeli/ordered_varint/ordered_varint_reading.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/ordered_varint/ordered_varint_reading.h\"\n\n#include <stdint.h>\n\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/endian/endian_reading.h\"\n\nnamespace riegeli::ordered_varint_internal {\n\nbool ReadOrderedVarint32Slow(Reader& src, uint32_t& dest) {\n  RIEGELI_ASSERT_GT(src.available(), 0u)\n      << \"Failed precondition of ReadOrderedVarint32Slow(): no data available\";\n  const uint8_t first_byte = static_cast<uint8_t>(*src.cursor());\n  RIEGELI_ASSERT_GE(first_byte, 0x80)\n      << \"Failed precondition of ReadOrderedVarint32Slow(): length is 1\";\n  if (first_byte < 0xc0) {\n    if (ABSL_PREDICT_FALSE(!src.Pull(2))) return false;\n    dest = ReadBigEndian<uint16_t>(src.cursor()) & ~(uint16_t{0x80} << 8);\n    if (ABSL_PREDICT_FALSE(dest < uint32_t{1} << 7)) return false;\n    src.move_cursor(2);\n    return true;\n  } else if (first_byte < 0xe0) {\n    if (ABSL_PREDICT_FALSE(!src.Pull(3))) return false;\n    dest = (static_cast<uint32_t>(static_cast<uint8_t>(src.cursor()[0]) &\n                                  ~uint8_t{0xc0})\n            << (2 * 8)) |\n           ReadBigEndian<uint16_t>(src.cursor() + 1);\n    if (ABSL_PREDICT_FALSE(dest < uint32_t{1} << (2 * 7))) return false;\n    src.move_cursor(3);\n    return true;\n  } else if (first_byte < 0xf0) {\n    if (ABSL_PREDICT_FALSE(!src.Pull(4))) return false;\n    dest = ReadBigEndian<uint32_t>(src.cursor()) & ~(uint32_t{0xe0} << (3 * 8));\n    if (ABSL_PREDICT_FALSE(dest < uint32_t{1} << (3 * 7))) return false;\n    src.move_cursor(4);\n    return true;\n  } else {\n    if (ABSL_PREDICT_FALSE(first_byte > 0xf0)) return false;\n    if (ABSL_PREDICT_FALSE(!src.Pull(5))) return false;\n    dest = ReadBigEndian<uint32_t>(src.cursor() + 1);\n    if (ABSL_PREDICT_FALSE(dest < uint32_t{1} << (4 * 7))) return false;\n    src.move_cursor(5);\n    return true;\n  }\n}\n\nbool ReadOrderedVarint64Slow(Reader& src, uint64_t& dest) {\n  RIEGELI_ASSERT_GT(src.available(), 0u)\n      << \"Failed precondition of ReadOrderedVarint64Slow(): no data available\";\n  const uint8_t first_byte = static_cast<uint8_t>(*src.cursor());\n  RIEGELI_ASSERT_GE(first_byte, 0x80)\n      << \"Failed precondition of ReadOrderedVarint64Slow(): length is 1\";\n  if (first_byte < 0xc0) {\n    if (ABSL_PREDICT_FALSE(!src.Pull(2))) return false;\n    dest = ReadBigEndian<uint16_t>(src.cursor()) & ~(uint16_t{0x80} << 8);\n    if (ABSL_PREDICT_FALSE(dest < uint32_t{1} << 7)) return false;\n    src.move_cursor(2);\n    return true;\n  } else if (first_byte < 0xe0) {\n    if (ABSL_PREDICT_FALSE(!src.Pull(3))) return false;\n    dest = (static_cast<uint32_t>(static_cast<uint8_t>(src.cursor()[0]) &\n                                  ~uint8_t{0xc0})\n            << (2 * 8)) |\n           ReadBigEndian<uint16_t>(src.cursor() + 1);\n    if (ABSL_PREDICT_FALSE(dest < uint32_t{1} << (2 * 7))) return false;\n    src.move_cursor(3);\n    return true;\n  } else if (first_byte < 0xf0) {\n    if (ABSL_PREDICT_FALSE(!src.Pull(4))) return false;\n    dest = ReadBigEndian<uint32_t>(src.cursor()) & ~(uint32_t{0xe0} << (3 * 8));\n    if (ABSL_PREDICT_FALSE(dest < uint32_t{1} << (3 * 7))) return false;\n    src.move_cursor(4);\n    return true;\n  } else if (first_byte < 0xf8) {\n    if (ABSL_PREDICT_FALSE(!src.Pull(5))) return false;\n    dest = (static_cast<uint64_t>(static_cast<uint8_t>(src.cursor()[0]) &\n                                  ~uint8_t{0xf0})\n            << (4 * 8)) |\n           ReadBigEndian<uint32_t>(src.cursor() + 1);\n    if (ABSL_PREDICT_FALSE(dest < uint64_t{1} << (4 * 7))) return false;\n    src.move_cursor(5);\n    return true;\n  } else if (first_byte < 0xfc) {\n    if (ABSL_PREDICT_FALSE(!src.Pull(6))) return false;\n    dest = (static_cast<uint64_t>(ReadBigEndian<uint16_t>(src.cursor()) &\n                                  ~(uint16_t{0xf8} << 8))\n            << (4 * 8)) |\n           ReadBigEndian<uint32_t>(src.cursor() + 2);\n    if (ABSL_PREDICT_FALSE(dest < uint64_t{1} << (5 * 7))) return false;\n    src.move_cursor(6);\n    return true;\n  } else if (first_byte < 0xfe) {\n    if (ABSL_PREDICT_FALSE(!src.Pull(7))) return false;\n    dest = (static_cast<uint64_t>(ReadBigEndian<uint32_t>(src.cursor()) &\n                                  ~(uint32_t{0xfc} << (3 * 8)))\n            << (3 * 8)) |\n           ReadBigEndian<uint32_t>(src.cursor() + 3);\n    if (ABSL_PREDICT_FALSE(dest < uint64_t{1} << (6 * 7))) return false;\n    src.move_cursor(7);\n    return true;\n  } else if (first_byte < 0xff) {\n    if (ABSL_PREDICT_FALSE(!src.Pull(8))) return false;\n    dest = ReadBigEndian<uint64_t>(src.cursor()) & ~(uint64_t{0xfe} << (7 * 8));\n    if (ABSL_PREDICT_FALSE(dest < uint64_t{1} << (7 * 7))) return false;\n    src.move_cursor(8);\n    return true;\n  } else {\n    if (ABSL_PREDICT_FALSE(!src.Pull(9))) return false;\n    dest = ReadBigEndian<uint64_t>(src.cursor() + 1);\n    if (ABSL_PREDICT_FALSE(dest < uint64_t{1} << (8 * 7))) return false;\n    src.move_cursor(9);\n    return true;\n  }\n}\n\n}  // namespace riegeli::ordered_varint_internal\n"
  },
  {
    "path": "riegeli/ordered_varint/ordered_varint_reading.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_ORDERED_VARINT_ORDERED_VARINT_READING_H_\n#define RIEGELI_ORDERED_VARINT_ORDERED_VARINT_READING_H_\n\n#include <stdint.h>\n\n#include \"absl/base/optimization.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/ordered_varint/ordered_varint_internal.h\"  // IWYU pragma: export\n\nnamespace riegeli {\n\n// An ordered varint represents an unsigned integer in a variable number of\n// bytes, such that smaller values are represented by lexicographically smaller\n// strings, and also smaller values tend to be represented by shorter strings.\n//\n// Decoding a 64-bit value X:\n//\n// Let F = the first byte of the encoding.\n//\n// Let N = the number of the highest order one bits in F, plus 1.\n// N is in the range [1..9]. X will be decoded from N bytes.\n//\n// Bits of X, from highest to lowest, consist of:\n//  * 8 - N lower order bits of F, if N < 8\n//  * the remaining N - 1 bytes of the encoding in big endian\n//\n// Only the canonical representation is accepted, i.e. the shortest: if N > 1\n// then X must be at least 1 << ((N - 1) * 7).\n\n// Reads an ordered varint.\n//\n// Return values:\n//  * `true`                     - success (`dest` is set)\n//  * `false` (when `src.ok()`)  - source ends\n//                                 (`src` position is unchanged,\n//                                 `dest` is undefined)\n//  * `false` (when `!src.ok()`) - failure\n//                                 (`src` position is unchanged,\n//                                 `dest` is undefined)\nbool ReadOrderedVarint32(Reader& src, uint32_t& dest);\nbool ReadOrderedVarint64(Reader& src, uint64_t& dest);\n\n// Implementation details follow.\n\nnamespace ordered_varint_internal {\n\nbool ReadOrderedVarint32Slow(Reader& src, uint32_t& dest);\nbool ReadOrderedVarint64Slow(Reader& src, uint64_t& dest);\n\n}  // namespace ordered_varint_internal\n\ninline bool ReadOrderedVarint32(Reader& src, uint32_t& dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthOrderedVarint32))) return false;\n  const uint8_t first_byte = static_cast<uint8_t>(*src.cursor());\n  if (ABSL_PREDICT_TRUE(first_byte < 0x80)) {\n    dest = first_byte;\n    src.move_cursor(1);\n    return true;\n  }\n  return ordered_varint_internal::ReadOrderedVarint32Slow(src, dest);\n}\n\ninline bool ReadOrderedVarint64(Reader& src, uint64_t& dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthOrderedVarint64))) return false;\n  const uint8_t first_byte = static_cast<uint8_t>(*src.cursor());\n  if (ABSL_PREDICT_TRUE(first_byte < 0x80)) {\n    dest = first_byte;\n    src.move_cursor(1);\n    return true;\n  }\n  return ordered_varint_internal::ReadOrderedVarint64Slow(src, dest);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_ORDERED_VARINT_ORDERED_VARINT_READING_H_\n"
  },
  {
    "path": "riegeli/ordered_varint/ordered_varint_writing.cc",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/ordered_varint/ordered_varint_writing.h\"\n\n#include <stdint.h>\n\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/endian/endian_writing.h\"\n\nnamespace riegeli::ordered_varint_internal {\n\nbool WriteOrderedVarint32Slow(uint32_t data, Writer& dest) {\n  RIEGELI_ASSERT_GE(data, uint32_t{1} << 7)\n      << \"Failed precondition of WriteOrderedVarint32Slow(): data too small\";\n  if (data < uint32_t{1} << (2 * 7)) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(2))) return false;\n    WriteBigEndian<uint16_t>(IntCast<uint16_t>(data) | uint16_t{0x80} << 8,\n                             dest.cursor());\n    dest.move_cursor(2);\n    return true;\n  } else if (data < uint32_t{1} << (3 * 7)) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(3))) return false;\n    dest.cursor()[0] = static_cast<char>(static_cast<uint8_t>(data >> (2 * 8)) |\n                                         uint8_t{0xc0});\n    WriteBigEndian<uint16_t>(static_cast<uint16_t>(data), dest.cursor() + 1);\n    dest.move_cursor(3);\n    return true;\n  } else if (data < uint32_t{1} << (4 * 7)) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(4))) return false;\n    WriteBigEndian<uint32_t>(data | uint32_t{0xe0} << (3 * 8), dest.cursor());\n    dest.move_cursor(4);\n    return true;\n  } else {\n    if (ABSL_PREDICT_FALSE(!dest.Push(5))) return false;\n    dest.cursor()[0] = static_cast<char>(0xf0);\n    WriteBigEndian<uint32_t>(data, dest.cursor() + 1);\n    dest.move_cursor(5);\n    return true;\n  }\n}\n\nbool WriteOrderedVarint64Slow(uint64_t data, Writer& dest) {\n  RIEGELI_ASSERT_GE(data, uint64_t{1} << 7)\n      << \"Failed precondition of WriteOrderedVarint64Slow(): data too small\";\n  if (data < uint64_t{1} << (2 * 7)) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(2))) return false;\n    WriteBigEndian<uint16_t>(IntCast<uint16_t>(data) | uint16_t{0x80} << 8,\n                             dest.cursor());\n    dest.move_cursor(2);\n    return true;\n  } else if (data < uint64_t{1} << (3 * 7)) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(3))) return false;\n    dest.cursor()[0] = static_cast<char>(static_cast<uint8_t>(data >> (2 * 8)) |\n                                         uint8_t{0xc0});\n    WriteBigEndian<uint16_t>(static_cast<uint16_t>(data), dest.cursor() + 1);\n    dest.move_cursor(3);\n    return true;\n  } else if (data < uint64_t{1} << (4 * 7)) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(4))) return false;\n    WriteBigEndian<uint32_t>(\n        IntCast<uint32_t>(data) | uint32_t{0xe0} << (3 * 8), dest.cursor());\n    dest.move_cursor(4);\n    return true;\n  } else if (data < uint64_t{1} << (5 * 7)) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(5))) return false;\n    dest.cursor()[0] = static_cast<char>(static_cast<uint8_t>(data >> (4 * 8)) |\n                                         uint8_t{0xf0});\n    WriteBigEndian<uint32_t>(static_cast<uint32_t>(data), dest.cursor() + 1);\n    dest.move_cursor(5);\n    return true;\n  } else if (data < uint64_t{1} << (6 * 7)) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(6))) return false;\n    WriteBigEndian<uint16_t>(\n        static_cast<uint16_t>(data >> (4 * 8)) | uint16_t{0xf8} << 8,\n        dest.cursor());\n    WriteBigEndian<uint32_t>(static_cast<uint32_t>(data), dest.cursor() + 2);\n    dest.move_cursor(6);\n    return true;\n  } else if (data < uint64_t{1} << (7 * 7)) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(7))) return false;\n    WriteBigEndian<uint32_t>(\n        static_cast<uint32_t>(data >> (3 * 8)) | uint32_t{0xfc} << (3 * 8),\n        dest.cursor());\n    WriteBigEndian<uint32_t>(static_cast<uint32_t>(data), dest.cursor() + 3);\n    dest.move_cursor(7);\n    return true;\n  } else if (data < uint64_t{1} << (8 * 7)) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(8))) return false;\n    WriteBigEndian<uint64_t>(data | uint64_t{0xfe} << (7 * 8), dest.cursor());\n    dest.move_cursor(8);\n    return true;\n  } else {\n    if (ABSL_PREDICT_FALSE(!dest.Push(9))) return false;\n    dest.cursor()[0] = static_cast<char>(0xff);\n    WriteBigEndian<uint64_t>(data, dest.cursor() + 1);\n    dest.move_cursor(9);\n    return true;\n  }\n}\n\n}  // namespace riegeli::ordered_varint_internal\n"
  },
  {
    "path": "riegeli/ordered_varint/ordered_varint_writing.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_ORDERED_VARINT_ORDERED_VARINT_WRITING_H_\n#define RIEGELI_ORDERED_VARINT_ORDERED_VARINT_WRITING_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/bits.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/ordered_varint/ordered_varint_internal.h\"  // IWYU pragma: export\n\nnamespace riegeli {\n\n// An ordered varint represents an unsigned integer in a variable number of\n// bytes, such that smaller values are represented by lexicographically smaller\n// strings, and also smaller values tend to be represented by shorter strings.\n//\n// Encoding a 64-bit value X:\n//\n// If X == 0, then let L = 0. Otherwise let L = floor(log2(X)). L is in the\n// range [0..63].\n//\n// If L == 63, then let N = 9. Otherwise let N = L / 7 + 1. N is in the range\n// [1..9]. X will be encoded into N bytes.\n//\n// The first byte of the encoding consists of the following bits, from highest\n// to lowest:\n//  * N - 1 one bits\n//  * 1 zero bit, if N < 9\n//  * 8 - N bits representing X >> (8 * (N - 1)), if N < 8\n//\n// The remaining N - 1 bytes represent lower order bytes of X in big endian.\n\n// Writes an ordered varint.\n//\n// Return values:\n//  * `true`  - success (`dest.ok()`)\n//  * `false` - failure (`!dest.ok()`)\nbool WriteOrderedVarint32(uint32_t data, Writer& dest);\nbool WriteOrderedVarint64(uint64_t data, Writer& dest);\n\n// Returns the length needed to write a given value as an ordered varint, which\n// is at most `kMaxLengthOrderedVarint{32,64}`.\nsize_t LengthOrderedVarint32(uint32_t data);\nsize_t LengthOrderedVarint64(uint64_t data);\n\n// Implementation details follow.\n\nnamespace ordered_varint_internal {\n\nbool WriteOrderedVarint32Slow(uint32_t data, Writer& dest);\nbool WriteOrderedVarint64Slow(uint64_t data, Writer& dest);\n\n}  // namespace ordered_varint_internal\n\ninline bool WriteOrderedVarint32(uint32_t data, Writer& dest) {\n  if (ABSL_PREDICT_TRUE(data < 0x80)) {\n    return dest.WriteByte(IntCast<uint8_t>(data));\n  }\n  return ordered_varint_internal::WriteOrderedVarint32Slow(data, dest);\n}\n\ninline bool WriteOrderedVarint64(uint64_t data, Writer& dest) {\n  if (ABSL_PREDICT_TRUE(data < 0x80)) {\n    return dest.WriteByte(IntCast<uint8_t>(data));\n  }\n  return ordered_varint_internal::WriteOrderedVarint64Slow(data, dest);\n}\n\ninline size_t LengthOrderedVarint32(uint32_t data) {\n  const size_t width = IntCast<size_t>(absl::bit_width(data | 1));\n  // This is the same as `(width + 6) / 7` for `width` in [1..32],\n  // but performs division by a power of 2.\n  return (width * 9 + 63) / 64;\n}\n\ninline size_t LengthOrderedVarint64(uint64_t data) {\n  const size_t width = IntCast<size_t>(absl::bit_width(data | 1));\n  // This is the same as `width == 64 ? 9 : (width + 6) / 7`\n  // for `width` in [1..64], but performs division by a power of 2\n  // and does not need a special case for 63.\n  return (width * 9 + 63) / 64;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_ORDERED_VARINT_ORDERED_VARINT_WRITING_H_\n"
  },
  {
    "path": "riegeli/records/BUILD",
    "content": "load(\"@com_google_protobuf//bazel:cc_proto_library.bzl\", \"cc_proto_library\")\nload(\"@com_google_protobuf//bazel:proto_library.bzl\", \"proto_library\")\nload(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"record_reader\",\n    srcs = [\"record_reader.cc\"],\n    hdrs = [\"record_reader.h\"],\n    deps = [\n        \":chunk_reader\",\n        \":record_position\",\n        \":records_metadata_cc_proto\",\n        \":skipped_region\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:binary_search\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:chain_backward_writer\",\n        \"//riegeli/bytes:chain_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/chunk_encoding:chunk\",\n        \"//riegeli/chunk_encoding:chunk_decoder\",\n        \"//riegeli/chunk_encoding:constants\",\n        \"//riegeli/chunk_encoding:field_projection\",\n        \"//riegeli/chunk_encoding:transpose_decoder\",\n        \"//riegeli/messages:parse_message\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/functional:function_ref\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_protobuf//:protobuf\",\n    ],\n)\n\ncc_library(\n    name = \"record_writer\",\n    srcs = [\"record_writer.cc\"],\n    hdrs = [\"record_writer.h\"],\n    deps = [\n        \":block\",\n        \":chunk_writer\",\n        \":record_position\",\n        \":records_metadata_cc_proto\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:options_parser\",\n        \"//riegeli/base:parallelism\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:stable_dependency\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:chain_writer\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/chunk_encoding:chunk\",\n        \"//riegeli/chunk_encoding:chunk_encoder\",\n        \"//riegeli/chunk_encoding:compressor_options\",\n        \"//riegeli/chunk_encoding:constants\",\n        \"//riegeli/chunk_encoding:deferred_encoder\",\n        \"//riegeli/chunk_encoding:simple_encoder\",\n        \"//riegeli/chunk_encoding:transpose_encoder\",\n        \"//riegeli/messages:serialize_message\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/container:flat_hash_set\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/synchronization\",\n        \"@com_google_protobuf//:protobuf\",\n    ],\n)\n\ncc_library(\n    name = \"record_position\",\n    srcs = [\"record_position.cc\"],\n    hdrs = [\"record_position.h\"],\n    deps = [\n        \":block\",\n        \":chunk_writer\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:shared_ptr\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:string_reader\",\n        \"//riegeli/bytes:string_writer\",\n        \"//riegeli/chunk_encoding:chunk\",\n        \"//riegeli/endian:endian_reading\",\n        \"//riegeli/ordered_varint:ordered_varint_reading\",\n        \"//riegeli/ordered_varint:ordered_varint_writing\",\n        \"@com_google_absl//absl/base\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"skipped_region\",\n    srcs = [\"skipped_region.cc\"],\n    hdrs = [\"skipped_region.h\"],\n    deps = [\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:string_ref\",\n        \"//riegeli/base:types\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"chunk_reader\",\n    srcs = [\"chunk_reader.cc\"],\n    hdrs = [\"chunk_reader.h\"],\n    deps = [\n        \":block\",\n        \":skipped_region\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/chunk_encoding:chunk\",\n        \"//riegeli/chunk_encoding:constants\",\n        \"//riegeli/chunk_encoding:hash\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"chunk_writer\",\n    srcs = [\"chunk_writer.cc\"],\n    hdrs = [\"chunk_writer.h\"],\n    deps = [\n        \":block\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:chain_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:string_reader\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/chunk_encoding:chunk\",\n        \"//riegeli/chunk_encoding:constants\",\n        \"//riegeli/chunk_encoding:hash\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"block\",\n    hdrs = [\"block.h\"],\n    visibility = [\"//riegeli:__subpackages__\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:types\",\n        \"//riegeli/chunk_encoding:chunk\",\n        \"//riegeli/chunk_encoding:hash\",\n        \"//riegeli/endian:endian_reading\",\n        \"//riegeli/endian:endian_writing\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\nproto_library(\n    name = \"records_metadata_proto\",\n    srcs = [\"records_metadata.proto\"],\n    deps = [\"@com_google_protobuf//:descriptor_proto\"],\n)\n\ncc_proto_library(\n    name = \"records_metadata_cc_proto\",\n    deps = [\":records_metadata_proto\"],\n)\n"
  },
  {
    "path": "riegeli/records/README.md",
    "content": "# Summary\n\nRiegeli/records is a file format for storing a sequence of records.\n\nThe format supports sequential writing, appending to a previously created file,\nsequential reading, and seeking while reading. Data are optionally compressed,\nwith special support for the case when records are proto messages. Data\ncorruption is detected and a reading can be resumed after skipping over a\ncorrupted region.\n"
  },
  {
    "path": "riegeli/records/block.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_RECORDS_BLOCK_H_\n#define RIEGELI_RECORDS_BLOCK_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/chunk_encoding/chunk.h\"\n#include \"riegeli/chunk_encoding/hash.h\"\n#include \"riegeli/endian/endian_reading.h\"\n#include \"riegeli/endian/endian_writing.h\"\n\nnamespace riegeli::records_internal {\n\nclass BlockHeader {\n public:\n  BlockHeader() = default;\n\n  explicit BlockHeader(uint64_t previous_chunk, uint64_t next_chunk) {\n    set_previous_chunk(previous_chunk);\n    set_next_chunk(next_chunk);\n    set_header_hash(computed_header_hash());\n  }\n\n  BlockHeader(const BlockHeader& that) = default;\n  BlockHeader& operator=(const BlockHeader& that) = default;\n\n  char* bytes() { return bytes_; }\n  const char* bytes() const { return bytes_; }\n  static constexpr size_t size() { return sizeof(bytes_); }\n\n  uint64_t computed_header_hash() const {\n    return chunk_encoding_internal::Hash(absl::string_view(\n        bytes() + sizeof(uint64_t), size() - sizeof(uint64_t)));\n  }\n  uint64_t stored_header_hash() const {\n    return ReadLittleEndian<uint64_t>(bytes_);\n  }\n  uint64_t previous_chunk() const {\n    return ReadLittleEndian<uint64_t>(bytes_ + sizeof(uint64_t));\n  }\n  uint64_t next_chunk() const {\n    return ReadLittleEndian<uint64_t>(bytes_ + 2 * sizeof(uint64_t));\n  }\n\n private:\n  void set_header_hash(uint64_t value) {\n    WriteLittleEndian<uint64_t>(value, bytes_);\n  }\n  void set_previous_chunk(uint64_t value) {\n    WriteLittleEndian<uint64_t>(value, bytes_ + sizeof(uint64_t));\n  }\n  void set_next_chunk(uint64_t value) {\n    WriteLittleEndian<uint64_t>(value, bytes_ + 2 * sizeof(uint64_t));\n  }\n\n  // Representation (Little Endian):\n  //  - `uint64_t`: `header_hash`\n  //  - `uint64_t`: `previous_chunk`\n  //  - `uint64_t`: `next_chunk`\n  char bytes_[3 * sizeof(uint64_t)];\n};\n\ninline constexpr Position kBlockSize = Position{1} << 16;\n\ninline constexpr Position kUsableBlockSize = kBlockSize - BlockHeader::size();\n\n// Whether `pos` is a block boundary (immediately before a block header).\ninline bool IsBlockBoundary(Position pos) { return pos % kBlockSize == 0; }\n\n// The nearest block boundary at or before `pos`.\ninline Position RoundDownToBlockBoundary(Position pos) {\n  return pos - pos % kBlockSize;\n}\n\n// How many bytes remain until the end of the block (0 at a block boundary).\ninline Position RemainingInBlock(Position pos) { return (-pos) % kBlockSize; }\n\n// Whether `pos` is a possible chunk boundary (not inside nor immediately after\n// a block header).\ninline bool IsPossibleChunkBoundary(Position pos) {\n  return RemainingInBlock(pos) < kUsableBlockSize;\n}\n\n// The nearest possible chunk boundary at or after `pos` (chunk boundaries are\n// not valid inside or immediately after a block header).\ninline Position RoundUpToPossibleChunkBoundary(Position pos) {\n  return pos + SaturatingSub(RemainingInBlock(pos), kUsableBlockSize - 1);\n}\n\n// If `pos` is immediately before or inside a block header, how many bytes\n// remain until the end of the block header, otherwise 0.\ninline size_t RemainingInBlockHeader(Position pos) {\n  return SaturatingSub(BlockHeader::size(), pos % kBlockSize);\n}\n\n// For a chunk beginning at `chunk_begin`, the position after `length`, adding\n// intervening block headers.\ninline Position AddWithOverhead(Position chunk_begin, Position length) {\n  RIEGELI_ASSERT_LT(RemainingInBlock(chunk_begin), kUsableBlockSize)\n      << \"Failed precondition of AddWithOverhead(): invalid chunk boundary\";\n  const Position num_overhead_blocks =\n      (length + (chunk_begin + kUsableBlockSize - 1) % kBlockSize) /\n      kUsableBlockSize;\n  return chunk_begin + length + num_overhead_blocks * BlockHeader::size();\n}\n\n// For a chunk beginning at `chunk_begin`, the length until `pos`, subtracting\n// intervening block headers.\ninline Position DistanceWithoutOverhead(Position chunk_begin, Position pos) {\n  RIEGELI_ASSERT_LE(chunk_begin, pos)\n      << \"Failed precondition of DistanceWithoutOverhead(): \"\n         \"positions in the wrong order\";\n  const Position num_overhead_blocks =\n      pos / kBlockSize - chunk_begin / kBlockSize;\n  return (pos - UnsignedMin(pos % kBlockSize, BlockHeader::size())) -\n         (chunk_begin -\n          UnsignedMin(chunk_begin % kBlockSize, BlockHeader::size())) -\n         num_overhead_blocks * BlockHeader::size();\n}\n\n// The position after a chunk which begins at `chunk_begin`.\ninline Position ChunkEnd(const ChunkHeader& header, Position chunk_begin) {\n  return UnsignedMax(\n      AddWithOverhead(chunk_begin, header.size() + header.data_size()),\n      RoundUpToPossibleChunkBoundary(chunk_begin + header.num_records()));\n}\n\n}  // namespace riegeli::records_internal\n\n#endif  // RIEGELI_RECORDS_BLOCK_H_\n"
  },
  {
    "path": "riegeli/records/chunk_reader.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/records/chunk_reader.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/chunk_encoding/chunk.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/chunk_encoding/hash.h\"\n#include \"riegeli/records/block.h\"\n#include \"riegeli/records/skipped_region.h\"\n\nnamespace riegeli {\n\nvoid DefaultChunkReaderBase::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of DefaultChunkReader: null Reader pointer\";\n  pos_ = src->pos();\n  if (ABSL_PREDICT_FALSE(!src->ok()) && src->available() == 0) {\n    FailWithoutAnnotation(src->status());\n    return;\n  }\n  if (ABSL_PREDICT_FALSE(!records_internal::IsPossibleChunkBoundary(pos_))) {\n    recoverable_ = Recoverable::kFindChunk;\n    recoverable_pos_ = pos_;\n    Fail(absl::InvalidArgumentError(\n        absl::StrCat(\"Invalid chunk boundary: \", pos_)));\n  }\n}\n\nvoid DefaultChunkReaderBase::Done() {\n  recoverable_ = Recoverable::kNo;\n  recoverable_pos_ = 0;\n  if (ABSL_PREDICT_FALSE(truncated_)) {\n    Reader& src = *SrcReader();\n    RIEGELI_ASSERT_GT(src.pos(), pos_)\n        << \"Failed invariant of DefaultChunkReader: a chunk beginning must \"\n           \"have been read for the chunk to be considered incomplete\";\n    recoverable_ = Recoverable::kHaveChunk;\n    recoverable_pos_ = src.pos();\n    const Position chunk_header_read =\n        records_internal::DistanceWithoutOverhead(pos_, recoverable_pos_);\n    Fail(absl::InvalidArgumentError(absl::StrCat(\n        \"Truncated Riegeli/records file, incomplete chunk at \", pos_,\n        \" with length \", recoverable_pos_ - pos_,\n        chunk_header_read < chunk_.header.size()\n            ? std::string()\n            : absl::StrCat(\n                  \" < \",\n                  records_internal::ChunkEnd(chunk_.header, pos_) - pos_))));\n  }\n  chunk_.Reset();\n}\n\ninline bool DefaultChunkReaderBase::FailReading(const Reader& src) {\n  if (ABSL_PREDICT_FALSE(!src.ok())) return FailWithoutAnnotation(src.status());\n  if (ABSL_PREDICT_FALSE(src.pos() > pos_)) truncated_ = true;\n  return false;\n}\n\ninline bool DefaultChunkReaderBase::FailSeeking(const Reader& src,\n                                                Position new_pos) {\n  if (ABSL_PREDICT_FALSE(!src.ok())) return FailWithoutAnnotation(src.status());\n  recoverable_ = Recoverable::kFindChunk;\n  recoverable_pos_ = src.pos();\n  return Fail(absl::InvalidArgumentError(absl::StrCat(\n      \"Position \", new_pos, \" exceeds file size: \", recoverable_pos_)));\n}\n\nabsl::Status DefaultChunkReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Reader& src = *SrcReader();\n    return src.AnnotateStatus(std::move(status));\n  }\n  return status;\n}\n\nbool DefaultChunkReaderBase::CheckFileFormat() {\n  return PullChunkHeader(nullptr);\n}\n\nbool DefaultChunkReaderBase::ReadChunk(Chunk& chunk) {\n  if (ABSL_PREDICT_FALSE(!PullChunkHeader(nullptr))) return false;\n  Reader& src = *SrcReader();\n  const Position chunk_end = records_internal::ChunkEnd(chunk_.header, pos_);\n  src.ReadHint(SaturatingIntCast<size_t>(chunk_end - src.pos()),\n               SaturatingIntCast<size_t>(records_internal::AddWithOverhead(\n                                             chunk_end, ChunkHeader::size()) -\n                                         src.pos()));\n\n  while (chunk_.data.size() < chunk_.header.data_size()) {\n    if (records_internal::RemainingInBlockHeader(src.pos()) > 0) {\n      const Position block_begin =\n          records_internal::RoundDownToBlockBoundary(src.pos());\n      if (ABSL_PREDICT_FALSE(!ReadBlockHeader())) return false;\n      if (ABSL_PREDICT_FALSE(block_header_.previous_chunk() !=\n                             block_begin - pos_)) {\n        if (block_header_.next_chunk() <= records_internal::kBlockSize) {\n          // Trust the rest of the block header: skip to the next chunk.\n          recoverable_ = Recoverable::kHaveChunk;\n          recoverable_pos_ = block_begin + block_header_.next_chunk();\n        } else {\n          // Skip to the next block header.\n          recoverable_ = Recoverable::kFindChunk;\n          recoverable_pos_ = src.pos();\n        }\n        return Fail(absl::InvalidArgumentError(absl::StrCat(\n            \"Invalid Riegeli/records file: chunk boundary is \", pos_,\n            \" but block header at \", block_begin,\n            \" implies a different previous chunk boundary: \",\n            block_begin >= block_header_.previous_chunk()\n                ? absl::StrCat(block_begin - block_header_.previous_chunk())\n                : absl::StrCat(\"-\",\n                               block_header_.previous_chunk() - block_begin))));\n      }\n      if (ABSL_PREDICT_FALSE(block_header_.next_chunk() !=\n                             chunk_end - block_begin)) {\n        recoverable_ = Recoverable::kFindChunk;\n        recoverable_pos_ = src.pos();\n        return Fail(absl::InvalidArgumentError(\n            absl::StrCat(\"Invalid Riegeli/records file: chunk boundary is \",\n                         chunk_end, \" but block header at \", block_begin,\n                         \" implies a different next chunk boundary: \",\n                         block_begin + block_header_.next_chunk())));\n      }\n    }\n    if (ABSL_PREDICT_FALSE(!src.ReadAndAppend(\n            IntCast<size_t>(\n                UnsignedMin(chunk_.header.data_size() - chunk_.data.size(),\n                            records_internal::RemainingInBlock(src.pos()))),\n            chunk_.data))) {\n      return FailReading(src);\n    }\n  }\n\n  if (ABSL_PREDICT_FALSE(!src.Seek(chunk_end))) return FailReading(src);\n\n  const uint64_t computed_data_hash =\n      chunk_encoding_internal::Hash(chunk_.data);\n  if (ABSL_PREDICT_FALSE(computed_data_hash != chunk_.header.data_hash())) {\n    // `Recoverable::kHaveChunk`, not `Recoverable::kFindChunk`, because while\n    // chunk data are invalid, chunk header has a correct hash, and thus the\n    // next chunk is believed to be present after this chunk.\n    recoverable_ = Recoverable::kHaveChunk;\n    recoverable_pos_ = chunk_end;\n    return Fail(absl::InvalidArgumentError(absl::StrCat(\n        \"Corrupted Riegeli/records file: chunk data hash mismatch (computed 0x\",\n        absl::Hex(computed_data_hash, absl::PadSpec::kZeroPad16), \", stored 0x\",\n        absl::Hex(chunk_.header.data_hash(), absl::PadSpec::kZeroPad16),\n        \"), chunk at \", pos_, \" with length \", chunk_end - pos_)));\n  }\n\n  chunk = std::move(chunk_);\n  pos_ = chunk_end;\n  chunk_.Clear();\n  return true;\n}\n\nbool DefaultChunkReaderBase::PullChunkHeader(const ChunkHeader** chunk_header) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  truncated_ = false;\n\n  if (ABSL_PREDICT_FALSE(src.pos() < pos_)) {\n    // Source ended in a skipped region.\n    if (!src.Pull()) {\n      // Source still ends at the same position.\n      if (ABSL_PREDICT_FALSE(!src.ok())) {\n        return FailWithoutAnnotation(src.status());\n      }\n      return false;\n    }\n    // Source has grown. Recovery can continue.\n    recoverable_ = Recoverable::kHaveChunk;\n    recoverable_pos_ = pos_;\n    pos_ = src.pos();\n    return Fail(absl::InvalidArgumentError(absl::StrCat(\n        \"Riegeli/records file ended at \", pos_,\n        \" but has grown and will be skipped until \", recoverable_pos_)));\n  }\n\n  const Position chunk_header_read =\n      records_internal::DistanceWithoutOverhead(pos_, src.pos());\n  if (chunk_header_read < chunk_.header.size()) {\n    if (ABSL_PREDICT_FALSE(!ReadChunkHeader())) return false;\n  }\n  if (chunk_header != nullptr) *chunk_header = &chunk_.header;\n  return true;\n}\n\ninline bool DefaultChunkReaderBase::ReadChunkHeader() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of DefaultChunkReaderBase::ReadChunkHeader()\";\n  Reader& src = *SrcReader();\n  RIEGELI_ASSERT_LT(records_internal::DistanceWithoutOverhead(pos_, src.pos()),\n                    chunk_.header.size())\n      << \"Failed precondition of DefaultChunkReaderBase::ReadChunkHeader(): \"\n         \"chunk header already read\";\n  size_t remaining_length;\n  size_t length_to_read;\n  do {\n    if (records_internal::RemainingInBlockHeader(src.pos()) > 0) {\n      const Position block_begin =\n          records_internal::RoundDownToBlockBoundary(src.pos());\n      if (ABSL_PREDICT_FALSE(!ReadBlockHeader())) return false;\n      if (ABSL_PREDICT_FALSE(block_header_.previous_chunk() !=\n                             block_begin - pos_)) {\n        if (block_header_.next_chunk() <= records_internal::kBlockSize) {\n          // Trust the rest of the block header: skip to the next chunk.\n          recoverable_ = Recoverable::kHaveChunk;\n          recoverable_pos_ = block_begin + block_header_.next_chunk();\n        } else {\n          // Skip to the next block header.\n          recoverable_ = Recoverable::kFindChunk;\n          recoverable_pos_ = src.pos();\n        }\n        return Fail(absl::InvalidArgumentError(absl::StrCat(\n            \"Invalid Riegeli/records file: chunk boundary is \", pos_,\n            \" but block header at \", block_begin,\n            \" implies a different previous chunk boundary: \",\n            block_begin >= block_header_.previous_chunk()\n                ? absl::StrCat(block_begin - block_header_.previous_chunk())\n                : absl::StrCat(\"-\",\n                               block_header_.previous_chunk() - block_begin))));\n      }\n    }\n    const size_t chunk_header_read = IntCast<size_t>(\n        records_internal::DistanceWithoutOverhead(pos_, src.pos()));\n    remaining_length = chunk_.header.size() - chunk_header_read;\n    length_to_read = UnsignedMin(remaining_length,\n                                 records_internal::RemainingInBlock(src.pos()));\n    if (ABSL_PREDICT_FALSE(!src.Read(\n            length_to_read, chunk_.header.bytes() + chunk_header_read))) {\n      return FailReading(src);\n    }\n  } while (length_to_read < remaining_length);\n\n  const uint64_t computed_header_hash = chunk_.header.computed_header_hash();\n  if (ABSL_PREDICT_FALSE(computed_header_hash !=\n                         chunk_.header.stored_header_hash())) {\n    recoverable_ = Recoverable::kFindChunk;\n    recoverable_pos_ = src.pos();\n    return Fail(absl::InvalidArgumentError(absl::StrCat(\n        \"Corrupted Riegeli/records file: chunk header hash mismatch \"\n        \"(computed 0x\",\n        absl::Hex(computed_header_hash, absl::PadSpec::kZeroPad16),\n        \", stored 0x\",\n        absl::Hex(chunk_.header.stored_header_hash(),\n                  absl::PadSpec::kZeroPad16),\n        \"), chunk at \", pos_)));\n  }\n  if (records_internal::RemainingInBlock(pos_) < chunk_.header.size()) {\n    // The chunk header was interrupted by a block header. Both headers have\n    // been read so verify that they agree.\n    const Position block_begin =\n        pos_ + records_internal::RemainingInBlock(pos_);\n    const Position chunk_end = records_internal::ChunkEnd(chunk_.header, pos_);\n    if (ABSL_PREDICT_FALSE(block_header_.next_chunk() !=\n                           chunk_end - block_begin)) {\n      recoverable_ = Recoverable::kFindChunk;\n      recoverable_pos_ = src.pos();\n      return Fail(absl::InvalidArgumentError(\n          absl::StrCat(\"Invalid Riegeli/records file: chunk boundary is \",\n                       chunk_end, \" but block header at \", block_begin,\n                       \" implies a different next chunk boundary: \",\n                       block_begin + block_header_.next_chunk())));\n    }\n  }\n  if (pos_ == 0) {\n    // Verify file signature.\n    if (ABSL_PREDICT_FALSE(chunk_.header.data_size() != 0 ||\n                           chunk_.header.chunk_type() !=\n                               ChunkType::kFileSignature ||\n                           chunk_.header.num_records() != 0 ||\n                           chunk_.header.decoded_data_size() != 0)) {\n      recoverable_ = Recoverable::kFindChunk;\n      recoverable_pos_ = src.pos();\n      return Fail(absl::InvalidArgumentError(\n          \"Invalid Riegeli/records file: missing file signature\"));\n    }\n  }\n  return true;\n}\n\ninline bool DefaultChunkReaderBase::ReadBlockHeader() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of DefaultChunkReaderBase::ReadBlockHeader()\";\n  Reader& src = *SrcReader();\n  const size_t remaining_length =\n      records_internal::RemainingInBlockHeader(src.pos());\n  RIEGELI_ASSERT_GT(remaining_length, 0u)\n      << \"Failed precondition of DefaultChunkReaderBase::ReadBlockHeader(): \"\n         \"not before nor inside a block header\";\n  if (ABSL_PREDICT_FALSE(!src.Read(\n          remaining_length,\n          block_header_.bytes() + block_header_.size() - remaining_length))) {\n    return FailReading(src);\n  }\n  const uint64_t computed_header_hash = block_header_.computed_header_hash();\n  if (ABSL_PREDICT_FALSE(computed_header_hash !=\n                         block_header_.stored_header_hash())) {\n    recoverable_ = Recoverable::kFindChunk;\n    recoverable_pos_ = src.pos();\n    return Fail(absl::InvalidArgumentError(absl::StrCat(\n        \"Corrupted Riegeli/records file: block header hash mismatch \"\n        \"(computed 0x\",\n        absl::Hex(computed_header_hash, absl::PadSpec::kZeroPad16),\n        \", stored 0x\",\n        absl::Hex(block_header_.stored_header_hash(),\n                  absl::PadSpec::kZeroPad16),\n        \"), block at \",\n        records_internal::RoundDownToBlockBoundary(recoverable_pos_))));\n  }\n  return true;\n}\n\nbool DefaultChunkReaderBase::Recover(SkippedRegion* skipped_region) {\n  if (recoverable_ == Recoverable::kNo) return false;\n  Reader& src = *SrcReader();\n  const Position region_begin = pos_;\nagain:\n  RIEGELI_ASSERT(!ok()) << \"Failed invariant of DefaultChunkReader: \"\n                           \"recovery applicable but DefaultChunkReader OK\";\n  const Recoverable recoverable = recoverable_;\n  recoverable_ = Recoverable::kNo;\n  Position recoverable_pos = recoverable_pos_;\n  recoverable_pos_ = 0;\n  std::string saved_message(status().message());\n  MarkNotFailed();\n  chunk_.Clear();\n  if (recoverable == Recoverable::kHaveChunk) {\n    pos_ = recoverable_pos;\n    if (ok()) {\n      if (ABSL_PREDICT_FALSE(!src.Seek(pos_))) {\n        if (ABSL_PREDICT_FALSE(!src.ok())) {\n          return FailWithoutAnnotation(src.status());\n        }\n        if (skipped_region != nullptr) {\n          *skipped_region =\n              SkippedRegion(region_begin, src.pos(), std::move(saved_message));\n        }\n        return true;\n      }\n      if (ABSL_PREDICT_FALSE(\n              !records_internal::IsPossibleChunkBoundary(pos_))) {\n        recoverable_ = Recoverable::kFindChunk;\n        recoverable_pos_ = pos_;\n        goto again;\n      }\n    }\n    if (skipped_region != nullptr) {\n      *skipped_region =\n          SkippedRegion(region_begin, pos_, std::move(saved_message));\n    }\n    return true;\n  }\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed invariant of DefaultChunkReader: \"\n         \"chunk boundary not reached yet but DefaultChunkReader is closed\";\n  pos_ = recoverable_pos;\n\nfind_chunk:\n  pos_ += records_internal::RemainingInBlock(pos_);\n  if (ABSL_PREDICT_FALSE(!src.Seek(pos_))) {\n    if (ABSL_PREDICT_FALSE(!src.ok())) {\n      return FailWithoutAnnotation(src.status());\n    }\n    if (skipped_region != nullptr) {\n      *skipped_region =\n          SkippedRegion(region_begin, src.pos(), std::move(saved_message));\n    }\n    return true;\n  }\n  if (ABSL_PREDICT_FALSE(!ReadBlockHeader())) {\n    if (recoverable_ != Recoverable::kNo) goto again;\n    if (ABSL_PREDICT_FALSE(!src.ok())) {\n      return FailWithoutAnnotation(src.status());\n    }\n  } else if (block_header_.previous_chunk() == 0) {\n    // A chunk boundary coincides with block boundary. Recovery is done.\n  } else {\n    pos_ += block_header_.next_chunk();\n    if (ABSL_PREDICT_FALSE(!records_internal::IsPossibleChunkBoundary(pos_))) {\n      goto find_chunk;\n    }\n    if (ABSL_PREDICT_FALSE(!src.Seek(pos_))) {\n      if (ABSL_PREDICT_FALSE(!src.ok())) {\n        return FailWithoutAnnotation(src.status());\n      }\n      if (skipped_region != nullptr) {\n        *skipped_region =\n            SkippedRegion(region_begin, src.pos(), std::move(saved_message));\n      }\n      return true;\n    }\n  }\n  if (skipped_region != nullptr) {\n    *skipped_region =\n        SkippedRegion(region_begin, pos_, std::move(saved_message));\n  }\n  return true;\n}\n\nbool DefaultChunkReaderBase::SupportsRandomAccess() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRandomAccess();\n}\n\nbool DefaultChunkReaderBase::Seek(Position new_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (pos_ == new_pos) return true;\n  Reader& src = *SrcReader();\n  truncated_ = false;\n  pos_ = new_pos;\n  chunk_.Clear();\n  if (ABSL_PREDICT_FALSE(!src.Seek(pos_))) return FailSeeking(src, pos_);\n  if (ABSL_PREDICT_FALSE(!records_internal::IsPossibleChunkBoundary(pos_))) {\n    recoverable_ = Recoverable::kFindChunk;\n    recoverable_pos_ = pos_;\n    return Fail(absl::InvalidArgumentError(\n        absl::StrCat(\"Invalid chunk boundary: \", pos_)));\n  }\n  return true;\n}\n\nbool DefaultChunkReaderBase::SeekToChunkContaining(Position new_pos) {\n  return SeekToChunk<WhichChunk::kContaining>(new_pos);\n}\n\nbool DefaultChunkReaderBase::SeekToChunkBefore(Position new_pos) {\n  return SeekToChunk<WhichChunk::kBefore>(new_pos);\n}\n\nbool DefaultChunkReaderBase::SeekToChunkAfter(Position new_pos) {\n  return SeekToChunk<WhichChunk::kAfter>(new_pos);\n}\n\ntemplate <DefaultChunkReaderBase::WhichChunk which_chunk>\nbool DefaultChunkReaderBase::SeekToChunk(Position new_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (pos_ == new_pos) return true;\n  Reader& src = *SrcReader();\n  truncated_ = false;\n  const Position block_begin =\n      records_internal::RoundDownToBlockBoundary(new_pos);\n  Position chunk_begin;\n  if (pos_ < new_pos) {\n    // The current chunk begins before `new_pos`. If it also ends at or after\n    // `block_begin`, it is better to start searching from the current position\n    // than to seek back to `block_begin`.\n    if (ABSL_PREDICT_FALSE(!PullChunkHeader(nullptr))) {\n      if (ABSL_PREDICT_FALSE(!ok())) return false;\n      truncated_ = false;\n      return FailSeeking(src, new_pos);\n    }\n    if (which_chunk == WhichChunk::kContaining &&\n        pos_ + chunk_.header.num_records() > new_pos) {\n      return true;\n    }\n    const Position chunk_end = records_internal::ChunkEnd(chunk_.header, pos_);\n    if (which_chunk == WhichChunk::kBefore && chunk_end > new_pos) return true;\n    if (chunk_end < block_begin) {\n      // The current chunk ends too early. Skip to `block_begin`.\n      goto read_block_header;\n    }\n    chunk_begin = chunk_end;\n    chunk_.Clear();\n  } else {\n  read_block_header:\n    pos_ = block_begin;\n    chunk_.Clear();\n    if (ABSL_PREDICT_FALSE(!src.Seek(pos_))) return FailSeeking(src, new_pos);\n    if (ABSL_PREDICT_FALSE(!ReadBlockHeader())) {\n      if (ABSL_PREDICT_FALSE(!ok())) return false;\n      if (ABSL_PREDICT_TRUE(!truncated_)) {\n        // File ends at this block boundary, so a chunk ends here too.\n        if (ABSL_PREDICT_TRUE(pos_ >= new_pos)) return true;\n      }\n      truncated_ = false;\n      return FailSeeking(src, new_pos);\n    }\n    if (block_header_.previous_chunk() == 0) {\n      // A chunk boundary coincides with block boundary. The current position is\n      // already before the chunk header; start searching from this chunk,\n      // skipping seeking back and reading the block header again.\n      goto check_current_chunk;\n    }\n    chunk_begin = block_begin + block_header_.next_chunk();\n    if (which_chunk != WhichChunk::kAfter && chunk_begin > new_pos) {\n      // `new_pos` is inside the chunk which contains this block boundary, so\n      // start the search from this chunk instead of the next chunk.\n      if (ABSL_PREDICT_FALSE(block_header_.previous_chunk() > block_begin)) {\n        recoverable_ = Recoverable::kFindChunk;\n        recoverable_pos_ = src.pos();\n        return Fail(absl::InvalidArgumentError(absl::StrCat(\n            \"Invalid Riegeli/records file: block header at \", block_begin,\n            \" implies a negative previous chunk boundary: -\",\n            block_header_.previous_chunk() - block_begin)));\n      }\n      chunk_begin = block_begin - block_header_.previous_chunk();\n    }\n    if (ABSL_PREDICT_FALSE(\n            !records_internal::IsPossibleChunkBoundary(chunk_begin))) {\n      recoverable_ = Recoverable::kFindChunk;\n      recoverable_pos_ = src.pos();\n      return Fail(absl::InvalidArgumentError(absl::StrCat(\n          \"Invalid Riegeli/records file: block header at \", block_begin,\n          \" implies an invalid chunk boundary: \", chunk_begin)));\n    }\n  }\n\n  for (;;) {\n    pos_ = chunk_begin;\n    if (ABSL_PREDICT_FALSE(!src.Seek(pos_))) return FailSeeking(src, new_pos);\n  check_current_chunk:\n    if (pos_ >= new_pos) return true;\n    if (ABSL_PREDICT_FALSE(!ReadChunkHeader())) {\n      if (ABSL_PREDICT_FALSE(!ok())) return false;\n      truncated_ = false;\n      return FailSeeking(src, new_pos);\n    }\n    if (which_chunk == WhichChunk::kContaining &&\n        pos_ + chunk_.header.num_records() > new_pos) {\n      return true;\n    }\n    const Position chunk_end = records_internal::ChunkEnd(chunk_.header, pos_);\n    if (which_chunk == WhichChunk::kBefore && chunk_end > new_pos) return true;\n    chunk_begin = chunk_end;\n  }\n}\n\nstd::optional<Position> DefaultChunkReaderBase::Size() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  Reader& src = *SrcReader();\n  const std::optional<Position> size = src.Size();\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n    FailWithoutAnnotation(src.status());\n  }\n  return size;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/records/chunk_reader.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_RECORDS_CHUNK_READER_H_\n#define RIEGELI_RECORDS_CHUNK_READER_H_\n\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/chunk_encoding/chunk.h\"\n#include \"riegeli/records/block.h\"\n#include \"riegeli/records/skipped_region.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `DefaultChunkReader`.\nclass DefaultChunkReaderBase : public Object {\n public:\n  // Returns the Riegeli/records file being read from. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Ensures that the file looks like a valid Riegeli/Records file.\n  //\n  // Reading the file already checks whether it is valid. `CheckFileFormat()`\n  // can verify this before (or instead of) performing other operations.\n  //\n  // Return values:\n  //  * `true`                 - success\n  //  * `false` (when `ok()`)  - source ends\n  //  * `false` (when `!ok()`) - failure\n  bool CheckFileFormat();\n\n  // Reads the next chunk.\n  //\n  // Return values:\n  //  * `true`                 - success (`chunk` is set)\n  //  * `false` (when `ok()`)  - source ends\n  //  * `false` (when `!ok()`) - failure\n  bool ReadChunk(Chunk& chunk);\n\n  // Reads the next chunk header, from same chunk which will be read by an\n  // immediately following `ReadChunk()`.\n  //\n  // If `chunk_header != nullptr`, `*chunk_header` is set to the chunk header,\n  // valid until the next non-const function of the `ChunkReader`.\n  //\n  // Return values:\n  //  * `true`                 - success (`*chunk_header` is set)\n  //  * `false` (when `ok()`)  - source ends\n  //  * `false` (when `!ok()`) - failure\n  bool PullChunkHeader(const ChunkHeader** chunk_header);\n\n  // If `!ok()` and the failure was caused by invalid file contents, then\n  // `Recover()` tries to recover from the failure and allow reading again by\n  // skipping over the invalid region.\n  //\n  // If `Close()` failed and the failure was caused by truncated file contents,\n  // then `Recover()` returns `true`. The `ChunkReader` remains closed.\n  //\n  // If `ok()`, or if `!ok()` but the failure was not caused by invalid file\n  // contents, then `Recover()` returns false.\n  //\n  // If `skipped_region != nullptr`, `*skipped_region` is set to the position of\n  // the skipped region on success.\n  //\n  // Return values:\n  //  * `true`  - success\n  //  * `false` - failure not caused by invalid file contents\n  bool Recover(SkippedRegion* skipped_region = nullptr);\n\n  // Returns the current position, which is a chunk boundary (except that if\n  // the source ends in a skipped region, it can be greater than file size and\n  // it can be a block boundary).\n  //\n  // `ReadChunk()` and `PullChunkHeader()` return a chunk which begins at\n  // `pos()` if they succeed.\n  //\n  // `pos()` is unchanged by `Close()`.\n  Position pos() const { return pos_; }\n\n  // Returns `true` if this `ChunkReader` supports `Seek()`,\n  // `SeekToChunkContaining()`, `SeekToChunkAfter()`, and `Size()`.\n  bool SupportsRandomAccess();\n\n  // Seeks to `new_pos`, which should be a chunk boundary.\n  //\n  // Return values:\n  //  * `true`  - success\n  //  * `false` - failure (`!ok()`)\n  bool Seek(Position new_pos);\n\n  // Seeks to the nearest chunk boundary before or at `new_pos` if the position\n  // corresponds to some numeric record position in the following chunk (i.e. is\n  // less than `num_records` bytes after chunk beginning), otherwise seeks to\n  // the nearest chunk boundary at or after the given position.\n  //\n  // Return values:\n  //  * `true`  - success\n  //  * `false` - failure (`!ok()`)\n  bool SeekToChunkContaining(Position new_pos);\n\n  // Seeks to the nearest chunk boundary at or before `new_pos`.\n  //\n  // Return values:\n  //  * `true`  - success\n  //  * `false` - failure (`!ok()`)\n  bool SeekToChunkBefore(Position new_pos);\n\n  // Seeks to the nearest chunk boundary at or after `new_pos`.\n  //\n  // Return values:\n  //  * `true`  - success\n  //  * `false` - failure (`!ok()`)\n  bool SeekToChunkAfter(Position new_pos);\n\n  // Returns the size of the file, i.e. the position corresponding to its end.\n  //\n  // Returns `std::nullopt` on failure (`!ok()`).\n  std::optional<Position> Size();\n\n protected:\n  using Object::Object;\n\n  DefaultChunkReaderBase(DefaultChunkReaderBase&& that) noexcept;\n  DefaultChunkReaderBase& operator=(DefaultChunkReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(Reader* src);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n\n private:\n  enum class Recoverable { kNo, kHaveChunk, kFindChunk };\n  enum class WhichChunk { kContaining, kBefore, kAfter };\n\n  // Interprets a `false` result from `src` reading or seeking function.\n  //\n  // End of file (i.e. if `ok()`) is propagated, setting `truncated_` if it was\n  // in the middle of a chunk.\n  //\n  // Always returns `false`.\n  bool FailReading(const Reader& src);\n\n  // Interprets a `false` result from `src` reading or seeking function.\n  //\n  // End of file (i.e. if `ok()`) fails the `ChunkReader`.\n  //\n  // Always returns `false`.\n  bool FailSeeking(const Reader& src, Position new_pos);\n\n  // Reads or continues reading `chunk_.header`.\n  bool ReadChunkHeader();\n\n  // Reads or continues reading `block_header_`.\n  //\n  // Preconditions:\n  //   `ok()`\n  //   `records_internal::RemainingInBlockHeader(SrcReader()->pos()) > 0`\n  bool ReadBlockHeader();\n\n  // Shared implementation of `SeekToChunkContaining()`, `SeekToChunkBefore()`,\n  // and `SeekToChunkAfter()`.\n  //\n  // This template is defined and used only in chunk_reader.cc.\n  template <WhichChunk which_chunk>\n  bool SeekToChunk(Position new_pos);\n\n  // If `true`, the source is truncated (in the middle of a chunk) at the\n  // current position. If the source does not grow, `Close()` will fail.\n  //\n  // Invariant: if `truncated_` then `SrcReader()->pos() > pos_`\n  bool truncated_ = false;\n\n  // Beginning of the current chunk.\n  //\n  // If `pos_ > SrcReader()->pos()`, the source ends in a skipped region. In\n  // this case `pos_` can be a block boundary instead of a chunk boundary.\n  Position pos_ = 0;\n\n  // Chunk header and chunk data, filled to the point derived from `pos_` and\n  // `SrcReader()->pos()`.\n  Chunk chunk_;\n\n  // Block header, filled to the point derived from `SrcReader()->pos()`.\n  records_internal::BlockHeader block_header_;\n\n  // Whether `Recover()` is applicable, and if so, how it should be performed:\n  //\n  //  * `Recoverable::kNo`        - `Recover()` is not applicable\n  //  * `Recoverable::kHaveChunk` - `Recover()` assumes that a chunk starts\n  //                                at `recoverable_pos_`\n  //  * `Recoverable::kFindChunk` - `Recover()` finds a block after\n  //                                `recoverable_pos_`, and a chunk after\n  //                                 the block\n  //\n  // Invariants:\n  //   if `ok()` then `recoverable_ == Recoverable::kNo`\n  //   if `!is_open()` then `recoverable_ == Recoverable::kNo ||\n  //                         recoverable_ == Recoverable::kHaveChunk`\n  Recoverable recoverable_ = Recoverable::kNo;\n\n  // If `recoverable_ != Recoverable::kNo`, the position to start recovery from.\n  //\n  // Invariant:\n  //   if `recoverable_ != Recoverable::kNo` then `recoverable_pos_ >= pos_`\n  Position recoverable_pos_ = 0;\n};\n\n// A `ChunkReader` reads chunks of a Riegeli/records file (rather than\n// individual records, as `RecordReader` does).\n//\n// TODO: If the need arises, `ChunkReader` can be made more abstract\n// than `DefaultChunkReaderBase`, similarly to `ChunkWriter`.\nusing ChunkReader = DefaultChunkReaderBase;\n\n// The default `ChunkReader`. Reads chunks from a byte `Reader`, expecting them\n// to be interleaved with block headers at multiples of the Riegeli/records\n// block size.\n//\n// `DefaultChunkReader` can be used together with `DefaultChunkWriter` to\n// rewrite Riegeli/records files without recompressing chunks, e.g. to\n// concatenate files.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the byte `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The byte `Reader` must not be accessed until the `DefaultChunkReader` is\n// closed or no longer used.\ntemplate <typename Src = Reader*>\nclass DefaultChunkReader : public DefaultChunkReaderBase {\n public:\n  // Creates a closed `DefaultChunkReader`.\n  explicit DefaultChunkReader(Closed) : DefaultChunkReaderBase(kClosed) {}\n\n  // Will read from the byte `Reader` provided by `src`.\n  explicit DefaultChunkReader(Initializer<Src> src);\n\n  DefaultChunkReader(DefaultChunkReader&& that) = default;\n  DefaultChunkReader& operator=(DefaultChunkReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `DefaultChunkReader`. This\n  // avoids constructing a temporary `DefaultChunkReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src);\n\n  // Returns the object providing and possibly owning the byte `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n\n private:\n  // The object providing and possibly owning the Riegeli/records file being\n  // read from.\n  Dependency<Reader*, Src> src_;\n};\n\nexplicit DefaultChunkReader(Closed) -> DefaultChunkReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit DefaultChunkReader(Src&& src) -> DefaultChunkReader<TargetT<Src>>;\n\n// Specialization of `DependencyImpl<ChunkReader*, Manager>` adapted from\n// `DependencyImpl<Reader*, Manager>` by wrapping `Manager` in\n// `DefaultChunkReader<Manager>`.\ntemplate <typename Manager>\nclass DependencyImpl<\n    ChunkReader*, Manager,\n    std::enable_if_t<SupportsDependency<Reader*, Manager>::value>> {\n public:\n  DependencyImpl() noexcept : chunk_reader_(kClosed) {}\n\n  explicit DependencyImpl(Initializer<Manager> manager)\n      : chunk_reader_(std::move(manager)) {}\n\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset() { chunk_reader_.Reset(kClosed); }\n\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Manager> manager) {\n    chunk_reader_.Reset(std::move(manager));\n  }\n\n  Manager& manager() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return chunk_reader_.src();\n  }\n  const Manager& manager() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return chunk_reader_.src();\n  }\n\n  DefaultChunkReader<Manager>* get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return &chunk_reader_;\n  }\n\n  static constexpr bool kIsOwning = true;\n\n  static constexpr bool kIsStable = false;\n\n protected:\n  DependencyImpl(DependencyImpl&& that) = default;\n  DependencyImpl& operator=(DependencyImpl&& that) = default;\n\n  ~DependencyImpl() = default;\n\n private:\n  mutable DefaultChunkReader<Manager> chunk_reader_;\n};\n\n// Implementation details follow.\n\ninline DefaultChunkReaderBase::DefaultChunkReaderBase(\n    DefaultChunkReaderBase&& that) noexcept\n    : Object(static_cast<Object&&>(that)),\n      truncated_(that.truncated_),\n      pos_(that.pos_),\n      chunk_(std::move(that.chunk_)),\n      block_header_(that.block_header_),\n      recoverable_(std::exchange(that.recoverable_, Recoverable::kNo)),\n      recoverable_pos_(that.recoverable_pos_) {}\n\ninline DefaultChunkReaderBase& DefaultChunkReaderBase::operator=(\n    DefaultChunkReaderBase&& that) noexcept {\n  Object::operator=(static_cast<Object&&>(that));\n  truncated_ = that.truncated_;\n  pos_ = that.pos_;\n  chunk_ = that.chunk_;\n  block_header_ = that.block_header_;\n  recoverable_ = std::exchange(that.recoverable_, Recoverable::kNo);\n  recoverable_pos_ = that.recoverable_pos_;\n  return *this;\n}\n\ninline void DefaultChunkReaderBase::Reset(Closed) {\n  Object::Reset(kClosed);\n  truncated_ = false;\n  pos_ = 0;\n  chunk_.Reset();\n  recoverable_ = Recoverable::kNo;\n  recoverable_pos_ = 0;\n}\n\ninline void DefaultChunkReaderBase::Reset() {\n  Object::Reset();\n  truncated_ = false;\n  pos_ = 0;\n  chunk_.Clear();\n  recoverable_ = Recoverable::kNo;\n  recoverable_pos_ = 0;\n}\n\ntemplate <typename Src>\ninline DefaultChunkReader<Src>::DefaultChunkReader(Initializer<Src> src)\n    : src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void DefaultChunkReader<Src>::Reset(Closed) {\n  DefaultChunkReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void DefaultChunkReader<Src>::Reset(Initializer<Src> src) {\n  DefaultChunkReaderBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\nvoid DefaultChunkReader<Src>::Done() {\n  DefaultChunkReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(src_->status());\n    }\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_RECORDS_CHUNK_READER_H_\n"
  },
  {
    "path": "riegeli/records/chunk_writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/records/chunk_writer.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/string_reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/chunk.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/chunk_encoding/hash.h\"\n#include \"riegeli/records/block.h\"\n\nnamespace riegeli {\n\nnamespace records_internal {\n\nPosition PosAfterPadding(Position pos, Position padding) {\n  if (padding <= 1) return pos;\n  const Position remainder = pos % padding;\n  if (remainder == 0) return pos;\n  Position length = padding - remainder;\n  while (length < ChunkHeader::size()) {\n    // Not enough space for the chunk header.\n    length += padding;\n  }\n  Position end_pos = pos + length;\n  while (!records_internal::IsPossibleChunkBoundary(end_pos)) {\n    // `end_pos` falls inside a block header.\n    end_pos += padding;\n  }\n  return end_pos;\n}\n\n}  // namespace records_internal\n\nChunkWriter::~ChunkWriter() {}\n\nvoid DefaultChunkWriterBase::Initialize(Writer* dest, Position pos) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of DefaultChunkWriter: null Writer pointer\";\n  if (ABSL_PREDICT_FALSE(!records_internal::IsPossibleChunkBoundary(pos))) {\n    const Position length = records_internal::RemainingInBlock(pos);\n    dest->Write(ByteFill(length));\n    pos += length;\n  }\n  set_pos(pos);\n  if (ABSL_PREDICT_FALSE(!dest->ok())) FailWithoutAnnotation(dest->status());\n}\n\nabsl::Status DefaultChunkWriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    return dest.AnnotateStatus(std::move(status));\n  }\n  return status;\n}\n\nbool DefaultChunkWriterBase::WriteChunk(const Chunk& chunk) {\n  RIEGELI_ASSERT_EQ(chunk.header.data_hash(),\n                    chunk_encoding_internal::Hash(chunk.data))\n      << \"Failed precondition of ChunkWriter::WriteChunk(): \"\n         \"Wrong chunk data hash\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  StringReader<> header_reader(chunk.header.bytes(), chunk.header.size());\n  ChainReader<> data_reader(&chunk.data);\n  const Position chunk_begin = pos();\n  const Position chunk_end = PosAfterWriteChunk(chunk.header);\n  if (ABSL_PREDICT_FALSE(\n          !WriteSection(header_reader, chunk_begin, chunk_end, dest))) {\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(\n          !WriteSection(data_reader, chunk_begin, chunk_end, dest))) {\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(!WritePadding(chunk_begin, chunk_end, dest))) {\n    return false;\n  }\n  RIEGELI_ASSERT_EQ(pos(), chunk_end)\n      << \"Unexpected position after writing chunk\";\n  return true;\n}\n\ninline bool DefaultChunkWriterBase::WriteSection(Reader& src,\n                                                 Position chunk_begin,\n                                                 Position chunk_end,\n                                                 Writer& dest) {\n  const std::optional<Position> size = src.Size();\n  RIEGELI_ASSERT(size != std::nullopt) << src.status();\n  RIEGELI_ASSERT_EQ(src.pos(), 0u) << \"Non-zero section reader position\";\n  while (src.pos() < *size) {\n    if (records_internal::IsBlockBoundary(pos())) {\n      records_internal::BlockHeader block_header(\n          IntCast<uint64_t>(pos() - chunk_begin),\n          IntCast<uint64_t>(chunk_end - pos()));\n      if (ABSL_PREDICT_FALSE(!dest.Write(\n              absl::string_view(block_header.bytes(), block_header.size())))) {\n        return FailWithoutAnnotation(dest.status());\n      }\n      move_pos(block_header.size());\n    }\n    const Position length = UnsignedMin(\n        *size - src.pos(), records_internal::RemainingInBlock(pos()));\n    if (ABSL_PREDICT_FALSE(!src.Copy(length, dest))) {\n      return FailWithoutAnnotation(dest.status());\n    }\n    move_pos(length);\n  }\n  RIEGELI_EVAL_ASSERT(src.Close()) << src.status();\n  return true;\n}\n\ninline bool DefaultChunkWriterBase::WritePadding(Position chunk_begin,\n                                                 Position chunk_end,\n                                                 Writer& dest) {\n  while (pos() < chunk_end) {\n    if (records_internal::IsBlockBoundary(pos())) {\n      records_internal::BlockHeader block_header(\n          IntCast<uint64_t>(pos() - chunk_begin),\n          IntCast<uint64_t>(chunk_end - pos()));\n      if (ABSL_PREDICT_FALSE(!dest.Write(\n              absl::string_view(block_header.bytes(), block_header.size())))) {\n        return FailWithoutAnnotation(dest.status());\n      }\n      move_pos(block_header.size());\n    }\n    const Position length = UnsignedMin(\n        chunk_end - pos(), records_internal::RemainingInBlock(pos()));\n    if (ABSL_PREDICT_FALSE(!dest.Write(ByteFill(length)))) {\n      return FailWithoutAnnotation(dest.status());\n    }\n    move_pos(length);\n  }\n  return true;\n}\n\nbool DefaultChunkWriterBase::WritePadding(Position padding) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const Position pos_after_pad =\n      records_internal::PosAfterPadding(pos(), padding);\n  if (pos_after_pad == pos()) return true;\n\n  // Excludes the chunk header and any intervening block headers.\n  const size_t padding_data_length =\n      records_internal::DistanceWithoutOverhead(pos(), pos_after_pad) -\n      ChunkHeader::size();\n  Chunk chunk;\n  chunk.data = Chain(ByteFill(padding_data_length));\n  chunk.header = ChunkHeader(chunk.data, ChunkType::kPadding, 0, 0);\n  return WriteChunk(chunk);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/records/chunk_writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_RECORDS_CHUNK_WRITER_H_\n#define RIEGELI_RECORDS_CHUNK_WRITER_H_\n\n#include <limits>\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/chunk.h\"\n#include \"riegeli/records/block.h\"\n\nnamespace riegeli {\n\nnamespace records_internal {\n// Returns the position after `padding` is written after `pos`.\nPosition PosAfterPadding(Position pos, Position padding);\n}  // namespace records_internal\n\nclass Reader;\n\n// A `ChunkWriter` writes chunks of a Riegeli/records file (rather than\n// individual records, as `RecordWriter` does) to a destination.\n//\n// A `ChunkWriter` object can manage a buffer of data to be pushed to the\n// destination, which amortizes the overhead of pushing data over multiple\n// writes.\nclass ChunkWriter : public Object {\n public:\n  ~ChunkWriter() override;\n\n  // Writes a chunk, pushing data to the destination as needed.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  virtual bool WriteChunk(const Chunk& chunk) = 0;\n\n  // Writes padding to reach a position which is a multiple of `padding`.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  virtual bool WritePadding(Position padding) = 0;\n\n  // Pushes buffered data to the destination.\n  //\n  // This makes data written so far visible, but in contrast to `Close()`,\n  // keeps the possibility to write more data later. What exactly does it mean\n  // for data to be visible depends on the destination.\n  //\n  // The scope of objects to flush and the intended data durability (without a\n  // guarantee) are specified by `flush_type`:\n  //  * `FlushType::kFromObject`  - Makes data written so far visible in other\n  //                                objects, propagating flushing through owned\n  //                                dependencies of the given writer.\n  //  * `FlushType::kFromProcess` - Makes data written so far visible outside\n  //                                the process, propagating flushing through\n  //                                dependencies of the given writer.\n  //                                This is the default.\n  //  * `FlushType::kFromMachine` - Makes data written so far visible outside\n  //                                the process and durable in case of operating\n  //                                system crash, propagating flushing through\n  //                                dependencies of the given writer.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool Flush(FlushType flush_type = FlushType::kFromProcess);\n\n  // Returns the current byte position. Unchanged by `Close()`.\n  Position pos() const { return pos_; }\n\n protected:\n  using Object::Object;\n\n  ChunkWriter(ChunkWriter&& that) noexcept;\n  ChunkWriter& operator=(ChunkWriter&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void set_pos(Position pos) { pos_ = pos; }\n  void move_pos(Position length) {\n    RIEGELI_ASSERT_LE(length, std::numeric_limits<Position>::max() - pos_)\n        << \"Failed precondition of ChunkWriter::move_pos(): position overflow\";\n    pos_ += length;\n  }\n  virtual bool FlushImpl(FlushType flush_type) = 0;\n\n  // Returns the expected position after `WriteChunk()` at the current position.\n  Position PosAfterWriteChunk(const ChunkHeader& chunk_header) const;\n\n private:\n  Position pos_ = 0;\n};\n\n// Template parameter independent part of `DefaultChunkWriter`.\nclass DefaultChunkWriterBase : public ChunkWriter {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // File position assumed initially.\n    //\n    // This can be used to prepare a file fragment which can be appended to the\n    // target file at the given position.\n    //\n    // `std::nullopt` means `DestWriter()->pos()`.\n    //\n    // Default: `std::nullopt`.\n    Options& set_assumed_pos(std::optional<Position> assumed_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      assumed_pos_ = assumed_pos;\n      return *this;\n    }\n    Options&& set_assumed_pos(std::optional<Position> assumed_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_assumed_pos(assumed_pos));\n    }\n    std::optional<Position> assumed_pos() const { return assumed_pos_; }\n\n   private:\n    std::optional<Position> assumed_pos_;\n  };\n\n  // Returns the Riegeli/records file being written to. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool WriteChunk(const Chunk& chunk) override;\n  bool WritePadding(Position padding) override;\n\n protected:\n  using ChunkWriter::ChunkWriter;\n\n  DefaultChunkWriterBase(DefaultChunkWriterBase&& that) noexcept;\n  DefaultChunkWriterBase& operator=(DefaultChunkWriterBase&& that) noexcept;\n\n  void Initialize(Writer* dest, Position pos);\n\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n\n private:\n  bool WriteSection(Reader& src, Position chunk_begin, Position chunk_end,\n                    Writer& dest);\n  bool WritePadding(Position chunk_begin, Position chunk_end, Writer& dest);\n};\n\n// The default `ChunkWriter`. Writes chunks to a byte `Writer`, interleaving\n// them with block headers at multiples of the Riegeli/records block size.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the byte `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The byte `Writer` must not be accessed until the `DefaultChunkWriter` is\n// closed or no longer used, except that it is allowed to read the destination\n// of the byte `Writer` immediately after `Flush()`.\ntemplate <typename Dest = Writer*>\nclass DefaultChunkWriter : public DefaultChunkWriterBase {\n public:\n  // Creates a closed `DefaultChunkWriter`.\n  explicit DefaultChunkWriter(Closed) noexcept\n      : DefaultChunkWriterBase(kClosed) {}\n\n  // Will write to the byte `Writer` provided by `dest`.\n  explicit DefaultChunkWriter(Initializer<Dest> dest,\n                              Options options = Options());\n\n  DefaultChunkWriter(DefaultChunkWriter&& that) = default;\n  DefaultChunkWriter& operator=(DefaultChunkWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `DefaultChunkWriter`. This\n  // avoids constructing a temporary `DefaultChunkWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the byte `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  // The object providing and possibly owning the Riegeli/records file being\n  // written to.\n  Dependency<Writer*, Dest> dest_;\n};\n\nexplicit DefaultChunkWriter(Closed) -> DefaultChunkWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit DefaultChunkWriter(\n    Dest&& dest,\n    DefaultChunkWriterBase::Options options = DefaultChunkWriterBase::Options())\n    -> DefaultChunkWriter<TargetT<Dest>>;\n\n// Specialization of `DependencyImpl<ChunkWriter*, Manager>` adapted from\n// `DependencyImpl<Writer*, Manager>` by wrapping `Manager` in\n// `DefaultChunkWriter<Manager>`.\ntemplate <typename Manager>\nclass DependencyImpl<\n    ChunkWriter*, Manager,\n    std::enable_if_t<SupportsDependency<Writer*, Manager>::value>> {\n public:\n  DependencyImpl() noexcept : chunk_writer_(kClosed) {}\n\n  explicit DependencyImpl(Initializer<Manager> manager)\n      : chunk_writer_(std::move(manager)) {}\n\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset() { chunk_writer_.Reset(kClosed); }\n\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Manager> manager) {\n    chunk_writer_.Reset(std::move(manager));\n  }\n\n  Manager& manager() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return chunk_writer_.dest();\n  }\n  const Manager& manager() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return chunk_writer_.dest();\n  }\n\n  DefaultChunkWriter<Manager>* get() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return &chunk_writer_;\n  }\n\n  static constexpr bool kIsOwning = true;\n\n  static constexpr bool kIsStable = false;\n\n protected:\n  DependencyImpl(DependencyImpl&& that) = default;\n  DependencyImpl& operator=(DependencyImpl&& that) = default;\n\n  ~DependencyImpl() = default;\n\n private:\n  mutable DefaultChunkWriter<Manager> chunk_writer_;\n};\n\n// Implementation details follow.\n\ninline ChunkWriter::ChunkWriter(ChunkWriter&& that) noexcept\n    : Object(static_cast<Object&&>(that)), pos_(that.pos_) {}\n\ninline ChunkWriter& ChunkWriter::operator=(ChunkWriter&& that) noexcept {\n  Object::operator=(static_cast<Object&&>(that));\n  pos_ = that.pos_;\n  return *this;\n}\n\ninline void ChunkWriter::Reset(Closed) {\n  Object::Reset(kClosed);\n  pos_ = 0;\n}\n\ninline void ChunkWriter::Reset() {\n  Object::Reset();\n  pos_ = 0;\n}\n\ninline bool ChunkWriter::Flush(FlushType flush_type) {\n  return FlushImpl(flush_type);\n}\n\ninline Position ChunkWriter::PosAfterWriteChunk(\n    const ChunkHeader& chunk_header) const {\n  return records_internal::ChunkEnd(chunk_header, pos_);\n}\n\ninline DefaultChunkWriterBase::DefaultChunkWriterBase(\n    DefaultChunkWriterBase&& that) noexcept\n    : ChunkWriter(static_cast<ChunkWriter&&>(that)) {}\n\ninline DefaultChunkWriterBase& DefaultChunkWriterBase::operator=(\n    DefaultChunkWriterBase&& that) noexcept {\n  ChunkWriter::operator=(static_cast<ChunkWriter&&>(that));\n  return *this;\n}\n\ntemplate <typename Dest>\ninline DefaultChunkWriter<Dest>::DefaultChunkWriter(Initializer<Dest> dest,\n                                                    Options options)\n    : dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.assumed_pos().value_or(dest_->pos()));\n}\n\ntemplate <typename Dest>\ninline void DefaultChunkWriter<Dest>::Reset(Closed) {\n  DefaultChunkWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void DefaultChunkWriter<Dest>::Reset(Initializer<Dest> dest,\n                                            Options options) {\n  DefaultChunkWriterBase::Reset();\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.assumed_pos().value_or(dest_->pos()));\n}\n\ntemplate <typename Dest>\nvoid DefaultChunkWriter<Dest>::Done() {\n  DefaultChunkWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(dest_->status());\n    }\n  }\n}\n\ntemplate <typename Dest>\nbool DefaultChunkWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Flush(flush_type))) {\n      return FailWithoutAnnotation(dest_->status());\n    }\n  }\n  return true;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_RECORDS_CHUNK_WRITER_H_\n"
  },
  {
    "path": "riegeli/records/record_position.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/records/record_position.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <future>\n#include <limits>\n#include <ostream>\n#include <string>\n#include <utility>\n#include <variant>\n#include <vector>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/numbers.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/shared_ptr.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/string_reader.h\"\n#include \"riegeli/bytes/string_writer.h\"\n#include \"riegeli/chunk_encoding/chunk.h\"\n#include \"riegeli/endian/endian_reading.h\"\n#include \"riegeli/ordered_varint/ordered_varint_reading.h\"\n#include \"riegeli/ordered_varint/ordered_varint_writing.h\"\n#include \"riegeli/records/block.h\"\n#include \"riegeli/records/chunk_writer.h\"\n\nnamespace riegeli {\n\nstd::string RecordPosition::ToString() const {\n  return absl::StrCat(chunk_begin_, \"/\", record_index_);\n}\n\nbool RecordPosition::FromString(absl::string_view serialized) {\n  const size_t sep = serialized.find('/');\n  if (ABSL_PREDICT_FALSE(sep == absl::string_view::npos)) return false;\n  uint64_t chunk_begin;\n  if (ABSL_PREDICT_FALSE(\n          !absl::SimpleAtoi(serialized.substr(0, sep), &chunk_begin))) {\n    return false;\n  }\n  uint64_t record_index;\n  if (ABSL_PREDICT_FALSE(\n          !absl::SimpleAtoi(serialized.substr(sep + 1), &record_index))) {\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(record_index >\n                         std::numeric_limits<uint64_t>::max() - chunk_begin)) {\n    return false;\n  }\n  chunk_begin_ = chunk_begin;\n  record_index_ = record_index;\n  return true;\n}\n\nstd::string RecordPosition::ToBytes() const {\n  std::string serialized;\n  StringWriter<> writer(&serialized);\n  WriteOrderedVarint64(chunk_begin_, writer);\n  WriteOrderedVarint64(record_index_, writer);\n  writer.Close();\n  return serialized;\n}\n\nbool RecordPosition::FromBytes(absl::string_view serialized) {\n  if (serialized.size() == 2 * sizeof(uint64_t)) {\n    // Reading the old format is temporarily supported too.\n    const uint64_t chunk_begin = ReadBigEndian<uint64_t>(serialized.data());\n    const uint64_t record_index =\n        ReadBigEndian<uint64_t>(serialized.data() + sizeof(uint64_t));\n    if (ABSL_PREDICT_FALSE(record_index > std::numeric_limits<uint64_t>::max() -\n                                              chunk_begin)) {\n      return false;\n    }\n    chunk_begin_ = chunk_begin;\n    record_index_ = record_index;\n    return true;\n  }\n  StringReader<> reader(serialized);\n  uint64_t chunk_begin, record_index;\n  if (ABSL_PREDICT_FALSE(!ReadOrderedVarint64(reader, chunk_begin))) {\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(!ReadOrderedVarint64(reader, record_index))) {\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(record_index >\n                         std::numeric_limits<uint64_t>::max() - chunk_begin)) {\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(!reader.VerifyEndAndClose())) return false;\n  chunk_begin_ = chunk_begin;\n  record_index_ = record_index;\n  return true;\n}\n\nvoid RecordPosition::Output(std::ostream& dest) const { dest << ToString(); }\n\nnamespace records_internal {\n\ninline FutureChunkBegin::Unresolved::Unresolved(Position pos_before_chunks,\n                                                std::vector<Action> actions)\n    : pos_before_chunks_(pos_before_chunks), actions_(std::move(actions)) {}\n\nvoid FutureChunkBegin::Unresolved::Resolve() const {\n  struct Visitor {\n    void operator()(const std::shared_future<ChunkHeader>& chunk_header) {\n      // Matches `ChunkWriter::PosAfterWriteChunk()`.\n      pos = records_internal::ChunkEnd(chunk_header.get(), pos);\n    }\n    void operator()(const WritePadding& write_padding) {\n      pos = records_internal::PosAfterPadding(pos, write_padding.padding);\n    }\n\n    Position pos;\n  };\n  Visitor visitor{pos_before_chunks_};\n  for (const Action& action : actions_) std::visit(visitor, action);\n  pos_before_chunks_ = visitor.pos;\n  actions_ = std::vector<Action>();\n}\n\nFutureChunkBegin::FutureChunkBegin(Position pos_before_chunks,\n                                   std::vector<Action> actions)\n    : unresolved_(actions.empty()\n                      ? nullptr\n                      : SharedPtr<const Unresolved>(riegeli::Maker(\n                            pos_before_chunks, std::move(actions)))),\n      resolved_(pos_before_chunks) {}\n\n}  // namespace records_internal\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/records/record_position.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_RECORDS_RECORD_POSITION_H_\n#define RIEGELI_RECORDS_RECORD_POSITION_H_\n\n#include <stdint.h>\n\n#include <future>\n#include <iosfwd>\n#include <limits>\n#include <string>\n#include <utility>\n#include <variant>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/call_once.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/shared_ptr.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/chunk_encoding/chunk.h\"\n\nnamespace riegeli {\n\n// `RecordPosition` represents the position of a record in a Riegeli/records\n// file, or a position between records.\n//\n// There are two ways of expressing positions, both strictly monotonic:\n//  * `RecordPosition` (a class) - Faster for seeking.\n//  * `Position` (an integer)    - Scaled between 0 and file size.\n//\n// `RecordPosition` can be converted to `Position` by `numeric()`.\n//\n// Working with `RecordPosition` is recommended, unless it is needed to seek to\n// an approximate position interpolated along the file, e.g. for splitting the\n// file into shards, or unless the position must be expressed as an integer from\n// the range [0..`file_size`] in order to fit into a preexisting API.\nclass RecordPosition : public WithCompare<RecordPosition> {\n public:\n  // Creates a `RecordPosition` corresponding to the first record.\n  constexpr RecordPosition() = default;\n\n  // Creates a `RecordPosition` corresponding to the given record of the chunk\n  // beginning at the given file position.\n  explicit RecordPosition(uint64_t chunk_begin, uint64_t record_index);\n\n  RecordPosition(const RecordPosition& that) = default;\n  RecordPosition& operator=(const RecordPosition& that) = default;\n\n  // File position of the beginning of the chunk containing the given record.\n  uint64_t chunk_begin() const { return chunk_begin_; }\n  // Index of the record within the chunk.\n  uint64_t record_index() const { return record_index_; }\n\n  // Converts `RecordPosition` to an integer scaled between 0 and file size.\n  // Distinct `RecordPosition`s of a valid file have distinct numeric values.\n  uint64_t numeric() const { return chunk_begin_ + record_index_; }\n\n  // Text format: \"<chunk_begin>/<record_index>\".\n  std::string ToString() const;\n  bool FromString(absl::string_view serialized);\n\n  // Binary format: `chunk_begin` and `record_index` as BigEndian-encoded 8-byte\n  // integers. Serialized strings have the same natural order as the\n  // corresponding positions.\n  std::string ToBytes() const;\n  bool FromBytes(absl::string_view serialized);\n\n  friend bool operator==(RecordPosition a, RecordPosition b) {\n    return a.chunk_begin() == b.chunk_begin() &&\n           a.record_index() == b.record_index();\n  }\n  friend StrongOrdering RIEGELI_COMPARE(const RecordPosition& a,\n                                        const RecordPosition& b) {\n    if (const StrongOrdering ordering =\n            riegeli::Compare(a.chunk_begin(), b.chunk_begin());\n        ordering != 0) {\n      return ordering;\n    }\n    return riegeli::Compare(a.record_index(), b.record_index());\n  }\n\n  template <typename HashState>\n  friend HashState AbslHashValue(HashState hash_state, RecordPosition self) {\n    return HashState::combine(std::move(hash_state), self.chunk_begin_,\n                              self.record_index_);\n  }\n\n  // Default stringification by `absl::StrCat()` etc.\n  //\n  // Writes `src.ToString()` to `dest`.\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, RecordPosition src) {\n    dest.Append(src.ToString());\n  }\n\n  // Writes `src.ToString()` to `dest`.\n  friend std::ostream& operator<<(std::ostream& dest, RecordPosition src) {\n    src.Output(dest);\n    return dest;\n  }\n\n private:\n  void Output(std::ostream& dest) const;\n\n  // Invariant:\n  //   `record_index_ <= std::numeric_limits<uint64_t>::max() - chunk_begin_`\n  uint64_t chunk_begin_ = 0;\n  uint64_t record_index_ = 0;\n};\n\n// `FutureChunkBegin` is similar to `std::shared_future<Position>`.\n//\n// It is used to implement `FutureRecordPosition` and internally in\n// `RecordWriter`.\n\nnamespace records_internal {\n\nclass ABSL_ATTRIBUTE_TRIVIAL_ABI FutureChunkBegin {\n public:\n  struct WritePadding {\n    Position padding;\n  };\n  using Action = std::variant<std::shared_future<ChunkHeader>, WritePadding>;\n\n  constexpr FutureChunkBegin() = default;\n\n  /*implicit*/ FutureChunkBegin(Position chunk_begin) noexcept;\n\n  explicit FutureChunkBegin(Position pos_before_chunks,\n                            std::vector<Action> actions);\n\n  FutureChunkBegin(const FutureChunkBegin& that) noexcept;\n  FutureChunkBegin& operator=(const FutureChunkBegin& that) noexcept;\n\n  FutureChunkBegin(FutureChunkBegin&& that) noexcept;\n  FutureChunkBegin& operator=(FutureChunkBegin&& that) noexcept;\n\n  // May block if returned by `RecordWriter` with `parallelism > 0`.\n  Position get() const;\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const FutureChunkBegin* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->unresolved_);\n  }\n\n private:\n  class Unresolved;\n\n  // `unresolved_` is a pointer to save memory in the common case when\n  // it is absent.\n  //\n  // The pointer uses shared ownership because `Unresolved` is not copyable,\n  // which is because its contents are resolved lazily in a const method,\n  // so a copy constructor would need to block.\n  SharedPtr<const Unresolved> unresolved_;\n  // If `unresolved_ == nullptr`, `chunk_begin_` is stored here,\n  // otherwise it is `unresolved_->get()`.\n  Position resolved_ = 0;\n};\n\n}  // namespace records_internal\n\n// `FutureRecordPosition` is similar to `std::shared_future<RecordPosition>`.\n//\n// `RecordWriter` returns `FutureRecordPosition` instead of `RecordPosition`\n// because with `parallelism > 0` the actual position is not known until pending\n// chunks finish encoding in background.\nclass ABSL_ATTRIBUTE_TRIVIAL_ABI FutureRecordPosition {\n public:\n  constexpr FutureRecordPosition() = default;\n\n  /*implicit*/ FutureRecordPosition(RecordPosition pos) noexcept;\n\n  explicit FutureRecordPosition(records_internal::FutureChunkBegin chunk_begin,\n                                uint64_t record_index);\n\n  FutureRecordPosition(const FutureRecordPosition& that) noexcept;\n  FutureRecordPosition& operator=(const FutureRecordPosition& that) noexcept;\n\n  FutureRecordPosition(FutureRecordPosition&& that) noexcept;\n  FutureRecordPosition& operator=(FutureRecordPosition&& that) noexcept;\n\n  // May block if returned by `RecordWriter` with `parallelism > 0`.\n  RecordPosition get() const;\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const FutureRecordPosition* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->chunk_begin_);\n  }\n\n private:\n  records_internal::FutureChunkBegin chunk_begin_;\n  uint64_t record_index_ = 0;\n};\n\n// Implementation details follow.\n\ninline RecordPosition::RecordPosition(uint64_t chunk_begin,\n                                      uint64_t record_index)\n    : chunk_begin_(chunk_begin), record_index_(record_index) {\n  RIEGELI_ASSERT_LE(record_index,\n                    std::numeric_limits<uint64_t>::max() - chunk_begin)\n      << \"RecordPosition overflow\";\n}\n\nnamespace records_internal {\n\nclass FutureChunkBegin::Unresolved {\n public:\n  explicit Unresolved(Position pos_before_chunks, std::vector<Action> actions);\n\n  Unresolved(const Unresolved&) = delete;\n  Unresolved& operator=(const Unresolved&) = delete;\n\n  Position get() const;\n\n  // Supports `MemoryEstimator`.\n  template <typename MemoryEstimator>\n  friend void RiegeliRegisterSubobjects(const Unresolved* self,\n                                        MemoryEstimator& memory_estimator) {\n    memory_estimator.RegisterSubobjects(&self->actions_);\n  }\n\n private:\n  void Resolve() const;\n\n  mutable absl::once_flag flag_;\n  // Position before writing chunks according to `actions_`.\n  mutable Position pos_before_chunks_ = 0;\n  // Headers of chunks to be written after `pos_before_chunks_`.\n  mutable std::vector<Action> actions_;\n};\n\ninline Position FutureChunkBegin::Unresolved::get() const {\n  absl::call_once(flag_, &Unresolved::Resolve, this);\n  RIEGELI_ASSERT(actions_.empty()) << \"FutureChunkBegin::Unresolved::Resolve() \"\n                                      \"did not clear actions_\";\n  return pos_before_chunks_;\n}\n\ninline FutureChunkBegin::FutureChunkBegin(Position chunk_begin) noexcept\n    : resolved_(chunk_begin) {}\n\ninline FutureChunkBegin::FutureChunkBegin(const FutureChunkBegin& that) noexcept\n    : unresolved_(that.unresolved_), resolved_(that.resolved_) {}\n\ninline FutureChunkBegin& FutureChunkBegin::operator=(\n    const FutureChunkBegin& that) noexcept {\n  unresolved_ = that.unresolved_;\n  resolved_ = that.resolved_;\n  return *this;\n}\n\ninline FutureChunkBegin::FutureChunkBegin(FutureChunkBegin&& that) noexcept\n    : unresolved_(std::move(that.unresolved_)),\n      resolved_(std::exchange(that.resolved_, 0)) {}\n\ninline FutureChunkBegin& FutureChunkBegin::operator=(\n    FutureChunkBegin&& that) noexcept {\n  unresolved_ = std::move(that.unresolved_);\n  resolved_ = std::exchange(that.resolved_, 0);\n  return *this;\n}\n\ninline Position FutureChunkBegin::get() const {\n  return unresolved_ == nullptr ? resolved_ : unresolved_->get();\n}\n\n}  // namespace records_internal\n\ninline FutureRecordPosition::FutureRecordPosition(RecordPosition pos) noexcept\n    : chunk_begin_(pos.chunk_begin()), record_index_(pos.record_index()) {}\n\ninline FutureRecordPosition::FutureRecordPosition(\n    records_internal::FutureChunkBegin chunk_begin, uint64_t record_index)\n    : chunk_begin_(std::move(chunk_begin)), record_index_(record_index) {}\n\ninline FutureRecordPosition::FutureRecordPosition(\n    const FutureRecordPosition& that) noexcept\n    : chunk_begin_(that.chunk_begin_), record_index_(that.record_index_) {}\n\ninline FutureRecordPosition& FutureRecordPosition::operator=(\n    const FutureRecordPosition& that) noexcept {\n  chunk_begin_ = that.chunk_begin_;\n  record_index_ = that.record_index_;\n  return *this;\n}\n\ninline FutureRecordPosition::FutureRecordPosition(\n    FutureRecordPosition&& that) noexcept\n    : chunk_begin_(std::move(that.chunk_begin_)),\n      record_index_(std::exchange(that.record_index_, 0)) {}\n\ninline FutureRecordPosition& FutureRecordPosition::operator=(\n    FutureRecordPosition&& that) noexcept {\n  chunk_begin_ = std::move(that.chunk_begin_);\n  record_index_ = std::exchange(that.record_index_, 0);\n  return *this;\n}\n\ninline RecordPosition FutureRecordPosition::get() const {\n  return RecordPosition(chunk_begin_.get(), record_index_);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_RECORDS_RECORD_POSITION_H_\n"
  },
  {
    "path": "riegeli/records/record_reader.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/records/record_reader.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <functional>\n#include <memory>\n#include <optional>\n#include <string>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/functional/function_ref.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/protobuf/descriptor.h\"\n#include \"google/protobuf/descriptor.pb.h\"\n#include \"google/protobuf/message.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/binary_search.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/chain_backward_writer.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/chunk_encoding/chunk.h\"\n#include \"riegeli/chunk_encoding/chunk_decoder.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/chunk_encoding/field_projection.h\"\n#include \"riegeli/chunk_encoding/transpose_decoder.h\"\n#include \"riegeli/messages/parse_message.h\"\n#include \"riegeli/records/chunk_reader.h\"\n#include \"riegeli/records/record_position.h\"\n#include \"riegeli/records/records_metadata.pb.h\"\n#include \"riegeli/records/skipped_region.h\"\n\nnamespace riegeli {\n\nclass RecordsMetadataDescriptors::ErrorCollector\n    : public google::protobuf::DescriptorPool::ErrorCollector {\n public:\n  void RecordError(absl::string_view filename, absl::string_view element_name,\n                   const google::protobuf::Message* descriptor,\n                   ErrorLocation location, absl::string_view message) override {\n    descriptors_->Fail(absl::InvalidArgumentError(\n        absl::StrCat(\"Error in file \", filename, \", element \", element_name,\n                     \": \", message)));\n  }\n\n private:\n  friend class RecordsMetadataDescriptors;\n\n  explicit ErrorCollector(RecordsMetadataDescriptors* descriptors)\n      : descriptors_(descriptors) {}\n\n  RecordsMetadataDescriptors* descriptors_;\n};\n\nRecordsMetadataDescriptors::RecordsMetadataDescriptors(\n    const RecordsMetadata& metadata)\n    : record_type_name_(metadata.record_type_name()) {\n  if (record_type_name_.empty() || metadata.file_descriptor().empty()) return;\n  pool_ = std::make_unique<google::protobuf::DescriptorPool>();\n  ErrorCollector error_collector(this);\n  for (const google::protobuf::FileDescriptorProto& file_descriptor :\n       metadata.file_descriptor()) {\n    if (ABSL_PREDICT_FALSE(pool_->BuildFileCollectingErrors(\n                               file_descriptor, &error_collector) == nullptr)) {\n      return;\n    }\n  }\n}\n\nconst google::protobuf::Descriptor* RecordsMetadataDescriptors::descriptor()\n    const {\n  if (pool_ == nullptr) return nullptr;\n  return pool_->FindMessageTypeByName(record_type_name_);\n}\n\nRecordReaderBase::RecordReaderBase(Closed) noexcept : Object(kClosed) {}\n\nRecordReaderBase::RecordReaderBase() noexcept {}\n\nRecordReaderBase::RecordReaderBase(RecordReaderBase&& that) noexcept\n    : Object(static_cast<Object&&>(that)),\n      chunk_begin_(that.chunk_begin_),\n      chunk_decoder_(std::move(that.chunk_decoder_)),\n      last_record_is_valid_(std::exchange(that.last_record_is_valid_, false)),\n      flatten_(std::exchange(that.flatten_, false)),\n      recoverable_(std::exchange(that.recoverable_, Recoverable::kNo)),\n      recovery_(std::move(that.recovery_)),\n      recycling_pool_options_(that.recycling_pool_options_) {}\n\nRecordReaderBase& RecordReaderBase::operator=(\n    RecordReaderBase&& that) noexcept {\n  Object::operator=(static_cast<Object&&>(that));\n  chunk_begin_ = that.chunk_begin_;\n  chunk_decoder_ = std::move(that.chunk_decoder_);\n  last_record_is_valid_ = std::exchange(that.last_record_is_valid_, false);\n  flatten_ = std::exchange(that.flatten_, false);\n  recoverable_ = std::exchange(that.recoverable_, Recoverable::kNo);\n  recovery_ = std::move(that.recovery_);\n  recycling_pool_options_ = that.recycling_pool_options_;\n  return *this;\n}\n\nvoid RecordReaderBase::Reset(Closed) {\n  Object::Reset(kClosed);\n  chunk_begin_ = 0;\n  chunk_decoder_.Reset();\n  last_record_is_valid_ = false;\n  flatten_ = false;\n  recoverable_ = Recoverable::kNo;\n  recovery_ = nullptr;\n  recycling_pool_options_ = RecyclingPoolOptions();\n}\n\nvoid RecordReaderBase::Reset() {\n  Object::Reset();\n  chunk_begin_ = 0;\n  chunk_decoder_.Clear();\n  last_record_is_valid_ = false;\n  flatten_ = false;\n  recoverable_ = Recoverable::kNo;\n  recovery_ = nullptr;\n  recycling_pool_options_ = RecyclingPoolOptions();\n}\n\nvoid RecordReaderBase::Initialize(ChunkReader* src, Options&& options) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of RecordReader: null ChunkReader pointer\";\n  if (ABSL_PREDICT_FALSE(!src->ok())) {\n    FailWithoutAnnotation(src->status());\n    return;\n  }\n  chunk_begin_ = src->pos();\n  chunk_decoder_.Reset(\n      ChunkDecoder::Options()\n          .set_field_projection(std::move(options.field_projection()))\n          .set_recycling_pool_options(options.recycling_pool_options()));\n  recovery_ = std::move(options.recovery());\n  recycling_pool_options_ = options.recycling_pool_options();\n}\n\nvoid RecordReaderBase::Done() {\n  last_record_is_valid_ = false;\n  recoverable_ = Recoverable::kNo;\n  if (ABSL_PREDICT_FALSE(!chunk_decoder_.Close())) {\n    Fail(chunk_decoder_.status());\n  }\n}\n\ninline bool RecordReaderBase::FailReading(const ChunkReader& src) {\n  recoverable_ = Recoverable::kRecoverChunkReader;\n  FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n  return TryRecovery();\n}\n\ninline bool RecordReaderBase::FailSeeking(const ChunkReader& src) {\n  chunk_begin_ = src.pos();\n  chunk_decoder_.Clear();\n  recoverable_ = Recoverable::kRecoverChunkReader;\n  FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n  return TryRecovery();\n}\n\nabsl::Status RecordReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    ChunkReader& src = *SrcChunkReader();\n    status = src.AnnotateStatus(std::move(status));\n  }\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status RecordReaderBase::AnnotateOverSrc(absl::Status status) {\n  return Annotate(status, absl::StrCat(\"at record \", pos().ToString()));\n}\n\nbool RecordReaderBase::CheckFileFormat() {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (chunk_decoder_.num_records() > 0) return true;\n  ChunkReader& src = *SrcChunkReader();\n  if (ABSL_PREDICT_FALSE(!src.CheckFileFormat())) {\n    chunk_decoder_.Clear();\n    if (ABSL_PREDICT_FALSE(!src.ok())) {\n      recoverable_ = Recoverable::kRecoverChunkReader;\n      return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    }\n    return false;\n  }\n  return true;\n}\n\nbool RecordReaderBase::ReadMetadata(RecordsMetadata& metadata) {\n  Chain serialized_metadata;\n  if (ABSL_PREDICT_FALSE(!ReadSerializedMetadata(serialized_metadata))) {\n    metadata.Clear();\n    return false;\n  }\n  if (absl::Status status = ParseMessage(serialized_metadata, metadata);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    metadata.Clear();\n    recoverable_ = Recoverable::kRecoverMetadata;\n    Fail(std::move(status));\n    if (!TryRecovery()) return false;\n    // Recovered metadata parsing, assume empty `RecordsMetadata`.\n  }\n  return true;\n}\n\nbool RecordReaderBase::ReadSerializedMetadata(Chain& metadata) {\n  metadata.Clear();\n  last_record_is_valid_ = false;\n  if (ABSL_PREDICT_FALSE(!ok())) {\n    if (!TryRecovery()) return false;\n    last_record_is_valid_ = true;\n    return ok();\n  }\n  ChunkReader& src = *SrcChunkReader();\n  if (ABSL_PREDICT_FALSE(src.pos() != 0)) {\n    return Fail(absl::FailedPreconditionError(\n        \"RecordReaderBase::ReadMetadata() must be called \"\n        \"while the RecordReader is at the beginning of the file\"));\n  }\n\n  chunk_begin_ = src.pos();\n  Chunk chunk;\n  if (ABSL_PREDICT_FALSE(!src.ReadChunk(chunk))) {\n    if (ABSL_PREDICT_FALSE(!src.ok())) return FailReading(src);\n    return false;\n  }\n  RIEGELI_ASSERT_EQ(chunk.header.chunk_type(), ChunkType::kFileSignature)\n      << \"Unexpected type of the first chunk: \"\n      << static_cast<unsigned>(chunk.header.chunk_type());\n\n  chunk_begin_ = src.pos();\n  const ChunkHeader* chunk_header;\n  if (ABSL_PREDICT_FALSE(!src.PullChunkHeader(&chunk_header))) {\n    if (ABSL_PREDICT_FALSE(!src.ok())) return FailReading(src);\n    return false;\n  }\n  if (chunk_header->chunk_type() != ChunkType::kFileMetadata) {\n    // Missing file metadata chunk, assume empty `RecordsMetadata`.\n    return true;\n  }\n  if (ABSL_PREDICT_FALSE(!src.ReadChunk(chunk))) {\n    if (ABSL_PREDICT_FALSE(!src.ok())) return FailReading(src);\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(!ParseMetadata(chunk, metadata))) {\n    metadata.Clear();\n    recoverable_ = Recoverable::kRecoverChunkDecoder;\n    if (!TryRecovery()) return false;\n    // Recovered metadata decoding, assume empty `RecordsMetadata`.\n  }\n  last_record_is_valid_ = true;\n  return true;\n}\n\ninline bool RecordReaderBase::ParseMetadata(const Chunk& chunk,\n                                            Chain& metadata) {\n  RIEGELI_ASSERT_EQ(chunk.header.chunk_type(), ChunkType::kFileMetadata)\n      << \"Failed precondition of RecordReaderBase::ParseMetadata(): \"\n         \"wrong chunk type\";\n  if (ABSL_PREDICT_FALSE(chunk.header.num_records() != 0)) {\n    return Fail(absl::InvalidArgumentError(absl::StrCat(\n        \"Invalid file metadata chunk: number of records is not zero: \",\n        chunk.header.num_records())));\n  }\n  ChainReader<> data_reader(&chunk.data);\n  TransposeDecoder transpose_decoder(\n      TransposeDecoder::Options().set_recycling_pool_options(\n          recycling_pool_options_));\n  ChainBackwardWriter<> serialized_metadata_writer(&metadata);\n  serialized_metadata_writer.SetWriteSizeHint(chunk.header.decoded_data_size());\n  std::vector<size_t> limits;\n  const bool decode_ok = transpose_decoder.Decode(\n      1, chunk.header.decoded_data_size(), FieldProjection::All(), data_reader,\n      serialized_metadata_writer, limits);\n  if (ABSL_PREDICT_FALSE(!serialized_metadata_writer.Close())) {\n    return Fail(serialized_metadata_writer.status());\n  }\n  if (ABSL_PREDICT_FALSE(!decode_ok)) return Fail(transpose_decoder.status());\n  if (ABSL_PREDICT_FALSE(!data_reader.VerifyEndAndClose())) {\n    return Fail(data_reader.status());\n  }\n  RIEGELI_ASSERT_EQ(limits.size(), 1u)\n      << \"Metadata chunk has unexpected record limits\";\n  RIEGELI_ASSERT_EQ(limits.back(), metadata.size())\n      << \"Metadata chunk has unexpected record limits\";\n  return true;\n}\n\nbool RecordReaderBase::ReadRecord(google::protobuf::MessageLite& record) {\n  flatten_ = false;\n  return ReadRecordImpl(record);\n}\n\nbool RecordReaderBase::ReadRecord(absl::string_view& record) {\n  flatten_ = true;\n  return ReadRecordImpl(record);\n}\n\nbool RecordReaderBase::ReadRecord(std::string& record) {\n  flatten_ = false;\n  return ReadRecordImpl(record);\n}\n\nbool RecordReaderBase::ReadRecord(Chain& record) {\n  flatten_ = false;\n  return ReadRecordImpl(record);\n}\n\nbool RecordReaderBase::ReadRecord(absl::Cord& record) {\n  flatten_ = false;\n  return ReadRecordImpl(record);\n}\n\ntemplate <typename Record>\ninline bool RecordReaderBase::ReadRecordImpl(Record& record) {\n  last_record_is_valid_ = false;\n  for (;;) {\n    if (ABSL_PREDICT_TRUE(chunk_decoder_.ReadRecord(record))) {\n      RIEGELI_ASSERT_GT(chunk_decoder_.index(), 0u)\n          << \"ChunkDecoder::ReadRecord() left record index at 0\";\n      last_record_is_valid_ = true;\n      return true;\n    }\n    if (ABSL_PREDICT_FALSE(!ok())) {\n      if (!TryRecovery()) return false;\n      continue;\n    }\n    if (ABSL_PREDICT_FALSE(!chunk_decoder_.ok())) {\n      recoverable_ = Recoverable::kRecoverChunkDecoder;\n      Fail(chunk_decoder_.status());\n      if (!TryRecovery()) return false;\n      continue;\n    }\n    if (ABSL_PREDICT_FALSE(!ReadChunk())) {\n      if (!TryRecovery()) return false;\n    }\n  }\n}\n\nbool RecordReaderBase::SetFieldProjection(\n    Initializer<FieldProjection> field_projection) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  ChunkReader& src = *SrcChunkReader();\n  const uint64_t record_index = chunk_decoder_.index();\n  chunk_decoder_.ClearAndSetFieldProjection(std::move(field_projection));\n  if (ABSL_PREDICT_FALSE(!src.Seek(chunk_begin_))) return FailSeeking(src);\n  if (record_index > 0) {\n    if (ABSL_PREDICT_FALSE(!ReadChunk())) return TryRecovery();\n    chunk_decoder_.SetIndex(record_index);\n  }\n  return true;\n}\n\nbool RecordReaderBase::Recover(SkippedRegion* skipped_region) {\n  if (recoverable_ == Recoverable::kNo) return false;\n  ChunkReader& src = *SrcChunkReader();\n  RIEGELI_ASSERT(!ok()) << \"Failed invariant of RecordReader: \"\n                           \"recovery applicable but RecordReader OK\";\n  const Recoverable recoverable = recoverable_;\n  recoverable_ = Recoverable::kNo;\n  if (recoverable != Recoverable::kRecoverChunkReader) {\n    RIEGELI_ASSERT(is_open()) << \"Failed invariant of RecordReader: \"\n                                 \"recovery does not apply to chunk reader \"\n                                 \"but RecordReader is closed\";\n  }\n  std::string saved_message(status().message());\n  MarkNotFailed();\n  switch (recoverable) {\n    case Recoverable::kNo:\n      RIEGELI_ASSUME_UNREACHABLE() << \"kNo handled above\";\n    case Recoverable::kRecoverChunkReader:\n      if (ABSL_PREDICT_FALSE(!src.Recover(skipped_region))) {\n        return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n      }\n      return true;\n    case Recoverable::kRecoverChunkDecoder: {\n      const uint64_t index_before = chunk_decoder_.index();\n      if (ABSL_PREDICT_FALSE(!chunk_decoder_.Recover())) chunk_decoder_.Clear();\n      if (skipped_region != nullptr) {\n        *skipped_region =\n            SkippedRegion(chunk_begin_ + index_before, pos().numeric(),\n                          std::move(saved_message));\n      }\n      return true;\n    }\n    case Recoverable::kRecoverMetadata:\n      if (skipped_region != nullptr) {\n        *skipped_region = SkippedRegion(chunk_begin_, pos().numeric(),\n                                        std::move(saved_message));\n      }\n      return true;\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown recoverable method: \" << static_cast<int>(recoverable);\n}\n\nbool RecordReaderBase::SupportsRandomAccess() {\n  ChunkReader* const src = SrcChunkReader();\n  return src != nullptr && src->SupportsRandomAccess();\n}\n\nbool RecordReaderBase::Seek(RecordPosition new_pos) {\n  last_record_is_valid_ = false;\n  if (ABSL_PREDICT_FALSE(!ok())) return TryRecovery();\n  ChunkReader& src = *SrcChunkReader();\n  if (new_pos.chunk_begin() == chunk_begin_) {\n    if (new_pos.record_index() == 0 || src.pos() > chunk_begin_) {\n      // Seeking to the beginning of a chunk does not need reading the chunk,\n      // which is important because it may be non-existent at end of file.\n      //\n      // If `src.pos() > chunk_begin_`, the chunk is already read.\n      goto skip_reading_chunk;\n    }\n  } else {\n    if (ABSL_PREDICT_FALSE(!src.Seek(new_pos.chunk_begin()))) {\n      return FailSeeking(src);\n    }\n    if (new_pos.record_index() == 0) {\n      // Seeking to the beginning of a chunk does not need reading the chunk,\n      // which is important because it may be non-existent at end of file.\n      chunk_begin_ = src.pos();\n      chunk_decoder_.Clear();\n      return true;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!ReadChunk())) return TryRecovery();\nskip_reading_chunk:\n  chunk_decoder_.SetIndex(new_pos.record_index());\n  return true;\n}\n\nbool RecordReaderBase::Seek(Position new_pos) {\n  last_record_is_valid_ = false;\n  if (ABSL_PREDICT_FALSE(!ok())) return TryRecovery();\n  ChunkReader& src = *SrcChunkReader();\n  if (new_pos >= chunk_begin_ && new_pos <= src.pos()) {\n    // Seeking inside or just after the current chunk which has been read,\n    // or to the beginning of the current chunk which has been located,\n    // or to the end of file which has been reached.\n  } else {\n    if (ABSL_PREDICT_FALSE(!src.SeekToChunkContaining(new_pos))) {\n      return FailSeeking(src);\n    }\n    if (src.pos() >= new_pos) {\n      // Seeking to the beginning of a chunk does not need reading the chunk,\n      // which is important because it may be non-existent at end of file.\n      //\n      // It is possible that the chunk position is greater than `new_pos` if\n      // `new_pos` falls after all records of the previous chunk. This also\n      // seeks to the beginning of the chunk.\n      chunk_begin_ = src.pos();\n      chunk_decoder_.Clear();\n      return true;\n    }\n    if (ABSL_PREDICT_FALSE(!ReadChunk())) return TryRecovery();\n  }\n  chunk_decoder_.SetIndex(IntCast<uint64_t>(new_pos - chunk_begin_));\n  return true;\n}\n\nbool RecordReaderBase::SeekBack() {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  last_record_is_valid_ = false;\n  if (ABSL_PREDICT_TRUE(chunk_decoder_.index() > 0)) {\n    chunk_decoder_.SetIndex(chunk_decoder_.index() - 1);\n    return true;\n  }\n  ChunkReader& src = *SrcChunkReader();\n  Position chunk_pos = chunk_begin_;\n  while (chunk_pos > 0) {\n    if (ABSL_PREDICT_FALSE(!src.SeekToChunkBefore(chunk_pos - 1))) {\n      // If recovery succeeds, continue searching back from the beginning of the\n      // skipped region.\n      chunk_pos = src.pos();\n      if (!FailSeeking(src)) return false;\n      continue;\n    }\n    chunk_pos = chunk_begin_;\n    if (ABSL_PREDICT_FALSE(!ReadChunk())) {\n      // If recovery succeeds, continue searching back from the beginning of the\n      // skipped region.\n      if (!TryRecovery()) return false;\n      continue;\n    }\n    if (ABSL_PREDICT_TRUE(chunk_decoder_.num_records() > 0)) {\n      chunk_decoder_.SetIndex(chunk_decoder_.num_records() - 1);\n      return true;\n    }\n    // The chunk has no records. Continue searching back from the beginning of\n    // the chunk.\n  }\n  return false;\n}\n\nstd::optional<Position> RecordReaderBase::Size() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  ChunkReader& src = *SrcChunkReader();\n  const std::optional<Position> size = src.Size();\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n    FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n  }\n  return size;\n}\n\n// Traits for `BinarySearch()`: searching for a chunk.\nclass RecordReaderBase::ChunkSearchTraits {\n public:\n  explicit ChunkSearchTraits(RecordReaderBase* self)\n      : self_(RIEGELI_EVAL_ASSERT_NOTNULL(self)) {}\n\n  using Pos = Position;\n\n  bool Empty(Position low, Position high) const { return low >= high; }\n\n  std::optional<Position> Middle(Position low, Position high) const {\n    if (low >= high) return std::nullopt;\n    ChunkReader& src = *self_->SrcChunkReader();\n    if (ABSL_PREDICT_FALSE(!src.SeekToChunkBefore(low + (high - low) / 2))) {\n      if (!self_->FailSeeking(src)) {\n        // There was a failure or unexpected end of file. Cancel the search.\n        return std::nullopt;\n      }\n      if (src.pos() >= high) {\n        // Skipped region after the middle ends after `high`. Find the next\n        // chunk after `low` instead.\n        if (ABSL_PREDICT_FALSE(!src.Seek(low))) {\n          if (!self_->FailSeeking(src) || src.pos() >= high) {\n            // There was a failure or unexpected end of file, or the whole range\n            // is skipped. Cancel the search.\n            return std::nullopt;\n          }\n        }\n      }\n    }\n    return src.pos();\n  }\n\n private:\n  RecordReaderBase* self_;\n};\n\nstd::optional<PartialOrdering> RecordReaderBase::SearchImpl(\n    absl::FunctionRef<std::optional<PartialOrdering>(RecordReaderBase& reader)>\n        test) {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  last_record_is_valid_ = false;\n  ChunkReader& src = *SrcChunkReader();\n  const std::optional<Position> size = src.Size();\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) {\n    FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    return std::nullopt;\n  }\n\n  struct ChunkSuffix {\n    Position chunk_begin;\n    uint64_t record_index;\n    uint64_t num_records;\n  };\n  std::optional<ChunkSuffix> less_found;\n  uint64_t greater_record_index = 0;\n  std::optional<SearchResult<Position>> greater_chunk_begin = BinarySearch(\n      0, *size,\n      [&](Position chunk_begin) -> std::optional<SearchGuide<Position>> {\n        if (ABSL_PREDICT_FALSE(!src.Seek(chunk_begin))) {\n          if (!FailSeeking(src)) return std::nullopt;\n          // Declare the skipped region unordered.\n          return SearchGuide<Position>(PartialOrdering::unordered, src.pos());\n        }\n        if (ABSL_PREDICT_FALSE(!ReadChunk())) {\n          if (!TryRecovery()) {\n            if (!ok()) return std::nullopt;\n            // The chunk is truncated. Continue the search before the chunk.\n            greater_record_index = 0;\n            return SearchGuide<Position>(PartialOrdering::greater, chunk_begin);\n          }\n          // Declare the skipped region unordered.\n          return SearchGuide<Position>(PartialOrdering::unordered, src.pos());\n        }\n        // `src.pos()` points to the next chunk. Adjust `chunk_begin` in case\n        // recovery moved it forwards.\n        chunk_begin = chunk_begin_;\n        const uint64_t num_records = chunk_decoder_.num_records();\n        // Judge the chunk by its earliest record which is not unordered.\n        for (uint64_t record_index = 0; record_index < num_records;\n             ++record_index) {\n          if (ABSL_PREDICT_FALSE(\n                  !Seek(RecordPosition(chunk_begin, record_index)))) {\n            return std::nullopt;\n          }\n          std::function<bool(const SkippedRegion&, RecordReaderBase&)>\n              recovery = std::exchange(recovery_, nullptr);\n          const std::optional<PartialOrdering> ordering = test(*this);\n          recovery_ = std::move(recovery);\n          if (ABSL_PREDICT_FALSE(!ok())) {\n            // Reading the record made the `RecordReader` not OK, probably\n            // because a message could not be parsed (or `test()` did something\n            // unusual).\n            if (!TryRecovery()) return std::nullopt;\n            // Declare the skipped record unordered.\n            continue;\n          }\n          if (ABSL_PREDICT_FALSE(ordering == std::nullopt)) return std::nullopt;\n          if (*ordering < 0) {\n            less_found =\n                ChunkSuffix{chunk_begin, record_index + 1, num_records};\n            return SearchGuide<Position>(PartialOrdering::less, src.pos());\n          }\n          if (*ordering >= 0) {\n            greater_record_index = record_index;\n            return SearchGuide<Position>(*ordering, chunk_begin);\n          }\n        }\n        // All records are unordered.\n        return SearchGuide<Position>(PartialOrdering::unordered, src.pos());\n      },\n      ChunkSearchTraits(this));\n\n  if (ABSL_PREDICT_FALSE(greater_chunk_begin == std::nullopt)) {\n    return std::nullopt;\n  }\n  if (greater_chunk_begin->ordering != 0 && less_found != std::nullopt) {\n    const std::optional<SearchResult<uint64_t>> less_record_index =\n        BinarySearch(\n            less_found->record_index, less_found->num_records,\n            [&](uint64_t record_index) -> std::optional<PartialOrdering> {\n              if (ABSL_PREDICT_FALSE(!Seek(\n                      RecordPosition(less_found->chunk_begin, record_index)))) {\n                return std::nullopt;\n              }\n              std::function<bool(const SkippedRegion&, RecordReaderBase&)>\n                  recovery = std::exchange(recovery_, nullptr);\n              const std::optional<PartialOrdering> ordering = test(*this);\n              recovery_ = std::move(recovery);\n              if (ABSL_PREDICT_FALSE(!ok())) {\n                // Reading the record made the `RecordReader` not OK, probably\n                // because a message could not be parsed (or `test()` did\n                // something unusual).\n                if (!TryRecovery()) return std::nullopt;\n                // Declare the skipped record unordered.\n                return PartialOrdering::unordered;\n              }\n              return ordering;\n            });\n    if (ABSL_PREDICT_FALSE(less_record_index == std::nullopt)) {\n      return std::nullopt;\n    }\n    if (less_record_index->ordering >= 0) {\n      greater_chunk_begin->ordering = less_record_index->ordering;\n      greater_chunk_begin->found = less_found->chunk_begin;\n      greater_record_index = less_record_index->found;\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!Seek(\n          RecordPosition(greater_chunk_begin->found, greater_record_index)))) {\n    Fail(absl::DataLossError(\"Riegeli/records file got truncated\"));\n    return std::nullopt;\n  }\n  return greater_chunk_begin->ordering;\n}\n\ninline bool RecordReaderBase::ReadChunk() {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of RecordReaderBase::ReadChunk()\";\n  ChunkReader& src = *SrcChunkReader();\n  chunk_begin_ = src.pos();\n  Chunk chunk;\n  if (ABSL_PREDICT_FALSE(!src.ReadChunk(chunk))) {\n    chunk_decoder_.Clear();\n    if (ABSL_PREDICT_FALSE(!src.ok())) {\n      recoverable_ = Recoverable::kRecoverChunkReader;\n      return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    }\n    return false;\n  }\n  if (ABSL_PREDICT_FALSE(!chunk_decoder_.Decode(chunk, flatten_))) {\n    recoverable_ = Recoverable::kRecoverChunkDecoder;\n    return Fail(chunk_decoder_.status());\n  }\n  return true;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/records/record_reader.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_RECORDS_RECORD_READER_H_\n#define RIEGELI_RECORDS_RECORD_READER_H_\n\n#include <stdint.h>\n\n#include <functional>\n#include <initializer_list>\n#include <memory>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/functional/function_ref.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/protobuf/descriptor.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/chunk_encoding/chunk.h\"\n#include \"riegeli/chunk_encoding/chunk_decoder.h\"\n#include \"riegeli/chunk_encoding/field_projection.h\"\n#include \"riegeli/records/chunk_reader.h\"\n#include \"riegeli/records/record_position.h\"\n#include \"riegeli/records/records_metadata.pb.h\"\n#include \"riegeli/records/skipped_region.h\"\n\nnamespace riegeli {\n\n// Interprets `record_type_name` and `file_descriptor` from metadata.\nclass RecordsMetadataDescriptors : public Object {\n public:\n  explicit RecordsMetadataDescriptors(const RecordsMetadata& metadata);\n\n  RecordsMetadataDescriptors(RecordsMetadataDescriptors&& that) noexcept;\n  RecordsMetadataDescriptors& operator=(RecordsMetadataDescriptors&& that);\n\n  // Returns message descriptor of the record type, or `nullptr` if not\n  // available.\n  //\n  // The message descriptor is valid as long as the `RecordsMetadataDescriptors`\n  // object is valid.\n  const google::protobuf::Descriptor* descriptor() const;\n\n  // Returns record type full name, or an empty string if not available.\n  absl::string_view record_type_name() const { return record_type_name_; }\n\n private:\n  class ErrorCollector;\n\n  std::string record_type_name_;\n  std::unique_ptr<google::protobuf::DescriptorPool> pool_;\n};\n\n// Template parameter independent part of `RecordReader`.\nclass RecordReaderBase : public Object {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // Specifies the set of fields to be included in returned records, allowing\n    // to exclude the remaining fields (but does not guarantee that they will be\n    // excluded). Excluding data makes reading faster.\n    //\n    // Projection is effective if the file has been written with\n    // `set_transpose(true)`. Additionally, `set_bucket_fraction()` with a lower\n    // value can make reading with projection faster.\n    //\n    // Default: `FieldProjection::All()`.\n    Options& set_field_projection(\n        Initializer<FieldProjection> field_projection) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      riegeli::Reset(field_projection_, std::move(field_projection));\n      return *this;\n    }\n    Options&& set_field_projection(\n        Initializer<FieldProjection> field_projection) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_field_projection(std::move(field_projection)));\n    }\n    Options& set_field_projection(\n        std::initializer_list<Field> field_projection) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      set_field_projection(Initializer<FieldProjection>(field_projection));\n      return *this;\n    }\n    Options&& set_field_projection(\n        std::initializer_list<Field> field_projection) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_field_projection(std::move(field_projection)));\n    }\n    FieldProjection& field_projection() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return field_projection_;\n    }\n    const FieldProjection& field_projection() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return field_projection_;\n    }\n\n    // Recovery function to be called after skipping over invalid file contents.\n    //\n    // If `nullptr`, then invalid file contents cause `RecordReader` to fail.\n    // `Recover()` can be used to skip over the invalid region.\n    //\n    // If not `nullptr`, then invalid file contents cause `RecordReader` to skip\n    // over the invalid region and call the recovery function. If the recovery\n    // function returns `true`, reading continues. If the recovery function\n    // returns `false`, reading ends as if the end of source was encountered.\n    //\n    // If `Close()` is called and file contents were truncated, the recovery\n    // function is called if set. The `RecordReader` remains closed.\n    //\n    // Calling the following functions may cause the recovery function to be\n    // called (in the same thread):\n    //  * `Close()` - returns `true`, ignores the result of the recovery\n    //                function\n    //  * `ReadMetadata()` - returns the result of the recovery function\n    //  * `ReadSerializedMetadata()` - returns the result of the recovery\n    //                                 function\n    //  * `ReadRecord()` - retried if the recovery function returns `true`,\n    //                     returns `false` if the recovery function returns\n    //                     `false`\n    //  * `Seek()` - returns the result of the recovery function\n    //  * `Search()` - skips invalid regions if the recovery function returns\n    //                 `true`, returns `std::nullopt` if the recovery function\n    //                 returns `false`\n    //\n    // Default: `nullptr`.\n    Options& set_recovery(\n        Initializer<\n            std::function<bool(const SkippedRegion&, RecordReaderBase&)>>\n            recovery) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      riegeli::Reset(recovery_, std::move(recovery));\n      return *this;\n    }\n    Options&& set_recovery(\n        Initializer<\n            std::function<bool(const SkippedRegion&, RecordReaderBase&)>>\n            recovery) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recovery(std::move(recovery)));\n    }\n    std::function<bool(const SkippedRegion&, RecordReaderBase&)>& recovery()\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recovery_;\n    }\n    const std::function<bool(const SkippedRegion&, RecordReaderBase&)>&\n    recovery() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recovery_;\n    }\n\n    // Options for a global `RecyclingPool` of decompression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // compression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    Options& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    Options&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    FieldProjection field_projection_ = FieldProjection::All();\n    std::function<bool(const SkippedRegion&, RecordReaderBase&)> recovery_;\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Returns the Riegeli/records file being read from. Unchanged by `Close()`.\n  virtual ChunkReader* SrcChunkReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Ensures that the file looks like a valid Riegeli/Records file.\n  //\n  // Reading functions already check the file format. `CheckFileFormat()` can\n  // verify the file format before (or instead of) performing other operations.\n  //\n  // This ignores the recovery function. If invalid file contents are skipped,\n  // then checking the file format is meaningless: any file can be read.\n  //\n  // Return values:\n  //  * `true`                 - success\n  //  * `false` (when `ok()`)  - source ends\n  //  * `false` (when `!ok()`) - failure\n  bool CheckFileFormat();\n\n  // Returns file metadata.\n  //\n  // `ReadMetadata()` must be called while the `RecordReader` is at the\n  // beginning of the file (calling `CheckFileFormat()` before is allowed).\n  //\n  // Record type in metadata can be conveniently interpreted by\n  // `RecordsMetadataDescriptors`.\n  //\n  // Return values:\n  //  * `true`                 - success (`metadata` is set)\n  //  * `false` (when `ok()`)  - source ends\n  //  * `false` (when `!ok()`) - failure\n  bool ReadMetadata(RecordsMetadata& metadata);\n\n  // Like `ReadMetadata()`, but metadata is returned in the serialized form.\n  //\n  // This is faster if the caller needs metadata already serialized.\n  bool ReadSerializedMetadata(Chain& metadata);\n\n  // Reads the next record.\n  //\n  // `ReadRecord(google::protobuf::MessageLite&)` parses raw bytes to a proto\n  // message after reading. The remaining overloads read raw bytes. For\n  // `ReadRecord(absl::string_view&)` the `absl::string_view` is valid until the\n  // next non-const operation on this `RecordReader`.\n  //\n  // Return values:\n  //  * `true`                 - success (`record` is set)\n  //  * `false` (when `ok()`)  - source ends\n  //  * `false` (when `!ok()`) - failure\n  bool ReadRecord(google::protobuf::MessageLite& record);\n  bool ReadRecord(absl::string_view& record);\n  bool ReadRecord(std::string& record);\n  bool ReadRecord(Chain& record);\n  bool ReadRecord(absl::Cord& record);\n\n  // Like `Options::set_field_projection()`, but can be done at any time.\n  //\n  // This may cause reading the current chunk again.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool SetFieldProjection(Initializer<FieldProjection> field_projection);\n  bool SetFieldProjection(std::initializer_list<Field> field_projection) {\n    return SetFieldProjection(Initializer<FieldProjection>(field_projection));\n  }\n\n  // Like `Options::set_recovery()`, but can be done at any time.\n  void set_recovery(\n      Initializer<std::function<bool(const SkippedRegion&, RecordReaderBase&)>>\n          recovery) {\n    riegeli::Reset(recovery_, std::move(recovery));\n  }\n\n  // Returns the function set by `Options::set_recovery` or `set_recovery()`.\n  const std::function<bool(const SkippedRegion&, RecordReaderBase&)>& recovery()\n      const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return recovery_;\n  }\n\n  // If `!ok()` and the failure was caused by invalid file contents, then\n  // `Recover()` tries to recover from the failure and allow reading again by\n  // skipping over the invalid region.\n  //\n  // If `Close()` failed and the failure was caused by truncated file contents,\n  // then `Recover()` returns `true`. The `RecordReader` remains closed.\n  //\n  // If `ok()`, or if `!ok()` but the failure was not caused by invalid file\n  // contents, then `Recover()` returns `false`.\n  //\n  // If `skipped_region != nullptr`, `*skipped_region` is set to the position of\n  // the skipped region on success.\n  //\n  // If a recovery function (`RecordReaderBase::Options::recovery()`) is set,\n  // then `Recover()` is called automatically. Otherwise `Recover()` can be\n  // called after one of the following functions returned `false`, and the\n  // function can be assumed to have returned `true` if `Recover()` returns\n  // `true`:\n  //  * `Close()`\n  //  * `ReadMetadata()`\n  //  * `ReadSerializedMetadata()`\n  //  * `ReadRecord()` - should be retried if `Recover()` returns `true`\n  //  * `Seek()`\n  //\n  // Return values:\n  //  * `true`  - success\n  //  * `false` - failure not caused by invalid file contents\n  bool Recover(SkippedRegion* skipped_region = nullptr);\n\n  // Returns the canonical position of the last record read.\n  //\n  // The canonical position is the largest among all equivalent positions.\n  // Seeking to any equivalent position leads to reading the same record.\n  //\n  // `last_pos().numeric()` returns the position as an integer of type\n  // `Position`.\n  //\n  // Precondition: a record was successfully read and there was no intervening\n  // call to `Close()`, `Seek()`, `SeekBack()`, or `Search()` (this can be\n  // checked with `last_record_is_valid()`).\n  RecordPosition last_pos() const;\n\n  // Returns `true` if calling `last_pos()` is valid.\n  bool last_record_is_valid() const { return last_record_is_valid_; }\n\n  // Returns a position of the next record (or the end of file if there is no\n  // next record).\n  //\n  // A position which is not canonical can be smaller than the equivalent\n  // canonical position. Seeking to any equivalent position leads to reading the\n  // same record.\n  //\n  // `pos().numeric()` returns the position as an integer of type `Position`.\n  //\n  // `pos()` is unchanged by `Close()`.\n  RecordPosition pos() const;\n\n  // Returns `true` if this `RecordReader` supports `Seek()`, `SeekBack()`,\n  // `Size()`, and `Search()`.\n  bool SupportsRandomAccess();\n\n  // Seeks to a position.\n  //\n  // In `Seek(RecordPosition)` the position should have been obtained by `pos()`\n  // for the same file.\n  //\n  // In `Seek(Position)` the position can be any integer between 0 and file\n  // size. If it points between records, it is interpreted as the next record.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool Seek(RecordPosition new_pos);\n  bool Seek(Position new_pos);\n\n  // Seeks back by one record.\n  //\n  // Return values:\n  //  * `true`                 - success (`ok()`)\n  //  * `false` (when `ok()`)  - beginning of the source reached\n  //  * `false` (when `!ok()`) - failure\n  bool SeekBack();\n\n  // Returns the size of the file in bytes, i.e. the position corresponding to\n  // its end.\n  //\n  // Returns `std::nullopt` on failure (`!ok()`).\n  std::optional<Position> Size();\n\n  // Searches the file for a desired record, or for a desired position between\n  // records, given that it is possible to determine whether a given record is\n  // before or after the desired position.\n  //\n  // The current position before calling `Search()` does not matter.\n  //\n  // The `test` function takes `*this` as a parameter, seeked to some record,\n  // and returns `std::nullopt` or an ordering (a value comparable with literal\n  // 0, such as `{Partial,Strong}Ordering`,\n  // `{std,absl}::{partial,weak,strong}_ordering`, or `int`):\n  //  * `std::nullopt` - Cancel the search.\n  //  * `less`          - The current record is before the desired position.\n  //  * `equivalent`    - The current record is desired, searching can stop.\n  //  * `greater`       - The current record is after the desired position.\n  //  * `unordered`     - It could not be determined which is the case.\n  //                      The current record will be skipped.\n  //\n  // The recovery function is set to `nullptr` while calling `test()`. Recovery\n  // is handled outside calling `test()`.\n  //\n  // Preconditions:\n  //  * All `less` records precede all `equivalent` records.\n  //  * All `equivalent` records precede all `greater` records.\n  //  * All `less` records precede all `greater` records,\n  //    even if there are no `equivalent` records.\n  //\n  // Return values:\n  //  * `std::nullopt` - Reading failed (`!ok()`)\n  //                      or the search was cancelled (`ok()`).\n  //  * `equivalent`    - There is some `equivalent` record,\n  //                      and `Search()` points to some such record.\n  //  * `greater`       - There are no `equivalent` records\n  //                      but there is some `greater` record,\n  //                      and `Search()` points to the earliest such record.\n  //  * `less`          - There are no `equivalent` nor `greater` records\n  //                      but there is some `less` record,\n  //                      and `Search()` points to the end of file.\n  //  * `unordered`     - All records are `unordered`,\n  //                      and `Search()` points to the end of file.\n  //\n  // To find the earliest `equivalent` record instead of an arbitrary one,\n  // `test()` can be changed to return `greater` in place of `equivalent`.\n  //\n  // Further guarantees:\n  //  * If a `test()` returns `equivalent`, `Search()` seeks back to the record\n  //    before `test()` and returns.\n  //  * If a `test()` returns `less`, `test()` will not be called again at\n  //    earlier positions.\n  //  * If a `test()` returns `greater`, `test()` will not be called again at\n  //    later positions.\n  //  * `test()` will not be called again at the same position.\n  //\n  // It follows that if a `test()` returns `equivalent` or `greater`, `Search()`\n  // points to the record before the last `test()` call with one of these\n  // results. This allows to communicate additional context of an `equivalent`\n  // or `greater` result by a side effect of `test()`.\n  //\n  // For skipping invalid file regions during `Search()`, a recovery function\n  // (`RecordReaderBase::Options::recovery()`) can be set, but `Recover()`\n  // resumes only simple operations and is not applicable here.\n  template <typename Test>\n  std::optional<PartialOrdering> Search(Test&& test);\n\n  // A variant of `Search()` which reads a record before calling `test()`,\n  // instead of letting `test()` read the record.\n  //\n  // The `Record` type must be supported by `ReadRecord()`. The `test` function\n  // takes `Record&` or `const Record&` as a parameter, and returns\n  // `std::nullopt` or an ordering.\n  template <typename Record, typename Test>\n  std::optional<PartialOrdering> Search(Test&& test);\n\n protected:\n  enum class Recoverable {\n    kNo,\n    kRecoverChunkReader,\n    kRecoverChunkDecoder,\n    kRecoverMetadata\n  };\n\n  explicit RecordReaderBase(Closed) noexcept;\n\n  RecordReaderBase() noexcept;\n\n  RecordReaderBase(RecordReaderBase&& that) noexcept;\n  RecordReaderBase& operator=(RecordReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(ChunkReader* src, Options&& options);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n\n  bool TryRecovery();\n\n  // Position of the beginning of the current chunk or end of file, except when\n  // `Seek(Position)` failed to locate the chunk containing the position, in\n  // which case this is that position.\n  Position chunk_begin_ = 0;\n\n  // Current chunk if a chunk has been read, empty otherwise.\n  //\n  // Invariants:\n  //   if `ok()` then `chunk_decoder_.ok()`\n  //   if `!ok()` then\n  //       `!chunk_decoder_.ok() ||\n  //        chunk_decoder_.index() == chunk_decoder_.num_records()`\n  ChunkDecoder chunk_decoder_;\n\n  bool last_record_is_valid_ = false;\n\n  // If `true`, prefer making records readily available as `absl::string_view`.\n  bool flatten_ = false;\n\n  // Whether `Recover()` is applicable, and if so, how it should be performed:\n  //\n  //  * `Recoverable::kNo`                  - `Recover()` is not applicable\n  //  * `Recoverable::kRecoverChunkReader`  - `Recover()` tries to recover\n  //                                          `chunk_reader_`\n  //  * `Recoverable::kRecoverChunkDecoder` - `Recover()` tries to recover\n  //                                          `chunk_decoder_`, skips the chunk\n  //                                          if that failed\n  //\n  // Invariants:\n  //   if `ok()` then `recoverable_ == Recoverable::kNo`\n  //   if `!is_open()` then `recoverable_ == Recoverable::kNo ||\n  //                         recoverable_ == Recoverable::kRecoverChunkReader`\n  Recoverable recoverable_ = Recoverable::kNo;\n\n  std::function<bool(const SkippedRegion&, RecordReaderBase&)> recovery_;\n\n  RecyclingPoolOptions recycling_pool_options_;\n\n private:\n  class ChunkSearchTraits;\n\n  bool FailReading(const ChunkReader& src);\n  bool FailSeeking(const ChunkReader& src);\n\n  bool ParseMetadata(const Chunk& chunk, Chain& metadata);\n\n  template <typename Record>\n  bool ReadRecordImpl(Record& record);\n\n  // Reads the next chunk from `chunk_reader_` and decodes it into\n  // `chunk_decoder_` and `chunk_begin_`. On failure resets `chunk_decoder_`.\n  //\n  // Precondition: `ok()`\n  bool ReadChunk();\n\n  std::optional<PartialOrdering> SearchImpl(\n      absl::FunctionRef<\n          std::optional<PartialOrdering>(RecordReaderBase& reader)>\n          test);\n};\n\n// `RecordReader` reads records of a Riegeli/records file. A record is\n// conceptually a binary string; usually it is a serialized proto message.\n//\n// `RecordReader` supports reading records sequentially, querying for the\n// current position, and seeking to continue reading from another position.\n//\n// For reading records sequentially, this kind of loop can be used:\n// ```\n//   SomeProto record;\n//   while (record_reader_.ReadRecord(record)) {\n//     ... Process record.\n//   }\n//   if (!record_reader_.Close()) {\n//     ... Failed with reason: record_reader_.status()\n//   }\n// ```\n//\n// For reading records while skipping errors, pass options like these:\n// ```\n//       riegeli::RecordReaderBase::Options().set_recovery(\n//           [&skipped_bytes](const riegeli::SkippedRegion& skipped_region,\n//                            riegeli::RecordReaderBase& record_reader) {\n//             skipped_bytes += skipped_region.length();\n//             return true;\n//           })\n// ```\n//\n// An equivalent lower level implementation, without callbacks:\n// ```\n//   riegeli::Position skipped_bytes = 0;\n//   SomeProto record;\n//   for (;;) {\n//     if (!record_reader_.ReadRecord(record)) {\n//       riegeli::SkippedRegion skipped_region;\n//       if (record_reader_.Recover(&skipped_region)) {\n//         skipped_bytes += skipped_region.length();\n//         continue;\n//       }\n//       break;\n//     }\n//     ... Process record.\n//   }\n//   if (!record_reader_.Close()) {\n//     riegeli::SkippedRegion skipped_region;\n//     if (record_reader_.Recover(&skipped_region)) {\n//       skipped_bytes += skipped_region.length();\n//     } else {\n//       ... Failed with reason: record_reader_.status()\n//     }\n//   }\n// ```\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the byte `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// `Src` may also specify a `ChunkReader` instead of a byte `Reader`. In this\n// case `Src` must support `Dependency<ChunkReader*, Src>`, e.g.\n// `ChunkReader*` (not owned), `DefaultChunkReader<>` (owned),\n// `std::unique_ptr<ChunkReader>` (owned), `Any<ChunkReader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The byte `Reader` or `ChunkReader` must not be accessed until the\n// `RecordReader` is closed or no longer used.\ntemplate <typename Src = Reader*>\nclass RecordReader : public RecordReaderBase {\n public:\n  // Creates a closed `RecordReader`.\n  explicit RecordReader(Closed) noexcept : RecordReaderBase(kClosed) {}\n\n  // Will read from the byte `Reader` or `ChunkReader` provided by `src`.\n  explicit RecordReader(Initializer<Src> src, Options options = Options());\n\n  RecordReader(RecordReader&& that) = default;\n  RecordReader& operator=(RecordReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `RecordReader`. This avoids\n  // constructing a temporary `RecordReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the byte `Reader` or\n  // `ChunkReader`. Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  ChunkReader* SrcChunkReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n  // An optimized implementation in a derived class, avoiding a virtual call.\n  RecordPosition pos() const;\n\n protected:\n  void Done() override;\n\n private:\n  // The object providing and possibly owning the byte `Reader` or\n  // `ChunkReader`.\n  Dependency<ChunkReader*, Src> src_;\n};\n\nexplicit RecordReader(Closed) -> RecordReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit RecordReader(\n    Src&& src, RecordReaderBase::Options options = RecordReaderBase::Options())\n    -> RecordReader<TargetT<Src>>;\n\n// Implementation details follow.\n\ninline RecordsMetadataDescriptors::RecordsMetadataDescriptors(\n    RecordsMetadataDescriptors&& that) noexcept\n    : Object(static_cast<Object&&>(that)),\n      record_type_name_(std::move(that.record_type_name_)),\n      pool_(std::move(that.pool_)) {}\n\ninline RecordsMetadataDescriptors& RecordsMetadataDescriptors::operator=(\n    RecordsMetadataDescriptors&& that) {\n  Object::operator=(static_cast<Object&&>(that));\n  record_type_name_ = std::move(that.record_type_name_),\n  pool_ = std::move(that.pool_);\n  return *this;\n}\n\ninline bool RecordReaderBase::TryRecovery() {\n  if (recovery_ == nullptr) return false;\n  SkippedRegion skipped_region;\n  return Recover(&skipped_region) && recovery_(skipped_region, *this);\n}\n\ninline RecordPosition RecordReaderBase::last_pos() const {\n  RIEGELI_ASSERT(last_record_is_valid())\n      << \"Failed precondition of RecordReaderBase::last_pos(): \"\n         \"no record was recently read\";\n  // `chunk_decoder_.index() == 0` after reading metadata.\n  return RecordPosition(chunk_begin_,\n                        SaturatingSub(chunk_decoder_.index(), uint64_t{1}));\n}\n\ninline RecordPosition RecordReaderBase::pos() const {\n  if (ABSL_PREDICT_TRUE(chunk_decoder_.index() <\n                        chunk_decoder_.num_records()) ||\n      ABSL_PREDICT_FALSE(recoverable_ == Recoverable::kRecoverChunkDecoder)) {\n    return RecordPosition(chunk_begin_, chunk_decoder_.index());\n  }\n  return RecordPosition(SrcChunkReader()->pos(), 0);\n}\n\nnamespace record_reader_internal {\n\ntemplate <typename T>\ninline std::optional<PartialOrdering> AsOptionalPartialOrdering(T test_result) {\n  return AsPartialOrdering(test_result);\n}\n\ntemplate <typename T>\ninline std::optional<PartialOrdering> AsOptionalPartialOrdering(\n    std::optional<T> test_result) {\n  if (test_result == std::nullopt) return std::nullopt;\n  return AsPartialOrdering(*test_result);\n}\n\n}  // namespace record_reader_internal\n\ntemplate <typename Test>\nstd::optional<PartialOrdering> RecordReaderBase::Search(Test&& test) {\n  return SearchImpl([&](RecordReaderBase& self) {\n    return record_reader_internal::AsOptionalPartialOrdering(test(self));\n  });\n}\n\ntemplate <typename Record, typename Test>\nstd::optional<PartialOrdering> RecordReaderBase::Search(Test&& test) {\n  Record record;\n  return SearchImpl(\n      [&](RecordReaderBase& self) -> std::optional<PartialOrdering> {\n        if (ABSL_PREDICT_FALSE(!self.ReadRecord(record))) {\n          return PartialOrdering::unordered;\n        }\n        return record_reader_internal::AsOptionalPartialOrdering(test(record));\n      });\n}\n\ntemplate <typename Src>\ninline RecordReader<Src>::RecordReader(Initializer<Src> src, Options options)\n    : src_(std::move(src)) {\n  Initialize(src_.get(), std::move(options));\n}\n\ntemplate <typename Src>\ninline void RecordReader<Src>::Reset(Closed) {\n  RecordReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void RecordReader<Src>::Reset(Initializer<Src> src, Options options) {\n  RecordReaderBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(src_.get(), std::move(options));\n}\n\ntemplate <typename Src>\nvoid RecordReader<Src>::Done() {\n  RecordReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      recoverable_ = Recoverable::kRecoverChunkReader;\n      FailWithoutAnnotation(AnnotateOverSrc(src_->status()));\n      TryRecovery();\n    }\n  }\n}\n\ntemplate <typename Src>\ninline RecordPosition RecordReader<Src>::pos() const {\n  if (ABSL_PREDICT_TRUE(chunk_decoder_.index() <\n                        chunk_decoder_.num_records()) ||\n      ABSL_PREDICT_FALSE(recoverable_ == Recoverable::kRecoverChunkDecoder)) {\n    return RecordPosition(chunk_begin_, chunk_decoder_.index());\n  }\n  return RecordPosition(src_->pos(), 0);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_RECORDS_RECORD_READER_H_\n"
  },
  {
    "path": "riegeli/records/record_writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/records/record_writer.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <cmath>\n#include <deque>\n#include <future>\n#include <limits>\n#include <memory>\n#include <optional>\n#include <string>\n#include <utility>\n#include <variant>\n#include <vector>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/base/thread_annotations.h\"\n#include \"absl/container/flat_hash_set.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/synchronization/mutex.h\"\n#include \"google/protobuf/descriptor.h\"\n#include \"google/protobuf/descriptor.pb.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/options_parser.h\"\n#include \"riegeli/base/parallelism.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/chain_writer.h\"\n#include \"riegeli/chunk_encoding/chunk.h\"\n#include \"riegeli/chunk_encoding/chunk_encoder.h\"\n#include \"riegeli/chunk_encoding/compressor_options.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/chunk_encoding/deferred_encoder.h\"\n#include \"riegeli/chunk_encoding/simple_encoder.h\"\n#include \"riegeli/chunk_encoding/transpose_encoder.h\"\n#include \"riegeli/messages/serialize_message.h\"\n#include \"riegeli/records/chunk_writer.h\"\n#include \"riegeli/records/record_position.h\"\n#include \"riegeli/records/records_metadata.pb.h\"\n\nnamespace riegeli {\n\nnamespace {\n\nclass FileDescriptorCollector {\n public:\n  explicit FileDescriptorCollector(\n      google::protobuf::RepeatedPtrField<google::protobuf::FileDescriptorProto>*\n          file_descriptors)\n      : file_descriptors_(RIEGELI_EVAL_ASSERT_NOTNULL(file_descriptors)) {}\n\n  void AddFile(const google::protobuf::FileDescriptor* file_descriptor) {\n    if (!files_seen_.emplace(file_descriptor->name()).second) return;\n    for (int i = 0; i < file_descriptor->dependency_count(); ++i) {\n      AddFile(file_descriptor->dependency(i));\n    }\n    file_descriptor->CopyTo(file_descriptors_->Add());\n  }\n\n private:\n  google::protobuf::RepeatedPtrField<google::protobuf::FileDescriptorProto>*\n      file_descriptors_;\n  absl::flat_hash_set<std::string> files_seen_;\n};\n\n}  // namespace\n\nvoid SetRecordType(const google::protobuf::Descriptor& descriptor,\n                   RecordsMetadata& metadata) {\n  metadata.set_record_type_name(descriptor.full_name());\n  metadata.clear_file_descriptor();\n  FileDescriptorCollector collector(metadata.mutable_file_descriptor());\n  collector.AddFile(descriptor.file());\n}\n\nabsl::Status RecordWriterBase::Options::FromString(absl::string_view text) {\n  std::string compressor_text;\n  OptionsParser options_parser;\n  options_parser.AddOption(\"default\", ValueParser::FailIfAnySeen());\n  options_parser.AddOption(\n      \"transpose\",\n      ValueParser::Enum({{\"\", true}, {\"true\", true}, {\"false\", false}},\n                        &transpose_));\n  options_parser.AddOption(\"uncompressed\",\n                           ValueParser::CopyTo(&compressor_text));\n  options_parser.AddOption(\"brotli\", ValueParser::CopyTo(&compressor_text));\n  options_parser.AddOption(\"zstd\", ValueParser::CopyTo(&compressor_text));\n  options_parser.AddOption(\"snappy\", ValueParser::CopyTo(&compressor_text));\n  options_parser.AddOption(\"window_log\", ValueParser::CopyTo(&compressor_text));\n  options_parser.AddOption(\"brotli_encoder\",\n                           ValueParser::CopyTo(&compressor_text));\n  uint64_t chunk_size;\n  options_parser.AddOption(\n      \"chunk_size\",\n      ValueParser::Or(\n          ValueParser::Enum({{\"auto\", std::nullopt}}, &chunk_size_),\n          ValueParser::And(\n              ValueParser::Bytes(1, std::numeric_limits<uint64_t>::max(),\n                                 &chunk_size),\n              [this, &chunk_size](ValueParser&) {\n                chunk_size_ = chunk_size;\n                return true;\n              })));\n  options_parser.AddOption(\"bucket_fraction\",\n                           ValueParser::Real(0.0, 1.0, &bucket_fraction_));\n  uint64_t padding;\n  options_parser.AddOption(\n      \"padding\",\n      ValueParser::And(\n          ValueParser::Or(\n              ValueParser::Empty(uint64_t{kImplicitPadding}, &padding),\n              ValueParser::Bytes(1, std::numeric_limits<uint64_t>::max(),\n                                 &padding)),\n          [this, &padding](ValueParser&) {\n            set_padding(padding);\n            return true;\n          }));\n  options_parser.AddOption(\n      \"initial_padding\",\n      ValueParser::And(\n          ValueParser::Or(\n              ValueParser::Empty(uint64_t{kImplicitPadding}, &padding),\n              ValueParser::Bytes(1, std::numeric_limits<uint64_t>::max(),\n                                 &padding)),\n          [this, &padding](ValueParser&) {\n            set_initial_padding(padding);\n            return true;\n          }));\n  options_parser.AddOption(\n      \"final_padding\",\n      ValueParser::And(\n          ValueParser::Or(\n              ValueParser::Empty(uint64_t{kImplicitPadding}, &padding),\n              ValueParser::Bytes(1, std::numeric_limits<uint64_t>::max(),\n                                 &padding)),\n          [this, &padding](ValueParser&) {\n            set_final_padding(padding);\n            return true;\n          }));\n  options_parser.AddOption(\n      \"parallelism\",\n      ValueParser::Int(0, std::numeric_limits<int>::max(), &parallelism_));\n  if (ABSL_PREDICT_FALSE(!options_parser.FromString(text))) {\n    return options_parser.status();\n  }\n  return compressor_options_.FromString(compressor_text);\n}\n\nclass RecordWriterBase::Worker {\n public:\n  explicit Worker(ChunkWriter* chunk_writer, Options&& options);\n\n  virtual ~Worker();\n\n  bool Close();\n  virtual absl::Status status() const = 0;\n  virtual absl::Status AnnotateStatus(absl::Status status) = 0;\n\n  // Precondition for `Close()`: chunk is not open.\n\n  // Precondition: chunk is not open.\n  virtual void OpenChunk() = 0;\n\n  // Precondition: chunk is open.\n  template <typename Record>\n  bool AddRecord(Record&& record);\n  bool AddRecord(const google::protobuf::MessageLite& record,\n                 SerializeMessageOptions serialize_options);\n\n  // Precondition: chunk is open.\n  //\n  // If the result is `false` then `!ok()`.\n  virtual bool CloseChunk() = 0;\n\n  bool MaybePadToFinalBoundary();\n\n  // Precondition: chunk is not open.\n  virtual bool Flush(FlushType flush_type) = 0;\n\n  // Precondition: chunk is not open.\n  virtual FutureStatus FutureFlush(FlushType flush_type) = 0;\n\n  virtual FutureRecordPosition LastPos() const = 0;\n\n  virtual FutureRecordPosition Pos() const = 0;\n\n  virtual Position EstimatedSize() const = 0;\n\n protected:\n  void Initialize(Position initial_pos);\n\n  virtual void Done() {}\n  virtual bool ok() const = 0;\n  ABSL_ATTRIBUTE_COLD bool Fail(absl::Status status);\n  virtual bool FailWithoutAnnotation(absl::Status status) = 0;\n\n  virtual bool WriteSignature() = 0;\n  virtual bool WriteMetadata() = 0;\n  virtual bool WritePadding(Position padding) = 0;\n\n  std::unique_ptr<ChunkEncoder> MakeChunkEncoder();\n  void EncodeSignature(Chunk& chunk);\n  bool EncodeMetadata(Chunk& chunk);\n  bool EncodeChunk(ChunkEncoder& chunk_encoder, Chunk& chunk);\n\n  ObjectState state_;\n  Options options_;\n  // Invariant: `chunk_writer_ != nullptr`\n  ChunkWriter* chunk_writer_;\n  // Invariant: if chunk is open then `chunk_encoder_ != nullptr`\n  std::unique_ptr<ChunkEncoder> chunk_encoder_;\n};\n\ninline RecordWriterBase::Worker::Worker(ChunkWriter* chunk_writer,\n                                        Options&& options)\n    : options_(std::move(options)),\n      chunk_writer_(RIEGELI_EVAL_ASSERT_NOTNULL(chunk_writer)),\n      chunk_encoder_(MakeChunkEncoder()) {\n  if (ABSL_PREDICT_FALSE(!chunk_writer_->ok())) {\n    // `FailWithoutAnnotation()` is pure virtual and must not be called from the\n    // constructor.\n    state_.Fail(chunk_writer_->status());\n  }\n}\n\nRecordWriterBase::Worker::~Worker() {}\n\nbool RecordWriterBase::Worker::Close() {\n  if (ABSL_PREDICT_FALSE(!state_.is_open())) return state_.not_failed();\n  Done();\n  return state_.MarkClosed();\n}\n\ninline bool RecordWriterBase::Worker::Fail(absl::Status status) {\n  return FailWithoutAnnotation(AnnotateStatus(std::move(status)));\n}\n\ninline void RecordWriterBase::Worker::Initialize(Position initial_pos) {\n  if (initial_pos == 0) {\n    if (ABSL_PREDICT_FALSE(!WriteSignature())) return;\n    if (ABSL_PREDICT_FALSE(!WriteMetadata())) return;\n  } else {\n    WritePadding(options_.initial_padding());\n  }\n}\n\ninline bool RecordWriterBase::Worker::MaybePadToFinalBoundary() {\n  return WritePadding(options_.final_padding());\n}\n\ninline std::unique_ptr<ChunkEncoder>\nRecordWriterBase::Worker::MakeChunkEncoder() {\n  std::unique_ptr<ChunkEncoder> chunk_encoder;\n  if (options_.transpose()) {\n    const long double long_double_bucket_size =\n        std::round(static_cast<long double>(options_.effective_chunk_size()) *\n                   static_cast<long double>(options_.bucket_fraction()));\n    const uint64_t bucket_size =\n        ABSL_PREDICT_FALSE(\n            long_double_bucket_size >=\n            static_cast<long double>(std::numeric_limits<uint64_t>::max()))\n            ? std::numeric_limits<uint64_t>::max()\n        : ABSL_PREDICT_TRUE(long_double_bucket_size >= 1.0L)\n            ? static_cast<uint64_t>(long_double_bucket_size)\n            : uint64_t{1};\n    chunk_encoder = std::make_unique<TransposeEncoder>(\n        options_.compressor_options(),\n        TransposeEncoder::TuningOptions()\n            .set_bucket_size(bucket_size)\n            .set_recycling_pool_options(options_.recycling_pool_options()));\n  } else {\n    chunk_encoder = std::make_unique<SimpleEncoder>(\n        options_.compressor_options(),\n        SimpleEncoder::TuningOptions()\n            .set_size_hint(options_.effective_chunk_size())\n            .set_recycling_pool_options(options_.recycling_pool_options()));\n  }\n  if (options_.parallelism() == 0) {\n    return chunk_encoder;\n  } else {\n    return std::make_unique<DeferredEncoder>(std::move(chunk_encoder));\n  }\n}\n\ninline void RecordWriterBase::Worker::EncodeSignature(Chunk& chunk) {\n  chunk.header = ChunkHeader(chunk.data, ChunkType::kFileSignature, 0, 0);\n}\n\ninline bool RecordWriterBase::Worker::EncodeMetadata(Chunk& chunk) {\n  TransposeEncoder transpose_encoder(\n      options_.compressor_options(),\n      TransposeEncoder::TuningOptions().set_recycling_pool_options(\n          options_.recycling_pool_options()));\n  if (ABSL_PREDICT_FALSE(\n          options_.metadata() != std::nullopt\n              ? !transpose_encoder.AddRecord(*options_.metadata())\n              : !transpose_encoder.AddRecord(\n                    *options_.serialized_metadata()))) {\n    return Fail(transpose_encoder.status());\n  }\n  ChainWriter<> data_writer(&chunk.data);\n  ChunkType chunk_type;\n  uint64_t num_records;\n  uint64_t decoded_data_size;\n  if (ABSL_PREDICT_FALSE(!transpose_encoder.EncodeAndClose(\n          data_writer, chunk_type, num_records, decoded_data_size))) {\n    return Fail(transpose_encoder.status());\n  }\n  if (ABSL_PREDICT_FALSE(!data_writer.Close())) {\n    return Fail(data_writer.status());\n  }\n  chunk.header =\n      ChunkHeader(chunk.data, ChunkType::kFileMetadata, 0, decoded_data_size);\n  return true;\n}\n\ntemplate <typename Record>\ninline bool RecordWriterBase::Worker::AddRecord(Record&& record) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(\n          !chunk_encoder_->AddRecord(std::forward<Record>(record)))) {\n    return Fail(chunk_encoder_->status());\n  }\n  return true;\n}\n\ninline bool RecordWriterBase::Worker::AddRecord(\n    const google::protobuf::MessageLite& record,\n    SerializeMessageOptions serialize_options) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(\n          !chunk_encoder_->AddRecord(record, std::move(serialize_options)))) {\n    return Fail(chunk_encoder_->status());\n  }\n  return true;\n}\n\ninline bool RecordWriterBase::Worker::EncodeChunk(ChunkEncoder& chunk_encoder,\n                                                  Chunk& chunk) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  ChunkType chunk_type;\n  uint64_t num_records;\n  uint64_t decoded_data_size;\n  ChainWriter<> data_writer(&chunk.data);\n  if (ABSL_PREDICT_FALSE(!chunk_encoder.EncodeAndClose(\n          data_writer, chunk_type, num_records, decoded_data_size))) {\n    return Fail(chunk_encoder.status());\n  }\n  if (ABSL_PREDICT_FALSE(!data_writer.Close())) {\n    return Fail(data_writer.status());\n  }\n  chunk.header =\n      ChunkHeader(chunk.data, chunk_type, num_records, decoded_data_size);\n  return true;\n}\n\nclass RecordWriterBase::SerialWorker : public Worker {\n public:\n  explicit SerialWorker(ChunkWriter* chunk_writer, Options&& options);\n\n  absl::Status status() const override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatus(absl::Status status) override;\n\n  void OpenChunk() override { chunk_encoder_->Clear(); }\n  bool CloseChunk() override;\n  bool Flush(FlushType flush_type) override;\n  FutureStatus FutureFlush(FlushType flush_type) override;\n  FutureRecordPosition LastPos() const override;\n  FutureRecordPosition Pos() const override;\n  Position EstimatedSize() const override;\n\n protected:\n  bool ok() const override;\n  ABSL_ATTRIBUTE_COLD bool FailWithoutAnnotation(absl::Status status) override;\n\n  bool WriteSignature() override;\n  bool WriteMetadata() override;\n  bool WritePadding(Position padding) override;\n};\n\ninline RecordWriterBase::SerialWorker::SerialWorker(ChunkWriter* chunk_writer,\n                                                    Options&& options)\n    : Worker(chunk_writer, std::move(options)) {\n  Initialize(chunk_writer_->pos());\n}\n\ninline bool RecordWriterBase::SerialWorker::ok() const { return state_.ok(); }\n\ninline absl::Status RecordWriterBase::SerialWorker::status() const {\n  return state_.status();\n}\n\nbool RecordWriterBase::SerialWorker::FailWithoutAnnotation(\n    absl::Status status) {\n  return state_.Fail(std::move(status));\n}\n\nabsl::Status RecordWriterBase::SerialWorker::AnnotateStatus(\n    absl::Status status) {\n  return chunk_writer_->AnnotateStatus(std::move(status));\n}\n\nbool RecordWriterBase::SerialWorker::WriteSignature() {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chunk chunk;\n  EncodeSignature(chunk);\n  if (ABSL_PREDICT_FALSE(!chunk_writer_->WriteChunk(chunk))) {\n    return FailWithoutAnnotation(chunk_writer_->status());\n  }\n  return true;\n}\n\nbool RecordWriterBase::SerialWorker::WriteMetadata() {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (options_.metadata() == std::nullopt &&\n      options_.serialized_metadata() == std::nullopt) {\n    return true;\n  }\n  Chunk chunk;\n  if (ABSL_PREDICT_FALSE(!EncodeMetadata(chunk))) return false;\n  if (ABSL_PREDICT_FALSE(!chunk_writer_->WriteChunk(chunk))) {\n    return FailWithoutAnnotation(chunk_writer_->status());\n  }\n  return true;\n}\n\nbool RecordWriterBase::SerialWorker::CloseChunk() {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chunk chunk;\n  if (ABSL_PREDICT_FALSE(!EncodeChunk(*chunk_encoder_, chunk))) return false;\n  if (ABSL_PREDICT_FALSE(!chunk_writer_->WriteChunk(chunk))) {\n    return FailWithoutAnnotation(chunk_writer_->status());\n  }\n  return true;\n}\n\nbool RecordWriterBase::SerialWorker::WritePadding(Position padding) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!chunk_writer_->WritePadding(padding))) {\n    return FailWithoutAnnotation(chunk_writer_->status());\n  }\n  return true;\n}\n\nbool RecordWriterBase::SerialWorker::Flush(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!chunk_writer_->Flush(flush_type))) {\n    return FailWithoutAnnotation(chunk_writer_->status());\n  }\n  return true;\n}\n\nRecordWriterBase::FutureStatus RecordWriterBase::SerialWorker::FutureFlush(\n    FlushType flush_type) {\n  std::promise<absl::Status> promise;\n  promise.set_value(ABSL_PREDICT_TRUE(Flush(flush_type)) ? absl::OkStatus()\n                                                         : status());\n  return promise.get_future();\n}\n\nFutureRecordPosition RecordWriterBase::SerialWorker::LastPos() const {\n  RIEGELI_ASSERT_GT(chunk_encoder_->num_records(), 0u)\n      << \"Failed invariant of RecordWriterBase::SerialWorker: \"\n         \"last position should be valid but no record was encoded\";\n  return RecordPosition(chunk_writer_->pos(),\n                        chunk_encoder_->num_records() - 1);\n}\n\nFutureRecordPosition RecordWriterBase::SerialWorker::Pos() const {\n  return RecordPosition(chunk_writer_->pos(), chunk_encoder_->num_records());\n}\n\nPosition RecordWriterBase::SerialWorker::EstimatedSize() const {\n  return chunk_writer_->pos();\n}\n\n// `ParallelWorker` uses parallelism internally, but the class is still only\n// thread-compatible, not thread-safe.\nclass RecordWriterBase::ParallelWorker : public Worker {\n public:\n  explicit ParallelWorker(ChunkWriter* chunk_writer, Options&& options);\n\n  ~ParallelWorker();\n\n  absl::Status status() const override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatus(absl::Status status) override;\n\n  void OpenChunk() override { chunk_encoder_ = MakeChunkEncoder(); }\n  bool CloseChunk() override;\n  bool Flush(FlushType flush_type) override;\n  FutureStatus FutureFlush(FlushType flush_type) override;\n  FutureRecordPosition LastPos() const override;\n  FutureRecordPosition Pos() const override;\n  Position EstimatedSize() const override;\n\n protected:\n  void Done() override;\n  bool ok() const override;\n  ABSL_ATTRIBUTE_COLD bool FailWithoutAnnotation(absl::Status status) override;\n\n  bool WriteSignature() override;\n  bool WriteMetadata() override;\n  bool WritePadding(Position padding) override;\n\n private:\n  struct ChunkPromises {\n    std::promise<ChunkHeader> chunk_header;\n    std::promise<Chunk> chunk;\n  };\n\n  // A request to the chunk writer thread.\n  struct DoneRequest {\n    std::promise<void> done;\n  };\n  struct AnnotateStatusRequest {\n    absl::Status status;\n    std::promise<absl::Status> done;\n  };\n  struct WriteChunkRequest {\n    std::shared_future<ChunkHeader> chunk_header;\n    std::future<Chunk> chunk;\n  };\n  struct WritePaddingRequest {\n    Position padding;\n  };\n  struct FlushRequest {\n    FlushType flush_type;\n    std::promise<absl::Status> done;\n  };\n  using ChunkWriterRequest =\n      std::variant<DoneRequest, AnnotateStatusRequest, WriteChunkRequest,\n                   WritePaddingRequest, FlushRequest>;\n\n  bool HasCapacityForRequest() const ABSL_EXCLUSIVE_LOCKS_REQUIRED(mutex_);\n  records_internal::FutureChunkBegin ChunkBegin() const;\n\n  mutable absl::Mutex mutex_;\n  std::deque<ChunkWriterRequest> chunk_writer_requests_ ABSL_GUARDED_BY(mutex_);\n  // Position before handling `chunk_writer_requests_`.\n  Position pos_before_chunks_ ABSL_GUARDED_BY(mutex_);\n};\n\ninline RecordWriterBase::ParallelWorker::ParallelWorker(\n    ChunkWriter* chunk_writer, Options&& options)\n    : Worker(chunk_writer, std::move(options)),\n      pos_before_chunks_(chunk_writer_->pos()) {\n  internal::ThreadPool::global().Schedule([this] {\n    struct Visitor {\n      bool operator()(DoneRequest& request) const {\n        request.done.set_value();\n        return false;\n      }\n\n      bool operator()(AnnotateStatusRequest& request) const {\n        request.done.set_value(\n            self->chunk_writer_->AnnotateStatus(request.status));\n        return true;\n      }\n\n      bool operator()(WriteChunkRequest& request) const {\n        // If `!ok()`, the chunk must still be waited for, to ensure that the\n        // chunk encoder thread exits before the chunk writer thread responds to\n        // `DoneRequest`.\n        const Chunk chunk = request.chunk.get();\n        if (ABSL_PREDICT_FALSE(!self->ok())) return true;\n        if (ABSL_PREDICT_FALSE(!self->chunk_writer_->WriteChunk(chunk))) {\n          self->FailWithoutAnnotation(self->chunk_writer_->status());\n        }\n        return true;\n      }\n\n      bool operator()(WritePaddingRequest& request) const {\n        if (ABSL_PREDICT_FALSE(!self->ok())) return true;\n        if (ABSL_PREDICT_FALSE(\n                !self->chunk_writer_->WritePadding(request.padding))) {\n          self->FailWithoutAnnotation(self->chunk_writer_->status());\n        }\n        return true;\n      }\n\n      bool operator()(FlushRequest& request) const {\n        if (ABSL_PREDICT_FALSE(!self->ok())) {\n          request.done.set_value(self->status());\n          return true;\n        }\n        if (ABSL_PREDICT_FALSE(\n                !self->chunk_writer_->Flush(request.flush_type))) {\n          self->FailWithoutAnnotation(self->chunk_writer_->status());\n          request.done.set_value(self->status());\n          return true;\n        }\n        request.done.set_value(absl::OkStatus());\n        return true;\n      }\n\n      ParallelWorker* self;\n    };\n\n    mutex_.lock();\n    for (;;) {\n      mutex_.Await(absl::Condition(\n          +[](std::deque<ChunkWriterRequest>* chunk_writer_requests) {\n            return !chunk_writer_requests->empty();\n          },\n          &chunk_writer_requests_));\n      ChunkWriterRequest& request = chunk_writer_requests_.front();\n      mutex_.unlock();\n      if (ABSL_PREDICT_FALSE(!std::visit(Visitor{this}, request))) return;\n      mutex_.lock();\n      chunk_writer_requests_.pop_front();\n      pos_before_chunks_ = chunk_writer_->pos();\n    }\n  });\n  Initialize(pos_before_chunks_);\n}\n\nRecordWriterBase::ParallelWorker::~ParallelWorker() {\n  if (ABSL_PREDICT_FALSE(state_.is_open())) Done();\n}\n\nvoid RecordWriterBase::ParallelWorker::Done() {\n  std::promise<void> done_promise;\n  std::future<void> done_future = done_promise.get_future();\n  {\n    absl::MutexLock lock(mutex_);\n    chunk_writer_requests_.emplace_back(DoneRequest{std::move(done_promise)});\n  }\n  done_future.get();\n}\n\ninline bool RecordWriterBase::ParallelWorker::ok() const {\n  absl::MutexLock lock(mutex_);\n  return state_.ok();\n}\n\ninline absl::Status RecordWriterBase::ParallelWorker::status() const {\n  absl::MutexLock lock(mutex_);\n  return state_.status();\n}\n\nbool RecordWriterBase::ParallelWorker::FailWithoutAnnotation(\n    absl::Status status) {\n  absl::MutexLock lock(mutex_);\n  return state_.Fail(std::move(status));\n}\n\nabsl::Status RecordWriterBase::ParallelWorker::AnnotateStatus(\n    absl::Status status) {\n  std::promise<absl::Status> done_promise;\n  std::future<absl::Status> done_future = done_promise.get_future();\n  {\n    absl::MutexLock lock(\n        mutex_, absl::Condition(this, &ParallelWorker::HasCapacityForRequest));\n    chunk_writer_requests_.emplace_back(\n        AnnotateStatusRequest{std::move(status), std::move(done_promise)});\n  }\n  return done_future.get();\n}\n\nbool RecordWriterBase::ParallelWorker::HasCapacityForRequest() const {\n  return chunk_writer_requests_.size() <\n         IntCast<size_t>(options_.parallelism());\n}\n\nbool RecordWriterBase::ParallelWorker::WriteSignature() {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Chunk chunk;\n  EncodeSignature(chunk);\n  ChunkPromises chunk_promises;\n  chunk_promises.chunk_header.set_value(chunk.header);\n  chunk_promises.chunk.set_value(std::move(chunk));\n  {\n    absl::MutexLock lock(\n        mutex_, absl::Condition(this, &ParallelWorker::HasCapacityForRequest));\n    chunk_writer_requests_.emplace_back(\n        WriteChunkRequest{chunk_promises.chunk_header.get_future(),\n                          chunk_promises.chunk.get_future()});\n  }\n  return true;\n}\n\nbool RecordWriterBase::ParallelWorker::WriteMetadata() {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (options_.metadata() == std::nullopt &&\n      options_.serialized_metadata() == std::nullopt) {\n    return true;\n  }\n  ChunkPromises* const chunk_promises = new ChunkPromises();\n  {\n    absl::MutexLock lock(\n        mutex_, absl::Condition(this, &ParallelWorker::HasCapacityForRequest));\n    chunk_writer_requests_.emplace_back(\n        WriteChunkRequest{chunk_promises->chunk_header.get_future(),\n                          chunk_promises->chunk.get_future()});\n  }\n  internal::ThreadPool::global().Schedule([this, chunk_promises] {\n    Chunk chunk;\n    EncodeMetadata(chunk);\n    chunk_promises->chunk_header.set_value(chunk.header);\n    chunk_promises->chunk.set_value(std::move(chunk));\n    delete chunk_promises;\n  });\n  return true;\n}\n\nbool RecordWriterBase::ParallelWorker::CloseChunk() {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  ChunkEncoder* const chunk_encoder = chunk_encoder_.release();\n  ChunkPromises* const chunk_promises = new ChunkPromises();\n  {\n    absl::MutexLock lock(\n        mutex_, absl::Condition(this, &ParallelWorker::HasCapacityForRequest));\n    chunk_writer_requests_.emplace_back(\n        WriteChunkRequest{chunk_promises->chunk_header.get_future(),\n                          chunk_promises->chunk.get_future()});\n  }\n  internal::ThreadPool::global().Schedule(\n      [this, chunk_encoder, chunk_promises] {\n        Chunk chunk;\n        EncodeChunk(*chunk_encoder, chunk);\n        delete chunk_encoder;\n        chunk_promises->chunk_header.set_value(chunk.header);\n        chunk_promises->chunk.set_value(std::move(chunk));\n        delete chunk_promises;\n      });\n  return true;\n}\n\nbool RecordWriterBase::ParallelWorker::WritePadding(Position padding) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  {\n    absl::MutexLock lock(\n        mutex_, absl::Condition(this, &ParallelWorker::HasCapacityForRequest));\n    chunk_writer_requests_.emplace_back(WritePaddingRequest{padding});\n  }\n  return true;\n}\n\nbool RecordWriterBase::ParallelWorker::Flush(FlushType flush_type) {\n  return FutureFlush(flush_type).get().ok();\n}\n\nRecordWriterBase::FutureStatus RecordWriterBase::ParallelWorker::FutureFlush(\n    FlushType flush_type) {\n  std::promise<absl::Status> done_promise;\n  FutureStatus done_future = done_promise.get_future();\n  {\n    absl::MutexLock lock(\n        mutex_, absl::Condition(this, &ParallelWorker::HasCapacityForRequest));\n    chunk_writer_requests_.emplace_back(\n        FlushRequest{flush_type, std::move(done_promise)});\n  }\n  return done_future;\n}\n\nrecords_internal::FutureChunkBegin\nRecordWriterBase::ParallelWorker::ChunkBegin() const {\n  struct Visitor {\n    void operator()(const DoneRequest&) {}\n    void operator()(const AnnotateStatusRequest&) {}\n    void operator()(const WriteChunkRequest& request) {\n      actions.emplace_back(request.chunk_header);\n    }\n    void operator()(const WritePaddingRequest& request) {\n      actions.emplace_back(\n          records_internal::FutureChunkBegin::WritePadding{request.padding});\n    }\n    void operator()(const FlushRequest&) {}\n\n    std::vector<records_internal::FutureChunkBegin::Action> actions;\n  };\n  Visitor visitor;\n  absl::MutexLock lock(mutex_);\n  visitor.actions.reserve(chunk_writer_requests_.size());\n  for (const ChunkWriterRequest& request : chunk_writer_requests_) {\n    std::visit(visitor, request);\n  }\n  return records_internal::FutureChunkBegin(pos_before_chunks_,\n                                            std::move(visitor.actions));\n}\n\nFutureRecordPosition RecordWriterBase::ParallelWorker::LastPos() const {\n  RIEGELI_ASSERT_NE(chunk_encoder_, nullptr)\n      << \"Failed invariant of RecordWriterBase::ParallelWorker: \"\n         \"last position should be valid but chunk is closed\";\n  RIEGELI_ASSERT_GT(chunk_encoder_->num_records(), 0u)\n      << \"Failed invariant of RecordWriterBase::ParallelWorker: \"\n         \"last position should be valid but no record was encoded\";\n  return FutureRecordPosition(ChunkBegin(), chunk_encoder_->num_records() - 1);\n}\n\nFutureRecordPosition RecordWriterBase::ParallelWorker::Pos() const {\n  // `chunk_encoder_` is `nullptr` when the current chunk is closed, e.g. when\n  // `RecordWriter` is closed or if `RecordWriter::Flush()` failed.\n  return FutureRecordPosition(ChunkBegin(),\n                              ABSL_PREDICT_FALSE(chunk_encoder_ == nullptr)\n                                  ? uint64_t{0}\n                                  : chunk_encoder_->num_records());\n}\n\nPosition RecordWriterBase::ParallelWorker::EstimatedSize() const {\n  absl::MutexLock lock(mutex_);\n  return pos_before_chunks_;\n}\n\nRecordWriterBase::RecordWriterBase(Closed) noexcept : Object(kClosed) {}\n\nRecordWriterBase::RecordWriterBase() noexcept {}\n\nvoid RecordWriterBase::Reset(Closed) {\n  DoneBackground();\n  Object::Reset(kClosed);\n  desired_chunk_size_ = 0;\n  chunk_size_so_far_ = 0;\n  last_record_ = LastRecordIsInvalid();\n}\n\nvoid RecordWriterBase::Reset() {\n  DoneBackground();\n  Object::Reset();\n  desired_chunk_size_ = 0;\n  chunk_size_so_far_ = 0;\n  last_record_ = LastRecordIsInvalid();\n}\n\nRecordWriterBase::RecordWriterBase(RecordWriterBase&& that) noexcept\n    : Object(static_cast<Object&&>(that)),\n      desired_chunk_size_(that.desired_chunk_size_),\n      chunk_size_so_far_(that.chunk_size_so_far_),\n      last_record_(std::exchange(that.last_record_, LastRecordIsInvalid())),\n      worker_(std::move(that.worker_)) {}\n\nRecordWriterBase& RecordWriterBase::operator=(\n    RecordWriterBase&& that) noexcept {\n  if (ABSL_PREDICT_TRUE(&that != this)) {\n    DoneBackground();\n    Object::operator=(static_cast<Object&&>(that));\n    desired_chunk_size_ = that.desired_chunk_size_;\n    chunk_size_so_far_ = that.chunk_size_so_far_;\n    last_record_ = std::exchange(that.last_record_, LastRecordIsInvalid());\n    worker_ = std::move(that.worker_);\n  }\n  return *this;\n}\n\nRecordWriterBase::~RecordWriterBase() {}\n\nvoid RecordWriterBase::Initialize(ChunkWriter* dest, Options&& options) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of RecordWriter: null ChunkWriter pointer\";\n  // Ensure that `num_records` does not overflow when `WriteRecordImpl()` keeps\n  // `num_records * sizeof(uint64_t)` under `desired_chunk_size_`.\n  desired_chunk_size_ = UnsignedMin(options.effective_chunk_size(),\n                                    kMaxNumRecords * sizeof(uint64_t));\n  if (options.parallelism() == 0) {\n    worker_ = std::make_unique<SerialWorker>(dest, std::move(options));\n  } else {\n    worker_ = std::make_unique<ParallelWorker>(dest, std::move(options));\n  }\n  if (absl::Status status = worker_->status();\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    FailWithoutAnnotation(std::move(status));\n  }\n}\n\nvoid RecordWriterBase::Done() {\n  if (ABSL_PREDICT_FALSE(worker_ == nullptr)) {\n    RIEGELI_ASSERT(!is_open()) << \"Failed invariant of RecordWriterBase: \"\n                                  \"null worker_ but RecordWriterBase is_open()\";\n    return;\n  }\n  if (chunk_size_so_far_ > 0) {\n    if (std::holds_alternative<LastRecordIsValid>(last_record_)) {\n      last_record_ = LastRecordIsValidAt{worker_->LastPos()};\n    }\n    if (ABSL_PREDICT_FALSE(!worker_->CloseChunk())) {\n      FailWithoutAnnotation(worker_->status());\n    }\n    chunk_size_so_far_ = 0;\n  }\n  if (ABSL_PREDICT_FALSE(!worker_->MaybePadToFinalBoundary())) {\n    FailWithoutAnnotation(worker_->status());\n  }\n  if (ABSL_PREDICT_FALSE(!worker_->Close())) {\n    FailWithoutAnnotation(worker_->status());\n  }\n}\n\nvoid RecordWriterBase::DoneBackground() { worker_.reset(); }\n\nabsl::Status RecordWriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    RIEGELI_ASSERT_NE(worker_, nullptr)\n        << \"Failed invariant of RecordWriterBase: \"\n           \"null worker_ but RecordWriterBase is_open()\";\n    status = worker_->AnnotateStatus(std::move(status));\n  }\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status RecordWriterBase::AnnotateOverDest(absl::Status status) {\n  return Annotate(status, absl::StrCat(\"at record \", Pos().get().ToString()));\n}\n\nbool RecordWriterBase::WriteRecord(const google::protobuf::MessageLite& record,\n                                   SerializeMessageOptions serialize_options) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const size_t size = serialize_options.GetByteSize(record);\n  return WriteRecordImpl(size, record, std::move(serialize_options));\n}\n\nbool RecordWriterBase::WriteRecord(BytesRef record) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  return WriteRecordImpl(record.size(), record);\n}\n\nbool RecordWriterBase::WriteRecord(ExternalRef record) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const size_t size = record.size();\n  return WriteRecordImpl(size, std::move(record));\n}\n\nbool RecordWriterBase::WriteRecord(const Chain& record) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  return WriteRecordImpl(record.size(), record);\n}\n\nbool RecordWriterBase::WriteRecord(Chain&& record) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const size_t size = record.size();\n  return WriteRecordImpl(size, std::move(record));\n}\n\nbool RecordWriterBase::WriteRecord(const absl::Cord& record) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  return WriteRecordImpl(record.size(), record);\n}\n\nbool RecordWriterBase::WriteRecord(absl::Cord&& record) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  const size_t size = record.size();\n  return WriteRecordImpl(size, std::move(record));\n}\n\ntemplate <typename... Args>\ninline bool RecordWriterBase::WriteRecordImpl(size_t size, Args&&... args) {\n  last_record_ = LastRecordIsInvalid();\n  // Decoding a chunk writes records to one array, and their positions to\n  // another array. We limit the size of both arrays together, to include\n  // attempts to accumulate an unbounded number of empty records.\n  const uint64_t added_size = uint64_t{size} + uint64_t{sizeof(uint64_t)};\n  if (ABSL_PREDICT_FALSE(chunk_size_so_far_ + added_size >\n                         desired_chunk_size_) &&\n      chunk_size_so_far_ > 0) {\n    if (ABSL_PREDICT_FALSE(!worker_->CloseChunk())) {\n      return FailWithoutAnnotation(worker_->status());\n    }\n    worker_->OpenChunk();\n    chunk_size_so_far_ = 0;\n  }\n  chunk_size_so_far_ += added_size;\n  if (ABSL_PREDICT_FALSE(!worker_->AddRecord(std::forward<Args>(args)...))) {\n    return FailWithoutAnnotation(worker_->status());\n  }\n  if (ABSL_PREDICT_FALSE(chunk_size_so_far_ + uint64_t{sizeof(uint64_t)} >\n                         desired_chunk_size_)) {\n    // No more records will fit in this chunk, most likely a single record\n    // exceeds the desired chunk size. Write the chunk now to avoid keeping a\n    // large chunk in memory.\n    last_record_ = LastRecordIsValidAt{worker_->LastPos()};\n    if (ABSL_PREDICT_FALSE(!worker_->CloseChunk())) {\n      return FailWithoutAnnotation(worker_->status());\n    }\n    worker_->OpenChunk();\n    chunk_size_so_far_ = 0;\n    return true;\n  }\n  last_record_ = LastRecordIsValid();\n  return true;\n}\n\nbool RecordWriterBase::Flush(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (chunk_size_so_far_ > 0) {\n    if (std::holds_alternative<LastRecordIsValid>(last_record_)) {\n      last_record_ = LastRecordIsValidAt{worker_->LastPos()};\n    }\n    if (ABSL_PREDICT_FALSE(!worker_->CloseChunk())) {\n      return FailWithoutAnnotation(worker_->status());\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!worker_->MaybePadToFinalBoundary())) {\n    return FailWithoutAnnotation(worker_->status());\n  }\n  if (flush_type != FlushType::kFromObject || IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!worker_->Flush(flush_type))) {\n      return FailWithoutAnnotation(worker_->status());\n    }\n  }\n  if (chunk_size_so_far_ > 0) {\n    worker_->OpenChunk();\n    chunk_size_so_far_ = 0;\n  }\n  return true;\n}\n\nRecordWriterBase::FutureStatus RecordWriterBase::FutureFlush(\n    FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ok())) {\n    std::promise<absl::Status> promise;\n    promise.set_value(status());\n    return promise.get_future();\n  }\n  if (chunk_size_so_far_ > 0) {\n    if (std::holds_alternative<LastRecordIsValid>(last_record_)) {\n      last_record_ = LastRecordIsValidAt{worker_->LastPos()};\n    }\n    if (ABSL_PREDICT_FALSE(!worker_->CloseChunk())) {\n      FailWithoutAnnotation(worker_->status());\n      std::promise<absl::Status> promise;\n      promise.set_value(status());\n      return promise.get_future();\n    }\n  }\n  if (ABSL_PREDICT_FALSE(!worker_->MaybePadToFinalBoundary())) {\n    FailWithoutAnnotation(worker_->status());\n    std::promise<absl::Status> promise;\n    promise.set_value(status());\n    return promise.get_future();\n  }\n  FutureStatus result;\n  if (flush_type == FlushType::kFromObject && !IsOwning()) {\n    std::promise<absl::Status> promise;\n    promise.set_value(absl::OkStatus());\n    result = promise.get_future();\n  } else {\n    result = worker_->FutureFlush(flush_type);\n  }\n  if (chunk_size_so_far_ > 0) {\n    worker_->OpenChunk();\n    chunk_size_so_far_ = 0;\n  }\n  return result;\n}\n\nFutureRecordPosition RecordWriterBase::LastPos() const {\n  RIEGELI_ASSERT(last_record_is_valid())\n      << \"Failed precondition of RecordWriterBase::LastPos(): \"\n         \"no record was recently written\";\n  if (const LastRecordIsValidAt* const last_record_at_pos =\n          std::get_if<LastRecordIsValidAt>(&last_record_)) {\n    return last_record_at_pos->pos;\n  }\n  RIEGELI_ASSERT_NE(worker_, nullptr)\n      << \"Failed invariant of RecordWriterBase: \"\n         \"last position should be valid but worker is null\";\n  return worker_->LastPos();\n}\n\nFutureRecordPosition RecordWriterBase::Pos() const {\n  if (ABSL_PREDICT_FALSE(worker_ == nullptr)) return FutureRecordPosition();\n  return worker_->Pos();\n}\n\nPosition RecordWriterBase::EstimatedSize() const {\n  if (ABSL_PREDICT_FALSE(worker_ == nullptr)) return 0;\n  return worker_->EstimatedSize();\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/records/record_writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_RECORDS_RECORD_WRITER_H_\n#define RIEGELI_RECORDS_RECORD_WRITER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <future>\n#include <memory>\n#include <optional>\n#include <type_traits>\n#include <utility>\n#include <variant>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/protobuf/descriptor.h\"\n#include \"google/protobuf/message_lite.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/stable_dependency.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/compressor_options.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/messages/serialize_message.h\"\n#include \"riegeli/records/block.h\"\n#include \"riegeli/records/chunk_writer.h\"\n#include \"riegeli/records/record_position.h\"\n#include \"riegeli/records/records_metadata.pb.h\"\n\nnamespace riegeli {\n\n// Sets `record_type_name` and `file_descriptor` in metadata, based on the\n// message descriptor of the type of records.\n//\n// TODO: This currently includes whole file descriptors. It would be\n// better to prune them to keep only what is needed for the message descriptor.\nvoid SetRecordType(const google::protobuf::Descriptor& descriptor,\n                   RecordsMetadata& metadata);\n\n// Template parameter independent part of `RecordWriter`.\nclass RecordWriterBase : public Object {\n public:\n  constexpr static Position kImplicitPadding = records_internal::kBlockSize;\n\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // Parses options from text:\n    // ```\n    //   options ::= option? (\",\" option?)*\n    //   option ::=\n    //     \"default\" |\n    //     \"transpose\" (\":\" (\"true\" | \"false\"))? |\n    //     \"uncompressed\" |\n    //     \"brotli\" (\":\" brotli_level)? |\n    //     \"zstd\" (\":\" zstd_level)? |\n    //     \"snappy\" (\":\" snappy_level)? |\n    //     \"window_log\" \":\" window_log |\n    //     \"brotli_encoder\" \":\" (\"rbrotli_or_cbrotli\" | \"cbrotli\" | \"rbrotli\") |\n    //     \"chunk_size\" \":\" chunk_size |\n    //     \"bucket_fraction\" \":\" bucket_fraction |\n    //     \"padding\" (\":\" padding)? |\n    //     \"initial_padding\" (\":\" padding)? |\n    //     \"final_padding\" (\":\" padding)? |\n    //     \"parallelism\" \":\" parallelism\n    //   brotli_level ::= integer in the range [0..11] (default 6)\n    //   zstd_level ::= integer in the range [-131072..22] (default 3)\n    //   snappy_level ::= integer in the range [1..2] (default 1)\n    //   window_log ::= \"auto\" or integer in the range [10..31]\n    //   chunk_size ::= \"auto\" or positive integer expressed as real with\n    //     optional suffix [BkKMGTPE]\n    //   bucket_fraction ::= real in the range [0..1]\n    //   padding ::= positive integer expressed as real with optional suffix\n    //     [BkKMGTPE] (default 64K)\n    //   parallelism ::= non-negative integer\n    // ```\n    //\n    // An empty string is the same as \"default\".\n    //\n    // Options are documented below, and also at\n    // https://github.com/google/riegeli/blob/master/doc/record_writer_options.md\n    //\n    // Returns status:\n    //  * `status.ok()`  - success\n    //  * `!status.ok()` - failure\n    absl::Status FromString(absl::string_view text);\n\n    // If `false`, records can be arbitrary strings. A chunk of records is\n    // stored in a simple format, directly or with compression.\n    //\n    // If `true`, records should be serialized proto messages, but nothing\n    // breaks if they are not. A chunk of records is processed in a way which\n    // allows for better compression of proto messages, by putting next to each\n    // other occurrences of the same field across records or across elements of\n    // a repeated field.\n    //\n    // Default: `false`.\n    Options& set_transpose(bool transpose) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      transpose_ = transpose;\n      return *this;\n    }\n    Options&& set_transpose(bool transpose) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_transpose(transpose));\n    }\n    bool transpose() const { return transpose_; }\n\n    // Changes compression algorithm to Uncompressed (turns compression off).\n    Options& set_uncompressed() & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      compressor_options_.set_uncompressed();\n      return *this;\n    }\n    Options&& set_uncompressed() && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_uncompressed());\n    }\n\n    // Changes compression algorithm to Brotli. Sets compression level which\n    // tunes the tradeoff between compression density and compression speed\n    // (higher = better density but slower).\n    //\n    // `compression_level` must be between `kMinBrotli` (0) and\n    // `kMaxBrotli` (11). Default: `kDefaultBrotli` (6).\n    //\n    // This is the default compression algorithm.\n    static constexpr int kMinBrotli = CompressorOptions::kMinBrotli;\n    static constexpr int kMaxBrotli = CompressorOptions::kMaxBrotli;\n    static constexpr int kDefaultBrotli = CompressorOptions::kDefaultBrotli;\n    Options& set_brotli(int compression_level = kDefaultBrotli) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      compressor_options_.set_brotli(compression_level);\n      return *this;\n    }\n    Options&& set_brotli(int compression_level = kDefaultBrotli) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_brotli(compression_level));\n    }\n\n    // Changes compression algorithm to Zstd. Sets compression level which tunes\n    // the tradeoff between compression density and compression speed (higher =\n    // better density but slower).\n    //\n    // `compression_level` must be between `kMinZstd` (-131072) and\n    // `kMaxZstd` (22). Level 0 is currently equivalent to 3.\n    // Default: `kDefaultZstd` (3).\n    static constexpr int kMinZstd = CompressorOptions::kMinZstd;\n    static constexpr int kMaxZstd = CompressorOptions::kMaxZstd;\n    static constexpr int kDefaultZstd = CompressorOptions::kDefaultZstd;\n    Options& set_zstd(int compression_level = kDefaultZstd) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      compressor_options_.set_zstd(compression_level);\n      return *this;\n    }\n    Options&& set_zstd(int compression_level = kDefaultZstd) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_zstd(compression_level));\n    }\n\n    // Changes compression algorithm to Snappy.\n    //\n    // There are no Snappy compression levels to tune.\n    static constexpr int kMinSnappy = CompressorOptions::kMinSnappy;\n    static constexpr int kMaxSnappy = CompressorOptions::kMaxSnappy;\n    static constexpr int kDefaultSnappy = CompressorOptions::kDefaultSnappy;\n    Options& set_snappy(int compression_level = kDefaultSnappy) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      compressor_options_.set_snappy(compression_level);\n      return *this;\n    }\n    Options&& set_snappy(int compression_level = kDefaultSnappy) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_snappy(compression_level));\n    }\n\n    CompressionType compression_type() const {\n      return compressor_options_.compression_type();\n    }\n\n    int compression_level() const {\n      return compressor_options_.compression_level();\n    }\n\n    // Logarithm of the LZ77 sliding window size. This tunes the tradeoff\n    // between compression density and memory usage (higher = better density but\n    // more memory).\n    //\n    // Special value `std::nullopt` means to keep the default (Brotli: 22,\n    // Zstd: derived from compression level and chunk size).\n    //\n    // For Uncompressed and Snappy, `window_log` must be `std::nullopt`.\n    //\n    // For Brotli, `window_log` must be `std::nullopt` or between\n    // `BrotliWriterBase::Options::kMinWindowLog` (10) and\n    // `BrotliWriterBase::Options::kMaxWindowLog` (30).\n    //\n    // For Zstd, `window_log` must be `std::nullopt` or between\n    // `ZstdWriterBase::Options::kMinWindowLog` (10) and\n    // `ZstdWriterBase::Options::kMaxWindowLog` (30 in 32-bit build,\n    // 31 in 64-bit build).\n    //\n    // Default: `std::nullopt`.\n    static constexpr int kMinWindowLog = CompressorOptions::kMinWindowLog;\n    static constexpr int kMaxWindowLog = CompressorOptions::kMaxWindowLog;\n    Options& set_window_log(std::optional<int> window_log) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      compressor_options_.set_window_log(window_log);\n      return *this;\n    }\n    Options&& set_window_log(std::optional<int> window_log) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_window_log(window_log));\n    }\n    std::optional<int> window_log() const {\n      return compressor_options_.window_log();\n    }\n\n    // Returns grouped compression options.\n    CompressorOptions& compressor_options() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return compressor_options_;\n    }\n    const CompressorOptions& compressor_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return compressor_options_;\n    }\n\n    // Desired uncompressed size of a chunk which groups messages to be\n    // transposed, compressed, and written together.\n    //\n    // A larger chunk size improves compression density; a smaller chunk size\n    // allows to read pieces of the file independently with finer granularity,\n    // and reduces memory usage of both writer and reader.\n    //\n    // Special value `std::nullopt` means to keep the default\n    // (compressed: 1M, uncompressed: 4k).\n    //\n    // Default: `std::nullopt`.\n    Options& set_chunk_size(std::optional<uint64_t> chunk_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      if (chunk_size != std::nullopt) {\n        RIEGELI_ASSERT_GT(*chunk_size, 0u)\n            << \"Failed precondition of \"\n               \"RecordWriterBase::Options::set_chunk_size(): \"\n               \"zero chunk size\";\n      }\n      chunk_size_ = chunk_size;\n      return *this;\n    }\n    Options&& set_chunk_size(std::optional<uint64_t> chunk_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_chunk_size(chunk_size));\n    }\n    std::optional<uint64_t> chunk_size() const { return chunk_size_; }\n    uint64_t effective_chunk_size() const {\n      if (chunk_size_ == std::nullopt) {\n        return compression_type() == CompressionType::kNone ? uint64_t{4} << 10\n                                                            : uint64_t{1} << 20;\n      }\n      return *chunk_size_;\n    }\n\n    // Desired uncompressed size of a bucket which groups values of several\n    // fields of the given wire type to be compressed together, relative to the\n    // desired chunk size, on the scale between 0.0 (compress each field\n    // separately) to 1.0 (put all fields of the same wire type in the same\n    // bucket).\n    //\n    // This is meaningful if transpose and compression are enabled. A larger\n    // bucket size improves compression density; a smaller bucket size makes\n    // reading with projection faster, allowing to skip decompression of values\n    // of fields which are not included.\n    //\n    // Default: 1.0.\n    Options& set_bucket_fraction(double bucket_fraction) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(bucket_fraction, 0.0)\n          << \"Failed precondition of \"\n             \"RecordWriterBase::Options::set_bucket_fraction(): \"\n             \"negative bucket fraction\";\n      RIEGELI_ASSERT_LE(bucket_fraction, 1.0)\n          << \"Failed precondition of \"\n             \"RecordWriterBase::Options::set_bucket_fraction(): \"\n             \"fraction larger than 1\";\n      bucket_fraction_ = bucket_fraction;\n      return *this;\n    }\n    Options&& set_bucket_fraction(double bucket_fraction) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_bucket_fraction(bucket_fraction));\n    }\n    double bucket_fraction() const { return bucket_fraction_; }\n\n    // If not `std::nullopt`, sets file metadata to be written at the\n    // beginning.\n    //\n    // Metadata are written only when the file is written from the beginning,\n    // not when it is appended to.\n    //\n    // Record type in metadata can be conveniently set by `SetRecordType()`.\n    //\n    // Default: `std::nullopt`.\n    Options& set_metadata(\n        Initializer<std::optional<RecordsMetadata>> metadata) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      riegeli::Reset(metadata_, std::move(metadata));\n      serialized_metadata_ = std::nullopt;\n      return *this;\n    }\n    Options&& set_metadata(\n        Initializer<std::optional<RecordsMetadata>> metadata) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_metadata(std::move(metadata)));\n    }\n    std::optional<RecordsMetadata>& metadata() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return metadata_;\n    }\n    const std::optional<RecordsMetadata>& metadata() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return metadata_;\n    }\n\n    // Like `set_metadata()`, but metadata are passed in the serialized form.\n    //\n    // This is faster if the caller has metadata already serialized.\n    Options& set_serialized_metadata(\n        Initializer<std::optional<Chain>> serialized_metadata) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      metadata_ = std::nullopt;\n      riegeli::Reset(serialized_metadata_, std::move(serialized_metadata));\n      return *this;\n    }\n    Options&& set_serialized_metadata(\n        Initializer<std::optional<Chain>> serialized_metadata) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_serialized_metadata(std::move(serialized_metadata)));\n    }\n    std::optional<Chain>& serialized_metadata() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return serialized_metadata_;\n    }\n    const std::optional<Chain>& serialized_metadata() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return serialized_metadata_;\n    }\n\n    // If `padding > 1`, padding is written at the beginning, when flushing,\n    // and at the end of the file, for the absolute position to reach a multiple\n    // of `padding`.\n    //\n    // Consequences if `padding` is a multiple of 64KB:\n    //\n    //  1. Physical concatenation of separately written files yields a valid\n    //     file (setting metadata in subsequent files is wasteful but harmless).\n    //\n    //  2. Even if the existing file was corrupted or truncated, data appended\n    //     to it will be recoverable.\n    //\n    // The cost is that up to `padding` bytes is wasted when padding is written.\n    //\n    // `set_padding(padding)` is a shortcut for `set_initial_padding(padding)`\n    // with `set_final_padding(padding)`.\n    //\n    // `set_padding()` without the parameter assumes 64KB.\n    //\n    // Default: 1 (no padding).\n    Options& set_padding(Position padding = kImplicitPadding) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GT(padding, 0u)\n          << \"Failed precondition of RecordWriterBase::Options::set_padding(): \"\n             \"padding must be positive\";\n      initial_padding_ = padding;\n      final_padding_ = padding;\n      return *this;\n    }\n    Options&& set_padding(Position padding = kImplicitPadding) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_padding(padding));\n    }\n\n    // If `initial_padding > 1`, padding is written at the beginning of the\n    // file, for the absolute position to reach a multiple of `initial_padding`.\n    //\n    // See `set_padding()` for details.\n    //\n    // `set_initial_padding()` without the parameter assumes 64KB.\n    //\n    // Default: 1 (no padding).\n    Options& set_initial_padding(Position initial_padding = kImplicitPadding) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GT(initial_padding, 0u)\n          << \"Failed precondition of \"\n             \"RecordWriterBase::Options::set_initial_padding(): \"\n             \"padding must be positive\";\n      initial_padding_ = initial_padding;\n      return *this;\n    }\n    Options&& set_initial_padding(\n        Position initial_padding = kImplicitPadding) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_initial_padding(initial_padding));\n    }\n    Position initial_padding() const { return initial_padding_; }\n\n    // If `final_padding > 1`, padding is written when flushing and at the end\n    // of the file, for the absolute position to reach a multiple of\n    // `final_padding`.\n    //\n    // See `set_padding()` for details.\n    //\n    // `set_final_padding()` without the parameter assumes 64KB.\n    //\n    // Default: 1 (no padding).\n    Options& set_final_padding(Position final_padding = kImplicitPadding) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GT(final_padding, 0u)\n          << \"Failed precondition of \"\n             \"RecordWriterBase::Options::set_final_padding(): \"\n             \"padding must be positive\";\n      final_padding_ = final_padding;\n      return *this;\n    }\n    Options&& set_final_padding(Position final_padding = kImplicitPadding) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_final_padding(final_padding));\n    }\n    Position final_padding() const { return final_padding_; }\n\n    // Maximum number of chunks being encoded in parallel in background. Larger\n    // parallelism can increase throughput, up to a point where it no longer\n    // matters; smaller parallelism reduces memory usage.\n    //\n    // If `parallelism > 0`, chunks are written to the byte `Writer` in\n    // background and reporting writing errors is delayed.\n    //\n    // Default: 0.\n    Options& set_parallelism(int parallelism) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(parallelism, 0)\n          << \"Failed precondition of \"\n             \"RecordWriterBase::Options::set_parallelism(): \"\n             \"negative parallelism\";\n      parallelism_ = parallelism;\n      return *this;\n    }\n    Options&& set_parallelism(int parallelism) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_parallelism(parallelism));\n    }\n    int parallelism() const { return parallelism_; }\n\n    // Options for a global `RecyclingPool` of compression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // compression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    Options& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    Options&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    bool transpose_ = false;\n    CompressorOptions compressor_options_;\n    std::optional<uint64_t> chunk_size_;\n    double bucket_fraction_ = 1.0;\n    std::optional<RecordsMetadata> metadata_;\n    std::optional<Chain> serialized_metadata_;\n    Position initial_padding_ = 1;\n    Position final_padding_ = 1;\n    int parallelism_ = 0;\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // `get()` returns the resolved value. Can block.\n  using FutureStatus = std::shared_future<absl::Status>;\n\n  ~RecordWriterBase();\n\n  // Returns the Riegeli/records file being written to. Unchanged by `Close()`.\n  virtual ChunkWriter* DestChunkWriter() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Writes the next record.\n  //\n  // `WriteRecord(google::protobuf::MessageLite)` serializes a proto message to\n  // raw bytes beforehand. The remaining overloads accept raw bytes.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool WriteRecord(\n      const google::protobuf::MessageLite& record,\n      SerializeMessageOptions serialize_options = SerializeMessageOptions());\n  bool WriteRecord(BytesRef record);\n  bool WriteRecord(ExternalRef record);\n  template <typename Src,\n            std::enable_if_t<SupportsExternalRefWhole<Src>::value, int> = 0>\n  bool WriteRecord(Src&& src);\n  bool WriteRecord(const Chain& record);\n  bool WriteRecord(Chain&& record);\n  bool WriteRecord(const absl::Cord& record);\n  bool WriteRecord(absl::Cord&& record);\n\n  // Finalizes any open chunk and pushes buffered data to the destination.\n  // If `Options::parallelism() > 0`, waits for any background writing to\n  // complete.\n  //\n  // This makes data written so far visible, but in contrast to `Close()`,\n  // keeps the possibility to write more data later. What exactly does it mean\n  // for data to be visible depends on the destination.\n  //\n  // This degrades compression density if used too often.\n  //\n  // The scope of objects to flush and the intended data durability (without a\n  // guarantee) are specified by `flush_type`:\n  //  * `FlushType::kFromObject`  - Makes data written so far visible in other\n  //                                objects, propagating flushing through owned\n  //                                dependencies of the given writer.\n  //  * `FlushType::kFromProcess` - Makes data written so far visible outside\n  //                                the process, propagating flushing through\n  //                                dependencies of the given writer.\n  //                                This is the default.\n  //  * `FlushType::kFromMachine` - Makes data written so far visible outside\n  //                                the process and durable in case of operating\n  //                                system crash, propagating flushing through\n  //                                dependencies of the given writer.\n  //\n  // Return values:\n  //  * `true`  - success (`ok()`)\n  //  * `false` - failure (`!ok()`)\n  bool Flush(FlushType flush_type = FlushType::kFromProcess);\n\n  // Like `Flush()`, but if `Options::parallelism() > 0`, does not wait for\n  // background writing to complete. Returns a `FutureStatus` which can be used\n  // to wait for background writing to complete.\n  //\n  // Like any member function, `FutureFlush()` must not be called concurrently\n  // with other member functions, but there are no concurrency restrictions on\n  // calling `get()` on the result.\n  //\n  // `Flush()` is similar to `FutureFlush().get().ok()`, except that `Flush()`\n  // also propagates the status to the `RecordWriter`.\n  FutureStatus FutureFlush(FlushType flush_type = FlushType::kFromProcess);\n\n  // Returns the canonical position of the last record written.\n  //\n  // The canonical position is the largest among all equivalent positions.\n  // Seeking to any equivalent position leads to reading the same record.\n  //\n  // `LastPos().get().numeric()` returns the position as an integer of type\n  // `Position`.\n  //\n  // Precondition: a record was successfully written (this can be checked with\n  // `last_record_is_valid()`).\n  FutureRecordPosition LastPos() const;\n\n  // Returns `true` if calling `LastPos()` is valid.\n  bool last_record_is_valid() const {\n    return !std::holds_alternative<LastRecordIsInvalid>(last_record_);\n  }\n\n  // Returns a position of the next record (or the end of file if there is no\n  // next record).\n  //\n  // A position which is not canonical can be smaller than the equivalent\n  // canonical position. Seeking to any equivalent position leads to reading the\n  // same record.\n  //\n  // `Pos().get().numeric()` returns the position as an integer of type\n  // `Position`.\n  //\n  // After opening the file, `Close()`, or `Flush()`, `Pos()` is the canonical\n  // position of the next record, and `Pos().get().record_index() == 0`.\n  FutureRecordPosition Pos() const;\n\n  // Returns an estimation of the file size if no more data is written, without\n  // affecting data representation (i.e. without closing the current chunk) and\n  // without blocking.\n  //\n  // This is an underestimation because pending work is not taken into account:\n  //  * The currently open chunk.\n  //  * If `Options::parallelism() > 0`, chunks being encoded in background.\n  //\n  // The exact file size can be found by `FutureFlush(FlushType::kFromObject)`\n  // which closes the currently open chunk, and `Pos().get().chunk_begin()`\n  // (`record_index() == 0` after flushing) which might need to wait for some\n  // background work to complete.\n  Position EstimatedSize() const;\n\n protected:\n  explicit RecordWriterBase(Closed) noexcept;\n\n  RecordWriterBase() noexcept;\n\n  RecordWriterBase(RecordWriterBase&& that) noexcept;\n  RecordWriterBase& operator=(RecordWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n\n  virtual bool IsOwning() const = 0;\n\n  void Initialize(ChunkWriter* dest, Options&& options);\n  void DoneBackground();\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n\n private:\n  class ParallelWorker;\n  class SerialWorker;\n  class Worker;\n\n  struct LastRecordIsInvalid {};\n  struct LastRecordIsValid {};  // At one record before `Pos()`.\n  struct LastRecordIsValidAt {\n    FutureRecordPosition pos;\n  };\n\n  template <typename... Args>\n  bool WriteRecordImpl(size_t size, Args&&... args);\n\n  uint64_t desired_chunk_size_ = 0;\n  uint64_t chunk_size_so_far_ = 0;\n  std::variant<LastRecordIsInvalid, LastRecordIsValid, LastRecordIsValidAt>\n      last_record_ = LastRecordIsInvalid();\n  // Invariant: if `is_open()` then `worker_ != nullptr`.\n  std::unique_ptr<Worker> worker_;\n};\n\n// `RecordWriter` writes records to a Riegeli/records file. A record is\n// conceptually a binary string; usually it is a serialized proto message.\n//\n// For writing records, this kind of loop can be used:\n// ```\n//   SomeProto record;\n//   while (more records to write) {\n//     ... Compute record.\n//     if (!record_writer_.Write(record)) {\n//       // record_writer_.Close() will fail below.\n//       break;\n//     }\n//   }\n//   if (!record_writer_.Close()) {\n//     ... Failed with reason: record_writer_.status()\n//   }\n// ```\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the byte `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// `Dest` can also specify a `ChunkWriter` instead of a byte `Writer`. In this\n// case `Dest` must support `Dependency<ChunkWriter*, Dest>`, e.g.\n// `ChunkWriter*` (not owned), `DefaultChunkWriter<>` (owned),\n// `std::unique_ptr<ChunkWriter>` (owned), `Any<ChunkWriter*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The byte `Writer` or `ChunkWriter` must not be accessed until the\n// `RecordWriter` is closed or (when parallelism in options is 0) no longer\n// used.\ntemplate <typename Dest = Writer*>\nclass RecordWriter : public RecordWriterBase {\n public:\n  // Creates a closed `RecordWriter`.\n  explicit RecordWriter(Closed) noexcept : RecordWriterBase(kClosed) {}\n\n  // Will write to the byte `Writer` or `ChunkWriter` provided by `dest`.\n  explicit RecordWriter(Initializer<Dest> dest, Options options = Options());\n\n  RecordWriter(RecordWriter&& that) = default;\n  RecordWriter& operator=(RecordWriter&& that) = default;\n\n  ~RecordWriter() override { DoneBackground(); }\n\n  // Makes `*this` equivalent to a newly constructed `RecordWriter`. This avoids\n  // constructing a temporary `RecordWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the byte `Writer` or\n  // `ChunkWriter`. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  ChunkWriter* DestChunkWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n\n  bool IsOwning() const override { return dest_.IsOwning(); }\n\n private:\n  // The object providing and possibly owning the byte `Writer` or\n  // `ChunkWriter`.\n  StableDependency<ChunkWriter*, Dest> dest_;\n};\n\nexplicit RecordWriter(Closed) -> RecordWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit RecordWriter(Dest&& dest, RecordWriterBase::Options options =\n                                       RecordWriterBase::Options())\n    -> RecordWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ntemplate <typename Src,\n          std::enable_if_t<SupportsExternalRefWhole<Src>::value, int>>\ninline bool RecordWriterBase::WriteRecord(Src&& src) {\n  return WriteRecord(ExternalRef(std::forward<Src>(src)));\n}\n\ntemplate <typename Dest>\ninline RecordWriter<Dest>::RecordWriter(Initializer<Dest> dest, Options options)\n    : dest_(std::move(dest)) {\n  Initialize(dest_.get(), std::move(options));\n}\n\ntemplate <typename Dest>\ninline void RecordWriter<Dest>::Reset(Closed) {\n  RecordWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void RecordWriter<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  RecordWriterBase::Reset();\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), std::move(options));\n}\n\ntemplate <typename Dest>\nvoid RecordWriter<Dest>::Done() {\n  RecordWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_RECORDS_RECORD_WRITER_H_\n"
  },
  {
    "path": "riegeli/records/records_metadata.proto",
    "content": "edition = \"2024\";\n\npackage riegeli;\n\nimport \"google/protobuf/descriptor.proto\";\n\n// Information about a Riegeli/records file, which may be helpful to interpret\n// file contents.\nmessage RecordsMetadata {\n  // Human-readable explanation of what the file contains.\n  string file_comment = 1;\n\n  // If records are proto messages of a fixed type, the full name of their type.\n  string record_type_name = 2;\n\n  // If `record_type_name` is set, proto file descriptors which should contain\n  // the definition of that type and their dependencies (each file comes after\n  // all its dependencies).\n  //\n  // If `file_descriptor` is empty but `record_type_name` is set (not\n  // recommended), `record_type_name` can be interpreted in the context of an\n  // unspecified proto descriptor database.\n  repeated google.protobuf.FileDescriptorProto file_descriptor = 3;\n\n  // Options originally used to encode the file:\n  // https://github.com/google/riegeli/blob/master/doc/record_writer_options.md\n  //\n  // They are informative here, they are never necessary to decode the file.\n  string record_writer_options = 4;\n\n  // Number of records in the file, so that the reader can tune for it.\n  //\n  // This is informative, the actual number of records may differ.\n  int64 num_records = 5;\n\n  // Clients can define custom metadata in extensions of this message.\n  extensions 1000 to max;\n}\n"
  },
  {
    "path": "riegeli/records/skipped_region.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/records/skipped_region.h\"\n\n#include <ostream>\n#include <string>\n\n#include \"absl/strings/str_cat.h\"\n\nnamespace riegeli {\n\nstd::string SkippedRegion::ToString() const {\n  return absl::StrCat(\"[\", begin_, \"..\", end_, \"): \", message_);\n}\n\nvoid SkippedRegion::Output(std::ostream& dest) const { dest << ToString(); }\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/records/skipped_region.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_RECORDS_SKIPPED_REGION_H_\n#define RIEGELI_RECORDS_SKIPPED_REGION_H_\n\n#include <iosfwd>\n#include <string>\n#include <utility>\n\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/string_ref.h\"\n#include \"riegeli/base/types.h\"\n\nnamespace riegeli {\n\n// Details about a skipped region of invalid file contents.\nclass SkippedRegion {\n public:\n  SkippedRegion() = default;\n\n  // Creates a `SkippedRegion` with the given region location and message\n  // explaining why the region is invalid.\n  explicit SkippedRegion(Position begin, Position end,\n                         StringInitializer message);\n\n  SkippedRegion(const SkippedRegion& that) = default;\n  SkippedRegion& operator=(const SkippedRegion& that) = default;\n\n  SkippedRegion(SkippedRegion&&) = default;\n  SkippedRegion& operator=(SkippedRegion&&) = default;\n\n  // File position of the beginning of the skipped region, inclusive.\n  Position begin() const { return begin_; }\n  // File position of the end of the skipped region, exclusive.\n  Position end() const { return end_; }\n\n  // Length of the skipped region, in bytes.\n  Position length() const { return end_ - begin_; }\n\n  // Message explaining why the region is invalid.\n  absl::string_view message() const { return message_; }\n\n  // Formats `SkippedRegion` as string: \"[<begin>..<end>): <message>\".\n  std::string ToString() const;\n\n  // Default stringification by `absl::StrCat()` etc.\n  //\n  // Writes `src.ToString()` to `dest`.\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const SkippedRegion& src) {\n    dest.Append(src.ToString());\n  }\n\n  // Writes `src.ToString()` to `out`.\n  friend std::ostream& operator<<(std::ostream& dest,\n                                  const SkippedRegion& src) {\n    src.Output(dest);\n    return dest;\n  }\n\n private:\n  void Output(std::ostream& dest) const;\n\n  Position begin_ = 0;\n  Position end_ = 0;\n  std::string message_;\n};\n\n// Implementation details follow.\n\ninline SkippedRegion::SkippedRegion(Position begin, Position end,\n                                    StringInitializer message)\n    : begin_(begin), end_(end), message_(std::move(message)) {\n  RIEGELI_ASSERT_LE(begin, end)\n      << \"Failed precondition of SkippedRegion::SkippedRegion: \"\n         \"positions in the wrong order\";\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_RECORDS_SKIPPED_REGION_H_\n"
  },
  {
    "path": "riegeli/records/tools/BUILD",
    "content": "load(\"@com_google_protobuf//bazel:cc_proto_library.bzl\", \"cc_proto_library\")\nload(\"@com_google_protobuf//bazel:proto_library.bzl\", \"proto_library\")\nload(\"@rules_cc//cc:defs.bzl\", \"cc_binary\", \"cc_library\")\n\npackage(features = [\"header_modules\"])\n\nlicenses([\"notice\"])\n\ncc_binary(\n    name = \"describe_riegeli_file\",\n    srcs = [\"describe_riegeli_file.cc\"],\n    deps = [\n        \":riegeli_summary_cc_proto\",\n        \"//riegeli/base:any\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:backward_writer\",\n        \"//riegeli/bytes:chain_backward_writer\",\n        \"//riegeli/bytes:chain_reader\",\n        \"//riegeli/bytes:fd_reader\",\n        \"//riegeli/bytes:limiting_reader\",\n        \"//riegeli/bytes:null_backward_writer\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:std_io\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/chunk_encoding:chunk\",\n        \"//riegeli/chunk_encoding:constants\",\n        \"//riegeli/chunk_encoding:decompressor\",\n        \"//riegeli/chunk_encoding:field_projection\",\n        \"//riegeli/chunk_encoding:transpose_decoder\",\n        \"//riegeli/lines:line_writing\",\n        \"//riegeli/lines:text_writer\",\n        \"//riegeli/messages:parse_message\",\n        \"//riegeli/messages:text_print_message\",\n        \"//riegeli/records:chunk_reader\",\n        \"//riegeli/records:records_metadata_cc_proto\",\n        \"//riegeli/records:skipped_region\",\n        \"//riegeli/varint:varint_reading\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/cleanup\",\n        \"@com_google_absl//absl/flags:flag\",\n        \"@com_google_absl//absl/flags:parse\",\n        \"@com_google_absl//absl/flags:usage\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_protobuf//:protobuf_lite\",\n    ],\n)\n\nproto_library(\n    name = \"riegeli_summary_proto\",\n    srcs = [\"riegeli_summary.proto\"],\n    deps = [\"//riegeli/records:records_metadata_proto\"],\n)\n\ncc_proto_library(\n    name = \"riegeli_summary_cc_proto\",\n    deps = [\":riegeli_summary_proto\"],\n)\n\ncc_binary(\n    name = \"records_benchmark\",\n    srcs = [\"records_benchmark.cc\"],\n    deps = [\n        \":tfrecord_recognizer\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:options_parser\",\n        \"//riegeli/bytes:fd_reader\",\n        \"//riegeli/bytes:fd_writer\",\n        \"//riegeli/bytes:std_io\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/lines:line_writing\",\n        \"//riegeli/lines:newline\",\n        \"//riegeli/lines:text_writer\",\n        \"//riegeli/records:chunk_reader\",\n        \"//riegeli/records:record_reader\",\n        \"//riegeli/records:record_writer\",\n        \"//riegeli/text:ascii_align\",\n        \"//riegeli/varint:varint_writing\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/flags:flag\",\n        \"@com_google_absl//absl/flags:parse\",\n        \"@com_google_absl//absl/flags:usage\",\n        \"@com_google_absl//absl/functional:function_ref\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:str_format\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n        \"@local_config_tf//:libtensorflow_framework\",\n        \"@local_config_tf//:tf_header_lib\",\n    ],\n)\n\ncc_library(\n    name = \"tfrecord_recognizer\",\n    srcs = [\"tfrecord_recognizer.cc\"],\n    hdrs = [\"tfrecord_recognizer.h\"],\n    deps = [\n        \"//riegeli/base:any\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/endian:endian_reading\",\n        \"//riegeli/zlib:zlib_reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@local_config_tf//:tf_header_lib\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/records/tools/describe_riegeli_file.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <optional>\n#include <string>\n#include <vector>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/cleanup/cleanup.h\"\n#include \"absl/flags/flag.h\"\n#include \"absl/flags/parse.h\"\n#include \"absl/flags/usage.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/escaping.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"google/protobuf/repeated_ptr_field.h\"\n#include \"riegeli/base/any.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/chain_backward_writer.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/fd_reader.h\"\n#include \"riegeli/bytes/limiting_reader.h\"\n#include \"riegeli/bytes/null_backward_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/std_io.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/chunk_encoding/chunk.h\"\n#include \"riegeli/chunk_encoding/constants.h\"\n#include \"riegeli/chunk_encoding/decompressor.h\"\n#include \"riegeli/chunk_encoding/field_projection.h\"\n#include \"riegeli/chunk_encoding/transpose_decoder.h\"\n#include \"riegeli/lines/line_writing.h\"\n#include \"riegeli/lines/text_writer.h\"\n#include \"riegeli/messages/parse_message.h\"\n#include \"riegeli/messages/text_print_message.h\"\n#include \"riegeli/records/chunk_reader.h\"\n#include \"riegeli/records/records_metadata.pb.h\"\n#include \"riegeli/records/skipped_region.h\"\n#include \"riegeli/records/tools/riegeli_summary.pb.h\"\n#include \"riegeli/varint/varint_reading.h\"\n\nABSL_FLAG(bool, show_records_metadata, false,\n          \"If true, show parsed file metadata.\");\nABSL_FLAG(bool, show_record_sizes, false,\n          \"If true, show the list of record sizes in each chunk.\");\nABSL_FLAG(bool, show_records, false,\n          \"If true, show contents of records in each chunk.\");\n\nnamespace riegeli::tools {\nnamespace {\n\nabsl::Status DescribeFileMetadataChunk(const Chunk& chunk,\n                                       RecordsMetadata& records_metadata) {\n  // Based on `RecordReaderBase::ParseMetadata()`.\n  if (ABSL_PREDICT_FALSE(chunk.header.num_records() != 0)) {\n    return absl::InvalidArgumentError(absl::StrCat(\n        \"Invalid file metadata chunk: number of records is not zero: \",\n        chunk.header.num_records()));\n  }\n  ChainReader data_reader(&chunk.data);\n  TransposeDecoder transpose_decoder;\n  ChainBackwardWriter serialized_metadata_writer;\n  serialized_metadata_writer.SetWriteSizeHint(chunk.header.decoded_data_size());\n  std::vector<size_t> limits;\n  const bool decode_ok = transpose_decoder.Decode(\n      1, chunk.header.decoded_data_size(), FieldProjection::All(), data_reader,\n      serialized_metadata_writer, limits);\n  if (ABSL_PREDICT_FALSE(!serialized_metadata_writer.Close())) {\n    return serialized_metadata_writer.status();\n  }\n  if (ABSL_PREDICT_FALSE(!decode_ok)) return transpose_decoder.status();\n  if (ABSL_PREDICT_FALSE(!data_reader.VerifyEndAndClose())) {\n    return data_reader.status();\n  }\n  const Chain& serialized_metadata = serialized_metadata_writer.dest();\n  RIEGELI_ASSERT_EQ(limits.size(), 1u)\n      << \"Metadata chunk has unexpected record limits\";\n  RIEGELI_ASSERT_EQ(limits.back(), serialized_metadata.size())\n      << \"Metadata chunk has unexpected record limits\";\n  return ParseMessage(serialized_metadata, records_metadata);\n}\n\nabsl::Status ReadRecords(\n    Reader& src, const std::vector<size_t>& limits,\n    google::protobuf::RepeatedPtrField<std::string>& dest) {\n  // Based on `ChunkDecoder::ReadRecord()`.\n  const Position initial_pos = src.pos();\n  for (const size_t limit : limits) {\n    const size_t start = IntCast<size_t>(src.pos() - initial_pos);\n    RIEGELI_ASSERT_LE(start, limit) << \"Record end positions not sorted\";\n    if (ABSL_PREDICT_FALSE(!src.Read(limit - start, *dest.Add()))) {\n      return src.StatusOrAnnotate(\n          absl::InvalidArgumentError(\"Reading record failed\"));\n    }\n  }\n  return absl::OkStatus();\n}\n\nabsl::Status DescribeSimpleChunk(const Chunk& chunk,\n                                 summary::SimpleChunk& simple_chunk) {\n  ChainReader src(&chunk.data);\n  const bool show_record_sizes = absl::GetFlag(FLAGS_show_record_sizes);\n  const bool show_records = absl::GetFlag(FLAGS_show_records);\n\n  // Based on `SimpleDecoder::Decode()`.\n  uint8_t compression_type_byte;\n  if (ABSL_PREDICT_FALSE(!src.ReadByte(compression_type_byte))) {\n    return absl::InvalidArgumentError(\"Reading compression type failed\");\n  }\n  const CompressionType compression_type =\n      static_cast<CompressionType>(compression_type_byte);\n  simple_chunk.set_compression_type(\n      static_cast<summary::CompressionType>(compression_type));\n\n  if (show_record_sizes || show_records) {\n    uint64_t sizes_size;\n    if (ABSL_PREDICT_FALSE(!ReadVarint64(src, sizes_size))) {\n      return absl::InvalidArgumentError(\"Reading size of sizes failed\");\n    }\n\n    chunk_encoding_internal::Decompressor<LimitingReader<>> sizes_decompressor(\n        riegeli::Maker(\n            &src, LimitingReaderBase::Options().set_exact_length(sizes_size)),\n        compression_type);\n    if (ABSL_PREDICT_FALSE(!sizes_decompressor.ok())) {\n      return sizes_decompressor.status();\n    }\n    if (show_record_sizes) {\n      if (ABSL_PREDICT_FALSE(chunk.header.num_records() >\n                             unsigned{std::numeric_limits<int>::max()})) {\n        return absl::ResourceExhaustedError(\"Too many records\");\n      }\n      simple_chunk.mutable_record_sizes()->Reserve(\n          IntCast<int>(chunk.header.num_records()));\n    }\n    std::vector<size_t> limits;\n    if (show_records) {\n      if (ABSL_PREDICT_FALSE(chunk.header.num_records() > limits.max_size())) {\n        return absl::ResourceExhaustedError(\"Too many records\");\n      }\n      limits.reserve(IntCast<size_t>(chunk.header.num_records()));\n    }\n    size_t limit = 0;\n    for (uint64_t i = 0; i < chunk.header.num_records(); ++i) {\n      uint64_t size;\n      if (ABSL_PREDICT_FALSE(\n              !ReadVarint64(sizes_decompressor.reader(), size))) {\n        return sizes_decompressor.reader().StatusOrAnnotate(\n            absl::InvalidArgumentError(\"Reading record size failed\"));\n      }\n      if (ABSL_PREDICT_FALSE(size > chunk.header.decoded_data_size() - limit)) {\n        return absl::InvalidArgumentError(\n            \"Decoded data size larger than expected\");\n      }\n      limit += IntCast<size_t>(size);\n      if (show_record_sizes) simple_chunk.add_record_sizes(size);\n      if (show_records) limits.push_back(limit);\n    }\n    if (ABSL_PREDICT_FALSE(!sizes_decompressor.VerifyEndAndClose())) {\n      return sizes_decompressor.status();\n    }\n    if (ABSL_PREDICT_FALSE(limit != chunk.header.decoded_data_size())) {\n      return absl::InvalidArgumentError(\n          \"Decoded data size smaller than expected\");\n    }\n\n    if (show_records) {\n      chunk_encoding_internal::Decompressor<> records_decompressor(\n          &src, compression_type);\n      if (ABSL_PREDICT_FALSE(!records_decompressor.ok())) {\n        return records_decompressor.status();\n      }\n      if (absl::Status status =\n              ReadRecords(records_decompressor.reader(), limits,\n                          *simple_chunk.mutable_records());\n          !status.ok()) {\n        return status;\n      }\n      if (ABSL_PREDICT_FALSE(!records_decompressor.VerifyEndAndClose())) {\n        return records_decompressor.status();\n      }\n      if (ABSL_PREDICT_FALSE(!src.VerifyEndAndClose())) return src.status();\n    }\n  }\n  return absl::OkStatus();\n}\n\nabsl::Status DescribeTransposedChunk(\n    const Chunk& chunk, summary::TransposedChunk& transposed_chunk) {\n  ChainReader src(&chunk.data);\n  const bool show_record_sizes = absl::GetFlag(FLAGS_show_record_sizes);\n  const bool show_records = absl::GetFlag(FLAGS_show_records);\n\n  // Based on `TransposeDecoder::Decode()`.\n  uint8_t compression_type_byte;\n  if (ABSL_PREDICT_FALSE(!src.ReadByte(compression_type_byte))) {\n    return absl::InvalidArgumentError(\"Reading compression type failed\");\n  }\n  transposed_chunk.set_compression_type(\n      static_cast<summary::CompressionType>(compression_type_byte));\n\n  if (show_record_sizes || show_records) {\n    // Based on `ChunkDecoder::Parse()`.\n    src.Seek(0);\n    TransposeDecoder transpose_decoder;\n    Chain dest;\n    Any<BackwardWriter*>::Inlining<ChainBackwardWriter<>, NullBackwardWriter>\n        dest_writer;\n    if (show_records) {\n      dest_writer = riegeli::Maker<ChainBackwardWriter>(&dest);\n    } else {\n      dest_writer = riegeli::Maker<NullBackwardWriter>();\n    }\n    dest_writer->SetWriteSizeHint(chunk.header.decoded_data_size());\n    std::vector<size_t> limits;\n    const bool decode_ok = transpose_decoder.Decode(\n        chunk.header.num_records(), chunk.header.decoded_data_size(),\n        FieldProjection::All(), src, *dest_writer, limits);\n    if (ABSL_PREDICT_FALSE(!dest_writer->Close())) return dest_writer->status();\n    if (ABSL_PREDICT_FALSE(!decode_ok)) return transpose_decoder.status();\n    if (show_record_sizes) {\n      if (ABSL_PREDICT_FALSE(limits.size() >\n                             unsigned{std::numeric_limits<int>::max()})) {\n        return absl::ResourceExhaustedError(\"Too many records\");\n      }\n      transposed_chunk.mutable_record_sizes()->Reserve(\n          IntCast<int>(limits.size()));\n      size_t prev_limit = 0;\n      for (const size_t next_limit : limits) {\n        RIEGELI_ASSERT_LE(prev_limit, next_limit)\n            << \"Failed postcondition of TransposeDecoder: \"\n               \"record end positions not sorted\";\n        transposed_chunk.add_record_sizes(next_limit - prev_limit);\n        prev_limit = next_limit;\n      }\n    }\n\n    if (show_records) {\n      ChainReader records_reader(&dest);\n      if (absl::Status status = ReadRecords(\n              records_reader, limits, *transposed_chunk.mutable_records());\n          !status.ok()) {\n        return status;\n      }\n      if (ABSL_PREDICT_FALSE(!records_reader.VerifyEndAndClose())) {\n        return records_reader.status();\n      }\n    }\n    if (ABSL_PREDICT_FALSE(!src.VerifyEndAndClose())) return src.status();\n  }\n  return absl::OkStatus();\n}\n\nvoid DescribeFile(absl::string_view filename, Writer& report) {\n  WriteLine(\"file {\", report);\n  WriteLine(\"  filename: \\\"\", absl::Utf8SafeCEscape(filename), '\"', report);\n  DefaultChunkReader chunk_reader(riegeli::Maker<FdReader>(filename));\n  if (chunk_reader.SupportsRandomAccess()) {\n    const std::optional<Position> size = chunk_reader.Size();\n    if (size != std::nullopt) WriteLine(\"  file_size: \", *size, report);\n  }\n  TextPrintMessageOptions print_options;\n  print_options.printer().SetInitialIndentLevel(2);\n  print_options.printer().SetUseShortRepeatedPrimitives(true);\n  print_options.printer().SetUseUtf8StringEscaping(true);\n  for (;;) {\n    report.Flush();\n\n    const ChunkHeader* chunk_header;\n    if (ABSL_PREDICT_FALSE(!chunk_reader.PullChunkHeader(&chunk_header))) {\n      SkippedRegion skipped_region;\n      if (chunk_reader.Recover(&skipped_region)) {\n        WriteLine(\"  # FILE CORRUPTED: \", skipped_region.ToString(), report);\n        continue;\n      }\n      break;\n    }\n\n    summary::Chunk chunk_summary;\n    chunk_summary.set_chunk_begin(chunk_reader.pos());\n    chunk_summary.set_chunk_type(\n        static_cast<summary::ChunkType>(chunk_header->chunk_type()));\n    chunk_summary.set_data_size(chunk_header->data_size());\n    chunk_summary.set_num_records(chunk_header->num_records());\n    chunk_summary.set_decoded_data_size(chunk_header->decoded_data_size());\n\n    WriteLine(\"  chunk {\", report);\n    absl::Cleanup report_chunk = [&] {\n      TextPrintMessage(chunk_summary, TextWriter(&report), print_options)\n          .IgnoreError();\n      WriteLine(\"  }\", report);\n    };\n\n    Chunk chunk;\n    if (ABSL_PREDICT_FALSE(!chunk_reader.ReadChunk(chunk))) {\n      SkippedRegion skipped_region;\n      if (chunk_reader.Recover(&skipped_region)) {\n        WriteLine(\"    # FILE CORRUPTED: \", skipped_region.ToString(), report);\n        continue;\n      }\n      break;\n    }\n\n    absl::Status status;\n    switch (chunk_summary.chunk_type()) {\n      case summary::FILE_METADATA:\n        if (absl::GetFlag(FLAGS_show_records_metadata)) {\n          status = DescribeFileMetadataChunk(\n              chunk, *chunk_summary.mutable_file_metadata_chunk());\n        }\n        break;\n      case summary::SIMPLE:\n        status =\n            DescribeSimpleChunk(chunk, *chunk_summary.mutable_simple_chunk());\n        break;\n      case summary::TRANSPOSED:\n        status = DescribeTransposedChunk(\n            chunk, *chunk_summary.mutable_transposed_chunk());\n        break;\n      default:\n        break;\n    }\n    if (ABSL_PREDICT_FALSE(!status.ok())) {\n      WriteLine(\"    # FILE CORRUPTED: \",\n                Annotate(chunk_reader.AnnotateStatus(status),\n                         absl::StrCat(\"at record \", chunk_summary.chunk_begin(),\n                                      \"/0\"))\n                    .message(),\n                report);\n    }\n  }\n  if (!chunk_reader.Close()) {\n    WriteLine(\"  # FILE READ ERROR: \", chunk_reader.status().message(), report);\n  }\n  WriteLine('}', report);\n  report.Flush();\n}\n\nconst char kUsage[] =\n    \"Usage: describe_riegeli_file (OPTION|FILE)...\\n\"\n    \"\\n\"\n    \"Shows summary of Riegeli/records file contents.\\n\";\n\n}  // namespace\n}  // namespace riegeli::tools\n\nint main(int argc, char** argv) {\n  absl::SetProgramUsageMessage(riegeli::tools::kUsage);\n  const std::vector<char*> args = absl::ParseCommandLine(argc, argv);\n  riegeli::StdOut std_out;\n  for (size_t i = 1; i < args.size(); ++i) {\n    riegeli::tools::DescribeFile(args[i], std_out);\n  }\n  std_out.Close();\n}\n"
  },
  {
    "path": "riegeli/records/tools/records_benchmark.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// Make file offsets 64-bit even on 32-bit systems.\n#undef _FILE_OFFSET_BITS\n#define _FILE_OFFSET_BITS 64\n\n#include <stddef.h>\n#include <stdint.h>\n#include <sys/stat.h>\n#include <sys/time.h>\n#include <sys/types.h>\n#include <time.h>\n\n#include <algorithm>\n#include <array>\n#include <cerrno>\n#include <cstdlib>\n#include <memory>\n#include <string>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/flags/flag.h\"\n#include \"absl/flags/parse.h\"\n#include \"absl/flags/usage.h\"\n#include \"absl/functional/function_ref.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/str_format.h\"\n#include \"absl/strings/str_split.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/options_parser.h\"\n#include \"riegeli/bytes/fd_reader.h\"\n#include \"riegeli/bytes/fd_writer.h\"\n#include \"riegeli/bytes/std_io.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/lines/line_writing.h\"\n#include \"riegeli/lines/newline.h\"\n#include \"riegeli/lines/text_writer.h\"\n#include \"riegeli/records/chunk_reader.h\"\n#include \"riegeli/records/record_reader.h\"\n#include \"riegeli/records/record_writer.h\"\n#include \"riegeli/records/tools/tfrecord_recognizer.h\"\n#include \"riegeli/text/ascii_align.h\"\n#include \"riegeli/varint/varint_writing.h\"\n#include \"tensorflow/core/lib/io/compression.h\"\n#include \"tensorflow/core/lib/io/record_reader.h\"\n#include \"tensorflow/core/lib/io/record_writer.h\"\n#include \"tensorflow/core/platform/env.h\"\n#include \"tensorflow/core/platform/file_system.h\"\n#include \"tensorflow/core/platform/tstring.h\"\n\nABSL_FLAG(std::string, tfrecord_benchmarks, \"uncompressed gzip\",\n          \"Whitespace-separated TFRecord RecordWriter/RecordReader options\");\nABSL_FLAG(std::string, riegeli_benchmarks,\n          \"uncompressed \"\n          \"brotli:0 \"\n          \"brotli:6 \"\n          \"brotli:6,parallelism:10 \"\n          \"brotli:9 \"\n          \"zstd:1 \"\n          \"zstd:3 \"\n          \"zstd:15 \"\n          \"snappy \"\n          \"snappy:2 \"\n          \"transpose,uncompressed \"\n          \"transpose,brotli:6 \"\n          \"transpose,brotli:6,parallelism:10 \"\n          \"transpose,zstd:3 \"\n          \"transpose,snappy\",\n          \"Whitespace-separated Riegeli RecordWriter options\");\nABSL_FLAG(uint64_t, max_size, uint64_t{100} * 1000 * 1000,\n          \"Maximum size of records to read, in bytes\");\nABSL_FLAG(std::string, output_dir, \"/tmp\",\n          \"Directory to write files to (files are named record_benchmark_*)\");\nABSL_FLAG(int32_t, repetitions, 5, \"Number of times to repeat each benchmark\");\n\nnamespace {\n\nclass SizeLimiter {\n public:\n  explicit SizeLimiter(size_t limit);\n\n  bool Accept(size_t size);\n\n private:\n  size_t limit_;\n  size_t remaining_;\n};\n\nSizeLimiter::SizeLimiter(size_t limit) : limit_(limit), remaining_(limit) {}\n\nbool SizeLimiter::Accept(size_t size) {\n  if (ABSL_PREDICT_TRUE(size < remaining_)) {\n    remaining_ -= size;\n    return true;\n  }\n  if (remaining_ == limit_) {\n    // Return at least one item.\n    remaining_ = 0;\n    return true;\n  }\n  return false;\n}\n\nuint64_t FileSize(const std::string& filename) {\n  struct stat stat_info;\n  RIEGELI_CHECK_EQ(stat(filename.c_str(), &stat_info), 0)\n      << absl::ErrnoToStatus(errno, \"stat() failed\").message();\n  return riegeli::IntCast<uint64_t>(stat_info.st_size);\n}\n\nuint64_t CpuTimeNow_ns() {\n  struct timespec time_info;\n  RIEGELI_CHECK_EQ(clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &time_info), 0);\n  return riegeli::IntCast<uint64_t>(time_info.tv_sec) * uint64_t{1000000000} +\n         riegeli::IntCast<uint64_t>(time_info.tv_nsec);\n}\n\nuint64_t RealTimeNow_ns() {\n  struct timespec time_info;\n  RIEGELI_CHECK_EQ(clock_gettime(CLOCK_MONOTONIC, &time_info), 0);\n  return riegeli::IntCast<uint64_t>(time_info.tv_sec) * uint64_t{1000000000} +\n         riegeli::IntCast<uint64_t>(time_info.tv_nsec);\n}\n\nclass Stats {\n public:\n  void Add(double value);\n\n  double Median();\n\n private:\n  std::vector<double> samples_;\n};\n\nvoid Stats::Add(double value) { samples_.push_back(value); }\n\ndouble Stats::Median() {\n  RIEGELI_CHECK(!samples_.empty()) << \"No data\";\n  const size_t middle = samples_.size() / 2;\n  std::nth_element(samples_.begin(), samples_.begin() + middle, samples_.end());\n  return samples_[middle];\n}\n\nclass Benchmarks {\n public:\n  static bool ReadFile(absl::string_view filename,\n                       std::vector<std::string>* records,\n                       SizeLimiter* size_limiter, riegeli::Writer& report);\n\n  explicit Benchmarks(std::vector<std::string> records, std::string output_dir,\n                      int repetitions);\n\n  void RegisterTFRecord(absl::string_view tfrecord_options);\n  void RegisterRiegeli(absl::string_view riegeli_options);\n\n  void RunAll(riegeli::Writer& report);\n\n private:\n  static void WriteTFRecord(\n      absl::string_view filename,\n      const tensorflow::io::RecordWriterOptions& record_writer_options,\n      absl::Span<const std::string> records);\n  static bool ReadTFRecord(\n      absl::string_view filename,\n      const tensorflow::io::RecordReaderOptions& record_reader_options,\n      std::vector<std::string>* records, SizeLimiter* size_limiter = nullptr);\n\n  static void WriteRiegeli(\n      absl::string_view filename,\n      riegeli::RecordWriterBase::Options record_writer_options,\n      absl::Span<const std::string> records);\n  static bool ReadRiegeli(\n      absl::string_view filename,\n      riegeli::RecordReaderBase::Options record_reader_options,\n      std::vector<std::string>* records, SizeLimiter* size_limiter = nullptr);\n\n  void RunOne(\n      absl::string_view name,\n      absl::FunctionRef<void(absl::string_view, absl::Span<const std::string>)>\n          write_records,\n      absl::FunctionRef<void(absl::string_view, std::vector<std::string>*)>\n          read_records,\n      riegeli::Writer& report);\n\n  static std::string Filename(absl::string_view name);\n\n  std::vector<std::string> records_;\n  size_t original_size_;\n  std::string output_dir_;\n  int repetitions_;\n  std::vector<std::pair<std::string, const char*>> tfrecord_benchmarks_;\n  std::vector<std::pair<std::string, riegeli::RecordWriterBase::Options>>\n      riegeli_benchmarks_;\n  size_t max_name_width_ = 0;\n};\n\nbool Benchmarks::ReadFile(absl::string_view filename,\n                          std::vector<std::string>* records,\n                          SizeLimiter* size_limiter, riegeli::Writer& report) {\n  riegeli::FdReader<> file_reader(filename);\n  if (ABSL_PREDICT_FALSE(!file_reader.ok())) {\n    riegeli::StdErr errors;\n    riegeli::WriteLine(\"Could not open file: \", file_reader.status().ToString(),\n                       errors);\n    errors.Close();\n    std::exit(1);\n  }\n  {\n    riegeli::TFRecordRecognizer tfrecord_recognizer(&file_reader);\n    tensorflow::io::RecordReaderOptions record_reader_options;\n    if (tfrecord_recognizer.CheckFileFormat(record_reader_options)) {\n      RIEGELI_CHECK(tfrecord_recognizer.Close())\n          << tfrecord_recognizer.status();\n      RIEGELI_CHECK(file_reader.Close()) << file_reader.status();\n      riegeli::WriteLine(\"Reading TFRecord: \", filename, report);\n      report.Flush();\n      return ReadTFRecord(filename, record_reader_options, records,\n                          size_limiter);\n    }\n  }\n  RIEGELI_CHECK(file_reader.Seek(0)) << file_reader.status();\n  {\n    riegeli::DefaultChunkReader<> chunk_reader(&file_reader);\n    if (chunk_reader.CheckFileFormat()) {\n      RIEGELI_CHECK(chunk_reader.Close()) << chunk_reader.status();\n      RIEGELI_CHECK(file_reader.Close()) << file_reader.status();\n      riegeli::WriteLine(\"Reading Riegeli/records: \", filename, report);\n      report.Flush();\n      return ReadRiegeli(filename, riegeli::RecordReaderBase::Options(),\n                         records, size_limiter);\n    }\n  }\n  riegeli::StdErr errors;\n  riegeli::WriteLine(\"Unknown file format: \", filename, errors);\n  errors.Close();\n  std::exit(1);\n}\n\nvoid Benchmarks::WriteTFRecord(\n    absl::string_view filename,\n    const tensorflow::io::RecordWriterOptions& record_writer_options,\n    absl::Span<const std::string> records) {\n  tensorflow::Env* const env = tensorflow::Env::Default();\n  std::unique_ptr<tensorflow::WritableFile> file_writer;\n  {\n    const absl::Status status =\n        env->NewWritableFile(std::string(filename), &file_writer);\n    RIEGELI_CHECK_OK(status);\n  }\n  tensorflow::io::RecordWriter record_writer(file_writer.get(),\n                                             record_writer_options);\n  for (const absl::string_view record : records) {\n    const absl::Status status = record_writer.WriteRecord(record);\n    RIEGELI_CHECK_OK(status);\n  }\n  const absl::Status status = record_writer.Close();\n  RIEGELI_CHECK_OK(status);\n}\n\nbool Benchmarks::ReadTFRecord(\n    absl::string_view filename,\n    const tensorflow::io::RecordReaderOptions& record_reader_options,\n    std::vector<std::string>* records, SizeLimiter* size_limiter) {\n  tensorflow::Env* const env = tensorflow::Env::Default();\n  std::unique_ptr<tensorflow::RandomAccessFile> file_reader;\n  {\n    const absl::Status status =\n        env->NewRandomAccessFile(std::string(filename), &file_reader);\n    RIEGELI_CHECK_OK(status);\n  }\n  tensorflow::io::SequentialRecordReader record_reader(file_reader.get(),\n                                                       record_reader_options);\n  tensorflow::tstring record;\n  for (;;) {\n    const absl::Status status = record_reader.ReadRecord(&record);\n    if (!status.ok()) {\n      RIEGELI_CHECK(absl::IsOutOfRange(status)) << status;\n      break;\n    }\n    if (size_limiter != nullptr &&\n        ABSL_PREDICT_FALSE(!size_limiter->Accept(\n            riegeli::LengthVarint64(record.size()) + record.size()))) {\n      return false;\n    }\n    records->push_back(std::move(record));\n  }\n  return true;\n}\n\nvoid Benchmarks::WriteRiegeli(\n    absl::string_view filename,\n    riegeli::RecordWriterBase::Options record_writer_options,\n    absl::Span<const std::string> records) {\n  riegeli::RecordWriter<riegeli::FdWriter<>> record_writer(\n      riegeli::Maker(filename), std::move(record_writer_options));\n  for (const absl::string_view record : records) {\n    RIEGELI_CHECK(record_writer.WriteRecord(record)) << record_writer.status();\n  }\n  RIEGELI_CHECK(record_writer.Close()) << record_writer.status();\n}\n\nbool Benchmarks::ReadRiegeli(\n    absl::string_view filename,\n    riegeli::RecordReaderBase::Options record_reader_options,\n    std::vector<std::string>* records, SizeLimiter* size_limiter) {\n  riegeli::RecordReader<riegeli::FdReader<>> record_reader(\n      riegeli::Maker(filename), std::move(record_reader_options));\n  std::string record;\n  while (record_reader.ReadRecord(record)) {\n    if (size_limiter != nullptr &&\n        ABSL_PREDICT_FALSE(!size_limiter->Accept(\n            riegeli::LengthVarint64(record.size()) + record.size()))) {\n      return false;\n    }\n    records->push_back(std::move(record));\n  }\n  RIEGELI_CHECK(record_reader.Close()) << record_reader.status();\n  return true;\n}\n\nstd::string Benchmarks::Filename(absl::string_view name) {\n  std::string filename(name);\n  for (char& ch : filename) {\n    if (!(ch == '-' || ch == '.' || (ch >= '0' && ch <= '9') ||\n          (ch >= 'A' && ch <= 'Z') || ch == '_' || (ch >= 'a' && ch <= 'z'))) {\n      ch = '_';\n    }\n  }\n  return filename;\n}\n\nBenchmarks::Benchmarks(std::vector<std::string> records, std::string output_dir,\n                       int repetitions)\n    : records_(std::move(records)),\n      original_size_(0),\n      output_dir_(std::move(output_dir)),\n      repetitions_(repetitions) {\n  for (const absl::string_view record : records_) {\n    original_size_ += riegeli::LengthVarint64(record.size()) + record.size();\n  }\n}\n\nvoid Benchmarks::RegisterTFRecord(absl::string_view tfrecord_options) {\n  max_name_width_ = riegeli::UnsignedMax(\n      max_name_width_,\n      absl::string_view(\"tfrecord \").size() + tfrecord_options.size());\n  const char* compression = tensorflow::io::compression::kNone;\n  riegeli::OptionsParser options_parser;\n  options_parser.AddOption(\n      \"uncompressed\",\n      riegeli::ValueParser::And(\n          riegeli::ValueParser::FailIfSeen(\"gzip\"),\n          riegeli::ValueParser::Empty(tensorflow::io::compression::kNone,\n                                      &compression)));\n  options_parser.AddOption(\n      \"gzip\", riegeli::ValueParser::And(\n                  riegeli::ValueParser::FailIfSeen(\"uncompressed\"),\n                  riegeli::ValueParser::Empty(\n                      tensorflow::io::compression::kGzip, &compression)));\n  RIEGELI_CHECK(options_parser.FromString(tfrecord_options))\n      << options_parser.status();\n  tfrecord_benchmarks_.emplace_back(tfrecord_options, compression);\n}\n\nvoid Benchmarks::RegisterRiegeli(absl::string_view riegeli_options) {\n  max_name_width_ = riegeli::UnsignedMax(\n      max_name_width_,\n      absl::string_view(\"riegeli \").size() + riegeli_options.size());\n  riegeli::RecordWriterBase::Options options;\n  RIEGELI_CHECK_OK(options.FromString(riegeli_options));\n  riegeli_benchmarks_.emplace_back(riegeli_options, std::move(options));\n}\n\nvoid Benchmarks::RunAll(riegeli::Writer& report) {\n  report.Write(\"Original uncompressed size: \");\n  absl::Format(&report, \"%.3f\",\n               static_cast<double>(original_size_) / 1000000.0);\n  riegeli::WriteLine(\" MB\", report);\n  riegeli::WriteLine(\"Creating files \", output_dir_, \"/record_benchmark_*\",\n                     report);\n  riegeli::WriteLine(riegeli::AsciiLeft(max_name_width_),\n                     \"  Compr.    Write       Read\", report);\n  riegeli::WriteLine(riegeli::AsciiLeft(max_name_width_),\n                     \"  ratio    CPU Real   CPU Real\", report);\n  riegeli::WriteLine(riegeli::AsciiLeft(max_name_width_),\n                     \"    %     MB/s MB/s  MB/s MB/s\", report);\n  riegeli::WriteLine(riegeli::AsciiLeft(riegeli::AlignOptions()\n                                            .set_width(max_name_width_ + 30)\n                                            .set_fill('-')),\n                     report);\n\n  for (const std::pair<std::string, const char*>& tfrecord_options :\n       tfrecord_benchmarks_) {\n    RunOne(\n        absl::StrCat(\"tfrecord \", tfrecord_options.first),\n        [&](absl::string_view filename, absl::Span<const std::string> records) {\n          WriteTFRecord(\n              filename,\n              tensorflow::io::RecordWriterOptions::CreateRecordWriterOptions(\n                  tfrecord_options.second),\n              records);\n        },\n        [&](absl::string_view filename, std::vector<std::string>* records) {\n          return ReadTFRecord(\n              filename,\n              tensorflow::io::RecordReaderOptions::CreateRecordReaderOptions(\n                  tfrecord_options.second),\n              records);\n        },\n        report);\n  }\n  for (const std::pair<std::string, riegeli::RecordWriterBase::Options>&\n           riegeli_options : riegeli_benchmarks_) {\n    RunOne(\n        absl::StrCat(\"riegeli \", riegeli_options.first),\n        [&](absl::string_view filename, absl::Span<const std::string> records) {\n          WriteRiegeli(filename, riegeli_options.second, records);\n        },\n        [&](absl::string_view filename, std::vector<std::string>* records) {\n          return ReadRiegeli(filename, riegeli::RecordReaderBase::Options(),\n                             records);\n        },\n        report);\n  }\n}\n\nvoid Benchmarks::RunOne(\n    absl::string_view name,\n    absl::FunctionRef<void(absl::string_view, absl::Span<const std::string>)>\n        write_records,\n    absl::FunctionRef<void(absl::string_view, std::vector<std::string>*)>\n        read_records,\n    riegeli::Writer& report) {\n  report.Write(riegeli::AsciiLeft(name, max_name_width_), ' ');\n  report.Flush();\n  const std::string filename =\n      absl::StrCat(output_dir_, \"/record_benchmark_\", Filename(name));\n\n  Stats compression;\n  Stats writing_cpu_speed;\n  Stats writing_real_speed;\n  Stats reading_cpu_speed;\n  Stats reading_real_speed;\n  for (int i = 0; i < repetitions_ + 1; ++i) {\n    const uint64_t cpu_time_before_ns = CpuTimeNow_ns();\n    const uint64_t real_time_before_ns = RealTimeNow_ns();\n    write_records(filename, records_);\n    const uint64_t cpu_time_after_ns = CpuTimeNow_ns();\n    const uint64_t real_time_after_ns = RealTimeNow_ns();\n    if (i == 0) {\n      // Warm-up.\n    } else {\n      compression.Add(static_cast<double>(FileSize(filename)) /\n                      static_cast<double>(original_size_) * 100.0);\n      writing_cpu_speed.Add(\n          static_cast<double>(original_size_) /\n          static_cast<double>(cpu_time_after_ns - cpu_time_before_ns) * 1000.0);\n      writing_real_speed.Add(\n          static_cast<double>(original_size_) /\n          static_cast<double>(real_time_after_ns - real_time_before_ns) *\n          1000.0);\n    }\n  }\n  for (int i = 0; i < repetitions_ + 1; ++i) {\n    std::vector<std::string> decoded_records;\n    const uint64_t cpu_time_before_ns = CpuTimeNow_ns();\n    const uint64_t real_time_before_ns = RealTimeNow_ns();\n    read_records(filename, &decoded_records);\n    const uint64_t cpu_time_after_ns = CpuTimeNow_ns();\n    const uint64_t real_time_after_ns = RealTimeNow_ns();\n    if (i == 0) {\n      // Warm-up and correctness check.\n      RIEGELI_CHECK(decoded_records == records_)\n          << \"Decoded records do not match for \" << name;\n    } else {\n      reading_cpu_speed.Add(\n          static_cast<double>(original_size_) /\n          static_cast<double>(cpu_time_after_ns - cpu_time_before_ns) * 1000.0);\n      reading_real_speed.Add(\n          static_cast<double>(original_size_) /\n          static_cast<double>(real_time_after_ns - real_time_before_ns) *\n          1000.0);\n    }\n  }\n\n  absl::Format(&report, \"%7.3f\", compression.Median());\n  for (const std::array<Stats*, 2>& stats_cpu_real :\n       {std::array<Stats*, 2>{{&writing_cpu_speed, &writing_real_speed}},\n        std::array<Stats*, 2>{{&reading_cpu_speed, &reading_real_speed}}}) {\n    report.Write(' ');\n    for (Stats* const stats : stats_cpu_real) {\n      report.Write(' ');\n      absl::Format(&report, \"%4.0f\", stats->Median());\n    }\n  }\n  riegeli::WriteLine(report);\n}\n\nconst char kUsage[] =\n    \"Usage: records_benchmark (OPTION|FILE)...\\n\"\n    \"\\n\"\n    \"FILEs may be TFRecord or Riegeli/records files.\\n\";\n\ntemplate <typename Function>\nvoid ForEachWord(absl::string_view words, Function&& f) {\n  for (const absl::string_view word :\n       absl::StrSplit(words, absl::ByAnyChar(\"\\t\\n \"), absl::SkipEmpty())) {\n    f(word);\n  }\n}\n\n}  // namespace\n\nint main(int argc, char** argv) {\n  absl::SetProgramUsageMessage(kUsage);\n  const std::vector<char*> args = absl::ParseCommandLine(argc, argv);\n  std::vector<std::string> records;\n  if (args.size() <= 1) {\n    riegeli::TextWriter<riegeli::WriteNewline::kNative, riegeli::StdErr>\n        std_err(riegeli::Maker());\n    std_err.Write(kUsage, '\\n');\n    std_err.Close();\n    return 1;\n  }\n  riegeli::StdOut std_out;\n  SizeLimiter size_limiter(\n      riegeli::IntCast<size_t>(absl::GetFlag(FLAGS_max_size)));\n  for (size_t i = 1; i < args.size(); ++i) {\n    if (!Benchmarks::ReadFile(args[i], &records, &size_limiter, std_out)) break;\n  }\n  Benchmarks benchmarks(std::move(records), absl::GetFlag(FLAGS_output_dir),\n                        absl::GetFlag(FLAGS_repetitions));\n  ForEachWord(absl::GetFlag(FLAGS_tfrecord_benchmarks),\n              [&](absl::string_view tfrecord_options) {\n                benchmarks.RegisterTFRecord(tfrecord_options);\n              });\n  ForEachWord(absl::GetFlag(FLAGS_riegeli_benchmarks),\n              [&](absl::string_view riegeli_options) {\n                benchmarks.RegisterRiegeli(riegeli_options);\n              });\n  benchmarks.RunAll(std_out);\n  std_out.Close();\n}\n"
  },
  {
    "path": "riegeli/records/tools/riegeli_summary.proto",
    "content": "edition = \"2024\";\n\npackage riegeli.summary;\n\nimport \"riegeli/records/records_metadata.proto\";\n\n// Summary of a Riegeli/records file contents.\n//\n// This is currently used merely to format the output of describe_riegeli_file\n// as structured data.\n\nenum ChunkType {\n  CHUNK_TYPE_ZERO = 0;\n  FILE_SIGNATURE = 0x73;\n  FILE_METADATA = 0x6d;\n  PADDING = 0x70;\n  SIMPLE = 0x72;\n  TRANSPOSED = 0x74;\n}\n\nenum CompressionType {\n  NONE = 0;\n  BROTLI = 0x62;\n  ZSTD = 0x7a;\n  SNAPPY = 0x73;\n}\n\nmessage SimpleChunk {\n  CompressionType compression_type = 1;\n  repeated uint64 record_sizes = 2;\n  repeated bytes records = 3;\n}\n\nmessage TransposedChunk {\n  CompressionType compression_type = 1;\n  repeated uint64 record_sizes = 2;\n  repeated bytes records = 3;\n}\n\nmessage Chunk {\n  uint64 chunk_begin = 1;\n  ChunkType chunk_type = 2;\n  uint64 data_size = 3;\n  uint64 num_records = 4;\n  uint64 decoded_data_size = 5;\n\n  oneof data {\n    riegeli.RecordsMetadata file_metadata_chunk = 6;\n    SimpleChunk simple_chunk = 7;\n    TransposedChunk transposed_chunk = 8;\n  }\n}\n\n// This is not used because each chunk is printed on the fly, so that the output\n// appears incrementally.\n//\n// message File {\n//   string filename = 1;\n//   uint64 file_size = 2;\n//   repeated Chunk chunk = 3;\n// }\n"
  },
  {
    "path": "riegeli/records/tools/tfrecord_recognizer.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/records/tools/tfrecord_recognizer.h\"\n\n#include <stdint.h>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/any.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/endian/endian_reading.h\"\n#include \"riegeli/zlib/zlib_reader.h\"\n#include \"tensorflow/core/lib/hash/crc32c.h\"\n#include \"tensorflow/core/lib/io/record_reader.h\"\n\nnamespace riegeli {\n\nbool TFRecordRecognizer::CheckFileFormat(\n    tensorflow::io::RecordReaderOptions& record_reader_options) {\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!byte_reader_->Pull())) {\n    if (ABSL_PREDICT_FALSE(!byte_reader_->ok())) {\n      return Fail(byte_reader_->status());\n    }\n    // Empty file: return `false` but leave `ok()` as `true`. This mimics the\n    // behavior of reading functions at end of file.\n    return false;\n  }\n\n  Any<Reader*>::Inlining<ZlibReader<>> reader;\n  if (RecognizeZlib(*byte_reader_)) {\n    record_reader_options.compression_type =\n        tensorflow::io::RecordReaderOptions::ZLIB_COMPRESSION;\n    record_reader_options.zlib_options =\n        tensorflow::io::ZlibCompressionOptions::DEFAULT();\n    record_reader_options.zlib_options.window_bits = 32;\n    reader = riegeli::Maker<ZlibReader<>>(byte_reader_);\n  } else {\n    record_reader_options.compression_type =\n        tensorflow::io::RecordReaderOptions::NONE;\n    reader = byte_reader_;\n  }\n\n  if (ABSL_PREDICT_FALSE(!reader->Pull(sizeof(uint64_t) + sizeof(uint32_t)))) {\n    if (ABSL_PREDICT_FALSE(!reader->ok())) return Fail(reader->status());\n    return Fail(absl::InvalidArgumentError(\"Truncated TFRecord file\"));\n  }\n  if (tensorflow::crc32c::Unmask(\n          ReadLittleEndian<uint32_t>(reader->cursor() + sizeof(uint64_t))) !=\n      tensorflow::crc32c::Value(reader->cursor(), sizeof(uint64_t))) {\n    return Fail(absl::InvalidArgumentError(\"Corrupted TFRecord file\"));\n  }\n  return true;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/records/tools/tfrecord_recognizer.h",
    "content": "#include \"absl/base/attributes.h\"\n// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_RECORDS_TOOLS_TFRECORD_DETECTOR_H_\n#define RIEGELI_RECORDS_TOOLS_TFRECORD_DETECTOR_H_\n\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"tensorflow/core/lib/io/record_reader.h\"\n\nnamespace riegeli {\n\nclass TFRecordRecognizer : public Object {\n public:\n  explicit TFRecordRecognizer(\n      Reader* byte_reader ABSL_ATTRIBUTE_LIFETIME_BOUND);\n\n  // Ensures that the file looks like a valid TFRecord file.\n  //\n  // Updates `record_reader_options` on success.\n  //\n  // Return values:\n  //  * `true`                 - success (`record_reader_options` is updated)\n  //  * `false` (when `ok()`)  - source ends\n  //  * `false` (when `!ok()`) - failure\n  bool CheckFileFormat(\n      tensorflow::io::RecordReaderOptions& record_reader_options);\n\n private:\n  Reader* byte_reader_;\n};\n\n// Implementation details follow.\n\ninline TFRecordRecognizer::TFRecordRecognizer(\n    Reader* byte_reader ABSL_ATTRIBUTE_LIFETIME_BOUND)\n    : byte_reader_(RIEGELI_EVAL_ASSERT_NOTNULL(byte_reader)) {}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_RECORDS_TOOLS_TFRECORD_DETECTOR_H_\n"
  },
  {
    "path": "riegeli/snappy/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"snappy_reader\",\n    srcs = [\"snappy_reader.cc\"],\n    hdrs = [\"snappy_reader.h\"],\n    deps = [\n        \":snappy_streams\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:chain_reader\",\n        \"//riegeli/bytes:chain_writer\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/varint:varint_reading\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@snappy\",\n    ],\n)\n\ncc_library(\n    name = \"snappy_writer\",\n    srcs = [\"snappy_writer.cc\"],\n    hdrs = [\"snappy_writer.h\"],\n    deps = [\n        \":snappy_streams\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:chain_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/types:span\",\n        \"@snappy\",\n    ],\n)\n\ncc_library(\n    name = \"snappy_streams\",\n    srcs = [\"snappy_streams.cc\"],\n    hdrs = [\"snappy_streams.h\"],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@snappy\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/snappy/framed/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"framed_snappy_reader\",\n    srcs = [\"framed_snappy_reader.cc\"],\n    hdrs = [\"framed_snappy_reader.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffer\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:moving_dependency\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:pullable_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/endian:endian_reading\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/crc:crc32c\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@snappy\",\n    ],\n)\n\ncc_library(\n    name = \"framed_snappy_writer\",\n    srcs = [\"framed_snappy_writer.cc\"],\n    hdrs = [\"framed_snappy_writer.h\"],\n    deps = [\n        \":framed_snappy_reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffer\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:pushable_writer\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/digests:crc32c_digester\",\n        \"//riegeli/digests:digesting_writer\",\n        \"//riegeli/endian:endian_writing\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@snappy\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/snappy/framed/framed_snappy_reader.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/snappy/framed/framed_snappy_reader.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/crc/crc32c.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pullable_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/endian/endian_reading.h\"\n#include \"snappy.h\"\n\nnamespace riegeli {\n\nnamespace {\n\n// https://github.com/google/snappy/blob/e9e11b84e629c3e06fbaa4f0a86de02ceb9d6992/framing_format.txt#L39\ninline uint32_t MaskChecksum(uint32_t x) {\n  return ((x >> 15) | (x << 17)) + 0xa282ead8;\n}\n\n}  // namespace\n\nvoid FramedSnappyReaderBase::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of FramedSnappyReader: null Reader pointer\";\n  if (ABSL_PREDICT_FALSE(!src->ok()) && src->available() == 0) {\n    FailWithoutAnnotation(AnnotateOverSrc(src->status()));\n    return;\n  }\n  initial_compressed_pos_ = src->pos();\n}\n\nvoid FramedSnappyReaderBase::Done() {\n  if (ABSL_PREDICT_FALSE(truncated_)) {\n    Reader& src = *SrcReader();\n    FailWithoutAnnotation(\n        AnnotateOverSrc(src.AnnotateStatus(absl::InvalidArgumentError(\n            \"Truncated FramedSnappy-compressed stream\"))));\n  }\n  PullableReader::Done();\n  uncompressed_ = Buffer();\n}\n\ninline bool FramedSnappyReaderBase::FailInvalidStream(\n    absl::string_view message) {\n  return Fail(absl::InvalidArgumentError(\n      absl::StrCat(\"Invalid FramedSnappy-compressed stream: \", message)));\n}\n\nabsl::Status FramedSnappyReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    if (ABSL_PREDICT_FALSE(truncated_)) {\n      status =\n          Annotate(status, \"reading truncated FramedSnappy-compressed stream\");\n    }\n    Reader& src = *SrcReader();\n    status = src.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `src` with the compressed position.\n  // Clarify that the current position is the uncompressed position instead of\n  // delegating to `PullableReader::AnnotateStatusImpl()`.\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status FramedSnappyReaderBase::AnnotateOverSrc(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nbool FramedSnappyReaderBase::PullBehindScratch(size_t recommended_length) {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"some data available, use Pull() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  truncated_ = false;\n  while (src.Pull(sizeof(uint32_t))) {\n    const uint32_t chunk_header = ReadLittleEndian<uint32_t>(src.cursor());\n    const uint8_t chunk_type = static_cast<uint8_t>(chunk_header);\n    const size_t chunk_length = IntCast<size_t>(chunk_header >> 8);\n    if (ABSL_PREDICT_FALSE(!src.Pull(sizeof(uint32_t) + chunk_length))) {\n      set_buffer();\n      if (ABSL_PREDICT_FALSE(!src.ok())) {\n        return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n      }\n      truncated_ = true;\n      return false;\n    }\n    if (ABSL_PREDICT_FALSE(src.pos() == 0 &&\n                           chunk_type != 0xff /* Stream identifier */)) {\n      set_buffer();\n      return FailInvalidStream(\"missing stream identifier\");\n    }\n    switch (chunk_type) {\n      case 0x00: {  // Compressed data.\n        if (ABSL_PREDICT_FALSE(chunk_length < sizeof(uint32_t))) {\n          set_buffer();\n          return FailInvalidStream(\"compressed data too short\");\n        }\n        const uint32_t checksum =\n            ReadLittleEndian<uint32_t>(src.cursor() + sizeof(uint32_t));\n        const char* const compressed_data = src.cursor() + 2 * sizeof(uint32_t);\n        const size_t compressed_length = chunk_length - sizeof(uint32_t);\n        size_t uncompressed_length;\n        if (ABSL_PREDICT_FALSE(!snappy::GetUncompressedLength(\n                compressed_data, compressed_length, &uncompressed_length))) {\n          set_buffer();\n          return FailInvalidStream(\"invalid uncompressed length\");\n        }\n        if (ABSL_PREDICT_FALSE(uncompressed_length > snappy::kBlockSize)) {\n          set_buffer();\n          return FailInvalidStream(\"uncompressed length too large\");\n        }\n        uncompressed_.Reset(uncompressed_length);\n        if (ABSL_PREDICT_FALSE(!snappy::RawUncompress(\n                compressed_data, compressed_length, uncompressed_.data()))) {\n          set_buffer();\n          return FailInvalidStream(\"invalid compressed data\");\n        }\n        if (ABSL_PREDICT_FALSE(\n                MaskChecksum(static_cast<uint32_t>(absl::ComputeCrc32c(\n                    absl::string_view(uncompressed_.data(),\n                                      uncompressed_length)))) != checksum)) {\n          set_buffer();\n          return FailInvalidStream(\n              \"Invalid FramedSnappy-compressed stream: wrong checksum\");\n        }\n        src.move_cursor(sizeof(uint32_t) + chunk_length);\n        if (ABSL_PREDICT_FALSE(uncompressed_length == 0)) continue;\n        const Position max_length =\n            std::numeric_limits<Position>::max() - limit_pos();\n        if (ABSL_PREDICT_FALSE(uncompressed_length > max_length)) {\n          set_buffer(uncompressed_.data(), IntCast<size_t>(max_length));\n          move_limit_pos(available());\n          return FailOverflow();\n        }\n        set_buffer(uncompressed_.data(), uncompressed_length);\n        move_limit_pos(available());\n        return true;\n      }\n      case 0x01: {  // Uncompressed data.\n        if (ABSL_PREDICT_FALSE(chunk_length < sizeof(uint32_t))) {\n          set_buffer();\n          return FailInvalidStream(\"uncompressed data too short\");\n        }\n        const uint32_t checksum =\n            ReadLittleEndian<uint32_t>(src.cursor() + sizeof(uint32_t));\n        const char* const uncompressed_data =\n            src.cursor() + 2 * sizeof(uint32_t);\n        const size_t uncompressed_length = chunk_length - sizeof(uint32_t);\n        if (ABSL_PREDICT_FALSE(uncompressed_length > snappy::kBlockSize)) {\n          set_buffer();\n          return FailInvalidStream(\"uncompressed length too large\");\n        }\n        if (ABSL_PREDICT_FALSE(\n                MaskChecksum(static_cast<uint32_t>(absl::ComputeCrc32c(\n                    absl::string_view(uncompressed_data,\n                                      uncompressed_length)))) != checksum)) {\n          set_buffer();\n          return FailInvalidStream(\"wrong checksum\");\n        }\n        src.move_cursor(sizeof(uint32_t) + chunk_length);\n        if (ABSL_PREDICT_FALSE(uncompressed_length == 0)) continue;\n        const Position max_length =\n            std::numeric_limits<Position>::max() - limit_pos();\n        if (ABSL_PREDICT_FALSE(uncompressed_length > max_length)) {\n          set_buffer(uncompressed_data, IntCast<size_t>(max_length));\n          move_limit_pos(available());\n          return FailOverflow();\n        }\n        set_buffer(uncompressed_data, uncompressed_length);\n        move_limit_pos(available());\n        return true;\n      }\n      case 0xff:  // Stream identifier.\n        if (ABSL_PREDICT_FALSE(\n                absl::string_view(src.cursor() + sizeof(uint32_t),\n                                  chunk_length) !=\n                absl::string_view(\"sNaPpY\", 6))) {\n          set_buffer();\n          return FailInvalidStream(\"invalid stream identifier\");\n        }\n        src.move_cursor(sizeof(uint32_t) + chunk_length);\n        continue;\n      default:\n        if (ABSL_PREDICT_FALSE(chunk_type < 0x80)) {\n          set_buffer();\n          return FailInvalidStream(\"reserved unskippable chunk\");\n        }\n        src.move_cursor(sizeof(uint32_t) + chunk_length);\n        continue;\n    }\n  }\n  set_buffer();\n  if (ABSL_PREDICT_FALSE(!src.ok())) {\n    return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n  }\n  if (ABSL_PREDICT_FALSE(src.available() > 0)) truncated_ = true;\n  return false;\n}\n\nbool FramedSnappyReaderBase::ToleratesReadingAhead() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->ToleratesReadingAhead();\n}\n\nbool FramedSnappyReaderBase::SupportsRewind() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRewind();\n}\n\nbool FramedSnappyReaderBase::SeekBehindScratch(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"scratch used\";\n  if (new_pos <= limit_pos()) {\n    // Seeking backwards.\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    Reader& src = *SrcReader();\n    truncated_ = false;\n    set_buffer();\n    set_limit_pos(0);\n    if (ABSL_PREDICT_FALSE(!src.Seek(initial_compressed_pos_))) {\n      return FailWithoutAnnotation(\n          AnnotateOverSrc(src.StatusOrAnnotate(absl::DataLossError(\n              \"FramedSnappy-compressed stream got truncated\"))));\n    }\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    if (new_pos == 0) return true;\n  }\n  return PullableReader::SeekBehindScratch(new_pos);\n}\n\nbool FramedSnappyReaderBase::SupportsNewReader() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsNewReader();\n}\n\nstd::unique_ptr<Reader> FramedSnappyReaderBase::NewReaderImpl(\n    Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point\n  // if `SrcReader()->SupportsNewReader()`.\n  Reader& src = *SrcReader();\n  std::unique_ptr<Reader> compressed_reader =\n      src.NewReader(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    return nullptr;\n  }\n  std::unique_ptr<Reader> reader =\n      std::make_unique<FramedSnappyReader<std::unique_ptr<Reader>>>(\n          std::move(compressed_reader));\n  reader->Seek(initial_pos);\n  return reader;\n}\n\nbool RecognizeFramedSnappy(Reader& src) {\n  const absl::string_view kSignature(\n      \"\\xff\\x06\\x00\\x00\"\n      \"sNaPpY\",\n      10);\n  return src.Pull(kSignature.size()) &&\n         absl::string_view(src.cursor(), kSignature.size()) == kSignature;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/snappy/framed/framed_snappy_reader.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_SNAPPY_FRAMED_FRAMED_SNAPPY_READER_H_\n#define RIEGELI_SNAPPY_FRAMED_FRAMED_SNAPPY_READER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/moving_dependency.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pullable_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `FramedSnappyReader`.\nclass FramedSnappyReaderBase : public PullableReader {\n public:\n  class Options {};\n\n  // Returns the compressed `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRewind() override;\n  bool SupportsNewReader() override;\n\n protected:\n  using PullableReader::PullableReader;\n\n  FramedSnappyReaderBase(FramedSnappyReaderBase&& that) noexcept;\n  FramedSnappyReaderBase& operator=(FramedSnappyReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(Reader* src);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PullBehindScratch(size_t recommended_length) override;\n  bool SeekBehindScratch(Position new_pos) override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  ABSL_ATTRIBUTE_COLD bool FailInvalidStream(absl::string_view message);\n\n  Position initial_compressed_pos_ = 0;\n  // If `true`, the source is truncated (without a clean end of the compressed\n  // stream) at the current position. If the source does not grow, `Close()`\n  // will fail.\n  bool truncated_ = false;\n  // Buffered uncompressed data.\n  Buffer uncompressed_;\n\n  // Invariant if scratch is not used:\n  //   `start() == nullptr` or `start() == uncompressed_.data()` or\n  //   `limit() == SrcReader()->cursor()`\n};\n\n// A `Reader` which decompresses data with framed Snappy format after getting\n// it from another `Reader`:\n// https://github.com/google/snappy/blob/master/framing_format.txt\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Reader` must not be accessed until the `FramedSnappyReader`\n// is closed or no longer used.\ntemplate <typename Src = Reader*>\nclass FramedSnappyReader : public FramedSnappyReaderBase {\n public:\n  // Creates a closed `FramedSnappyReader`.\n  explicit FramedSnappyReader(Closed) noexcept\n      : FramedSnappyReaderBase(kClosed) {}\n\n  // Will read from the compressed `Reader` provided by `src`.\n  explicit FramedSnappyReader(Initializer<Src> src,\n                              Options options = Options());\n\n  FramedSnappyReader(FramedSnappyReader&& that) = default;\n  FramedSnappyReader& operator=(FramedSnappyReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `FramedSnappyReader`. This\n  // avoids constructing a temporary `FramedSnappyReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  void VerifyEndImpl() override;\n\n private:\n  class Mover;\n\n  // The object providing and possibly owning the compressed `Reader`.\n  MovingDependency<Reader*, Src, Mover> src_;\n};\n\nexplicit FramedSnappyReader(Closed) -> FramedSnappyReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit FramedSnappyReader(Src&& src, FramedSnappyReaderBase::Options options =\n                                           FramedSnappyReaderBase::Options())\n    -> FramedSnappyReader<TargetT<Src>>;\n\n// Returns `true` if the data look like they have been FramedSnappy-compressed.\n//\n// The current position of `src` is unchanged.\nbool RecognizeFramedSnappy(Reader& src);\n\n// Implementation details follow.\n\ninline FramedSnappyReaderBase::FramedSnappyReaderBase(\n    FramedSnappyReaderBase&& that) noexcept\n    : PullableReader(static_cast<PullableReader&&>(that)),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      truncated_(that.truncated_),\n      uncompressed_(std::move(that.uncompressed_)) {}\n\ninline FramedSnappyReaderBase& FramedSnappyReaderBase::operator=(\n    FramedSnappyReaderBase&& that) noexcept {\n  PullableReader::operator=(static_cast<PullableReader&&>(that));\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  truncated_ = that.truncated_;\n  uncompressed_ = std::move(that.uncompressed_);\n  return *this;\n}\n\ninline void FramedSnappyReaderBase::Reset(Closed) {\n  PullableReader::Reset(kClosed);\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  uncompressed_ = Buffer();\n}\n\ninline void FramedSnappyReaderBase::Reset() {\n  PullableReader::Reset();\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n}\n\ntemplate <typename Src>\nclass FramedSnappyReader<Src>::Mover {\n public:\n  static auto member() { return &FramedSnappyReader::src_; }\n\n  explicit Mover(FramedSnappyReader& self, FramedSnappyReader& that)\n      : behind_scratch_(&self),\n        // Buffer pointers are already moved so `limit()` is taken from `self`.\n        // `src_` is not moved yet so `src_` is taken from `that`.\n        reads_uncompressed_(ABSL_PREDICT_TRUE(self.is_open()) &&\n                            self.limit() == that.src_->cursor()) {\n    if (reads_uncompressed_) {\n      available_ = self.available();\n      that.src_->set_cursor(that.src_->cursor() - available_);\n    }\n  }\n\n  void Done(FramedSnappyReader& self) {\n    if (reads_uncompressed_) {\n      if (ABSL_PREDICT_FALSE(!self.src_->Pull(available_))) {\n        self.FailWithoutAnnotation(self.AnnotateOverSrc(self.src_->status()));\n        return;\n      }\n      self.set_buffer(self.src_->cursor(), available_);\n      self.src_->move_cursor(available_);\n    }\n  }\n\n private:\n  BehindScratch behind_scratch_;\n  bool reads_uncompressed_;\n  size_t available_;\n};\n\ntemplate <typename Src>\ninline FramedSnappyReader<Src>::FramedSnappyReader(\n    Initializer<Src> src, ABSL_ATTRIBUTE_UNUSED Options options)\n    : src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void FramedSnappyReader<Src>::Reset(Closed) {\n  FramedSnappyReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void FramedSnappyReader<Src>::Reset(\n    Initializer<Src> src, ABSL_ATTRIBUTE_UNUSED Options options) {\n  FramedSnappyReaderBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\nvoid FramedSnappyReader<Src>::Done() {\n  FramedSnappyReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(AnnotateOverSrc(src_->status()));\n    }\n  }\n}\n\ntemplate <typename Src>\nvoid FramedSnappyReader<Src>::SetReadAllHintImpl(bool read_all_hint) {\n  if (src_.IsOwning()) src_->SetReadAllHint(read_all_hint);\n}\n\ntemplate <typename Src>\nvoid FramedSnappyReader<Src>::VerifyEndImpl() {\n  FramedSnappyReaderBase::VerifyEndImpl();\n  if (src_.IsOwning() && ABSL_PREDICT_TRUE(ok())) src_->VerifyEnd();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_SNAPPY_FRAMED_FRAMED_SNAPPY_READER_H_\n"
  },
  {
    "path": "riegeli/snappy/framed/framed_snappy_writer.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/snappy/framed/framed_snappy_writer.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <cstring>\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pushable_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/digests/crc32c_digester.h\"\n#include \"riegeli/digests/digesting_writer.h\"\n#include \"riegeli/endian/endian_writing.h\"\n#include \"riegeli/snappy/framed/framed_snappy_reader.h\"\n#include \"snappy.h\"\n\nnamespace riegeli {\n\nvoid FramedSnappyWriterBase::Initialize(Writer* dest, int compression_level) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of FramedSnappyWriter: null Writer pointer\";\n  compression_level_ = compression_level;\n  if (ABSL_PREDICT_FALSE(!dest->ok())) {\n    FailWithoutAnnotation(AnnotateOverDest(dest->status()));\n    return;\n  }\n  initial_compressed_pos_ = dest->pos();\n  if (dest->pos() == 0) {\n    // Stream identifier.\n    if (ABSL_PREDICT_FALSE(!dest->Write(absl::string_view(\"\\xff\\x06\\x00\\x00\"\n                                                          \"sNaPpY\",\n                                                          10)))) {\n      FailWithoutAnnotation(AnnotateOverDest(dest->status()));\n    }\n  }\n}\n\nvoid FramedSnappyWriterBase::Done() {\n  PushableWriter::Done();\n  uncompressed_ = Buffer();\n  associated_reader_.Reset();\n}\n\nabsl::Status FramedSnappyWriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    status = dest.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `dest` with the compressed\n  // position. Clarify that the current position is the uncompressed position\n  // instead of delegating to `PushableWriter::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status FramedSnappyWriterBase::AnnotateOverDest(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nvoid FramedSnappyWriterBase::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (write_size_hint == std::nullopt) {\n    size_hint_ = std::nullopt;\n  } else {\n    size_hint_ = SaturatingAdd(pos(), *write_size_hint);\n  }\n}\n\nbool FramedSnappyWriterBase::PushBehindScratch(size_t recommended_length) {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PushableWriter::PushBehindScratch(): \"\n         \"some space available, use Push() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::PushBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!PushInternal(dest))) return false;\n  if (ABSL_PREDICT_FALSE(start_pos() == std::numeric_limits<Position>::max())) {\n    return FailOverflow();\n  }\n  const size_t length = UnsignedMin(\n      ApplyBufferConstraints(\n          ApplySizeHint(snappy::kBlockSize, size_hint_, start_pos()), 1,\n          recommended_length, snappy::kBlockSize),\n      std::numeric_limits<Position>::max() - start_pos());\n  uncompressed_.Reset(length);\n  set_buffer(uncompressed_.data(), length);\n  return true;\n}\n\ninline bool FramedSnappyWriterBase::PushInternal(Writer& dest) {\n  const size_t uncompressed_length = start_to_cursor();\n  RIEGELI_ASSERT_LE(uncompressed_length, snappy::kBlockSize)\n      << \"Failed invariant of FramedSnappyWriterBase: buffer too large\";\n  if (uncompressed_length == 0) return true;\n  set_cursor(start());\n  const char* const uncompressed_data = cursor();\n  if (ABSL_PREDICT_FALSE(\n          !dest.Push(2 * sizeof(uint32_t) +\n                     snappy::MaxCompressedLength(uncompressed_length)))) {\n    return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n  char* const compressed_chunk = dest.cursor();\n  size_t compressed_length;\n  snappy::RawCompress(uncompressed_data, uncompressed_length,\n                      compressed_chunk + 2 * sizeof(uint32_t),\n                      &compressed_length, {/*level=*/compression_level_});\n  if (compressed_length < uncompressed_length) {\n    WriteLittleEndian<uint32_t>(\n        IntCast<uint32_t>(0x00 /* Compressed data */ |\n                          ((sizeof(uint32_t) + compressed_length) << 8)),\n        compressed_chunk);\n  } else {\n    std::memcpy(compressed_chunk + 2 * sizeof(uint32_t), uncompressed_data,\n                uncompressed_length);\n    compressed_length = uncompressed_length;\n    WriteLittleEndian<uint32_t>(\n        IntCast<uint32_t>(0x01 /* Uncompressed data */ |\n                          ((sizeof(uint32_t) + compressed_length) << 8)),\n        compressed_chunk);\n  }\n  WriteLittleEndian<uint32_t>(\n      MaskCrc32c(\n          DigestFrom(absl::string_view(uncompressed_data, uncompressed_length),\n                     Crc32cDigester())),\n      compressed_chunk + sizeof(uint32_t));\n  dest.move_cursor(2 * sizeof(uint32_t) + compressed_length);\n  move_start_pos(uncompressed_length);\n  return true;\n}\n\nbool FramedSnappyWriterBase::FlushBehindScratch(FlushType flush_type) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::FlushBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  return PushInternal(dest);\n}\n\nbool FramedSnappyWriterBase::SupportsReadMode() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsReadMode();\n}\n\nReader* FramedSnappyWriterBase::ReadModeBehindScratch(Position initial_pos) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::ReadModeBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!FramedSnappyWriterBase::FlushBehindScratch(\n          FlushType::kFromObject))) {\n    return nullptr;\n  }\n  Writer& dest = *DestWriter();\n  Reader* const compressed_reader = dest.ReadMode(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    return nullptr;\n  }\n  FramedSnappyReader<>* const reader =\n      associated_reader_.ResetReader(compressed_reader);\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/snappy/framed/framed_snappy_writer.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_SNAPPY_FRAMED_FRAMED_SNAPPY_WRITER_H_\n#define RIEGELI_SNAPPY_FRAMED_FRAMED_SNAPPY_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pushable_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"snappy.h\"\n\nnamespace riegeli {\n\ntemplate <typename Src>\nclass FramedSnappyReader;\nclass Reader;\n\n// Template parameter independent part of `FramedSnappyWriter`.\nclass FramedSnappyWriterBase : public PushableWriter {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // Tunes the tradeoff between compression density and compression speed\n    // (higher = better density but slower).\n    //\n    // `compression_level` must be between `kMinCompressionLevel` (1) and\n    // `kMaxCompressionLevel` (2).\n    static constexpr int kMinCompressionLevel =\n        snappy::CompressionOptions::MinCompressionLevel();\n    static constexpr int kMaxCompressionLevel =\n        snappy::CompressionOptions::MaxCompressionLevel();\n    static constexpr int kDefaultCompressionLevel =\n        snappy::CompressionOptions::DefaultCompressionLevel();\n    Options& set_compression_level(int compression_level) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(compression_level, kMinCompressionLevel)\n          << \"Failed precondition of \"\n             \"FramedSnappyWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      RIEGELI_ASSERT_LE(compression_level, kMaxCompressionLevel)\n          << \"Failed precondition of \"\n             \"FramedSnappyWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      compression_level_ = compression_level;\n      return *this;\n    }\n    Options&& set_compression_level(int compression_level) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_compression_level(compression_level));\n    }\n    int compression_level() const { return compression_level_; }\n\n   private:\n    int compression_level_ = kDefaultCompressionLevel;\n  };\n\n  // Returns the compressed `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsReadMode() override;\n\n protected:\n  explicit FramedSnappyWriterBase(Closed) noexcept : PushableWriter(kClosed) {}\n\n  explicit FramedSnappyWriterBase() {}\n\n  FramedSnappyWriterBase(FramedSnappyWriterBase&& that) noexcept;\n  FramedSnappyWriterBase& operator=(FramedSnappyWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(Writer* dest, int compression_level);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool PushBehindScratch(size_t recommended_length) override;\n  bool FlushBehindScratch(FlushType flush_type) override;\n  Reader* ReadModeBehindScratch(Position initial_pos) override;\n\n private:\n  // Compresses buffered data, but unlike `PushSlow()`, does not ensure that a\n  // buffer is allocated.\n  //\n  // Precondition: `ok()`\n  //\n  // Postcondition: `start_to_cursor() == 0`\n  bool PushInternal(Writer& dest);\n\n  int compression_level_ = Options::kDefaultCompressionLevel;\n  std::optional<Position> size_hint_;\n  Position initial_compressed_pos_ = 0;\n  // Buffered uncompressed data.\n  Buffer uncompressed_;\n\n  AssociatedReader<FramedSnappyReader<Reader*>> associated_reader_;\n\n  // Invariants if scratch is not used:\n  //   `start() == nullptr` or `start() == uncompressed_.data()`\n  //   `start_to_limit() <= snappy::kBlockSize`\n};\n\n// A `Writer` which compresses data with framed Snappy format before passing it\n// to another `Writer`:\n// https://github.com/google/snappy/blob/master/framing_format.txt\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Writer` must not be accessed until the `FramedSnappyWriter`\n// is closed or no longer used.\ntemplate <typename Dest = Writer*>\nclass FramedSnappyWriter : public FramedSnappyWriterBase {\n public:\n  // Creates a closed `FramedSnappyWriter`.\n  explicit FramedSnappyWriter(Closed) noexcept\n      : FramedSnappyWriterBase(kClosed) {}\n\n  // Will write to the compressed `Writer` provided by `dest`.\n  explicit FramedSnappyWriter(Initializer<Dest> dest,\n                              Options options = Options());\n\n  FramedSnappyWriter(FramedSnappyWriter&& that) = default;\n  FramedSnappyWriter& operator=(FramedSnappyWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `FramedSnappyWriter`. This\n  // avoids constructing a temporary `FramedSnappyWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  // The object providing and possibly owning the compressed `Writer`.\n  Dependency<Writer*, Dest> dest_;\n};\n\nexplicit FramedSnappyWriter(Closed) -> FramedSnappyWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit FramedSnappyWriter(\n    Dest&& dest,\n    FramedSnappyWriterBase::Options options = FramedSnappyWriterBase::Options())\n    -> FramedSnappyWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline FramedSnappyWriterBase::FramedSnappyWriterBase(\n    FramedSnappyWriterBase&& that) noexcept\n    : PushableWriter(static_cast<PushableWriter&&>(that)),\n      compression_level_(that.compression_level_),\n      size_hint_(that.size_hint_),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      uncompressed_(std::move(that.uncompressed_)),\n      associated_reader_(std::move(that.associated_reader_)) {}\n\ninline FramedSnappyWriterBase& FramedSnappyWriterBase::operator=(\n    FramedSnappyWriterBase&& that) noexcept {\n  PushableWriter::operator=(static_cast<PushableWriter&&>(that));\n  compression_level_ = that.compression_level_;\n  size_hint_ = that.size_hint_;\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  uncompressed_ = std::move(that.uncompressed_);\n  associated_reader_ = std::move(that.associated_reader_);\n  return *this;\n}\n\ninline void FramedSnappyWriterBase::Reset(Closed) {\n  PushableWriter::Reset(kClosed);\n  compression_level_ = Options::kDefaultCompressionLevel;\n  size_hint_ = std::nullopt;\n  initial_compressed_pos_ = 0;\n  uncompressed_ = Buffer();\n  associated_reader_.Reset();\n}\n\ninline void FramedSnappyWriterBase::Reset() {\n  PushableWriter::Reset();\n  compression_level_ = Options::kDefaultCompressionLevel;\n  size_hint_ = std::nullopt;\n  initial_compressed_pos_ = 0;\n  associated_reader_.Reset();\n}\n\ntemplate <typename Dest>\ninline FramedSnappyWriter<Dest>::FramedSnappyWriter(Initializer<Dest> dest,\n                                                    Options options)\n    : dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.compression_level());\n}\n\ntemplate <typename Dest>\ninline void FramedSnappyWriter<Dest>::Reset(Closed) {\n  FramedSnappyWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void FramedSnappyWriter<Dest>::Reset(Initializer<Dest> dest,\n                                            Options options) {\n  FramedSnappyWriterBase::Reset();\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.compression_level());\n}\n\ntemplate <typename Dest>\nvoid FramedSnappyWriter<Dest>::Done() {\n  FramedSnappyWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <typename Dest>\nbool FramedSnappyWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!FramedSnappyWriterBase::FlushImpl(flush_type))) {\n    return false;\n  }\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Flush(flush_type))) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n  return true;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_SNAPPY_FRAMED_FRAMED_SNAPPY_WRITER_H_\n"
  },
  {
    "path": "riegeli/snappy/hadoop/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"hadoop_snappy_reader\",\n    srcs = [\"hadoop_snappy_reader.cc\"],\n    hdrs = [\"hadoop_snappy_reader.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffer\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:pullable_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/endian:endian_reading\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@snappy\",\n    ],\n)\n\ncc_library(\n    name = \"hadoop_snappy_writer\",\n    srcs = [\"hadoop_snappy_writer.cc\"],\n    hdrs = [\"hadoop_snappy_writer.h\"],\n    deps = [\n        \":hadoop_snappy_reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffer\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:pushable_writer\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/endian:endian_writing\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@snappy\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/snappy/hadoop/hadoop_snappy_reader.cc",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/snappy/hadoop/hadoop_snappy_reader.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pullable_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/endian/endian_reading.h\"\n#include \"snappy.h\"\n\nnamespace riegeli {\n\nvoid HadoopSnappyReaderBase::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of HadoopSnappyReader: null Reader pointer\";\n  if (ABSL_PREDICT_FALSE(!src->ok()) && src->available() == 0) {\n    FailWithoutAnnotation(AnnotateOverSrc(src->status()));\n    return;\n  }\n  initial_compressed_pos_ = src->pos();\n}\n\nvoid HadoopSnappyReaderBase::Done() {\n  if (ABSL_PREDICT_FALSE(truncated_)) {\n    Reader& src = *SrcReader();\n    FailWithoutAnnotation(\n        AnnotateOverSrc(src.AnnotateStatus(absl::InvalidArgumentError(\n            \"Truncated HadoopSnappy-compressed stream\"))));\n  }\n  PullableReader::Done();\n  uncompressed_ = Buffer();\n}\n\ninline bool HadoopSnappyReaderBase::FailInvalidStream(\n    absl::string_view message) {\n  return Fail(absl::InvalidArgumentError(\n      absl::StrCat(\"Invalid HadoopSnappy-compressed stream: \", message)));\n}\n\nabsl::Status HadoopSnappyReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    if (ABSL_PREDICT_FALSE(truncated_)) {\n      status =\n          Annotate(status, \"reading truncated HadoopSnappy-compressed stream\");\n    }\n    Reader& src = *SrcReader();\n    status = src.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `src` with the compressed position.\n  // Clarify that the current position is the uncompressed position instead of\n  // delegating to `PullableReader::AnnotateStatusImpl()`.\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status HadoopSnappyReaderBase::AnnotateOverSrc(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nbool HadoopSnappyReaderBase::PullBehindScratch(size_t recommended_length) {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"some data available, use Pull() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::PullBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Reader& src = *SrcReader();\n  truncated_ = false;\n  while (remaining_chunk_length_ == 0) {\n    if (ABSL_PREDICT_FALSE(!src.Pull(sizeof(uint32_t)))) {\n      set_buffer();\n      if (ABSL_PREDICT_FALSE(!src.ok())) {\n        return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n      }\n      if (ABSL_PREDICT_FALSE(src.available() > 0)) truncated_ = true;\n      return false;\n    }\n    remaining_chunk_length_ = ReadBigEndian<uint32_t>(src.cursor());\n    src.move_cursor(sizeof(uint32_t));\n  }\n  size_t uncompressed_length;\n  char* uncompressed_data;\n  do {\n    if (ABSL_PREDICT_FALSE(!src.Pull(sizeof(uint32_t)))) {\n      set_buffer();\n      if (ABSL_PREDICT_FALSE(!src.ok())) {\n        return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n      }\n      truncated_ = true;\n      return false;\n    }\n    const uint32_t compressed_length = ReadBigEndian<uint32_t>(src.cursor());\n    if (ABSL_PREDICT_FALSE(compressed_length >\n                           std::numeric_limits<uint32_t>::max() -\n                               sizeof(uint32_t))) {\n      set_buffer();\n      return FailInvalidStream(\"compressed length too large\");\n    }\n    if (ABSL_PREDICT_FALSE(!src.Pull(sizeof(uint32_t) + compressed_length))) {\n      set_buffer();\n      if (ABSL_PREDICT_FALSE(!src.ok())) {\n        return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n      }\n      truncated_ = true;\n      return false;\n    }\n    const char* const compressed_data = src.cursor() + sizeof(uint32_t);\n    if (ABSL_PREDICT_FALSE(!snappy::GetUncompressedLength(\n            compressed_data, compressed_length, &uncompressed_length))) {\n      set_buffer();\n      return FailInvalidStream(\"invalid uncompressed length\");\n    }\n    if (ABSL_PREDICT_FALSE(uncompressed_length > remaining_chunk_length_)) {\n      set_buffer();\n      return FailInvalidStream(\"uncompressed length too large\");\n    }\n    uncompressed_.Reset(uncompressed_length);\n    uncompressed_data = uncompressed_.data();\n    if (ABSL_PREDICT_FALSE(!snappy::RawUncompress(\n            compressed_data, compressed_length, uncompressed_data))) {\n      set_buffer();\n      return FailInvalidStream(\"invalid compressed data\");\n    }\n    src.move_cursor(sizeof(uint32_t) + compressed_length);\n  } while (uncompressed_length == 0);\n  remaining_chunk_length_ -= uncompressed_length;\n  const Position max_length =\n      std::numeric_limits<Position>::max() - limit_pos();\n  if (ABSL_PREDICT_FALSE(uncompressed_length > max_length)) {\n    set_buffer(uncompressed_data, IntCast<size_t>(max_length));\n    move_limit_pos(available());\n    return FailOverflow();\n  }\n  set_buffer(uncompressed_data, uncompressed_length);\n  move_limit_pos(available());\n  return true;\n}\n\nbool HadoopSnappyReaderBase::ToleratesReadingAhead() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->ToleratesReadingAhead();\n}\n\nbool HadoopSnappyReaderBase::SupportsRewind() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRewind();\n}\n\nbool HadoopSnappyReaderBase::SeekBehindScratch(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PullableReader::SeekBehindScratch(): \"\n         \"scratch used\";\n  if (new_pos <= limit_pos()) {\n    // Seeking backwards.\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    Reader& src = *SrcReader();\n    truncated_ = false;\n    remaining_chunk_length_ = 0;\n    set_buffer();\n    set_limit_pos(0);\n    if (ABSL_PREDICT_FALSE(!src.Seek(initial_compressed_pos_))) {\n      return FailWithoutAnnotation(\n          AnnotateOverSrc(src.StatusOrAnnotate(absl::DataLossError(\n              \"HadoopSnappy-compressed stream got truncated\"))));\n    }\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    if (new_pos == 0) return true;\n  }\n  return PullableReader::SeekBehindScratch(new_pos);\n}\n\nbool HadoopSnappyReaderBase::SupportsNewReader() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsNewReader();\n}\n\nstd::unique_ptr<Reader> HadoopSnappyReaderBase::NewReaderImpl(\n    Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point\n  // if `SrcReader()->SupportsNewReader()`.\n  Reader& src = *SrcReader();\n  std::unique_ptr<Reader> compressed_reader =\n      src.NewReader(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    return nullptr;\n  }\n  std::unique_ptr<Reader> reader =\n      std::make_unique<HadoopSnappyReader<std::unique_ptr<Reader>>>(\n          std::move(compressed_reader));\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/snappy/hadoop/hadoop_snappy_reader.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_SNAPPY_HADOOP_HADOOP_SNAPPY_READER_H_\n#define RIEGELI_SNAPPY_HADOOP_HADOOP_SNAPPY_READER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <memory>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pullable_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `HadoopSnappyReader`.\nclass HadoopSnappyReaderBase : public PullableReader {\n public:\n  class Options {};\n\n  // Returns the compressed `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRewind() override;\n  bool SupportsNewReader() override;\n\n protected:\n  using PullableReader::PullableReader;\n\n  HadoopSnappyReaderBase(HadoopSnappyReaderBase&& that) noexcept;\n  HadoopSnappyReaderBase& operator=(HadoopSnappyReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(Reader* src);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool PullBehindScratch(size_t recommended_length) override;\n  bool SeekBehindScratch(Position new_pos) override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  ABSL_ATTRIBUTE_COLD bool FailInvalidStream(absl::string_view message);\n\n  Position initial_compressed_pos_ = 0;\n  // If `true`, the source is truncated (without a clean end of the compressed\n  // stream) at the current position. If the source does not grow, `Close()`\n  // will fail.\n  bool truncated_ = false;\n  // Remaining number of uncompressed bytes in the current chunk.\n  uint32_t remaining_chunk_length_ = 0;\n  // Buffered uncompressed data.\n  Buffer uncompressed_;\n\n  // Invariant if scratch is not used:\n  //   `start() == nullptr` or `start() == uncompressed_.data()`\n};\n\n// A `Reader` which decompresses data with Hadoop Snappy format after getting\n// it from another `Reader`:\n// https://github.com/kubo/snzip#hadoop-snappy-format\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Reader` must not be accessed until the `HadoopSnappyReader`\n// is closed or no longer used.\ntemplate <typename Src = Reader*>\nclass HadoopSnappyReader : public HadoopSnappyReaderBase {\n public:\n  // Creates a closed `HadoopSnappyReader`.\n  explicit HadoopSnappyReader(Closed) noexcept\n      : HadoopSnappyReaderBase(kClosed) {}\n\n  // Will read from the compressed `Reader` provided by `src`.\n  explicit HadoopSnappyReader(Initializer<Src> src,\n                              Options options = Options());\n\n  HadoopSnappyReader(HadoopSnappyReader&& that) = default;\n  HadoopSnappyReader& operator=(HadoopSnappyReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `HadoopSnappyReader`. This\n  // avoids constructing a temporary `HadoopSnappyReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  void VerifyEndImpl() override;\n\n private:\n  // The object providing and possibly owning the compressed `Reader`.\n  Dependency<Reader*, Src> src_;\n};\n\nexplicit HadoopSnappyReader(Closed) -> HadoopSnappyReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit HadoopSnappyReader(Src&& src, HadoopSnappyReaderBase::Options options =\n                                           HadoopSnappyReaderBase::Options())\n    -> HadoopSnappyReader<TargetT<Src>>;\n\n// Implementation details follow.\n\ninline HadoopSnappyReaderBase::HadoopSnappyReaderBase(\n    HadoopSnappyReaderBase&& that) noexcept\n    : PullableReader(static_cast<PullableReader&&>(that)),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      truncated_(that.truncated_),\n      remaining_chunk_length_(that.remaining_chunk_length_),\n      uncompressed_(std::move(that.uncompressed_)) {}\n\ninline HadoopSnappyReaderBase& HadoopSnappyReaderBase::operator=(\n    HadoopSnappyReaderBase&& that) noexcept {\n  PullableReader::operator=(static_cast<PullableReader&&>(that));\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  truncated_ = that.truncated_;\n  remaining_chunk_length_ = that.remaining_chunk_length_;\n  uncompressed_ = std::move(that.uncompressed_);\n  return *this;\n}\n\ninline void HadoopSnappyReaderBase::Reset(Closed) {\n  PullableReader::Reset(kClosed);\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  remaining_chunk_length_ = 0;\n  uncompressed_ = Buffer();\n}\n\ninline void HadoopSnappyReaderBase::Reset() {\n  PullableReader::Reset();\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  remaining_chunk_length_ = 0;\n}\n\ntemplate <typename Src>\ninline HadoopSnappyReader<Src>::HadoopSnappyReader(\n    Initializer<Src> src, ABSL_ATTRIBUTE_UNUSED Options options)\n    : src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void HadoopSnappyReader<Src>::Reset(Closed) {\n  HadoopSnappyReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void HadoopSnappyReader<Src>::Reset(\n    Initializer<Src> src, ABSL_ATTRIBUTE_UNUSED Options options) {\n  HadoopSnappyReaderBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\nvoid HadoopSnappyReader<Src>::Done() {\n  HadoopSnappyReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(AnnotateOverSrc(src_->status()));\n    }\n  }\n}\n\ntemplate <typename Src>\nvoid HadoopSnappyReader<Src>::SetReadAllHintImpl(bool read_all_hint) {\n  if (src_.IsOwning()) src_->SetReadAllHint(read_all_hint);\n}\n\ntemplate <typename Src>\nvoid HadoopSnappyReader<Src>::VerifyEndImpl() {\n  HadoopSnappyReaderBase::VerifyEndImpl();\n  if (src_.IsOwning() && ABSL_PREDICT_TRUE(ok())) src_->VerifyEnd();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_SNAPPY_HADOOP_HADOOP_SNAPPY_READER_H_\n"
  },
  {
    "path": "riegeli/snappy/hadoop/hadoop_snappy_writer.cc",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/snappy/hadoop/hadoop_snappy_writer.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pushable_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/endian/endian_writing.h\"\n#include \"riegeli/snappy/hadoop/hadoop_snappy_reader.h\"\n#include \"snappy.h\"\n\nnamespace riegeli {\n\nvoid HadoopSnappyWriterBase::Initialize(Writer* dest, int compression_level) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of HadoopSnappyWriter: null Writer pointer\";\n  compression_level_ = compression_level;\n  if (ABSL_PREDICT_FALSE(!dest->ok())) {\n    FailWithoutAnnotation(AnnotateOverDest(dest->status()));\n  }\n}\n\nvoid HadoopSnappyWriterBase::Done() {\n  PushableWriter::Done();\n  uncompressed_ = Buffer();\n  associated_reader_.Reset();\n}\n\nabsl::Status HadoopSnappyWriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    status = dest.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `dest` with the compressed\n  // position. Clarify that the current position is the uncompressed position\n  // instead of delegating to `PushableWriter::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status HadoopSnappyWriterBase::AnnotateOverDest(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nvoid HadoopSnappyWriterBase::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (write_size_hint == std::nullopt) {\n    size_hint_ = std::nullopt;\n  } else {\n    size_hint_ = SaturatingAdd(pos(), *write_size_hint);\n  }\n}\n\nbool HadoopSnappyWriterBase::PushBehindScratch(size_t recommended_length) {\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of PushableWriter::PushBehindScratch(): \"\n         \"some space available, use Push() instead\";\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::PushBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  if (ABSL_PREDICT_FALSE(!PushInternal(dest))) return false;\n  if (ABSL_PREDICT_FALSE(start_pos() == std::numeric_limits<Position>::max())) {\n    return FailOverflow();\n  }\n  const size_t length = UnsignedMin(\n      ApplyBufferConstraints(\n          ApplySizeHint(snappy::kBlockSize, size_hint_, start_pos()), 1,\n          recommended_length, snappy::kBlockSize),\n      std::numeric_limits<Position>::max() - start_pos());\n  uncompressed_.Reset(length);\n  set_buffer(uncompressed_.data(), length);\n  return true;\n}\n\ninline bool HadoopSnappyWriterBase::PushInternal(Writer& dest) {\n  const size_t uncompressed_length = start_to_cursor();\n  RIEGELI_ASSERT_LE(uncompressed_length, snappy::kBlockSize)\n      << \"Failed invariant of HadoopSnappyWriterBase: buffer too large\";\n  if (uncompressed_length == 0) return true;\n  set_cursor(start());\n  const char* const uncompressed_data = cursor();\n  if (ABSL_PREDICT_FALSE(\n          !dest.Push(2 * sizeof(uint32_t) +\n                     snappy::MaxCompressedLength(uncompressed_length)))) {\n    return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n  }\n  char* const compressed_chunk = dest.cursor();\n  WriteBigEndian<uint32_t>(IntCast<uint32_t>(uncompressed_length),\n                           compressed_chunk);\n  size_t compressed_length;\n  snappy::RawCompress(uncompressed_data, uncompressed_length,\n                      compressed_chunk + 2 * sizeof(uint32_t),\n                      &compressed_length, {/*level=*/compression_level_});\n  WriteBigEndian<uint32_t>(IntCast<uint32_t>(compressed_length),\n                           compressed_chunk + sizeof(uint32_t));\n  dest.move_cursor(2 * sizeof(uint32_t) + compressed_length);\n  move_start_pos(uncompressed_length);\n  return true;\n}\n\nbool HadoopSnappyWriterBase::FlushBehindScratch(FlushType flush_type) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::FlushBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  return PushInternal(dest);\n}\n\nbool HadoopSnappyWriterBase::SupportsReadMode() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsReadMode();\n}\n\nReader* HadoopSnappyWriterBase::ReadModeBehindScratch(Position initial_pos) {\n  RIEGELI_ASSERT(!scratch_used())\n      << \"Failed precondition of PushableWriter::ReadModeBehindScratch(): \"\n         \"scratch used\";\n  if (ABSL_PREDICT_FALSE(!HadoopSnappyWriterBase::FlushBehindScratch(\n          FlushType::kFromObject))) {\n    return nullptr;\n  }\n  Writer& dest = *DestWriter();\n  Reader* const compressed_reader = dest.ReadMode(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    return nullptr;\n  }\n  HadoopSnappyReader<>* const reader =\n      associated_reader_.ResetReader(compressed_reader);\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/snappy/hadoop/hadoop_snappy_writer.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_SNAPPY_HADOOP_HADOOP_SNAPPY_WRITER_H_\n#define RIEGELI_SNAPPY_HADOOP_HADOOP_SNAPPY_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffer.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/pushable_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"snappy.h\"\n\nnamespace riegeli {\n\ntemplate <typename Src>\nclass HadoopSnappyReader;\nclass Reader;\n\n// Template parameter independent part of `HadoopSnappyWriter`.\nclass HadoopSnappyWriterBase : public PushableWriter {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // Tunes the tradeoff between compression density and compression speed\n    // (higher = better density but slower).\n    //\n    // `compression_level` must be between `kMinCompressionLevel` (1) and\n    // `kMaxCompressionLevel` (2).\n    static constexpr int kMinCompressionLevel =\n        snappy::CompressionOptions::MinCompressionLevel();\n    static constexpr int kMaxCompressionLevel =\n        snappy::CompressionOptions::MaxCompressionLevel();\n    static constexpr int kDefaultCompressionLevel =\n        snappy::CompressionOptions::DefaultCompressionLevel();\n    Options& set_compression_level(int compression_level) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(compression_level, kMinCompressionLevel)\n          << \"Failed precondition of \"\n             \"HadoopSnappyWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      RIEGELI_ASSERT_LE(compression_level, kMaxCompressionLevel)\n          << \"Failed precondition of \"\n             \"HadoopSnappyWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      compression_level_ = compression_level;\n      return *this;\n    }\n    Options&& set_compression_level(int compression_level) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_compression_level(compression_level));\n    }\n    int compression_level() const { return compression_level_; }\n\n   private:\n    int compression_level_ = kDefaultCompressionLevel;\n  };\n\n  // Returns the compressed `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsReadMode() override;\n\n protected:\n  explicit HadoopSnappyWriterBase(Closed) noexcept : PushableWriter(kClosed) {}\n\n  explicit HadoopSnappyWriterBase() {}\n\n  HadoopSnappyWriterBase(HadoopSnappyWriterBase&& that) noexcept;\n  HadoopSnappyWriterBase& operator=(HadoopSnappyWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(Writer* dest, int compression_level);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool PushBehindScratch(size_t recommended_length) override;\n  bool FlushBehindScratch(FlushType flush_type) override;\n  Reader* ReadModeBehindScratch(Position initial_pos) override;\n\n private:\n  // Compresses buffered data, but unlike `PushSlow()`, does not ensure that a\n  // buffer is allocated.\n  //\n  // Precondition: `ok()`\n  //\n  // Postcondition: `start_to_cursor() == 0`\n  bool PushInternal(Writer& dest);\n\n  int compression_level_ = Options::kDefaultCompressionLevel;\n  std::optional<Position> size_hint_;\n  Position initial_compressed_pos_ = 0;\n  // Buffered uncompressed data.\n  Buffer uncompressed_;\n\n  AssociatedReader<HadoopSnappyReader<Reader*>> associated_reader_;\n\n  // Invariants if scratch is not used:\n  //   `start() == nullptr` or `start() == uncompressed_.data()`\n  //   `start_to_limit() <= snappy::kBlockSize`\n};\n\n// A `Writer` which compresses data with Hadoop Snappy format before passing it\n// to another `Writer`:\n// https://github.com/kubo/snzip#hadoop-snappy-format\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Writer` must not be accessed until the `HadoopSnappyWriter`\n// is closed or no longer used.\ntemplate <typename Dest = Writer*>\nclass HadoopSnappyWriter : public HadoopSnappyWriterBase {\n public:\n  // Creates a closed `HadoopSnappyWriter`.\n  explicit HadoopSnappyWriter(Closed) noexcept\n      : HadoopSnappyWriterBase(kClosed) {}\n\n  // Will write to the compressed `Writer` provided by `dest`.\n  explicit HadoopSnappyWriter(Initializer<Dest> dest,\n                              Options options = Options());\n\n  HadoopSnappyWriter(HadoopSnappyWriter&& that) = default;\n  HadoopSnappyWriter& operator=(HadoopSnappyWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `HadoopSnappyWriter`. This\n  // avoids constructing a temporary `HadoopSnappyWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  // The object providing and possibly owning the compressed `Writer`.\n  Dependency<Writer*, Dest> dest_;\n};\n\nexplicit HadoopSnappyWriter(Closed) -> HadoopSnappyWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit HadoopSnappyWriter(\n    Dest&& dest,\n    HadoopSnappyWriterBase::Options options = HadoopSnappyWriterBase::Options())\n    -> HadoopSnappyWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline HadoopSnappyWriterBase::HadoopSnappyWriterBase(\n    HadoopSnappyWriterBase&& that) noexcept\n    : PushableWriter(static_cast<PushableWriter&&>(that)),\n      compression_level_(that.compression_level_),\n      size_hint_(that.size_hint_),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      uncompressed_(std::move(that.uncompressed_)),\n      associated_reader_(std::move(that.associated_reader_)) {}\n\ninline HadoopSnappyWriterBase& HadoopSnappyWriterBase::operator=(\n    HadoopSnappyWriterBase&& that) noexcept {\n  PushableWriter::operator=(static_cast<PushableWriter&&>(that));\n  compression_level_ = that.compression_level_;\n  size_hint_ = that.size_hint_;\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  uncompressed_ = std::move(that.uncompressed_);\n  associated_reader_ = std::move(that.associated_reader_);\n  return *this;\n}\n\ninline void HadoopSnappyWriterBase::Reset(Closed) {\n  PushableWriter::Reset(kClosed);\n  compression_level_ = Options::kDefaultCompressionLevel;\n  size_hint_ = std::nullopt;\n  initial_compressed_pos_ = 0;\n  uncompressed_ = Buffer();\n  associated_reader_.Reset();\n}\n\ninline void HadoopSnappyWriterBase::Reset() {\n  PushableWriter::Reset();\n  compression_level_ = Options::kDefaultCompressionLevel;\n  size_hint_ = std::nullopt;\n  initial_compressed_pos_ = 0;\n  associated_reader_.Reset();\n}\n\ntemplate <typename Dest>\ninline HadoopSnappyWriter<Dest>::HadoopSnappyWriter(Initializer<Dest> dest,\n                                                    Options options)\n    : dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.compression_level());\n}\n\ntemplate <typename Dest>\ninline void HadoopSnappyWriter<Dest>::Reset(Closed) {\n  HadoopSnappyWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void HadoopSnappyWriter<Dest>::Reset(Initializer<Dest> dest,\n                                            Options options) {\n  HadoopSnappyWriterBase::Reset();\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.compression_level());\n}\n\ntemplate <typename Dest>\nvoid HadoopSnappyWriter<Dest>::Done() {\n  HadoopSnappyWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <typename Dest>\nbool HadoopSnappyWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!HadoopSnappyWriterBase::FlushImpl(flush_type))) {\n    return false;\n  }\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Flush(flush_type))) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n  return true;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_SNAPPY_HADOOP_HADOOP_SNAPPY_WRITER_H_\n"
  },
  {
    "path": "riegeli/snappy/snappy_reader.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/snappy/snappy_reader.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/chain_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/snappy/snappy_streams.h\"\n#include \"riegeli/varint/varint_reading.h\"\n#include \"snappy.h\"\n\nnamespace riegeli {\n\nvoid SnappyReaderBase::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of SnappyReader: null Reader pointer\";\n  if (ABSL_PREDICT_FALSE(!src->ok()) && src->available() == 0) {\n    FailWithoutAnnotation(AnnotateOverSrc(src->status()));\n    return;\n  }\n  Chain decompressed;\n  if (absl::Status status =\n          SnappyDecompress(*src, ChainWriter<>(&decompressed));\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    FailWithoutAnnotation(std::move(status));\n    return;\n  }\n  // `SnappyReaderBase` derives from `ChainReader<Chain>` but the `Chain` to\n  // read from was not known in `SnappyReaderBase` constructor. This sets the\n  // `Chain` and updates the `ChainReader` to read from it.\n  ChainReader::Reset(std::move(decompressed));\n}\n\nvoid SnappyReaderBase::Done() {\n  ChainReader::Done();\n  ChainReader::src() = Chain();\n}\n\nabsl::Status SnappyReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Reader& src = *SrcReader();\n    status = src.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `src` with the compressed position.\n  // Clarify that the current position is the uncompressed position instead of\n  // delegating to `ChainReader::AnnotateStatusImpl()`.\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status SnappyReaderBase::AnnotateOverSrc(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nnamespace snappy_internal {\n\nabsl::Status SnappyDecompressImpl(Reader& src, Writer& dest) {\n  const std::optional<Position> size = src.Size();\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) return src.status();\n  ReaderSnappySource source(&src, *size);\n  WriterSnappySink sink(&dest);\n  const bool uncompress_ok = snappy::Uncompress(&source, &sink);\n  if (ABSL_PREDICT_FALSE(!dest.ok())) return dest.status();\n  if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n  if (ABSL_PREDICT_FALSE(!uncompress_ok)) {\n    return Annotate(src.AnnotateStatus(absl::InvalidArgumentError(\n                        \"Invalid snappy-compressed stream\")),\n                    absl::StrCat(\"at uncompressed byte \", dest.pos()));\n  }\n  return absl::OkStatus();\n}\n\n}  // namespace snappy_internal\n\nstd::optional<size_t> SnappyUncompressedSize(Reader& src) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthVarint32))) return std::nullopt;\n  if (ABSL_PREDICT_FALSE(src.available() < kMaxLengthVarint32)) {\n    size_t length = 1;\n    while (length < kMaxLengthVarint32 &&\n           static_cast<uint8_t>(src.cursor()[length - 1]) >= 0x80) {\n      ++length;\n      if (ABSL_PREDICT_FALSE(!src.Pull(length, kMaxLengthVarint32))) {\n        return std::nullopt;\n      }\n    }\n  }\n  uint32_t size;\n  if (ABSL_PREDICT_FALSE(ReadVarint32(src.cursor(), src.available(), size) ==\n                         0)) {\n    return std::nullopt;\n  }\n  return size;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/snappy/snappy_reader.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_SNAPPY_SNAPPY_READER_H_\n#define RIEGELI_SNAPPY_SNAPPY_READER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nclass Writer;\n\n// Template parameter independent part of `SnappyReader`.\nclass SnappyReaderBase : public ChainReader<Chain> {\n public:\n  class Options {};\n\n  // Returns the compressed `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n protected:\n  explicit SnappyReaderBase(Closed) noexcept : ChainReader(kClosed) {}\n\n  SnappyReaderBase();\n\n  SnappyReaderBase(SnappyReaderBase&& that) noexcept;\n  SnappyReaderBase& operator=(SnappyReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(Reader* src);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n};\n\n// A `Reader` which decompresses data with Snappy after getting it from another\n// `Reader`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Reader` must support `Size()`. To supply or override this\n// size, the `Reader` can be wrapped in a `LimitingReader` with\n// `LimitingReaderBase::Options().set_exact_length(size)`.\n//\n// The compressed `Reader` must not be accessed until the `SnappyReader` is\n// closed or no longer used.\n//\n// `SnappyReader` does not decompress incrementally but reads compressed data\n// and decompresses them all in the constructor.\n//\n// `SnappyReader` does not support reading from a growing source. If source is\n// truncated, decompression fails.\ntemplate <typename Src = Reader*>\nclass SnappyReader : public SnappyReaderBase {\n public:\n  // Creates a closed `SnappyReader`.\n  explicit SnappyReader(Closed) noexcept : SnappyReaderBase(kClosed) {}\n\n  // Will read from the compressed `Reader` provided by `src`.\n  explicit SnappyReader(Initializer<Src> src, Options options = Options());\n\n  SnappyReader(SnappyReader&& that) = default;\n  SnappyReader& operator=(SnappyReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `SnappyReader`. This avoids\n  // constructing a temporary `SnappyReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  void VerifyEndImpl() override;\n\n private:\n  // The object providing and possibly owning the compressed `Reader`.\n  Dependency<Reader*, Src> src_;\n};\n\nexplicit SnappyReader(Closed) -> SnappyReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit SnappyReader(\n    Src&& src, SnappyReaderBase::Options options = SnappyReaderBase::Options())\n    -> SnappyReader<TargetT<Src>>;\n\n// An alternative interface to Snappy which avoids buffering uncompressed data.\n// Calling `SnappyDecompress()` is equivalent to copying all data from a\n// `SnappyReader<Src&&>` to `dest`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Reader`. `Src` must support\n// `DependencyRef<Reader*, Src>`, e.g. `Reader&` (not owned),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `AnyRef<Reader*>` (maybe owned).\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the uncompressed `Writer`. `Dest` must support\n// `DependencyRef<Writer*, Dest>`, e.g. `Writer&` (not owned),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `AnyRef<Writer*>` (maybe owned).\n//\n// The compressed `Reader` must support `Size()`. To supply or override this\n// size, the `Reader` can be wrapped in a `LimitingReader` with\n// `LimitingReaderBase::Options().set_exact_length(size)`.\ntemplate <typename Src, typename Dest,\n          std::enable_if_t<\n              std::conjunction_v<TargetRefSupportsDependency<Reader*, Src>,\n                                 TargetRefSupportsDependency<Writer*, Dest>>,\n              int> = 0>\nabsl::Status SnappyDecompress(Src&& src, Dest&& dest);\n\n// Returns the claimed uncompressed size of Snappy-compressed data.\n//\n// Returns `std::nullopt` on failure.\n//\n// The current position of `src` is unchanged.\nstd::optional<size_t> SnappyUncompressedSize(Reader& src);\n\n// Implementation details follow.\n\ninline SnappyReaderBase::SnappyReaderBase()\n    // Empty `Chain` as the `ChainReader` source is a placeholder, it will be\n    // set by `Initialize()`.\n    : ChainReader(riegeli::Maker()) {}\n\ninline SnappyReaderBase::SnappyReaderBase(SnappyReaderBase&& that) noexcept\n    : ChainReader(static_cast<ChainReader&&>(that)) {}\n\ninline SnappyReaderBase& SnappyReaderBase::operator=(\n    SnappyReaderBase&& that) noexcept {\n  ChainReader::operator=(static_cast<ChainReader&&>(that));\n  return *this;\n}\n\ninline void SnappyReaderBase::Reset(Closed) { ChainReader::Reset(kClosed); }\n\ninline void SnappyReaderBase::Reset() {\n  // Empty `Chain` as the `ChainReader` source is a placeholder, it will be set\n  // by `Initialize()`.\n  ChainReader::Reset(riegeli::Maker());\n}\n\ntemplate <typename Src>\ninline SnappyReader<Src>::SnappyReader(Initializer<Src> src,\n                                       ABSL_ATTRIBUTE_UNUSED Options options)\n    : src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void SnappyReader<Src>::Reset(Closed) {\n  SnappyReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void SnappyReader<Src>::Reset(Initializer<Src> src,\n                                     ABSL_ATTRIBUTE_UNUSED Options options) {\n  SnappyReaderBase::Reset();\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\nvoid SnappyReader<Src>::Done() {\n  SnappyReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(AnnotateOverSrc(src_->status()));\n    }\n  }\n}\n\ntemplate <typename Src>\nvoid SnappyReader<Src>::VerifyEndImpl() {\n  SnappyReaderBase::VerifyEndImpl();\n  if (src_.IsOwning() && ABSL_PREDICT_TRUE(ok())) src_->VerifyEnd();\n}\n\nnamespace snappy_internal {\n\nabsl::Status SnappyDecompressImpl(Reader& src, Writer& dest);\n\n}  // namespace snappy_internal\n\ntemplate <typename Src, typename Dest,\n          std::enable_if_t<\n              std::conjunction_v<TargetRefSupportsDependency<Reader*, Src>,\n                                 TargetRefSupportsDependency<Writer*, Dest>>,\n              int>>\ninline absl::Status SnappyDecompress(Src&& src, Dest&& dest) {\n  DependencyRef<Reader*, Src> src_dep(std::forward<Src>(src));\n  DependencyRef<Writer*, Dest> dest_dep(std::forward<Dest>(dest));\n  if (src_dep.IsOwning()) src_dep->SetReadAllHint(true);\n  if (dest_dep.IsOwning()) {\n    dest_dep->SetWriteSizeHint(SnappyUncompressedSize(*src_dep));\n  }\n  absl::Status status =\n      snappy_internal::SnappyDecompressImpl(*src_dep, *dest_dep);\n  if (dest_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_dep->Close())) {\n      status.Update(dest_dep->status());\n    }\n  }\n  if (src_dep.IsOwning()) {\n    if (ABSL_PREDICT_TRUE(status.ok())) src_dep->VerifyEnd();\n    if (ABSL_PREDICT_FALSE(!src_dep->Close())) status.Update(src_dep->status());\n  }\n  return status;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_SNAPPY_SNAPPY_READER_H_\n"
  },
  {
    "path": "riegeli/snappy/snappy_streams.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/snappy/snappy_streams.h\"\n\n#include <stddef.h>\n\n#include <functional>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli::snappy_internal {\n\nvoid WriterSnappySink::Append(const char* src, size_t length) {\n  RIEGELI_ASSERT(std::less_equal<>()(src, dest_->cursor()) ||\n                 std::greater_equal<>()(src, dest_->limit()))\n      << \"Failed precondition of Sink::Append(): \"\n         \"appending a pointer to the middle of GetAppendBuffer()\";\n  // Check also `dest_->available()` because `dest_->cursor()` might point to\n  // the end of a memory block, and `src` to the beginning of an unrelated\n  // memory block.\n  if (src == dest_->cursor() && ABSL_PREDICT_TRUE(dest_->available() > 0)) {\n    // Appending a prefix of the result of `GetAppendBuffer()`.\n    RIEGELI_ASSERT_LE(length, dest_->available())\n        << \"Failed precondition of Sink::Append(): \"\n           \"appending the result of GetAppendBuffer() with length too large\";\n    dest_->move_cursor(length);\n  } else {\n    dest_->Write(absl::string_view(src, length));\n  }\n}\n\nchar* WriterSnappySink::GetAppendBuffer(size_t length, char* scratch) {\n  if (ABSL_PREDICT_TRUE(dest_->Push(length))) {\n    return dest_->cursor();\n  } else {\n    return scratch;\n  }\n}\n\nvoid WriterSnappySink::AppendAndTakeOwnership(\n    char* src, size_t length, void (*deleter)(void*, const char*, size_t),\n    void* deleter_arg) {\n  dest_->Write(ExternalRef::From(\n      [deleter, deleter_arg](absl::string_view data) {\n        deleter(deleter_arg, data.data(), data.size());\n      },\n      absl::string_view(src, length)));\n}\n\nchar* WriterSnappySink::GetAppendBufferVariable(size_t min_length,\n                                                size_t recommended_length,\n                                                char* scratch,\n                                                size_t scratch_length,\n                                                size_t* result_length) {\n  RIEGELI_ASSERT_GE(scratch_length, min_length)\n      << \"Failed precondition of Sink::GetAppendBufferVariable(): \"\n         \"scratch length too small\";\n  if (ABSL_PREDICT_TRUE(dest_->Push(min_length, recommended_length))) {\n    *result_length = dest_->available();\n    return dest_->cursor();\n  } else {\n    *result_length = scratch_length;\n    return scratch;\n  }\n}\n\nsize_t ReaderSnappySource::Available() const {\n  return SaturatingIntCast<size_t>(SaturatingSub(size_, src_->pos()));\n}\n\nconst char* ReaderSnappySource::Peek(size_t* length) {\n  src_->Pull();\n  *length = src_->available();\n  return src_->cursor();\n}\n\nvoid ReaderSnappySource::Skip(size_t length) { src_->Skip(length); }\n\n}  // namespace riegeli::snappy_internal\n"
  },
  {
    "path": "riegeli/snappy/snappy_streams.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_SNAPPY_SNAPPY_STREAMS_H_\n#define RIEGELI_SNAPPY_SNAPPY_STREAMS_H_\n\n#include <stddef.h>\n\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"snappy-sinksource.h\"\n\nnamespace riegeli::snappy_internal {\n\n// Adapts a `Writer` to a `snappy::Sink`.\nclass WriterSnappySink : public snappy::Sink {\n public:\n  explicit WriterSnappySink(Writer* dest)\n      : dest_(RIEGELI_EVAL_ASSERT_NOTNULL(dest)) {}\n\n  WriterSnappySink(const WriterSnappySink&) = delete;\n  WriterSnappySink& operator=(const WriterSnappySink&) = delete;\n\n  void Append(const char* src, size_t length) override;\n  char* GetAppendBuffer(size_t length, char* scratch) override;\n  void AppendAndTakeOwnership(char* src, size_t length,\n                              void (*deleter)(void*, const char*, size_t),\n                              void* deleter_arg) override;\n  char* GetAppendBufferVariable(size_t min_length, size_t recommended_length,\n                                char* scratch, size_t scratch_length,\n                                size_t* result_length) override;\n\n private:\n  Writer* dest_;\n};\n\n// Adapts a `Reader` to a `snappy::Source`.\nclass ReaderSnappySource : public snappy::Source {\n public:\n  explicit ReaderSnappySource(Reader* src, Position size)\n      : src_(RIEGELI_EVAL_ASSERT_NOTNULL(src)), size_(size) {}\n\n  ReaderSnappySource(const ReaderSnappySource&) = delete;\n  ReaderSnappySource& operator=(const ReaderSnappySource&) = delete;\n\n  size_t Available() const override;\n  const char* Peek(size_t* length) override;\n  void Skip(size_t length) override;\n\n private:\n  Reader* src_;\n  Position size_;\n};\n\n}  // namespace riegeli::snappy_internal\n\n#endif  // RIEGELI_SNAPPY_SNAPPY_STREAMS_H_\n"
  },
  {
    "path": "riegeli/snappy/snappy_writer.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/snappy/snappy_writer.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/chain_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/snappy/snappy_streams.h\"\n#include \"snappy.h\"\n\nnamespace riegeli {\n\nvoid SnappyWriterBase::Done() {\n  if (ABSL_PREDICT_TRUE(ok())) SyncBuffer();\n  Writer::Done();\n  if (ABSL_PREDICT_TRUE(ok())) {\n    Writer& dest = *DestWriter();\n    if (absl::Status status = SnappyCompress(\n            ChainReader<>(&uncompressed_), dest,\n            SnappyCompressOptions().set_compression_level(compression_level_));\n        ABSL_PREDICT_FALSE(!status.ok())) {\n      FailWithoutAnnotation(std::move(status));\n    }\n  }\n  uncompressed_ = Chain();\n  associated_reader_.Reset();\n}\n\ninline size_t SnappyWriterBase::MaxBytesToCopy() const {\n  const size_t max_bytes_to_copy = ~IntCast<size_t>(pos()) & (kBlockSize - 1);\n  if (options_.size_hint() != std::nullopt &&\n      IntCast<size_t>(pos()) < *options_.size_hint()) {\n    return UnsignedMin(*options_.size_hint() - IntCast<size_t>(pos()) - 1,\n                       max_bytes_to_copy);\n  }\n  return max_bytes_to_copy;\n}\n\ninline bool SnappyWriterBase::SyncBuffer() {\n  set_start_pos(pos());\n  uncompressed_.RemoveSuffix(available());\n  set_buffer();\n  if (ABSL_PREDICT_FALSE(IntCast<size_t>(start_pos()) >\n                         std::numeric_limits<uint32_t>::max())) {\n    return FailOverflow();\n  }\n  return true;\n}\n\nabsl::Status SnappyWriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    status = dest.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `dest` with the compressed\n  // position. Clarify that the current position is the uncompressed position\n  // instead of delegating to `Writer::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status SnappyWriterBase::AnnotateOverDest(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nvoid SnappyWriterBase::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  if (write_size_hint == std::nullopt) {\n    options_.set_size_hint(std::nullopt);\n  } else {\n    options_.set_size_hint(\n        SaturatingIntCast<size_t>(SaturatingAdd(pos(), *write_size_hint)));\n  }\n}\n\nbool SnappyWriterBase::PushSlow(size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n  if (ABSL_PREDICT_FALSE(min_length > std::numeric_limits<size_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  const absl::Span<char> buffer = uncompressed_.AppendFixedBuffer(\n      UnsignedMax(\n          ApplySizeHint(\n              RoundUp<kBlockSize>(IntCast<size_t>(start_pos()) + min_length) -\n                  IntCast<size_t>(start_pos()),\n              options_.size_hint(), IntCast<size_t>(start_pos())),\n          min_length),\n      options_);\n  set_buffer(buffer.data(), buffer.size());\n  return true;\n}\n\nbool SnappyWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  if (src.size() <= MaxBytesToCopy()) return Writer::WriteSlow(std::move(src));\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<uint32_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  uncompressed_.Append(std::move(src), options_);\n  return true;\n}\n\nbool SnappyWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (src.size() <= MaxBytesToCopy()) return Writer::WriteSlow(src);\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<uint32_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  uncompressed_.Append(src, options_);\n  return true;\n}\n\nbool SnappyWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  if (src.size() <= MaxBytesToCopy()) {\n    // Not `std::move(src)`: forward to `Writer::WriteSlow(const Chain&)`,\n    // because `Writer::WriteSlow(Chain&&)` would forward to\n    // `SnappyWriterBase::WriteSlow(const Chain&)`.\n    return Writer::WriteSlow(src);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<uint32_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  uncompressed_.Append(std::move(src), options_);\n  return true;\n}\n\nbool SnappyWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (src.size() <= MaxBytesToCopy()) return Writer::WriteSlow(src);\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<uint32_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  uncompressed_.Append(src, options_);\n  return true;\n}\n\nbool SnappyWriterBase::WriteSlow(absl::Cord&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord&&): \"\n         \"enough space available, use Write(Cord&&) instead\";\n  if (src.size() <= MaxBytesToCopy()) {\n    // Not `std::move(src)`: forward to `Writer::WriteSlow(const absl::Cord&)`,\n    // because `Writer::WriteSlow(absl::Cord&&)` would forward to\n    // `SnappyWriterBase::WriteSlow(const absl::Cord&)`.\n    return Writer::WriteSlow(src);\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n  if (ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<uint32_t>::max() -\n                                          IntCast<size_t>(start_pos()))) {\n    return FailOverflow();\n  }\n  move_start_pos(src.size());\n  uncompressed_.Append(std::move(src), options_);\n  return true;\n}\n\nbool SnappyWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(IntCast<size_t>(pos()) >\n                             std::numeric_limits<uint32_t>::max() ||\n                         src.size() > std::numeric_limits<uint32_t>::max() -\n                                          IntCast<size_t>(pos()))) {\n    return FailOverflow();\n  }\n  const size_t first_length = UnsignedMin(\n      RoundUp<kBlockSize>(IntCast<size_t>(pos())) - IntCast<size_t>(pos()),\n      IntCast<size_t>(src.size()));\n  if (ABSL_PREDICT_FALSE(!Push(first_length))) return false;\n  riegeli::null_safe_memset(cursor(), src.fill(), first_length);\n  move_cursor(first_length);\n  src.Extract(first_length);\n  Write(src.Extract(RoundDown<kBlockSize>(IntCast<size_t>(src.size()))));\n  const size_t last_length = IntCast<size_t>(src.size());\n  if (ABSL_PREDICT_FALSE(!Push(last_length))) return false;\n  riegeli::null_safe_memset(cursor(), src.fill(), last_length);\n  move_cursor(last_length);\n  return true;\n}\n\nReader* SnappyWriterBase::ReadModeImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return nullptr;\n  ChainReader<>* const reader = associated_reader_.ResetReader(&uncompressed_);\n  reader->Seek(initial_pos);\n  return reader;\n}\n\nnamespace snappy_internal {\n\nabsl::Status SnappyCompressImpl(Reader& src, Writer& dest,\n                                SnappyCompressOptions options) {\n  const std::optional<Position> size = src.Size();\n  if (ABSL_PREDICT_FALSE(size == std::nullopt)) return src.status();\n  if (ABSL_PREDICT_FALSE(*size > std::numeric_limits<uint32_t>::max())) {\n    return absl::ResourceExhaustedError(absl::StrCat(\n        \"Uncompressed data too large for snappy compression: \", *size, \" > \",\n        std::numeric_limits<uint32_t>::max()));\n  }\n  ReaderSnappySource source(&src, *size);\n  WriterSnappySink sink(&dest);\n  snappy::Compress(&source, &sink, {/*level=*/options.compression_level()});\n  if (ABSL_PREDICT_FALSE(!dest.ok())) return dest.status();\n  if (ABSL_PREDICT_FALSE(!src.ok())) return src.status();\n  return absl::OkStatus();\n}\n\n}  // namespace snappy_internal\n\nsize_t SnappyMaxCompressedSize(size_t uncompressed_size) {\n  return snappy::MaxCompressedLength(uncompressed_size);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/snappy/snappy_writer.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_SNAPPY_SNAPPY_WRITER_H_\n#define RIEGELI_SNAPPY_SNAPPY_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"snappy.h\"\n\nnamespace riegeli {\n\ntemplate <typename Src>\nclass ChainReader;\nclass Reader;\n\n// Template parameter independent part of `SnappyWriter`.\nclass SnappyWriterBase : public Writer {\n public:\n  class Options {\n   public:\n    Options() noexcept {}\n\n    // Tunes the tradeoff between compression density and compression speed\n    // (higher = better density but slower).\n    //\n    // `compression_level` must be between `kMinCompressionLevel` (1) and\n    // `kMaxCompressionLevel` (2).\n    static constexpr int kMinCompressionLevel =\n        snappy::CompressionOptions::MinCompressionLevel();\n    static constexpr int kMaxCompressionLevel =\n        snappy::CompressionOptions::MaxCompressionLevel();\n    static constexpr int kDefaultCompressionLevel =\n        snappy::CompressionOptions::DefaultCompressionLevel();\n    Options& set_compression_level(int compression_level) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(compression_level, kMinCompressionLevel)\n          << \"Failed precondition of \"\n             \"SnappyWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      RIEGELI_ASSERT_LE(compression_level, kMaxCompressionLevel)\n          << \"Failed precondition of \"\n             \"SnappyWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      compression_level_ = compression_level;\n      return *this;\n    }\n    Options&& set_compression_level(int compression_level) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_compression_level(compression_level));\n    }\n    int compression_level() const { return compression_level_; }\n\n   private:\n    int compression_level_ = kDefaultCompressionLevel;\n  };\n\n  // Returns the compressed `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsReadMode() override { return true; }\n\n protected:\n  explicit SnappyWriterBase(Closed) noexcept : Writer(kClosed) {}\n\n  explicit SnappyWriterBase();\n\n  SnappyWriterBase(SnappyWriterBase&& that) noexcept;\n  SnappyWriterBase& operator=(SnappyWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset();\n  void Initialize(Writer* dest, int compression_level);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(absl::Cord&& src) override;\n  bool WriteSlow(ByteFill src) override;\n  Reader* ReadModeImpl(Position initial_pos) override;\n\n private:\n  // `snappy::kBlockSize`\n  static constexpr size_t kBlockSize = size_t{64} << 10;\n\n  // When deciding whether to copy an array of bytes or share memory, prefer\n  // copying up to this length.\n  size_t MaxBytesToCopy() const;\n\n  // Discards uninitialized space from the end of `uncompressed_`, so that it\n  // contains only actual data written.\n  bool SyncBuffer();\n\n  void MoveUncompressed(SnappyWriterBase& that);\n\n  int compression_level_ = Options::kDefaultCompressionLevel;\n  Chain::Options options_;\n\n  // `Writer` methods are similar to `ChainWriter` methods writing to\n  // `uncompressed_`.\n  //\n  // `snappy::Compress()` reads data in 64KB blocks, and copies a block to a\n  // scratch buffer if it is not contiguous. Hence `Writer` methods try to\n  // ensure that each 64KB block of `uncompressed_` is contiguous (unless that\n  // would require earlier memory copies).\n  Chain uncompressed_;\n\n  AssociatedReader<ChainReader<const Chain*>> associated_reader_;\n\n  // Invariant: `limit_pos() <= std::numeric_limits<size_t>::max()`\n};\n\n// A `Writer` which compresses data with Snappy before passing it to another\n// `Writer`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Writer` must not be accessed until the `SnappyWriter` is\n// closed or no longer used.\n//\n// `SnappyWriter` does not compress incrementally but buffers uncompressed data\n// and compresses them all in `Close()`.\n//\n// `Flush()` does nothing. It does not make data written so far visible.\ntemplate <typename Dest = Writer*>\nclass SnappyWriter : public SnappyWriterBase {\n public:\n  // Creates a closed `SnappyWriter`.\n  explicit SnappyWriter(Closed) noexcept : SnappyWriterBase(kClosed) {}\n\n  // Will write to the compressed `Writer` provided by `dest`.\n  explicit SnappyWriter(Initializer<Dest> dest, Options options = Options());\n\n  SnappyWriter(SnappyWriter&& that) = default;\n  SnappyWriter& operator=(SnappyWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `SnappyWriter`. This avoids\n  // constructing a temporary `SnappyWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n\n private:\n  // The object providing and possibly owning the compressed `Writer`.\n  Dependency<Writer*, Dest> dest_;\n};\n\nexplicit SnappyWriter(Closed) -> SnappyWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit SnappyWriter(Dest&& dest, SnappyWriterBase::Options options =\n                                       SnappyWriterBase::Options())\n    -> SnappyWriter<TargetT<Dest>>;\n\n// An alternative interface to Snappy which avoids buffering uncompressed data.\n// Calling `SnappyCompress()` is equivalent to copying all data from `src` to a\n// `SnappyWriter<Dest&&>`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the uncompressed `Reader`. `Src` must support\n// `DependencyRef<Reader*, Src>`, e.g. `Reader&` (not owned),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `AnyRef<Reader*>` (maybe owned).\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Writer`. `Dest` must support\n// `DependencyRef<Writer*, Dest>`, e.g. `Writer&` (not owned),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `AnyRef<Writer*>` (maybe owned).\n//\n// The uncompressed `Reader` must support `Size()`. To supply or override this\n// size, the `Reader` can be wrapped in a `LimitingReader` with\n// `LimitingReaderBase::Options().set_exact_length(size)`.\n\nusing SnappyCompressOptions = SnappyWriterBase::Options;\n\ntemplate <typename Src, typename Dest,\n          std::enable_if_t<\n              std::conjunction_v<TargetRefSupportsDependency<Reader*, Src>,\n                                 TargetRefSupportsDependency<Writer*, Dest>>,\n              int> = 0>\nabsl::Status SnappyCompress(\n    Src&& src, Dest&& dest,\n    SnappyCompressOptions options = SnappyCompressOptions());\n\n// Returns the maximum compressed size produced by the Snappy compressor for\n// data of the given uncompressed size.\nsize_t SnappyMaxCompressedSize(size_t uncompressed_size);\n\n// Implementation details follow.\n\ninline SnappyWriterBase::SnappyWriterBase()\n    : options_(Chain::Options().set_block_size(kBlockSize)) {}\n\ninline SnappyWriterBase::SnappyWriterBase(SnappyWriterBase&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)),\n      compression_level_(that.compression_level_),\n      options_(that.options_),\n      associated_reader_(std::move(that.associated_reader_)) {\n  MoveUncompressed(that);\n}\n\ninline SnappyWriterBase& SnappyWriterBase::operator=(\n    SnappyWriterBase&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  compression_level_ = that.compression_level_;\n  options_ = that.options_;\n  associated_reader_ = std::move(that.associated_reader_);\n  MoveUncompressed(that);\n  return *this;\n}\n\ninline void SnappyWriterBase::Reset(Closed) {\n  Writer::Reset(kClosed);\n  compression_level_ = Options::kDefaultCompressionLevel;\n  options_ = Chain::Options();\n  uncompressed_ = Chain();\n  associated_reader_.Reset();\n}\n\ninline void SnappyWriterBase::Reset() {\n  Writer::Reset();\n  compression_level_ = Options::kDefaultCompressionLevel;\n  options_ = Chain::Options().set_block_size(kBlockSize);\n  uncompressed_.Clear();\n  associated_reader_.Reset();\n}\n\ninline void SnappyWriterBase::Initialize(Writer* dest, int compression_level) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of SnappyWriter: null Writer pointer\";\n  compression_level_ = compression_level;\n  if (ABSL_PREDICT_FALSE(!dest->ok())) {\n    FailWithoutAnnotation(AnnotateOverDest(dest->status()));\n  }\n}\n\ninline void SnappyWriterBase::MoveUncompressed(SnappyWriterBase& that) {\n  const bool uses_buffer = start() != nullptr;\n  if (uses_buffer) {\n    RIEGELI_ASSERT(that.uncompressed_.blocks().back().data() +\n                       that.uncompressed_.blocks().back().size() ==\n                   limit())\n        << \"Failed invariant of SnappyWriter: \"\n           \"uncompressed data inconsistent with buffer pointers\";\n    RIEGELI_ASSERT_EQ(that.uncompressed_.size(), limit_pos())\n        << \"Failed invariant of SnappyWriter: \"\n           \"uncompressed data inconsistent with buffer pointers\";\n  }\n  const size_t saved_start_to_cursor = start_to_cursor();\n  uncompressed_ = std::move(that.uncompressed_);\n  if (uses_buffer) {\n    const size_t buffer_size =\n        uncompressed_.size() - IntCast<size_t>(start_pos());\n    const absl::string_view last_block = uncompressed_.blocks().back();\n    set_buffer(\n        const_cast<char*>(last_block.data() + last_block.size()) - buffer_size,\n        buffer_size, saved_start_to_cursor);\n  }\n}\n\ntemplate <typename Dest>\ninline SnappyWriter<Dest>::SnappyWriter(Initializer<Dest> dest, Options options)\n    : dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.compression_level());\n}\n\ntemplate <typename Dest>\ninline void SnappyWriter<Dest>::Reset(Closed) {\n  SnappyWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void SnappyWriter<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  SnappyWriterBase::Reset();\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.compression_level());\n}\n\ntemplate <typename Dest>\nvoid SnappyWriter<Dest>::Done() {\n  SnappyWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\nnamespace snappy_internal {\n\nabsl::Status SnappyCompressImpl(Reader& src, Writer& dest,\n                                SnappyCompressOptions options);\n\n}  // namespace snappy_internal\n\ntemplate <typename Src, typename Dest,\n          std::enable_if_t<\n              std::conjunction_v<TargetRefSupportsDependency<Reader*, Src>,\n                                 TargetRefSupportsDependency<Writer*, Dest>>,\n              int>>\ninline absl::Status SnappyCompress(Src&& src, Dest&& dest,\n                                   SnappyCompressOptions options) {\n  DependencyRef<Reader*, Src> src_dep(std::forward<Src>(src));\n  DependencyRef<Writer*, Dest> dest_dep(std::forward<Dest>(dest));\n  if (src_dep.IsOwning()) src_dep->SetReadAllHint(true);\n  absl::Status status =\n      snappy_internal::SnappyCompressImpl(*src_dep, *dest_dep, options);\n  if (dest_dep.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_dep->Close())) {\n      status.Update(dest_dep->status());\n    }\n  }\n  if (src_dep.IsOwning()) {\n    if (ABSL_PREDICT_TRUE(status.ok())) src_dep->VerifyEnd();\n    if (ABSL_PREDICT_FALSE(!src_dep->Close())) status.Update(src_dep->status());\n  }\n  return status;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_SNAPPY_SNAPPY_WRITER_H_\n"
  },
  {
    "path": "riegeli/tensorflow/BUILD",
    "content": "exports_files([\n    \"kernels/riegeli_dataset_ops.cc\",\n    \"ops/riegeli_dataset_ops.cc\",\n])\n"
  },
  {
    "path": "riegeli/tensorflow/io/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"file_reader\",\n    srcs = [\"file_reader.cc\"],\n    hdrs = [\"file_reader.h\"],\n    deps = [\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/types:span\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:null_safe_memcpy\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:sized_shared_buffer\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:backward_writer\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:path_ref\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        # TODO: Riegeli/TensorFlow build is currently broken\n        # in open source because TensorFlow does not support bzlmod yet.\n    ],\n)\n\ncc_library(\n    name = \"file_writer\",\n    srcs = [\"file_writer.cc\"],\n    hdrs = [\"file_writer.h\"],\n    deps = [\n        \":file_reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:buffering\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:external_ref\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:reset\",\n        \"//riegeli/base:shared_buffer\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:path_ref\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        # TODO: Riegeli/TensorFlow build is currently broken\n        # in open source because TensorFlow does not support bzlmod yet.\n    ],\n)\n\ncc_library(\n    name = \"tstring_writer\",\n    hdrs = [\"tstring_writer.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/bytes:resizable_writer\",\n        # TODO: Riegeli/TensorFlow build is currently broken\n        # in open source because TensorFlow does not support bzlmod yet.\n    ],\n)\n"
  },
  {
    "path": "riegeli/tensorflow/io/file_reader.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/tensorflow/io/file_reader.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <cstring>\n#include <limits>\n#include <memory>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/null_safe_memcpy.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/sized_shared_buffer.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/path_ref.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"tensorflow/compiler/xla/tsl/platform/env.h\"\n#include \"tensorflow/compiler/xla/tsl/platform/file_system.h\"\n#include \"tensorflow/core/public/version.h\"\n\nnamespace riegeli::tensorflow {\n\nbool FileReaderBase::InitializeFilename(tsl::RandomAccessFile* src) {\n  absl::string_view filename;\n  if (const absl::Status status = src->Name(&filename);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    filename_ = \"<unknown>\";\n    if (!absl::IsUnimplemented(status)) {\n      return FailOperation(status, \"RandomAccessFile::Name()\");\n    }\n    return true;\n  }\n  return InitializeFilename(filename);\n}\n\nbool FileReaderBase::InitializeFilename(PathInitializer filename) {\n  riegeli::Reset(filename_, std::move(filename));\n  if (const absl::Status status =\n          env_->GetFileSystemForFile(filename_, &file_system_);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return FailOperation(status, \"Env::GetFileSystemForFile()\");\n  }\n  return true;\n}\n\nstd::unique_ptr<tsl::RandomAccessFile> FileReaderBase::OpenFile() {\n  std::unique_ptr<tsl::RandomAccessFile> src;\n  if (const absl::Status status =\n          file_system_->NewRandomAccessFile(filename_, &src);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    Reader::Reset(kClosed);\n    FailOperation(status, \"FileSystem::NewRandomAccessFile()\");\n    RIEGELI_ASSERT(src == nullptr)\n        << \"FileSystem::NewRandomAccessFile() should store \"\n           \"null RandomAccessFile on failure\";\n  }\n  return src;\n}\n\nvoid FileReaderBase::InitializePos(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(initial_pos > std::numeric_limits<uint64_t>::max())) {\n    FailOverflow();\n    return;\n  }\n  set_limit_pos(initial_pos);\n  buffer_sizer_.BeginRun(limit_pos());\n}\n\nvoid FileReaderBase::Done() {\n  Reader::Done();\n  buffer_ = SizedSharedBuffer();\n}\n\ninline bool FileReaderBase::FailOperation(const absl::Status& status,\n                                          absl::string_view operation) {\n  RIEGELI_ASSERT(!status.ok())\n      << \"Failed precondition of FileReaderBase::FailOperation(): \"\n         \"status not failed\";\n  return Fail(\n      Annotate(absl::Status(static_cast<absl::StatusCode>(status.code()),\n#if TF_GRAPH_DEF_VERSION < 1467\n                            status.error_message()\n#else\n                            status.message()\n#endif\n                                ),\n               absl::StrCat(operation, \" failed\")));\n}\n\ninline absl::Status FileReaderBase::NoRandomAccessStatus() {\n  return absl::UnimplementedError(\"A filename required for random access\");\n}\n\nabsl::Status FileReaderBase::AnnotateStatusImpl(absl::Status status) {\n  return Reader::AnnotateStatusImpl(\n      Annotate(status, absl::StrCat(\"reading \", filename_)));\n}\n\ninline void FileReaderBase::SyncBuffer() {\n  buffer_.Clear();\n  set_buffer();\n}\n\nvoid FileReaderBase::SetReadAllHintImpl(bool read_all_hint) {\n  buffer_sizer_.set_read_all_hint(read_all_hint);\n}\n\nbool FileReaderBase::PullSlow(size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Reader::PullSlow(): \"\n         \"enough data available, use Pull() instead\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  tsl::RandomAccessFile* const src = SrcFile();\n  const size_t available_length = available();\n  const size_t buffer_length = buffer_sizer_.BufferLength(\n      limit_pos(), min_length - available_length,\n      SaturatingSub(recommended_length, available_length));\n  if (ABSL_PREDICT_FALSE(buffer_length == 0)) return false;\n  size_t cursor_index;\n  absl::Span<char> flat_buffer;\n  if (buffer_.empty()) {\n    // Copy available data to `buffer_` so that newly read data will be adjacent\n    // to available data.\n    cursor_index = 0;\n    flat_buffer = buffer_.AppendFixedBuffer(available_length + buffer_length);\n    riegeli::null_safe_memcpy(flat_buffer.data(), cursor(), available_length);\n    flat_buffer.remove_prefix(available_length);\n  } else {\n    cursor_index = start_to_cursor();\n    flat_buffer = buffer_.AppendBufferIfExisting(buffer_length);\n    if (flat_buffer.empty()) {\n      // Not enough space in `buffer_`. Resize `buffer_`, keeping available\n      // data.\n      buffer_.RemovePrefix(cursor_index);\n      buffer_.Shrink(available_length + buffer_length);\n      cursor_index = 0;\n      flat_buffer = buffer_.AppendFixedBuffer(buffer_length);\n    }\n  }\n  // Read more data, preferably into `buffer_`.\n  ReadToBuffer(cursor_index, src, flat_buffer);\n  return available() >= min_length;\n}\n\ninline bool FileReaderBase::ReadToDest(size_t length,\n                                       tsl::RandomAccessFile* src, char* dest) {\n  if (ABSL_PREDICT_FALSE(limit_pos() >= std::numeric_limits<uint64_t>::max())) {\n    return FailOverflow();\n  }\n  const size_t length_to_read =\n      UnsignedMin(length, std::numeric_limits<uint64_t>::max() - limit_pos());\n  absl::string_view result;\n  const absl::Status status = src->Read(IntCast<uint64_t>(limit_pos()), result,\n                                        absl::MakeSpan(dest, length_to_read));\n  RIEGELI_ASSERT_LE(result.size(), length_to_read)\n      << \"RandomAccessFile::Read() read more than requested\";\n  if (result.data() != dest) std::memcpy(dest, result.data(), result.size());\n  move_limit_pos(result.size());\n  if (ABSL_PREDICT_FALSE(!status.ok())) {\n    if (ABSL_PREDICT_FALSE(!absl::IsOutOfRange(status))) {\n      return FailOperation(status, \"RandomAccessFile::Read()\");\n    }\n    if (!growing_source_) set_exact_size(limit_pos());\n    return false;\n  }\n  RIEGELI_ASSERT_EQ(result.size(), length_to_read)\n      << \"RandomAccessFile::Read() succeeded but read less than requested\";\n  if (ABSL_PREDICT_FALSE(result.size() < length)) {\n    // `result.size() == length_to_read < length`, which implies that\n    // `std::numeric_limits<uint64_t>::max()` was reached.\n    RIEGELI_ASSERT_EQ(limit_pos(), std::numeric_limits<uint64_t>::max())\n        << \"Maximum position must have been reached\";\n    return FailOverflow();\n  }\n  return true;\n}\n\ninline bool FileReaderBase::ReadToBuffer(size_t cursor_index,\n                                         tsl::RandomAccessFile* src,\n                                         absl::Span<char> flat_buffer) {\n  RIEGELI_ASSERT(flat_buffer.data() + flat_buffer.size() ==\n                 buffer_.data() + buffer_.size())\n      << \"Failed precondition of FileReaderBase::ReadToBuffer(): \"\n         \"flat_buffer not a suffix of buffer_\";\n  if (ABSL_PREDICT_FALSE(limit_pos() >= std::numeric_limits<uint64_t>::max())) {\n    buffer_.RemoveSuffix(flat_buffer.size());\n    set_buffer(buffer_.data(), buffer_.size(), cursor_index);\n    return FailOverflow();\n  }\n  const size_t length_to_read = UnsignedMin(\n      flat_buffer.size(), std::numeric_limits<uint64_t>::max() - limit_pos());\n  absl::string_view result;\n  const absl::Status status =\n      src->Read(IntCast<uint64_t>(limit_pos()), result,\n                absl::MakeSpan(flat_buffer.data(), length_to_read));\n  RIEGELI_ASSERT_LE(result.size(), length_to_read)\n      << \"RandomAccessFile::Read() read more than requested\";\n  if (result.data() == flat_buffer.data()) {\n    buffer_.RemoveSuffix(flat_buffer.size() - result.size());\n    set_buffer(buffer_.data(), buffer_.size(), cursor_index);\n  } else if (buffer_.size() > cursor_index + flat_buffer.size()) {\n    // Copy newly read data to `buffer_` so that they are adjacent to previously\n    // available data.\n    std::memcpy(flat_buffer.data(), result.data(), result.size());\n    buffer_.RemoveSuffix(flat_buffer.size() - result.size());\n    set_buffer(buffer_.data(), buffer_.size(), cursor_index);\n  } else {\n    buffer_.Clear();\n    set_buffer(result.data(), result.size());\n  }\n  move_limit_pos(result.size());\n  if (ABSL_PREDICT_FALSE(!status.ok())) {\n    if (ABSL_PREDICT_FALSE(!absl::IsOutOfRange(status))) {\n      return FailOperation(status, \"RandomAccessFile::Read()\");\n    }\n    if (!growing_source_) set_exact_size(limit_pos());\n    return false;\n  }\n  RIEGELI_ASSERT_EQ(result.size(), length_to_read)\n      << \"RandomAccessFile::Read() succeeded but read less than requested\";\n  if (ABSL_PREDICT_FALSE(result.size() < flat_buffer.size())) {\n    // `result.size() == length_to_read < flat_buffer.size()`, which implies\n    // that `std::numeric_limits<uint64_t>::max()` was reached.\n    RIEGELI_ASSERT_EQ(limit_pos(), std::numeric_limits<uint64_t>::max())\n        << \"Maximum position must have been reached\";\n    return FailOverflow();\n  }\n  return true;\n}\n\nbool FileReaderBase::ReadSlow(size_t length, char* dest) {\n  RIEGELI_ASSERT_LT(available(), length)\n      << \"Failed precondition of Reader::ReadSlow(char*): \"\n         \"enough data available, use Read(char*) instead\";\n  if (length >= buffer_sizer_.BufferLength(pos())) {\n    // Read directly to `dest`.\n    const size_t available_length = available();\n    riegeli::null_safe_memcpy(dest, cursor(), available_length);\n    dest += available_length;\n    length -= available_length;\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    tsl::RandomAccessFile* const src = SrcFile();\n    SyncBuffer();\n    size_t length_to_read = length;\n    if (exact_size() != std::nullopt) {\n      if (ABSL_PREDICT_FALSE(limit_pos() >= *exact_size())) return false;\n      length_to_read = UnsignedMin(length_to_read, *exact_size() - limit_pos());\n    }\n    if (ABSL_PREDICT_FALSE(!ReadToDest(length_to_read, src, dest))) {\n      return false;\n    }\n    return length_to_read >= length;\n  }\n  return Reader::ReadSlow(length, dest);\n}\n\nbool FileReaderBase::ReadSlow(size_t length, Chain& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"enough data available, use Read(Chain&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Chain&): \"\n         \"Chain size overflow\";\n  tsl::RandomAccessFile* const src = SrcFile();\n  bool enough_read = true;\n  while (length > available()) {\n    const size_t available_length = available();\n    if (ABSL_PREDICT_FALSE(!ok())) {\n      // Read as much as is available.\n      enough_read = false;\n      length = available_length;\n      break;\n    }\n    const size_t buffer_length =\n        buffer_sizer_.BufferLength(limit_pos(), 1, length - available_length);\n    size_t cursor_index;\n    absl::Span<char> flat_buffer;\n    if (buffer_.empty()) {\n      // Do not extend `buffer_` if available data are outside of `buffer_`,\n      // because available data would be lost.\n      dest.Append(absl::string_view(cursor(), available_length));\n      length -= available_length;\n      if (ABSL_PREDICT_FALSE(buffer_length == 0)) {\n        set_buffer();\n        return false;\n      }\n      cursor_index = 0;\n      flat_buffer = buffer_.AppendFixedBuffer(buffer_length);\n    } else {\n      cursor_index = start_to_cursor();\n      flat_buffer = buffer_.AppendBufferIfExisting(buffer_length);\n      if (flat_buffer.empty()) {\n        // Not enough space in `buffer_`. Append available data to `dest` and\n        // make a new buffer.\n        dest.Append(ExternalRef(std::move(buffer_),\n                                absl::string_view(cursor(), available_length)));\n        length -= available_length;\n        buffer_.ClearAndShrink(buffer_length);\n        if (ABSL_PREDICT_FALSE(buffer_length == 0)) {\n          set_buffer();\n          return false;\n        }\n        cursor_index = 0;\n        flat_buffer = buffer_.AppendFixedBuffer(buffer_length);\n      }\n    }\n    // Read more data, preferably into `buffer_`.\n    if (ABSL_PREDICT_FALSE(!ReadToBuffer(cursor_index, src, flat_buffer))) {\n      // Read as much as is available.\n      enough_read = available() >= length;\n      if (ABSL_PREDICT_FALSE(!enough_read)) length = available();\n      break;\n    }\n  }\n  if (buffer_.empty()) {\n    dest.Append(absl::string_view(cursor(), length));\n  } else {\n    dest.Append(ExternalRef(buffer_, absl::string_view(cursor(), length)));\n  }\n  move_cursor(length);\n  return enough_read;\n}\n\nbool FileReaderBase::ReadSlow(size_t length, absl::Cord& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"enough data available, use Read(Cord&) instead\";\n  RIEGELI_ASSERT_LE(length, std::numeric_limits<size_t>::max() - dest.size())\n      << \"Failed precondition of Reader::ReadSlow(Cord&): \"\n         \"Cord size overflow\";\n  tsl::RandomAccessFile* const src = SrcFile();\n  bool enough_read = true;\n  while (length > available()) {\n    const size_t available_length = available();\n    if (ABSL_PREDICT_FALSE(!ok())) {\n      // Read as much as is available.\n      enough_read = false;\n      length = available_length;\n      break;\n    }\n    const size_t buffer_length =\n        buffer_sizer_.BufferLength(limit_pos(), 1, length - available_length);\n    size_t cursor_index;\n    absl::Span<char> flat_buffer;\n    if (buffer_.empty()) {\n      // Do not extend `buffer_` if available data are outside of `buffer_`,\n      // because available data would be lost.\n      dest.Append(absl::string_view(cursor(), available_length));\n      length -= available_length;\n      if (ABSL_PREDICT_FALSE(buffer_length == 0)) {\n        set_buffer();\n        return false;\n      }\n      cursor_index = 0;\n      flat_buffer = buffer_.AppendFixedBuffer(buffer_length);\n    } else {\n      cursor_index = start_to_cursor();\n      flat_buffer = buffer_.AppendBufferIfExisting(buffer_length);\n      if (flat_buffer.empty()) {\n        // Not enough space in `buffer_`. Append available data to `dest` and\n        // make a new buffer.\n        ExternalRef(std::move(buffer_),\n                    absl::string_view(cursor(), available_length))\n            .AppendTo(dest);\n        length -= available_length;\n        buffer_.ClearAndShrink(buffer_length);\n        if (ABSL_PREDICT_FALSE(buffer_length == 0)) {\n          set_buffer();\n          return false;\n        }\n        cursor_index = 0;\n        flat_buffer = buffer_.AppendFixedBuffer(buffer_length);\n      }\n    }\n    // Read more data, preferably into `buffer_`.\n    if (ABSL_PREDICT_FALSE(!ReadToBuffer(cursor_index, src, flat_buffer))) {\n      // Read as much as is available.\n      enough_read = available() >= length;\n      if (ABSL_PREDICT_FALSE(!enough_read)) length = available();\n      break;\n    }\n  }\n  if (buffer_.empty()) {\n    dest.Append(absl::string_view(cursor(), length));\n  } else {\n    ExternalRef(buffer_, absl::string_view(cursor(), length)).AppendTo(dest);\n  }\n  move_cursor(length);\n  return enough_read;\n}\n\nbool FileReaderBase::CopySlow(Position length, Writer& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(Writer&): \"\n         \"enough data available, use Copy(Writer&) instead\";\n  tsl::RandomAccessFile* const src = SrcFile();\n  bool enough_read = true;\n  while (length > available()) {\n    const size_t available_length = available();\n    if (ABSL_PREDICT_FALSE(!ok())) {\n      // Copy as much as is available.\n      length = available_length;\n      enough_read = false;\n      break;\n    }\n    const bool read_directly = length >= buffer_sizer_.BufferLength(pos());\n    if (read_directly) {\n      if (buffer_.empty() || available_length <= kMaxBytesToCopy) {\n        if (ABSL_PREDICT_FALSE(\n                !dest.Write(absl::string_view(cursor(), available_length)))) {\n          move_cursor(available_length);\n          return false;\n        }\n        length -= available_length;\n        SyncBuffer();\n        return CopyUsingPush(length, src, dest);\n      }\n      // It is better to write available data from `buffer_` as a `Chain` before\n      // reading directly to `dest`. Before that, `buffer_` might need to be\n      // filled more to avoid attaching a wasteful `Chain`.\n    }\n    const size_t buffer_length =\n        buffer_sizer_.BufferLength(limit_pos(), 1, length - available_length);\n    size_t cursor_index;\n    absl::Span<char> flat_buffer;\n    if (buffer_.empty()) {\n      // Do not extend `buffer_` if available data are outside of `buffer_`,\n      // because available data would be lost.\n      if (ABSL_PREDICT_FALSE(\n              !dest.Write(absl::string_view(cursor(), available_length)))) {\n        move_cursor(available_length);\n        return false;\n      }\n      length -= available_length;\n      if (ABSL_PREDICT_FALSE(buffer_length == 0)) {\n        set_buffer();\n        return false;\n      }\n      cursor_index = 0;\n      flat_buffer = buffer_.AppendFixedBuffer(buffer_length);\n    } else {\n      cursor_index = start_to_cursor();\n      flat_buffer = buffer_.AppendBufferIfExisting(buffer_length);\n      if (flat_buffer.empty()) {\n        // Not enough space in `buffer_`. Append available data to `dest` and\n        // make a new buffer.\n        if (available_length > 0) {\n          const bool write_ok = dest.Write(\n              ExternalRef(std::move(buffer_),\n                          absl::string_view(cursor(), available_length)));\n          if (ABSL_PREDICT_FALSE(!write_ok)) {\n            buffer_.ClearAndShrink(buffer_length);\n            set_buffer();\n            return false;\n          }\n          length -= available_length;\n        }\n        buffer_.ClearAndShrink(buffer_length);\n        if (ABSL_PREDICT_FALSE(buffer_length == 0)) {\n          set_buffer();\n          return false;\n        }\n        if (read_directly) {\n          set_buffer();\n          return CopyUsingPush(length, src, dest);\n        }\n        cursor_index = 0;\n        flat_buffer = buffer_.AppendFixedBuffer(buffer_length);\n      }\n    }\n    // Read more data, preferably into `buffer_`.\n    if (ABSL_PREDICT_FALSE(!ReadToBuffer(cursor_index, src, flat_buffer))) {\n      // Copy as much as is available.\n      enough_read = available() >= length;\n      if (ABSL_PREDICT_FALSE(!enough_read)) length = available();\n      break;\n    }\n  }\n  const bool write_ok =\n      buffer_.empty()\n          ? dest.Write(absl::string_view(cursor(), IntCast<size_t>(length)))\n          : dest.Write(ExternalRef(\n                buffer_, absl::string_view(cursor(), IntCast<size_t>(length))));\n  move_cursor(IntCast<size_t>(length));\n  return write_ok && enough_read;\n}\n\ninline bool FileReaderBase::CopyUsingPush(Position length,\n                                          tsl::RandomAccessFile* src,\n                                          Writer& dest) {\n  RIEGELI_ASSERT_GT(length, 0u)\n      << \"Failed precondition of FileReaderBase::CopyUsingPush(): \"\n         \"nothing to copy\";\n  do {\n    size_t length_to_read = SaturatingIntCast<size_t>(length);\n    if (exact_size() != std::nullopt) {\n      if (ABSL_PREDICT_FALSE(limit_pos() >= *exact_size())) return false;\n      length_to_read = UnsignedMin(length_to_read, *exact_size() - limit_pos());\n    }\n    if (ABSL_PREDICT_FALSE(!dest.Push(1, length_to_read))) return false;\n    const size_t length_to_copy = UnsignedMin(length_to_read, dest.available());\n    const Position pos_before = limit_pos();\n    const bool read_ok = ReadToDest(length_to_copy, src, dest.cursor());\n    const Position length_read = limit_pos() - pos_before;\n    dest.move_cursor(IntCast<size_t>(length_read));\n    if (ABSL_PREDICT_FALSE(!read_ok)) return false;\n    length -= length_read;\n  } while (length > 0);\n  return true;\n}\n\nbool FileReaderBase::CopySlow(size_t length, BackwardWriter& dest) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), length)\n      << \"Failed precondition of Reader::CopySlow(BackwardWriter&): \"\n         \"enough data available, use Copy(BackwardWriter&) instead\";\n  if (length <= available() && buffer_.empty()) {\n    // Avoid writing an `absl::string_view` if available data are in `buffer_`,\n    // because in this case it is better to write a `Chain`.\n    const absl::string_view data(cursor(), length);\n    move_cursor(length);\n    return dest.Write(data);\n  }\n  if (length <= kMaxBytesToCopy) {\n    if (ABSL_PREDICT_FALSE(!dest.Push(length))) return false;\n    dest.move_cursor(length);\n    if (ABSL_PREDICT_FALSE(!ReadSlow(length, dest.cursor()))) {\n      dest.set_cursor(dest.cursor() + length);\n      return false;\n    }\n    return true;\n  }\n  Chain data;\n  if (ABSL_PREDICT_FALSE(!ReadSlow(length, data))) return false;\n  return dest.Write(std::move(data));\n}\n\nbool FileReaderBase::ReadSomeSlow(size_t max_length, char* dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"nothing to read, use ReadSome(char*) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::ReadSomeSlow(char*): \"\n         \"some data available, use ReadSome(char*) instead\";\n  if (max_length >= buffer_sizer_.BufferLength(limit_pos())) {\n    // Read directly to `dest`.\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    tsl::RandomAccessFile* const src = SrcFile();\n    SyncBuffer();\n    if (exact_size() != std::nullopt) {\n      if (ABSL_PREDICT_FALSE(limit_pos() >= *exact_size())) return false;\n      max_length = UnsignedMin(max_length, *exact_size() - limit_pos());\n    }\n    const Position pos_before = limit_pos();\n    ReadToDest(max_length, src, dest);\n    RIEGELI_ASSERT_GE(limit_pos(), pos_before)\n        << \"FileReaderBase::ReadToDest() decreased limit_pos()\";\n    return limit_pos() != pos_before;\n  }\n  return Reader::ReadSomeSlow(max_length, dest);\n}\n\nbool FileReaderBase::CopySomeSlow(size_t max_length, Writer& dest) {\n  RIEGELI_ASSERT_GT(max_length, 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"nothing to read, use CopySome(Writer&) instead\";\n  RIEGELI_ASSERT_EQ(available(), 0u)\n      << \"Failed precondition of Reader::CopySomeSlow(Writer&): \"\n         \"some data available, use CopySome(Writer&) instead\";\n  if (max_length >= buffer_sizer_.BufferLength(limit_pos())) {\n    // Copy directly to `dest`.\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    tsl::RandomAccessFile* const src = SrcFile();\n    SyncBuffer();\n    if (exact_size() != std::nullopt) {\n      if (ABSL_PREDICT_FALSE(limit_pos() >= *exact_size())) return false;\n      max_length = UnsignedMin(max_length, *exact_size() - limit_pos());\n    }\n    if (ABSL_PREDICT_FALSE(!dest.Push(1, max_length))) return false;\n    max_length = UnsignedMin(max_length, dest.available());\n    const Position pos_before = limit_pos();\n    ReadToDest(max_length, src, dest.cursor());\n    RIEGELI_ASSERT_GE(limit_pos(), pos_before)\n        << \"BufferedReader::ReadInternal() decreased limit_pos()\";\n    const Position length_read = limit_pos() - pos_before;\n    RIEGELI_ASSERT_LE(length_read, max_length)\n        << \"BufferedReader::ReadInternal() read more than requested\";\n    dest.move_cursor(IntCast<size_t>(length_read));\n    return length_read > 0;\n  }\n  return Reader::CopySomeSlow(max_length, dest);\n}\n\nbool FileReaderBase::SyncImpl(SyncType sync_type) {\n  const Position new_pos = pos();\n  buffer_sizer_.EndRun(new_pos);\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  set_limit_pos(new_pos);\n  buffer_sizer_.BeginRun(limit_pos());\n  return true;\n}\n\nbool FileReaderBase::SeekSlow(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of Reader::SeekSlow(): \"\n         \"position in the buffer, use Seek() instead\";\n  if (ABSL_PREDICT_FALSE(!FileReaderBase::SupportsRandomAccess())) {\n    if (ABSL_PREDICT_FALSE(new_pos < start_pos())) {\n      if (ok()) Fail(NoRandomAccessStatus());\n      return false;\n    }\n    return Reader::SeekSlow(new_pos);\n  }\n  buffer_sizer_.EndRun(pos());\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  SyncBuffer();\n  if (new_pos > limit_pos()) {\n    // Seeking forwards.\n    uint64_t file_size;\n    if (exact_size() != std::nullopt) {\n      file_size = IntCast<uint64_t>(*exact_size());\n    } else {\n      if (const absl::Status status =\n              file_system_->GetFileSize(filename_, &file_size);\n          ABSL_PREDICT_FALSE(!status.ok())) {\n        return FailOperation(status, \"FileSystem::GetFileSize()\");\n      }\n      if (!growing_source_) set_exact_size(Position{file_size});\n    }\n    if (ABSL_PREDICT_FALSE(new_pos > file_size)) {\n      // File ends.\n      set_limit_pos(Position{file_size});\n      buffer_sizer_.BeginRun(limit_pos());\n      return false;\n    }\n  }\n  set_limit_pos(new_pos);\n  buffer_sizer_.BeginRun(limit_pos());\n  return true;\n}\n\nstd::optional<Position> FileReaderBase::SizeImpl() {\n  if (ABSL_PREDICT_FALSE(!ok())) return std::nullopt;\n  if (exact_size() != std::nullopt) return *exact_size();\n  if (ABSL_PREDICT_FALSE(!FileReaderBase::SupportsRandomAccess())) {\n    Fail(NoRandomAccessStatus());\n    return std::nullopt;\n  }\n  uint64_t file_size;\n  if (const absl::Status status =\n          file_system_->GetFileSize(filename_, &file_size);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    FailOperation(status, \"FileSystem::GetFileSize()\");\n    return std::nullopt;\n  }\n  if (!growing_source_) set_exact_size(Position{file_size});\n  return Position{file_size};\n}\n\nstd::unique_ptr<Reader> FileReaderBase::NewReaderImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!FileReaderBase::SupportsRandomAccess())) {\n    if (ok()) Fail(NoRandomAccessStatus());\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point.\n  tsl::RandomAccessFile* const src = SrcFile();\n  std::unique_ptr<FileReader<tsl::RandomAccessFile*>> reader =\n      std::make_unique<FileReader<tsl::RandomAccessFile*>>(\n          src, FileReaderBase::Options()\n                   .set_env(env_)\n                   .set_initial_pos(initial_pos)\n                   .set_growing_source(growing_source_)\n                   .set_buffer_options(buffer_sizer_.buffer_options()));\n  reader->set_exact_size(exact_size());\n  return reader;\n}\n\nstd::unique_ptr<Reader> FileReaderBase::NewReaderCurrentPosImpl() {\n  std::unique_ptr<Reader> reader = FileReaderBase::NewReaderImpl(pos());\n  if (ABSL_PREDICT_TRUE(reader != nullptr)) {\n    // Share `buffer_` with `*reader`.\n    FileReaderBase* const file_reader =\n        static_cast<FileReaderBase*>(reader.get());\n    file_reader->buffer_ = buffer_;\n    file_reader->set_buffer(start(), start_to_limit(), start_to_cursor());\n    file_reader->set_limit_pos(limit_pos());\n  }\n  return reader;\n}\n\n}  // namespace riegeli::tensorflow\n"
  },
  {
    "path": "riegeli/tensorflow/io/file_reader.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_TENSORFLOW_IO_FILE_READER_H_\n#define RIEGELI_TENSORFLOW_IO_FILE_READER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/types/span.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/sized_shared_buffer.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/path_ref.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"tensorflow/compiler/xla/tsl/platform/env.h\"\n#include \"tensorflow/compiler/xla/tsl/platform/file_system.h\"\n\nnamespace riegeli {\n\nclass BackwardWriter;\nclass Writer;\n\nnamespace tensorflow {\n\n// Template parameter independent part of `FileReader`.\nclass FileReaderBase : public Reader {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // `FileReader` has a larger `kDefaultMaxBufferSize` (1M) because remote\n    // file access may have high latency of each operation.\n    static constexpr size_t kDefaultMaxBufferSize = size_t{1} << 20;\n\n    // Overrides the TensorFlow environment.\n    //\n    // `nullptr` is interpreted as `tsl::Env::Default()`.\n    //\n    // Default: `nullptr`.\n    Options& set_env(tsl::Env* env) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      env_ = env;\n      return *this;\n    }\n    Options&& set_env(tsl::Env* env) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_env(env));\n    }\n    tsl::Env* env() const { return env_; }\n\n    // Reading will start from this position.\n    //\n    // Default: 0.\n    Options& set_initial_pos(Position initial_pos) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      initial_pos_ = initial_pos;\n      return *this;\n    }\n    Options&& set_initial_pos(Position initial_pos) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_initial_pos(initial_pos));\n    }\n    Position initial_pos() const { return initial_pos_; }\n\n    // If `true`, supports reading up to the end of the file, then retrying when\n    // the file has grown. This disables caching the file size.\n    //\n    // Default: `false`.\n    Options& set_growing_source(bool growing_source) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      growing_source_ = growing_source;\n      return *this;\n    }\n    Options&& set_growing_source(bool growing_source) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_growing_source(growing_source));\n    }\n    bool growing_source() const { return growing_source_; }\n\n   private:\n    tsl::Env* env_ = nullptr;\n    Position initial_pos_ = 0;\n    bool growing_source_ = false;\n  };\n\n  // Returns the `tsl::RandomAccessFile` being read from. If the\n  // `tsl::RandomAccessFile` is owned then changed to `nullptr` by\n  // `Close()`, otherwise unchanged.\n  virtual tsl::RandomAccessFile* SrcFile() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the name of the `tsl::RandomAccessFile` being read from.\n  // Unchanged by `Close()`.\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return filename_;\n  }\n\n  bool ToleratesReadingAhead() override {\n    return buffer_sizer_.read_all_hint() ||\n           FileReaderBase::SupportsRandomAccess();\n  }\n  bool SupportsRandomAccess() override { return file_system_ != nullptr; }\n  bool SupportsNewReader() override {\n    return FileReaderBase::SupportsRandomAccess();\n  }\n\n protected:\n  explicit FileReaderBase(Closed) noexcept : Reader(kClosed) {}\n\n  explicit FileReaderBase(BufferOptions buffer_options, tsl::Env* env,\n                          bool growing_source);\n\n  FileReaderBase(FileReaderBase&& that) noexcept;\n  FileReaderBase& operator=(FileReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, tsl::Env* env, bool growing_source);\n  void Initialize(tsl::RandomAccessFile* src, Position initial_pos);\n  bool InitializeFilename(tsl::RandomAccessFile* src);\n  bool InitializeFilename(PathInitializer filename);\n  std::unique_ptr<tsl::RandomAccessFile> OpenFile();\n  void InitializePos(Position initial_pos);\n\n  void Done() override;\n  absl::Status AnnotateStatusImpl(absl::Status status) override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  bool PullSlow(size_t min_length, size_t recommended_length) override;\n  using Reader::ReadSlow;\n  bool ReadSlow(size_t length, char* dest) override;\n  bool ReadSlow(size_t length, Chain& dest) override;\n  bool ReadSlow(size_t length, absl::Cord& dest) override;\n  using Reader::CopySlow;\n  bool CopySlow(Position length, Writer& dest) override;\n  bool CopySlow(size_t length, BackwardWriter& dest) override;\n  using Reader::ReadSomeSlow;\n  bool ReadSomeSlow(size_t max_length, char* dest) override;\n  using Reader::CopySomeSlow;\n  bool CopySomeSlow(size_t max_length, Writer& dest) override;\n  bool SyncImpl(SyncType sync_type) override;\n  bool SeekSlow(Position new_pos) override;\n  std::optional<Position> SizeImpl() override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n  std::unique_ptr<Reader> NewReaderCurrentPosImpl() override;\n\n private:\n  ABSL_ATTRIBUTE_COLD bool FailOperation(const absl::Status& status,\n                                         absl::string_view operation);\n  ABSL_ATTRIBUTE_COLD static absl::Status NoRandomAccessStatus();\n\n  void set_exact_size(std::optional<Position> exact_size) {\n    buffer_sizer_.set_exact_size(exact_size);\n  }\n  std::optional<Position> exact_size() const {\n    return buffer_sizer_.exact_size();\n  }\n\n  // Discards buffer contents.\n  void SyncBuffer();\n\n  // Clears `buffer_`. Reads `length` bytes from `*src`, from the physical file\n  // position which is `limit_pos()`, to `dest[]`.\n  //\n  // Increments `limit_pos()` by the length read. Returns `true` on success.\n  bool ReadToDest(size_t length, tsl::RandomAccessFile* src, char* dest);\n\n  // Reads `flat_buffer.size()` bytes from `*src`, from the physical file\n  // position which is `limit_pos()`, preferably to `flat_buffer.data()`. Newly\n  // read data are adjacent to previously available data in `buffer_`, if any.\n  // `cursor_index` is the amount of already read data before previously\n  // available data.\n  //\n  // Increments `limit_pos()` by the length read. Sets buffer pointers. Returns\n  // `true` on success.\n  //\n  // Precondition: `flat_buffer` is a suffix of `buffer_`\n  bool ReadToBuffer(size_t cursor_index, tsl::RandomAccessFile* src,\n                    absl::Span<char> flat_buffer);\n\n  // Implementation of `CopySlow(Writer&)` in terms of `Writer::Push()` and\n  // `ReadToDest()`. Does not use buffer pointers.\n  //\n  // Precondition: `length > 0`\n  bool CopyUsingPush(Position length, tsl::RandomAccessFile* src, Writer& dest);\n\n  std::string filename_{kDefaultFilename};\n  // Invariant: if `is_open()` then `env_ != nullptr`\n  tsl::Env* env_ = nullptr;\n  tsl::FileSystem* file_system_ = nullptr;\n  bool growing_source_ = false;\n  ReadBufferSizer buffer_sizer_;\n  // If `buffer_` is not empty, it contains buffered data, read directly before\n  // the physical source position which is `limit_pos()`. Otherwise buffered\n  // data are in memory managed by the `tsl::RandomAccessFile`. In any\n  // case `start()` points to them.\n  SizedSharedBuffer buffer_;\n\n  // Invariants if `!buffer_.empty()`:\n  //   `start() == buffer_.data()`\n  //   `start_to_limit() == buffer_.size()`\n};\n\n// A `Reader` which reads from a `tsl::RandomAccessFile`.\n//\n// It supports random access and `NewReader()` if the\n// `tsl::RandomAccessFile` supports\n// `tsl::RandomAccessFile::Name()` and the name is not empty.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the `tsl::RandomAccessFile` being read from. `Src`\n// must support `Dependency<tsl::RandomAccessFile*, Src>`, e.g.\n// `std::unique_ptr<tsl::RandomAccessFile>` (owned, default),\n// `tsl::RandomAccessFile*` (not owned),\n// `Any<tsl::RandomAccessFile*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The `tsl::RandomAccessFile` must not be closed until the\n// `FileReader` is closed or no longer used.\ntemplate <typename Src = std::unique_ptr<tsl::RandomAccessFile>>\nclass FileReader : public FileReaderBase {\n public:\n  // Creates a closed `FileReader`.\n  explicit FileReader(Closed) noexcept : FileReaderBase(kClosed) {}\n\n  // Will read from the `tsl::RandomAccessFile` provided by `src`.\n  explicit FileReader(Initializer<Src> src, Options options = Options());\n\n  // Opens a `tsl::RandomAccessFile` for reading.\n  //\n  // If opening the file fails, `FileReader` will be failed and closed.\n  explicit FileReader(PathInitializer filename, Options options = Options());\n\n  FileReader(FileReader&& that) = default;\n  FileReader& operator=(FileReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `FileReader`. This avoids\n  // constructing a temporary `FileReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n  ABSL_ATTRIBUTE_REINITIALIZES\n  void Reset(PathInitializer filename, Options options = Options());\n\n  // Returns the object providing and possibly owning the\n  // `tsl::RandomAccessFile` being read from. If the\n  // `tsl::RandomAccessFile` is owned then changed to `nullptr` by\n  // `Close()`, otherwise unchanged.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  tsl::RandomAccessFile* SrcFile() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n\n private:\n  using FileReaderBase::Initialize;\n  void Initialize(PathInitializer filename, Options&& options);\n\n  // The object providing and possibly owning the\n  // `tsl::RandomAccessFile` being read from.\n  Dependency<tsl::RandomAccessFile*, Src> src_;\n};\n\nexplicit FileReader(Closed) -> FileReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit FileReader(Src&& src,\n                    FileReaderBase::Options options = FileReaderBase::Options())\n    -> FileReader<std::conditional_t<\n        std::is_convertible_v<Src&&, PathInitializer>,\n        std::unique_ptr<tsl::RandomAccessFile>, TargetT<Src>>>;\n\n// Implementation details follow.\n\ninline FileReaderBase::FileReaderBase(BufferOptions buffer_options,\n                                      tsl::Env* env, bool growing_source)\n    : env_(env != nullptr ? env : tsl::Env::Default()),\n      growing_source_(growing_source),\n      buffer_sizer_(buffer_options) {}\n\ninline FileReaderBase::FileReaderBase(FileReaderBase&& that) noexcept\n    : Reader(static_cast<Reader&&>(that)),\n      filename_(std::exchange(that.filename_, std::string(kDefaultFilename))),\n      env_(that.env_),\n      file_system_(that.file_system_),\n      growing_source_(that.growing_source_),\n      buffer_sizer_(that.buffer_sizer_),\n      buffer_(std::move(that.buffer_)) {}\n\ninline FileReaderBase& FileReaderBase::operator=(\n    FileReaderBase&& that) noexcept {\n  Reader::operator=(static_cast<Reader&&>(that));\n  filename_ = std::exchange(that.filename_, std::string(kDefaultFilename));\n  env_ = that.env_;\n  file_system_ = that.file_system_;\n  growing_source_ = that.growing_source_;\n  buffer_sizer_ = that.buffer_sizer_;\n  buffer_ = std::move(that.buffer_);\n  return *this;\n}\n\ninline void FileReaderBase::Reset(Closed) {\n  Reader::Reset(kClosed);\n  filename_ = std::string(kDefaultFilename);\n  env_ = nullptr;\n  file_system_ = nullptr;\n  growing_source_ = false;\n  buffer_sizer_.Reset();\n  buffer_ = SizedSharedBuffer();\n}\n\ninline void FileReaderBase::Reset(BufferOptions buffer_options, tsl::Env* env,\n                                  bool growing_source) {\n  Reader::Reset();\n  // `filename_` will be set by `InitializeFilename()`.\n  env_ = env != nullptr ? env : tsl::Env::Default();\n  file_system_ = nullptr;\n  growing_source_ = growing_source;\n  buffer_sizer_.Reset(buffer_options);\n  buffer_.Clear();\n}\n\ninline void FileReaderBase::Initialize(tsl::RandomAccessFile* src,\n                                       Position initial_pos) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of FileReader: null RandomAccessFile pointer\";\n  if (ABSL_PREDICT_FALSE(!InitializeFilename(src))) return;\n  InitializePos(initial_pos);\n}\n\ntemplate <typename Src>\ninline FileReader<Src>::FileReader(Initializer<Src> src, Options options)\n    : FileReaderBase(options.buffer_options(), options.env(),\n                     options.growing_source()),\n      src_(std::move(src)) {\n  Initialize(src_.get(), options.initial_pos());\n}\n\ntemplate <typename Src>\ninline FileReader<Src>::FileReader(PathInitializer filename, Options options)\n    : FileReaderBase(options.buffer_options(), options.env(),\n                     options.growing_source()) {\n  Initialize(std::move(filename), std::move(options));\n}\n\ntemplate <typename Src>\ninline void FileReader<Src>::Reset(Closed) {\n  FileReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void FileReader<Src>::Reset(Initializer<Src> src, Options options) {\n  FileReaderBase::Reset(options.buffer_options(), options.env(),\n                        options.growing_source());\n  src_.Reset(std::move(src));\n  Initialize(src_.get(), options.initial_pos());\n}\n\ntemplate <typename Src>\ninline void FileReader<Src>::Reset(PathInitializer filename, Options options) {\n  FileReaderBase::Reset(options.buffer_options(), options.env(),\n                        options.growing_source());\n  Initialize(std::move(filename), std::move(options));\n}\n\ntemplate <typename Src>\ninline void FileReader<Src>::Initialize(PathInitializer filename,\n                                        Options&& options) {\n  if (ABSL_PREDICT_FALSE(!InitializeFilename(std::move(filename)))) return;\n  std::unique_ptr<tsl::RandomAccessFile> src = OpenFile();\n  if (ABSL_PREDICT_FALSE(src == nullptr)) return;\n  src_.Reset(riegeli::Maker(src.release()));\n  InitializePos(options.initial_pos());\n}\n\ntemplate <typename Src>\nvoid FileReader<Src>::Done() {\n  FileReaderBase::Done();\n  if (src_.IsOwning()) {\n    // The only way to close a `tsl::RandomAccessFile` is to delete it.\n    src_.Reset();\n  }\n}\n\n}  // namespace tensorflow\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_TENSORFLOW_IO_FILE_READER_H_\n"
  },
  {
    "path": "riegeli/tensorflow/io/file_writer.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/tensorflow/io/file_writer.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <string>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/buffering.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/reset.h\"\n#include \"riegeli/base/shared_buffer.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/path_ref.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/tensorflow/io/file_reader.h\"\n#include \"tensorflow/compiler/xla/tsl/platform/env.h\"\n#include \"tensorflow/compiler/xla/tsl/platform/file_system.h\"\n#include \"tensorflow/core/public/version.h\"\n\nnamespace riegeli::tensorflow {\n\nbool FileWriterBase::InitializeFilename(tsl::WritableFile* dest) {\n  absl::string_view filename;\n  if (const absl::Status status = dest->Name(&filename);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    filename_ = \"<unknown>\";\n    if (!absl::IsUnimplemented(status)) {\n      return FailOperation(status, \"WritableFile::Name()\");\n    }\n    return true;\n  }\n  return InitializeFilename(filename);\n}\n\nbool FileWriterBase::InitializeFilename(PathInitializer filename) {\n  riegeli::Reset(filename_, std::move(filename));\n  if (const absl::Status status =\n          env_->GetFileSystemForFile(filename_, &file_system_);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return FailOperation(status, \"Env::GetFileSystemForFile()\");\n  }\n  return true;\n}\n\nstd::unique_ptr<tsl::WritableFile> FileWriterBase::OpenFile(bool append) {\n  std::unique_ptr<tsl::WritableFile> dest;\n  if (const absl::Status status =\n          append ? file_system_->NewAppendableFile(filename_, &dest)\n                 : file_system_->NewWritableFile(filename_, &dest);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    Writer::Reset(kClosed);\n    FailOperation(status,\n                  append ? absl::string_view(\"FileSystem::NewAppendableFile()\")\n                         : absl::string_view(\"FileSystem::NewWritableFile()\"));\n    RIEGELI_ASSERT(dest == nullptr)\n        << (append ? absl::string_view(\"FileSystem::NewAppendableFile()\")\n                   : absl::string_view(\"FileSystem::NewWritableFile()\"))\n        << \" should store null WritableFile on failure\";\n  }\n  return dest;\n}\n\nvoid FileWriterBase::InitializePos(tsl::WritableFile* dest) {\n  int64_t file_pos;\n  if (const absl::Status status = dest->Tell(&file_pos);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    FailOperation(status, \"WritableFile::Tell()\");\n    return;\n  }\n  set_start_pos(IntCast<Position>(file_pos));\n  buffer_sizer_.BeginRun(start_pos());\n}\n\nvoid FileWriterBase::Done() {\n  SyncBuffer();\n  Writer::Done();\n  buffer_ = SharedBuffer();\n}\n\nbool FileWriterBase::FailOperation(const absl::Status& status,\n                                   absl::string_view operation) {\n  RIEGELI_ASSERT(!status.ok())\n      << \"Failed precondition of FileWriterBase::FailOperation(): \"\n         \"status not failed\";\n  return Fail(\n      Annotate(absl::Status(static_cast<absl::StatusCode>(status.code()),\n#if TF_GRAPH_DEF_VERSION < 1467\n                            status.error_message()\n#else\n                            status.message()\n#endif\n                                ),\n               absl::StrCat(operation, \" failed\")));\n}\n\nabsl::Status FileWriterBase::AnnotateStatusImpl(absl::Status status) {\n  return Writer::AnnotateStatusImpl(\n      Annotate(status, absl::StrCat(\"writing \", filename_)));\n}\n\ninline bool FileWriterBase::SyncBuffer() {\n  if (start_to_cursor() > kMaxBytesToCopy) {\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    const absl::Cord data(\n        ExternalRef(buffer_, absl::string_view(start(), start_to_cursor())));\n    set_buffer();\n    return WriteInternal(data);\n  }\n  const absl::string_view data(start(), start_to_cursor());\n  set_buffer();\n  if (data.empty()) return true;\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  return WriteInternal(data);\n}\n\nvoid FileWriterBase::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  buffer_sizer_.set_write_size_hint(pos(), write_size_hint);\n}\n\nbool FileWriterBase::PushSlow(size_t min_length, size_t recommended_length) {\n  RIEGELI_ASSERT_LT(available(), min_length)\n      << \"Failed precondition of Writer::PushSlow(): \"\n         \"enough space available, use Push() instead\";\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (ABSL_PREDICT_FALSE(min_length >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  const size_t buffer_length = UnsignedMin(\n      buffer_sizer_.BufferLength(start_pos(), min_length, recommended_length),\n      std::numeric_limits<Position>::max() - start_pos());\n  buffer_.Reset(buffer_length);\n  set_buffer(buffer_.mutable_data(), buffer_length);\n  return true;\n}\n\nbool FileWriterBase::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of FileWriterBase::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of FileWriterBase::WriteInternal()\";\n  tsl::WritableFile* const dest = DestFile();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  if (const absl::Status status = dest->Append(src);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return FailOperation(status, \"WritableFile::Append(string_view)\");\n  }\n  move_start_pos(src.size());\n  return true;\n}\n\nbool FileWriterBase::WriteSlow(absl::string_view src) {\n  RIEGELI_ASSERT_LT(available(), src.size())\n      << \"Failed precondition of Writer::WriteSlow(string_view): \"\n         \"enough space available, use Write(string_view) instead\";\n  if (src.size() >= buffer_sizer_.BufferLength(pos())) {\n    // Write directly from `src`.\n    if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    return WriteInternal(src);\n  }\n  return Writer::WriteSlow(src);\n}\n\nbool FileWriterBase::WriteSlow(ExternalRef src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ExternalRef): \"\n         \"enough space available, use Write(ExternalRef) instead\";\n  if (src.size() >= buffer_sizer_.BufferLength(pos())) {\n    // Write directly from `src`.\n    if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    return WriteInternal(absl::Cord(std::move(src)));\n  }\n  return Writer::WriteSlow(std::move(src));\n}\n\nbool FileWriterBase::WriteSlow(const Chain& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain): \"\n         \"enough space available, use Write(Chain) instead\";\n  if (src.size() >= buffer_sizer_.BufferLength(pos())) {\n    // Write directly from `src`.\n    if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    return WriteInternal(absl::Cord(src));\n  }\n  return Writer::WriteSlow(src);\n}\n\nbool FileWriterBase::WriteSlow(Chain&& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Chain&&): \"\n         \"enough space available, use Write(Chain&&) instead\";\n  if (src.size() >= buffer_sizer_.BufferLength(pos())) {\n    // Write directly from `src`.\n    if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    return WriteInternal(absl::Cord(std::move(src)));\n  }\n  // Not `std::move(src)`: forward to `Writer::WriteSlow(const Chain&)`,\n  // because `Writer::WriteSlow(Chain&&)` would forward to\n  // `FileWriterBase::WriteSlow(const Chain&)`.\n  return Writer::WriteSlow(src);\n}\n\nbool FileWriterBase::WriteSlow(const absl::Cord& src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(Cord): \"\n         \"enough space available, use Write(Cord) instead\";\n  if (src.size() >= buffer_sizer_.BufferLength(pos())) {\n    // Write directly from `src`.\n    if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    return WriteInternal(src);\n  }\n  return Writer::WriteSlow(src);\n}\n\nbool FileWriterBase::WriteSlow(ByteFill src) {\n  RIEGELI_ASSERT_LT(UnsignedMin(available(), kMaxBytesToCopy), src.size())\n      << \"Failed precondition of Writer::WriteSlow(ByteFill): \"\n         \"enough space available, use Write(ByteFill) instead\";\n  if (src.size() >= buffer_sizer_.BufferLength(pos())) {\n    // Write directly from `Cord(ByteFill)`.\n    if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    while (\n        ABSL_PREDICT_FALSE(src.size() > std::numeric_limits<size_t>::max())) {\n      if (ABSL_PREDICT_FALSE(!WriteInternal(\n              absl::Cord(src.Extract(std::numeric_limits<size_t>::max()))))) {\n        return false;\n      }\n    }\n    return WriteInternal(absl::Cord(src));\n  }\n  return Writer::WriteSlow(src);\n}\n\nbool FileWriterBase::WriteInternal(const absl::Cord& src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of FileWriterBase::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of FileWriterBase::WriteInternal()\";\n  tsl::WritableFile* const dest = DestFile();\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  if (absl::Status status = dest->Append(src);\n      ABSL_PREDICT_FALSE(!status.ok())) {\n    return FailOperation(status, \"WritableFile::Append(Cord)\");\n  }\n  move_start_pos(src.size());\n  return true;\n}\n\nbool FileWriterBase::FlushImpl(FlushType flush_type) {\n  buffer_sizer_.EndRun(pos());\n  if (ABSL_PREDICT_FALSE(!SyncBuffer())) return false;\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  buffer_sizer_.BeginRun(start_pos());\n  return true;\n}\n\nReader* FileWriterBase::ReadModeImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!FileWriterBase::SupportsReadMode())) {\n    Fail(absl::UnimplementedError(\"A filename required for read mode\"));\n    return nullptr;\n  }\n  if (ABSL_PREDICT_FALSE(!Flush())) return nullptr;\n  return associated_reader_.ResetReader(\n      filename_, FileReaderBase::Options()\n                     .set_env(env_)\n                     .set_initial_pos(initial_pos)\n                     .set_buffer_options(buffer_sizer_.buffer_options()));\n}\n\n}  // namespace riegeli::tensorflow\n"
  },
  {
    "path": "riegeli/tensorflow/io/file_writer.h",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_TENSORFLOW_IO_FILE_WRITER_H_\n#define RIEGELI_TENSORFLOW_IO_FILE_WRITER_H_\n\n#include <stddef.h>\n\n#include <memory>\n#include <optional>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/external_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/shared_buffer.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/path_ref.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"tensorflow/compiler/xla/tsl/platform/env.h\"\n#include \"tensorflow/compiler/xla/tsl/platform/file_system.h\"\n\nnamespace riegeli {\n\nclass Reader;\n\nnamespace tensorflow {\n\ntemplate <typename Src>\nclass FileReader;\n\n// Template parameter independent part of `FileWriter`.\nclass FileWriterBase : public Writer {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // `FileWriter` has a larger `kDefaultMaxBufferSize` (1M) because remote\n    // file access may have high latency of each operation.\n    static constexpr size_t kDefaultMaxBufferSize = size_t{1} << 20;\n\n    // Overrides the TensorFlow environment.\n    //\n    // `nullptr` is interpreted as `tsl::Env::Default()`.\n    //\n    // Default: `nullptr`.\n    Options& set_env(tsl::Env* env) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      env_ = env;\n      return *this;\n    }\n    Options&& set_env(tsl::Env* env) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_env(env));\n    }\n    tsl::Env* env() const { return env_; }\n\n    // If `false`, the file will be truncated to empty if it exists.\n    //\n    // If `true`, the file will not be truncated if it exists, and writing will\n    // always happen at its end, or will continue at its end, depending on the\n    // nature of the file.\n    //\n    // If `FileWriter` writes to an already open file, `append()` has no effect.\n    //\n    // Default: `false`.\n    Options& set_append(bool append) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      append_ = append;\n      return *this;\n    }\n    Options&& set_append(bool append) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_append(append));\n    }\n    bool append() const { return append_; }\n\n   private:\n    tsl::Env* env_ = nullptr;\n    bool append_ = false;\n  };\n\n  // Returns the `tsl::WritableFile` being written to. Unchanged by\n  // `Close()`.\n  virtual tsl::WritableFile* DestFile() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns the name of the `tsl::WritableFile` being written to.\n  // Unchanged by `Close()`.\n  absl::string_view filename() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return filename_;\n  }\n\n  bool SupportsReadMode() override { return file_system_ != nullptr; }\n\n protected:\n  explicit FileWriterBase(Closed) noexcept : Writer(kClosed) {}\n\n  explicit FileWriterBase(BufferOptions buffer_options, tsl::Env* env);\n\n  FileWriterBase(FileWriterBase&& that) noexcept;\n  FileWriterBase& operator=(FileWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, tsl::Env* env);\n  void Initialize(tsl::WritableFile* dest);\n  bool InitializeFilename(tsl::WritableFile* dest);\n  bool InitializeFilename(PathInitializer filename);\n  std::unique_ptr<tsl::WritableFile> OpenFile(bool append);\n  void InitializePos(tsl::WritableFile* dest);\n  ABSL_ATTRIBUTE_COLD bool FailOperation(const absl::Status& status,\n                                         absl::string_view operation);\n\n  void Done() override;\n  absl::Status AnnotateStatusImpl(absl::Status status) override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool PushSlow(size_t min_length, size_t recommended_length) override;\n  using Writer::WriteSlow;\n  bool WriteSlow(absl::string_view src) override;\n  bool WriteSlow(ExternalRef src) override;\n  bool WriteSlow(const Chain& src) override;\n  bool WriteSlow(Chain&& src) override;\n  bool WriteSlow(const absl::Cord& src) override;\n  bool WriteSlow(ByteFill src) override;\n  bool FlushImpl(FlushType flush_type) override;\n  Reader* ReadModeImpl(Position initial_pos) override;\n\n private:\n  // Writes `buffer_` to the file. Sets buffer pointers to `nullptr`.\n  bool SyncBuffer();\n\n  // Writes `src` to the destination.\n  //\n  // Does not use buffer pointers. Increments `start_pos()` by the length\n  // written, which must be `src.size()` on success. Returns `true` on success.\n  //\n  // Preconditions:\n  //   `!src.empty()`\n  //   `ok()`\n  bool WriteInternal(absl::string_view src);\n  bool WriteInternal(const absl::Cord& src);\n\n  std::string filename_{kDefaultFilename};\n  // Invariant: if `is_open()` then `env_ != nullptr`\n  tsl::Env* env_ = nullptr;\n  tsl::FileSystem* file_system_ = nullptr;\n  WriteBufferSizer buffer_sizer_;\n  // Buffered data to be written.\n  SharedBuffer buffer_;\n\n  AssociatedReader<FileReader<std::unique_ptr<tsl::RandomAccessFile>>>\n      associated_reader_;\n};\n\n// A `Writer` which writes to a `tsl::WritableFile`.\n//\n// It supports `ReadMode()` if the `tsl::WritableFile` supports\n// `tsl::WritableFile::Name()` and the name is not empty.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `tsl::WritableFile` being written to. `Dest`\n// must support `Dependency<tsl::WritableFile*, Dest>`, e.g.\n// `std::unique_ptr<tsl::WritableFile>` (owned, default),\n// `tsl::WritableFile*` (not owned),\n// `Any<tsl::WritableFile*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The `tsl::WritableFile` must not be closed until the `FileWriter` is\n// closed or no longer used. Until then the `tsl::WritableFile` may be\n// accessed, but not concurrently, `Flush()` is needed before switching to\n// another writer to the same `tsl::WritableFile`, and `pos()` does not\n// take other writers into account.\ntemplate <typename Dest = std::unique_ptr<tsl::WritableFile>>\nclass FileWriter : public FileWriterBase {\n public:\n  // Creates a closed `FileWriter`.\n  explicit FileWriter(Closed) noexcept : FileWriterBase(kClosed) {}\n\n  // Will write to the `tsl::WritableFile` provided by `dest`.\n  explicit FileWriter(Initializer<Dest> dest, Options options = Options());\n\n  // Opens a `tsl::WritableFile` for writing.\n  //\n  // If opening the file fails, `FileWriter` will be failed and closed.\n  explicit FileWriter(PathInitializer filename, Options options = Options());\n\n  FileWriter(FileWriter&& that) = default;\n  FileWriter& operator=(FileWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `FileWriter`. This avoids\n  // constructing a temporary `FileWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n  ABSL_ATTRIBUTE_REINITIALIZES\n  void Reset(PathInitializer filename, Options options = Options());\n\n  // Returns the object providing and possibly owning the\n  // `tsl::WritableFile` being written to. Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  tsl::WritableFile* DestFile() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  using FileWriterBase::Initialize;\n  void Initialize(PathInitializer filename, Options&& options);\n\n  // The object providing and possibly owning the `tsl::WritableFile`\n  // being written to.\n  Dependency<tsl::WritableFile*, Dest> dest_;\n};\n\nexplicit FileWriter(Closed) -> FileWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit FileWriter(Dest&& dest,\n                    FileWriterBase::Options options = FileWriterBase::Options())\n    -> FileWriter<\n        std::conditional_t<std::is_convertible_v<Dest&&, PathInitializer>,\n                           std::unique_ptr<tsl::WritableFile>, TargetT<Dest>>>;\n\n// Implementation details follow.\n\ninline FileWriterBase::FileWriterBase(BufferOptions buffer_options,\n                                      tsl::Env* env)\n    : env_(env != nullptr ? env : tsl::Env::Default()),\n      buffer_sizer_(buffer_options) {}\n\ninline FileWriterBase::FileWriterBase(FileWriterBase&& that) noexcept\n    : Writer(static_cast<Writer&&>(that)),\n      filename_(std::exchange(that.filename_, std::string(kDefaultFilename))),\n      env_(that.env_),\n      file_system_(that.file_system_),\n      buffer_sizer_(that.buffer_sizer_),\n      buffer_(std::move(that.buffer_)),\n      associated_reader_(std::move(that.associated_reader_)) {}\n\ninline FileWriterBase& FileWriterBase::operator=(\n    FileWriterBase&& that) noexcept {\n  Writer::operator=(static_cast<Writer&&>(that));\n  filename_ = std::exchange(that.filename_, std::string(kDefaultFilename));\n  env_ = that.env_;\n  file_system_ = that.file_system_;\n  buffer_sizer_ = that.buffer_sizer_;\n  buffer_ = std::move(that.buffer_);\n  associated_reader_ = std::move(that.associated_reader_);\n  return *this;\n}\n\ninline void FileWriterBase::Reset(Closed) {\n  Writer::Reset(kClosed);\n  filename_ = std::string(kDefaultFilename);\n  env_ = nullptr;\n  file_system_ = nullptr;\n  buffer_sizer_.Reset();\n  buffer_ = SharedBuffer();\n  associated_reader_.Reset();\n}\n\ninline void FileWriterBase::Reset(BufferOptions buffer_options, tsl::Env* env) {\n  Writer::Reset();\n  // `filename_` will be set by `InitializeFilename()`.\n  env_ = env != nullptr ? env : tsl::Env::Default();\n  file_system_ = nullptr;\n  buffer_sizer_.Reset(buffer_options);\n  associated_reader_.Reset();\n}\n\ninline void FileWriterBase::Initialize(tsl::WritableFile* dest) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of FileWriter: null WritableFile pointer\";\n  if (ABSL_PREDICT_FALSE(!InitializeFilename(dest))) return;\n  InitializePos(dest);\n}\n\ntemplate <typename Dest>\ninline FileWriter<Dest>::FileWriter(Initializer<Dest> dest, Options options)\n    : FileWriterBase(options.buffer_options(), options.env()),\n      dest_(std::move(dest)) {\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\ninline FileWriter<Dest>::FileWriter(PathInitializer filename, Options options)\n    : FileWriterBase(options.buffer_options(), options.env()) {\n  Initialize(std::move(filename), std::move(options));\n}\n\ntemplate <typename Dest>\ninline void FileWriter<Dest>::Reset(Closed) {\n  FileWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void FileWriter<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  FileWriterBase::Reset(options.buffer_options(), options.env());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get());\n}\n\ntemplate <typename Dest>\ninline void FileWriter<Dest>::Reset(PathInitializer filename, Options options) {\n  FileWriterBase::Reset(options.buffer_options(), options.env());\n  Initialize(std::move(filename), std::move(options));\n}\n\ntemplate <typename Dest>\ninline void FileWriter<Dest>::Initialize(PathInitializer filename,\n                                         Options&& options) {\n  if (ABSL_PREDICT_FALSE(!InitializeFilename(std::move(filename)))) return;\n  std::unique_ptr<tsl::WritableFile> dest = OpenFile(options.append());\n  if (ABSL_PREDICT_FALSE(dest == nullptr)) return;\n  dest_.Reset(riegeli::Maker(dest.release()));\n  InitializePos(dest_.get());\n}\n\ntemplate <typename Dest>\nvoid FileWriter<Dest>::Done() {\n  FileWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (const absl::Status status = dest_->Close();\n        ABSL_PREDICT_FALSE(!status.ok()) && ABSL_PREDICT_TRUE(ok())) {\n      FailOperation(status, \"WritableFile::Close()\");\n    }\n  }\n}\n\ntemplate <typename Dest>\nbool FileWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!FileWriterBase::FlushImpl(flush_type))) return false;\n  switch (flush_type) {\n    case FlushType::kFromObject:\n      if (!dest_.IsOwning()) return true;\n      ABSL_FALLTHROUGH_INTENDED;\n    case FlushType::kFromProcess:\n      if (const absl::Status status = dest_->Flush();\n          ABSL_PREDICT_FALSE(!status.ok())) {\n        return FailOperation(status, \"WritableFile::Flush()\");\n      }\n      return true;\n    case FlushType::kFromMachine:\n      if (const absl::Status status = dest_->Sync();\n          ABSL_PREDICT_FALSE(!status.ok())) {\n        return FailOperation(status, \"WritableFile::Sync()\");\n      }\n      return true;\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown flush type: \" << static_cast<int>(flush_type);\n}\n\n}  // namespace tensorflow\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_TENSORFLOW_IO_FILE_WRITER_H_\n"
  },
  {
    "path": "riegeli/tensorflow/io/tstring_writer.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_TENSORFLOW_IO_TSTRING_WRITER_H_\n#define RIEGELI_TENSORFLOW_IO_TSTRING_WRITER_H_\n\n#include <stddef.h>\n\n#include <type_traits>\n\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/bytes/resizable_writer.h\"\n#include \"tensorflow/tsl/platform/tstring.h\"\n\nnamespace riegeli::tensorflow {\n\nnamespace tstring_internal {\n\n// `ResizableTraits` for `tsl::tstring`.\nstruct TStringResizableTraits {\n  using Resizable = tsl::tstring;\n  static char* Data(Resizable& dest) { return dest.mdata(); }\n  static size_t Size(const Resizable& dest) { return dest.size(); }\n  static constexpr bool kIsStable = false;\n  static bool Resize(Resizable& dest, size_t new_size, size_t used_size) {\n    RIEGELI_ASSERT_LE(used_size, dest.size())\n        << \"Failed precondition of ResizableTraits::Resize(): \"\n           \"used size exceeds old size\";\n    RIEGELI_ASSERT_LE(used_size, new_size)\n        << \"Failed precondition of ResizableTraits::Resize(): \"\n           \"used size exceeds new size\";\n    Reserve(dest, new_size, used_size);\n    dest.resize_uninitialized(new_size);\n    return true;\n  }\n  static bool Grow(Resizable& dest, size_t new_size, size_t used_size) {\n    RIEGELI_ASSERT_GT(new_size, dest.size())\n        << \"Failed precondition of ResizableTraits::Grow(): \"\n           \"no need to grow\";\n    RIEGELI_ASSERT_LE(used_size, dest.size())\n        << \"Failed precondition of ResizableTraits::Grow(): \"\n           \"used size exceeds old size\";\n    RIEGELI_ASSERT_LE(used_size, new_size)\n        << \"Failed precondition of ResizableTraits::Grow(): \"\n           \"used size exceeds new size\";\n    Reserve(dest, new_size, used_size);\n    dest.resize_uninitialized(dest.capacity());\n    return true;\n  }\n  static bool GrowUnderCapacity(Resizable& dest, size_t new_size) {\n    RIEGELI_ASSERT_GT(new_size, dest.size())\n        << \"Failed precondition of ResizableTraits::GrowUnderCapacity(): \"\n           \"no need to grow\";\n    if (new_size > dest.capacity()) return false;\n    dest.resize_uninitialized(dest.capacity());\n    return true;\n  }\n\n private:\n  static void Reserve(Resizable& dest, size_t new_capacity, size_t used_size) {\n    if (new_capacity > dest.capacity()) {\n      dest.resize_uninitialized(used_size);\n      dest.reserve(dest.capacity() <= tsl::tstring().capacity()\n                       ? new_capacity\n                       : UnsignedMax(dest.capacity() + dest.capacity() / 2,\n                                     new_capacity));\n    }\n    RIEGELI_ASSUME_GE(dest.capacity(), new_capacity);\n  }\n};\n\n}  // namespace tstring_internal\n\n// Template parameter independent part of `TStringWriter`.\nusing TStringWriterBase = ResizableWriterBase;\n\n// A `Writer` which writes to a `tsl::tstring`. If `Options::append()`\n// is `false` (the default), replaces existing contents of the\n// `tsl::tstring`, clearing it first. If `Options::append()` is `true`,\n// appends to existing contents of the `tsl::tstring`.\n//\n// It supports `Seek()` and `ReadMode()`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the `tsl::tstring` being written to. `Dest` must\n// support `Dependency<tsl::tstring*, Dest>`, e.g.\n// `tsl::tstring*` (not owned,  default), `tsl::tstring` (owned),\n// `Any<tsl::tstring*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as\n// `tsl::tstring` if there are no constructor arguments or the only\n// argument is `Options`, otherwise as `TargetT` of the type of the first\n// constructor argument, except that CTAD is deleted if the first constructor\n// argument is a `tsl::tstring&` or `const tsl::tstring&` (to avoid writing to\n// an unintentionally separate copy of an existing object).\n//\n// The `tsl::tstring` must not be accessed until the `TStringWriter` is\n// closed or no longer used, except that it is allowed to read the\n// `tsl::tstring` immediately after `Flush()`.\ntemplate <typename Dest = tsl::tstring*>\nclass TStringWriter\n    : public ResizableWriter<tstring_internal::TStringResizableTraits, Dest> {\n public:\n  using TStringWriter::ResizableWriter::ResizableWriter;\n\n  TStringWriter(TStringWriter&& that) = default;\n  TStringWriter& operator=(TStringWriter&& that) = default;\n};\n\nexplicit TStringWriter(Closed) -> TStringWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit TStringWriter(Dest&& dest, TStringWriterBase::Options options =\n                                        TStringWriterBase::Options())\n    -> TStringWriter<std::conditional_t<\n        std::conjunction_v<std::is_lvalue_reference<Dest>,\n                           std::is_convertible<std::remove_reference_t<Dest>*,\n                                               const tsl::tstring*>>,\n        DeleteCtad<Dest&&>, TargetT<Dest>>>;\nexplicit TStringWriter(\n    TStringWriterBase::Options options = TStringWriterBase::Options())\n    -> TStringWriter<tsl::tstring>;\n\n}  // namespace riegeli::tensorflow\n\n#endif  // RIEGELI_TENSORFLOW_IO_TSTRING_WRITER_H_\n"
  },
  {
    "path": "riegeli/tensorflow/kernels/riegeli_dataset_ops.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <memory>\n#include <optional>\n#include <string>\n#include <utility>\n#include <vector>\n\n#include \"absl/base/thread_annotations.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/synchronization/mutex.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/records/record_position.h\"\n#include \"riegeli/records/record_reader.h\"\n#include \"riegeli/records/skipped_region.h\"\n#include \"riegeli/tensorflow/io/file_reader.h\"\n#include \"tensorflow/core/framework/allocator.h\"\n#include \"tensorflow/core/framework/dataset.h\"\n#include \"tensorflow/core/framework/op_kernel.h\"\n#include \"tensorflow/core/framework/op_requires.h\"\n#include \"tensorflow/core/framework/tensor.h\"\n#include \"tensorflow/core/framework/tensor_shape.h\"\n#include \"tensorflow/core/framework/tensor_types.h\"\n#include \"tensorflow/core/framework/types.h\"\n#include \"tensorflow/core/framework/types.pb.h\"\n#include \"tensorflow/core/graph/graph.h\"\n#include \"tensorflow/core/platform/macros.h\"\n#include \"tensorflow/core/platform/tstring.h\"\n#include \"util/task/status_macros.h\"\n\nnamespace riegeli::tensorflow {\nnamespace {\n\nclass RiegeliDatasetOp : public ::tensorflow::data::DatasetOpKernel {\n public:\n  using DatasetOpKernel::DatasetOpKernel;\n\n  void MakeDataset(::tensorflow::OpKernelContext* ctx,\n                   ::tensorflow::data::DatasetBase** output) override {\n    const ::tensorflow::Tensor* filenames_tensor;\n    OP_REQUIRES_OK(ctx, ctx->input(\"filenames\", &filenames_tensor));\n    OP_REQUIRES(ctx, filenames_tensor->dims() <= 1,\n                absl::InvalidArgumentError(\n                    \"`filenames` must be a scalar or a vector.\"));\n\n    std::vector<std::string> filenames;\n    filenames.reserve(IntCast<size_t>(filenames_tensor->NumElements()));\n    for (int i = 0; i < filenames_tensor->NumElements(); ++i) {\n      filenames.emplace_back(\n          filenames_tensor->flat<::tensorflow::tstring>()(i));\n    }\n\n    int64_t min_buffer_size;\n    OP_REQUIRES_OK(ctx, ::tensorflow::data::ParseScalarArgument<int64_t>(\n                            ctx, \"min_buffer_size\", &min_buffer_size));\n    int64_t max_buffer_size;\n    OP_REQUIRES_OK(ctx, ::tensorflow::data::ParseScalarArgument<int64_t>(\n                            ctx, \"max_buffer_size\", &max_buffer_size));\n\n    *output = new Dataset(ctx, std::move(filenames), min_buffer_size,\n                          max_buffer_size);\n  }\n\n private:\n  class Dataset : public ::tensorflow::data::DatasetBase {\n   public:\n    explicit Dataset(::tensorflow::OpKernelContext* ctx,\n                     std::vector<std::string> filenames,\n                     int64_t min_buffer_size, int64_t max_buffer_size)\n        : DatasetBase(::tensorflow::data::DatasetContext(ctx)),\n          filenames_(std::move(filenames)),\n          min_buffer_size_(min_buffer_size),\n          max_buffer_size_(max_buffer_size) {}\n\n    std::unique_ptr<::tensorflow::data::IteratorBase> MakeIteratorInternal(\n        const std::string& prefix) const override {\n      return std::unique_ptr<::tensorflow::data::IteratorBase>(\n          new Iterator({this, absl::StrCat(prefix, \"::Riegeli\")}));\n    }\n\n    const ::tensorflow::DataTypeVector& output_dtypes() const override {\n      static const ::tensorflow::DataTypeVector* const dtypes =\n          new ::tensorflow::DataTypeVector({::tensorflow::DT_STRING});\n      return *dtypes;\n    }\n\n    const std::vector<::tensorflow::PartialTensorShape>& output_shapes()\n        const override {\n      static const std::vector<::tensorflow::PartialTensorShape>* const shapes =\n          new std::vector<::tensorflow::PartialTensorShape>({{}});\n      return *shapes;\n    }\n\n    std::string DebugString() const override {\n      return \"RiegeliDatasetOp::Dataset\";\n    }\n\n    absl::Status CheckExternalState() const override {\n      return absl::OkStatus();\n    }\n\n    absl::Status InputDatasets(\n        std::vector<const ::tensorflow::data::DatasetBase*>* inputs)\n        const override {\n      inputs->clear();\n      return absl::OkStatus();\n    }\n\n   protected:\n    absl::Status AsGraphDefInternal(\n        ::tensorflow::data::SerializationContext* ctx,\n        DatasetGraphDefBuilder* b, ::tensorflow::Node** output) const override {\n      ::tensorflow::Node* filenames = nullptr;\n      RETURN_IF_ERROR(b->AddVector(filenames_, &filenames));\n      ::tensorflow::Node* min_buffer_size = nullptr;\n      RETURN_IF_ERROR(b->AddScalar(min_buffer_size_, &min_buffer_size));\n      ::tensorflow::Node* max_buffer_size = nullptr;\n      RETURN_IF_ERROR(b->AddScalar(max_buffer_size_, &max_buffer_size));\n      RETURN_IF_ERROR(b->AddDataset(\n          this, {filenames, min_buffer_size, max_buffer_size}, output));\n      return absl::OkStatus();\n    }\n\n   private:\n    class Iterator : public ::tensorflow::data::DatasetIterator<Dataset> {\n     public:\n      explicit Iterator(const Params& params) : DatasetIterator(params) {}\n\n      absl::Status GetNextInternal(\n          ::tensorflow::data::IteratorContext* ctx,\n          std::vector<::tensorflow::Tensor>* out_tensors,\n          bool* end_of_sequence) override ABSL_LOCKS_EXCLUDED(mu_) {\n        absl::MutexLock lock(mu_);\n        for (;;) {\n          if (reader_ != std::nullopt) {\n            // We are currently processing a file, so try to read the next\n            // record.\n            ::tensorflow::Tensor result_tensor(::tensorflow::cpu_allocator(),\n                                               ::tensorflow::DT_STRING, {});\n            absl::string_view value;\n            if (TF_PREDICT_TRUE(reader_->ReadRecord(value))) {\n              result_tensor.scalar<::tensorflow::tstring>()().assign(\n                  value.data(), value.size());\n              out_tensors->push_back(std::move(result_tensor));\n              *end_of_sequence = false;\n              return absl::OkStatus();\n            }\n            SkippedRegion skipped_region;\n            if (reader_->Recover(&skipped_region)) {\n              // File has invalid contents: return an error. Further iteration\n              // will resume reading the file after the invalid region has been\n              // skipped.\n              *end_of_sequence = false;\n              return absl::InvalidArgumentError(absl::StrCat(\n                  \"Skipping invalid region of a Riegeli/records file: \",\n                  skipped_region));\n            }\n            if (TF_PREDICT_FALSE(!reader_->Close())) {\n              // Failed to read the file: return an error.\n              absl::Status status = reader_->status();\n              // Further iteration will move on to the next file, if any.\n              reader_.reset();\n              ++current_file_index_;\n              *end_of_sequence =\n                  current_file_index_ == dataset()->filenames_.size();\n              return status;\n            }\n            // We have reached the end of the current file, so move on to the\n            // next file, if any.\n            reader_.reset();\n            ++current_file_index_;\n          }\n\n          // Iteration ends when there are no more files to process.\n          if (current_file_index_ == dataset()->filenames_.size()) {\n            *end_of_sequence = true;\n            return absl::OkStatus();\n          }\n\n          // Actually move on to next file.\n          OpenFile(ctx);\n        }\n      }\n\n     protected:\n      absl::Status SaveInternal(::tensorflow::data::SerializationContext* ctx,\n                                ::tensorflow::data::IteratorStateWriter* writer)\n          override ABSL_LOCKS_EXCLUDED(mu_) {\n        absl::MutexLock lock(mu_);\n        RETURN_IF_ERROR(\n            writer->WriteScalar(full_name(\"current_file_index\"),\n                                IntCast<int64_t>(current_file_index_)));\n        if (reader_ != std::nullopt) {\n          RETURN_IF_ERROR(writer->WriteScalar(full_name(\"current_pos\"),\n                                              reader_->pos().ToBytes()));\n        }\n        return absl::OkStatus();\n      }\n\n      absl::Status RestoreInternal(\n          ::tensorflow::data::IteratorContext* ctx,\n          ::tensorflow::data::IteratorStateReader* reader) override\n          ABSL_LOCKS_EXCLUDED(mu_) {\n        absl::MutexLock lock(mu_);\n        current_file_index_ = 0;\n        reader_.reset();\n\n        int64_t current_file_index;\n        RETURN_IF_ERROR(reader->ReadScalar(full_name(\"current_file_index\"),\n                                           &current_file_index));\n        if (TF_PREDICT_FALSE(current_file_index < 0 ||\n                             IntCast<uint64_t>(current_file_index) >\n                                 dataset()->filenames_.size())) {\n          return absl::InternalError(\"current_file_index out of range\");\n        }\n        current_file_index_ = IntCast<size_t>(current_file_index);\n\n        if (reader->Contains(full_name(\"current_pos\"))) {\n          if (TF_PREDICT_FALSE(current_file_index_ ==\n                               dataset()->filenames_.size())) {\n            return absl::InternalError(\"current_file_index out of range\");\n          }\n          ::tensorflow::tstring current_pos;\n          RETURN_IF_ERROR(\n              reader->ReadScalar(full_name(\"current_pos\"), &current_pos));\n          RecordPosition pos;\n          if (TF_PREDICT_FALSE(!pos.FromBytes(current_pos))) {\n            return absl::InternalError(\n                \"current_pos is not a valid RecordPosition\");\n          }\n          OpenFile(ctx);\n          reader_->Seek(pos);\n          // Any errors from seeking will be reported during reading.\n        }\n        return absl::OkStatus();\n      }\n\n     private:\n      void OpenFile(::tensorflow::data::IteratorContext* ctx)\n          ABSL_EXCLUSIVE_LOCKS_REQUIRED(mu_) {\n        reader_.emplace(riegeli::Maker(\n            dataset()->filenames_[current_file_index_],\n            tensorflow::FileReaderBase::Options()\n                .set_env(ctx->env())\n                .set_min_buffer_size(\n                    IntCast<size_t>(dataset()->min_buffer_size_))\n                .set_max_buffer_size(\n                    IntCast<size_t>(dataset()->max_buffer_size_))));\n      }\n\n      // Invariants:\n      //   `current_file_index_ <= dataset()->filenames_.size()`\n      //   if `current_file_index_ == dataset()->filenames_.size()` then\n      //       `reader_ == std::nullopt`\n\n      absl::Mutex mu_;\n      size_t current_file_index_ ABSL_GUARDED_BY(mu_) = 0;\n      // `std::nullopt` means not open yet.\n      std::optional<RecordReader<tensorflow::FileReader<>>> reader_\n          ABSL_GUARDED_BY(mu_);\n    };\n\n    const std::vector<std::string> filenames_;\n    const int64_t min_buffer_size_;\n    const int64_t max_buffer_size_;\n  };\n};\n\nREGISTER_KERNEL_BUILDER(Name(\"RiegeliDataset\").Device(::tensorflow::DEVICE_CPU),\n                        RiegeliDatasetOp);\n\n}  // namespace\n}  // namespace riegeli::tensorflow\n"
  },
  {
    "path": "riegeli/tensorflow/ops/riegeli_dataset_ops.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"tensorflow/core/framework/common_shape_fns.h\"\n#include \"tensorflow/core/framework/op.h\"\n#include \"tensorflow/core/framework/shape_inference.h\"\n\nnamespace riegeli::tensorflow {\n\nREGISTER_OP(\"RiegeliDataset\")\n    .Input(\"filenames: string\")\n    .Input(\"min_buffer_size: int64\")\n    .Input(\"max_buffer_size: int64\")\n    .Output(\"handle: variant\")\n    .SetIsStateful()\n    .SetShapeFn([](::tensorflow::shape_inference::InferenceContext* c) {\n      ::tensorflow::shape_inference::ShapeHandle unused;\n      // `filenames` must be a scalar or a vector.\n      TF_RETURN_IF_ERROR(c->WithRankAtMost(c->input(0), 1, &unused));\n      // `min_buffer_size` could only be a scalar.\n      TF_RETURN_IF_ERROR(c->WithRank(c->input(1), 0, &unused));\n      // `max_buffer_size` could only be a scalar.\n      TF_RETURN_IF_ERROR(c->WithRank(c->input(2), 0, &unused));\n      return ::tensorflow::shape_inference::ScalarShape(c);\n    })\n    .Doc(R\"doc(\nCreates a dataset that emits the records from one or more Riegeli/records files.\n\nfilenames: A scalar or vector containing the name(s) of the file(s) to be\n  read.\nmin_buffer_size: Tunes the minimal buffer size, which determines how much data\n  at a time is typically read from the file. The actual buffer size changes\n  between min_buffer_size and max_buffer_size depending on the access pattern.\nmax_buffer_size: Tunes the maximal buffer size, which determines how much data\n  at a time is typically read from the file. The actual buffer size changes\n  between min_buffer_size and max_buffer_size depending on the access pattern.\n)doc\");\n\n}  // namespace riegeli::tensorflow\n"
  },
  {
    "path": "riegeli/text/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"concat\",\n    hdrs = [\"concat.h\"],\n    deps = [\n        \"//riegeli/base:initializer\",\n        \"//riegeli/bytes:ostream_writer\",\n        \"//riegeli/bytes:stringify\",\n        \"//riegeli/bytes:stringify_writer\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n    ],\n)\n\ncc_library(\n    name = \"write_int\",\n    srcs = [\"write_int.cc\"],\n    hdrs = [\"write_int.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:ostream_writer\",\n        \"//riegeli/bytes:write_int_internal\",\n        \"//riegeli/bytes:writer\",\n        \"//riegeli/endian:endian_writing\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:bits\",\n        \"@com_google_absl//absl/numeric:int128\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"ascii_align\",\n    hdrs = [\"ascii_align.h\"],\n    deps = [\n        \":concat\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:byte_fill\",\n        \"//riegeli/base:chain\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:type_traits\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:ostream_writer\",\n        \"//riegeli/bytes:restricted_chain_writer\",\n        \"//riegeli/bytes:stringify\",\n        \"//riegeli/bytes:stringify_writer\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n    ],\n)\n\ncc_library(\n    name = \"join\",\n    hdrs = [\"join.h\"],\n    deps = [\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:iterable\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:ostream_writer\",\n        \"//riegeli/bytes:stringify\",\n        \"//riegeli/bytes:stringify_writer\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/text/ascii_align.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_TEXT_ASCII_ALIGN_H_\n#define RIEGELI_TEXT_ASCII_ALIGN_H_\n\n#include <stddef.h>\n\n#include <algorithm>\n#include <limits>\n#include <ostream>\n#include <tuple>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/byte_fill.h\"\n#include \"riegeli/base/chain.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/type_traits.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/ostream_writer.h\"\n#include \"riegeli/bytes/restricted_chain_writer.h\"\n#include \"riegeli/bytes/stringify.h\"\n#include \"riegeli/bytes/stringify_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/text/concat.h\"\n\nnamespace riegeli {\n\n// Options for `AsciiLeft()`, `AsciiCenter()`, and `AsciiRight()`.\nclass AlignOptions {\n public:\n  AlignOptions() noexcept {}\n\n  // Options can also be specified by the minimum width alone.\n  /*implicit*/ AlignOptions(Position width) : width_(width) {}\n\n  // Minimum width.\n  //\n  // Default: 0.\n  AlignOptions& set_width(Position width) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    width_ = width;\n    return *this;\n  }\n  AlignOptions&& set_width(Position width) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_width(width));\n  }\n  Position width() const { return width_; }\n\n  // The character to fill space before and/or after the value with.\n  //\n  // Default: ' '.\n  AlignOptions& set_fill(char fill) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    fill_ = fill;\n    return *this;\n  }\n  AlignOptions&& set_fill(char fill) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_fill(fill));\n  }\n  char fill() const { return fill_; }\n\n private:\n  Position width_ = 0;\n  char fill_ = ' ';\n};\n\n// The type returned by `riegeli::AsciiLeft()` and `riegeli::OwningAsciiLeft()`.\ntemplate <typename... T>\nclass AsciiLeftType {\n public:\n  explicit AsciiLeftType(std::tuple<Initializer<T>...> values,\n                         AlignOptions options)\n      : values_(std::move(values)), options_(std::move(options)) {}\n\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const AsciiLeftType& src) {\n    src.Stringify(dest);\n  }\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, AsciiLeftType&& src) {\n    std::move(src).Stringify(dest);\n  }\n\n  friend std::ostream& operator<<(std::ostream& dest,\n                                  const AsciiLeftType& src) {\n    OStreamWriter writer(&dest);\n    src.WriteTo(writer);\n    writer.Close();\n    return dest;\n  }\n  friend std::ostream& operator<<(std::ostream& dest, AsciiLeftType&& src) {\n    OStreamWriter writer(&dest);\n    std::move(src).WriteTo(writer);\n    writer.Close();\n    return dest;\n  }\n\n  friend auto RiegeliStringifiedSize(const AsciiLeftType& src) {\n    if constexpr (HasStringifiedSize<T...>::value) {\n      return UnsignedMax(riegeli::StringifiedSize(src.values_),\n                         src.options_.width());\n    }\n  }\n\n private:\n  template <typename Sink>\n  void Stringify(Sink& dest) const&;\n  template <typename Sink>\n  void Stringify(Sink& dest) &&;\n\n  // Faster implementation if `Sink` is `WriterStringifySink`.\n  void Stringify(WriterStringifySink& dest) const& { WriteTo(*dest.dest()); }\n  void Stringify(WriterStringifySink& dest) && {\n    std::move(*this).WriteTo(*dest.dest());\n  }\n\n  void WriteTo(Writer& dest) const&;\n  void WriteTo(Writer& dest) &&;\n\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS ConcatType<T...> values_;\n  AlignOptions options_;\n};\n\n// Wraps a sequence of values such that their concatenated stringified\n// representation is filled to at least the given width, with the values on the\n// left side of the field.\n//\n// The last argument is `AlignOptions` or the width. The remaining arguments are\n// the values.\n//\n// The width is measured in bytes, so this is suitable only for ASCII data.\n//\n// `riegeli::AsciiLeft()` does not own the values, even if they involve\n// temporaries, hence it should be stringified by the same expression which\n// constructed it, so that the temporaries outlive its usage. For storing\n// an `AsciiLeftType` in a variable or returning it from a function, use\n// `riegeli::OwningAsciiLeft()` or construct `AsciiLeftType` directly.\ntemplate <\n    typename... Args,\n    std::enable_if_t<\n        std::conjunction_v<\n            std::bool_constant<sizeof...(Args) != 2>,\n            std::is_convertible<GetTypeFromEndT<1, Args&&...>, AlignOptions>,\n            TupleElementsSatisfy<RemoveTypesFromEndT<1, TargetRefT<Args>...>,\n                                 IsStringifiable>>,\n        int> = 0>\ninline ApplyToTupleElementsT<AsciiLeftType,\n                             RemoveTypesFromEndT<1, TargetRefT<Args>...>>\nAsciiLeft(Args&&... args) {\n  return ApplyToTupleElementsT<AsciiLeftType,\n                               RemoveTypesFromEndT<1, TargetRefT<Args>...>>(\n      RemoveFromEnd<1>(std::forward<Args>(args)...),\n      GetFromEnd<1>(std::forward<Args>(args)...));\n}\n\n// A specialization for one stringifiable parameter which allows to annotate the\n// parameter with `ABSL_ATTRIBUTE_LIFETIME_BOUND`.\ntemplate <typename Arg,\n          std::enable_if_t<IsStringifiable<TargetRefT<Arg>>::value, int> = 0>\ninline AsciiLeftType<TargetRefT<Arg>> AsciiLeft(\n    Arg&& arg ABSL_ATTRIBUTE_LIFETIME_BOUND, AlignOptions options) {\n  return AsciiLeftType<TargetRefT<Arg>>(\n      std::forward_as_tuple(std::forward<Arg>(arg)), std::move(options));\n}\n\n// `riegeli::OwningAsciiLeft()` is like `riegeli::AsciiLeft()`, but the\n// arguments are stored by value instead of by reference. This is useful for\n// storing the `AsciiLeftType` in a variable or returning it from a function.\n//\n// If a particular argument is heavy and its lifetime is sufficient for storing\n// it by reference, convert `const std::string&` to `absl::string_view` or wrap\n// the argument in `std::cref()`.\ntemplate <\n    typename... Args,\n    std::enable_if_t<\n        std::conjunction_v<\n            std::is_convertible<GetTypeFromEndT<1, Args&&...>, AlignOptions>,\n            TupleElementsSatisfy<RemoveTypesFromEndT<1, TargetT<Args>...>,\n                                 IsStringifiable>>,\n        int> = 0>\ninline ApplyToTupleElementsT<AsciiLeftType,\n                             RemoveTypesFromEndT<1, TargetT<Args>...>>\nOwningAsciiLeft(Args&&... args) {\n  return ApplyToTupleElementsT<AsciiLeftType,\n                               RemoveTypesFromEndT<1, TargetT<Args>...>>(\n      RemoveFromEnd<1>(std::forward<Args>(args)...),\n      GetFromEnd<1>(std::forward<Args>(args)...));\n}\n\n// The type returned by `riegeli::AsciiCenter()` and\n// `riegeli::OwningAsciiCenter()`.\ntemplate <typename... T>\nclass AsciiCenterType {\n public:\n  explicit AsciiCenterType(std::tuple<Initializer<T>...> values,\n                           AlignOptions options)\n      : values_(std::move(values)), options_(std::move(options)) {}\n\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const AsciiCenterType& src) {\n    src.Stringify(dest);\n  }\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, AsciiCenterType&& src) {\n    std::move(src).Stringify(dest);\n  }\n\n  friend std::ostream& operator<<(std::ostream& dest,\n                                  const AsciiCenterType& src) {\n    OStreamWriter writer(&dest);\n    src.WriteTo(writer);\n    writer.Close();\n    return dest;\n  }\n  friend std::ostream& operator<<(std::ostream& dest, AsciiCenterType&& src) {\n    OStreamWriter writer(&dest);\n    std::move(src).WriteTo(writer);\n    writer.Close();\n    return dest;\n  }\n\n  friend auto RiegeliStringifiedSize(const AsciiCenterType& src) {\n    if constexpr (HasStringifiedSize<T...>::value) {\n      return UnsignedMax(riegeli::StringifiedSize(src.values_),\n                         src.options_.width());\n    }\n  }\n\n private:\n  template <typename Sink>\n  void Stringify(Sink& dest) const&;\n  template <typename Sink>\n  void Stringify(Sink& dest) &&;\n\n  // Faster implementation if `Sink` is `WriterStringifySink`.\n  void Stringify(WriterStringifySink& dest) const& { WriteTo(*dest.dest()); }\n  void Stringify(WriterStringifySink& dest) && {\n    std::move(*this).WriteTo(*dest.dest());\n  }\n\n  void WriteTo(Writer& dest) const&;\n  void WriteTo(Writer& dest) &&;\n\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS ConcatType<T...> values_;\n  AlignOptions options_;\n};\n\n// Wraps a sequence of values such that their concatenated stringified\n// representation is filled to at least the given width, with the values\n// centered in the field (with one fill character fewer on the left side if\n// there is an odd number of them).\n//\n// The last argument is `AlignOptions` or the width. The remaining arguments are\n// the values.\n//\n// The width is measured in bytes, so this is suitable only for ASCII data.\n//\n// `riegeli::AsciiCenter()` does not own the values, even if they involve\n// temporaries, hence it should be stringified by the same expression which\n// constructed it, so that the temporaries outlive its usage. For storing\n// an `AsciiCenterType` in a variable or returning it from a function, use\n// `riegeli::OwningAsciiCenter()` or construct `AsciiCenterType` directly.\ntemplate <\n    typename... Args,\n    std::enable_if_t<\n        std::conjunction_v<\n            std::bool_constant<sizeof...(Args) != 2>,\n            std::is_convertible<GetTypeFromEndT<1, Args&&...>, AlignOptions>,\n            TupleElementsSatisfy<RemoveTypesFromEndT<1, TargetRefT<Args>...>,\n                                 IsStringifiable>>,\n        int> = 0>\ninline ApplyToTupleElementsT<AsciiCenterType,\n                             RemoveTypesFromEndT<1, TargetRefT<Args>...>>\nAsciiCenter(Args&&... args) {\n  return ApplyToTupleElementsT<AsciiCenterType,\n                               RemoveTypesFromEndT<1, TargetRefT<Args>...>>(\n      RemoveFromEnd<1>(std::forward<Args>(args)...),\n      GetFromEnd<1>(std::forward<Args>(args)...));\n}\n\n// A specialization for one stringifiable parameter which allows to annotate the\n// parameter with `ABSL_ATTRIBUTE_LIFETIME_BOUND`.\ntemplate <typename Arg,\n          std::enable_if_t<IsStringifiable<TargetRefT<Arg>>::value, int> = 0>\ninline AsciiCenterType<TargetRefT<Arg>> AsciiCenter(\n    Arg&& arg ABSL_ATTRIBUTE_LIFETIME_BOUND, AlignOptions options) {\n  return AsciiCenterType<TargetRefT<Arg>>(\n      std::forward_as_tuple(std::forward<Arg>(arg)), std::move(options));\n}\n\n// `riegeli::OwningAsciiCenter()` is like `riegeli::AsciiCenter()`, but the\n// arguments are stored by value instead of by reference. This is useful for\n// storing the `AsciiCenterType` in a variable or returning it from a function.\n//\n// If a particular argument is heavy and its lifetime is sufficient for storing\n// it by reference, convert `const std::string&` to `absl::string_view` or wrap\n// the argument in `std::cref()`.\ntemplate <\n    typename... Args,\n    std::enable_if_t<\n        std::conjunction_v<\n            std::is_convertible<GetTypeFromEndT<1, Args&&...>, AlignOptions>,\n            TupleElementsSatisfy<RemoveTypesFromEndT<1, TargetT<Args>...>,\n                                 IsStringifiable>>,\n        int> = 0>\ninline ApplyToTupleElementsT<AsciiCenterType,\n                             RemoveTypesFromEndT<1, TargetT<Args>...>>\nOwningAsciiCenter(Args&&... args) {\n  return ApplyToTupleElementsT<AsciiCenterType,\n                               RemoveTypesFromEndT<1, TargetT<Args>...>>(\n      RemoveFromEnd<1>(std::forward<Args>(args)...),\n      GetFromEnd<1>(std::forward<Args>(args)...));\n}\n\n// The type returned by `riegeli::AsciiRight()` and\n// `riegeli::OwningAsciiRight()`.\ntemplate <typename... T>\nclass AsciiRightType {\n public:\n  explicit AsciiRightType(std::tuple<Initializer<T>...> values,\n                          AlignOptions options)\n      : values_(std::move(values)), options_(std::move(options)) {}\n\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const AsciiRightType& src) {\n    src.Stringify(dest);\n  }\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, AsciiRightType&& src) {\n    std::move(src).Stringify(dest);\n  }\n\n  friend std::ostream& operator<<(std::ostream& dest,\n                                  const AsciiRightType& src) {\n    OStreamWriter writer(&dest);\n    src.WriteTo(writer);\n    writer.Close();\n    return dest;\n  }\n  friend std::ostream& operator<<(std::ostream& dest, AsciiRightType&& src) {\n    OStreamWriter writer(&dest);\n    std::move(src).WriteTo(writer);\n    writer.Close();\n    return dest;\n  }\n\n  friend auto RiegeliStringifiedSize(const AsciiRightType& src) {\n    if constexpr (HasStringifiedSize<T...>::value) {\n      return UnsignedMax(riegeli::StringifiedSize(src.values_),\n                         src.options_.width());\n    }\n  }\n\n private:\n  template <typename Sink>\n  void Stringify(Sink& dest) const&;\n  template <typename Sink>\n  void Stringify(Sink& dest) &&;\n\n  // Faster implementation if `Sink` is `WriterStringifySink`.\n  void Stringify(WriterStringifySink& dest) const& { WriteTo(*dest.dest()); }\n  void Stringify(WriterStringifySink& dest) && {\n    std::move(*this).WriteTo(*dest.dest());\n  }\n\n  void WriteTo(Writer& dest) const&;\n  void WriteTo(Writer& dest) &&;\n\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS ConcatType<T...> values_;\n  AlignOptions options_;\n};\n\n// Wraps a sequence of values such that their concatenated stringified\n// representation is filled to at least the given width, with the values on the\n// right side of the field.\n//\n// The last argument is `AlignOptions` or the width. The remaining arguments are\n// the values.\n//\n// The width is measured in bytes, so this is suitable only for ASCII data.\n//\n// `riegeli::AsciiRight()` does not own the values, even if they involve\n// temporaries, hence it should be stringified by the same expression which\n// constructed it, so that the temporaries outlive its usage. For storing\n// an `AsciiRightType` in a variable or returning it from a function, use\n// `riegeli::OwningAsciiRight()` or construct `AsciiRightType` directly.\ntemplate <\n    typename... Args,\n    std::enable_if_t<\n        std::conjunction_v<\n            std::bool_constant<sizeof...(Args) != 2>,\n            std::is_convertible<GetTypeFromEndT<1, Args&&...>, AlignOptions>,\n            TupleElementsSatisfy<RemoveTypesFromEndT<1, TargetRefT<Args>...>,\n                                 IsStringifiable>>,\n        int> = 0>\ninline ApplyToTupleElementsT<AsciiRightType,\n                             RemoveTypesFromEndT<1, TargetRefT<Args>...>>\nAsciiRight(Args&&... args) {\n  return ApplyToTupleElementsT<AsciiRightType,\n                               RemoveTypesFromEndT<1, TargetRefT<Args>...>>(\n      RemoveFromEnd<1>(std::forward<Args>(args)...),\n      GetFromEnd<1>(std::forward<Args>(args)...));\n}\n\n// A specialization for one stringifiable parameter which allows to annotate the\n// parameter with `ABSL_ATTRIBUTE_LIFETIME_BOUND`.\ntemplate <typename Arg,\n          std::enable_if_t<IsStringifiable<TargetRefT<Arg>>::value, int> = 0>\ninline AsciiRightType<TargetRefT<Arg>> AsciiRight(\n    Arg&& arg ABSL_ATTRIBUTE_LIFETIME_BOUND, AlignOptions options) {\n  return AsciiRightType<TargetRefT<Arg>>(\n      std::forward_as_tuple(std::forward<Arg>(arg)), std::move(options));\n}\n\n// `riegeli::OwningAsciiRight()` is like `riegeli::AsciiRight()`, but the\n// arguments are stored by value instead of by reference. This is useful for\n// storing the `AsciiRightType` in a variable or returning it from a function.\n//\n// If a particular argument is heavy and its lifetime is sufficient for storing\n// it by reference, convert `const std::string&` to `absl::string_view` or wrap\n// the argument in `std::cref()`.\ntemplate <\n    typename... Args,\n    std::enable_if_t<\n        std::conjunction_v<\n            std::is_convertible<GetTypeFromEndT<1, Args&&...>, AlignOptions>,\n            TupleElementsSatisfy<RemoveTypesFromEndT<1, TargetT<Args>...>,\n                                 IsStringifiable>>,\n        int> = 0>\ninline ApplyToTupleElementsT<AsciiRightType,\n                             RemoveTypesFromEndT<1, TargetT<Args>...>>\nOwningAsciiRight(Args&&... args) {\n  return ApplyToTupleElementsT<AsciiRightType,\n                               RemoveTypesFromEndT<1, TargetT<Args>...>>(\n      RemoveFromEnd<1>(std::forward<Args>(args)...),\n      GetFromEnd<1>(std::forward<Args>(args)...));\n}\n\n// Implementation details follow.\n\nnamespace align_internal {\n\ntemplate <typename Sink>\ninline void WritePadding(Sink& dest, Position length, char fill) {\n  while (ABSL_PREDICT_FALSE(length > std::numeric_limits<size_t>::max())) {\n    dest.Append(std::numeric_limits<size_t>::max(), fill);\n    length -= std::numeric_limits<size_t>::max();\n  }\n  if (length > 0) dest.Append(IntCast<size_t>(length), fill);\n}\n\n}  // namespace align_internal\n\ntemplate <typename... T>\ntemplate <typename Sink>\ninline void AsciiLeftType<T...>::Stringify(Sink& dest) const& {\n  StringifyWriter writer(&dest);\n  writer.Write(values_);\n  if (ABSL_PREDICT_FALSE(!writer.Close())) return;\n  align_internal::WritePadding(\n      dest, SaturatingSub(options_.width(), writer.pos()), options_.fill());\n}\n\ntemplate <typename... T>\ntemplate <typename Sink>\ninline void AsciiLeftType<T...>::Stringify(Sink& dest) && {\n  StringifyWriter writer(&dest);\n  writer.Write(std::move(values_));\n  if (ABSL_PREDICT_FALSE(!writer.Close())) return;\n  align_internal::WritePadding(\n      dest, SaturatingSub(options_.width(), writer.pos()), options_.fill());\n}\n\ntemplate <typename... T>\ninline void AsciiLeftType<T...>::WriteTo(Writer& dest) const& {\n  const Position pos_before = dest.pos();\n  dest.Write(values_);\n  RIEGELI_ASSERT_GE(dest.pos(), pos_before)\n      << \"Writer::Write() decreased pos()\";\n  dest.Write(ByteFill(SaturatingSub(options_.width(), dest.pos() - pos_before),\n                      options_.fill()));\n}\n\ntemplate <typename... T>\ninline void AsciiLeftType<T...>::WriteTo(Writer& dest) && {\n  const Position pos_before = dest.pos();\n  dest.Write(std::move(values_));\n  RIEGELI_ASSERT_GE(dest.pos(), pos_before)\n      << \"Writer::Write() decreased pos()\";\n  dest.Write(ByteFill(SaturatingSub(options_.width(), dest.pos() - pos_before),\n                      options_.fill()));\n}\n\ntemplate <typename... T>\ntemplate <typename Sink>\ninline void AsciiCenterType<T...>::Stringify(Sink& dest) const& {\n  if constexpr (HasStringifiedSize<T...>::value) {\n    const Position padding =\n        SaturatingSub(options_.width(), riegeli::StringifiedSize(values_));\n    align_internal::WritePadding(dest, padding / 2, options_.fill());\n    StringifyWriter writer(&dest);\n    writer.Write(values_);\n    if (ABSL_PREDICT_FALSE(!writer.Close())) return;\n    align_internal::WritePadding(dest, padding - padding / 2, options_.fill());\n  } else {\n    RestrictedChainWriter chain_writer;\n    chain_writer.Write(values_);\n    if (ABSL_PREDICT_FALSE(!chain_writer.Close())) return;\n    const Position padding =\n        SaturatingSub(options_.width(), chain_writer.dest().size());\n    align_internal::WritePadding(dest, padding / 2, options_.fill());\n    AbslStringify(dest, chain_writer.dest());\n    align_internal::WritePadding(dest, padding - padding / 2, options_.fill());\n  }\n}\n\ntemplate <typename... T>\ntemplate <typename Sink>\ninline void AsciiCenterType<T...>::Stringify(Sink& dest) && {\n  if constexpr (HasStringifiedSize<T...>::value) {\n    const Position padding =\n        SaturatingSub(options_.width(), riegeli::StringifiedSize(values_));\n    align_internal::WritePadding(dest, padding / 2, options_.fill());\n    StringifyWriter writer(&dest);\n    writer.Write(std::move(values_));\n    if (ABSL_PREDICT_FALSE(!writer.Close())) return;\n    align_internal::WritePadding(dest, padding - padding / 2, options_.fill());\n  } else {\n    RestrictedChainWriter chain_writer;\n    chain_writer.Write(std::move(values_));\n    if (ABSL_PREDICT_FALSE(!chain_writer.Close())) return;\n    const Position padding =\n        SaturatingSub(options_.width(), chain_writer.dest().size());\n    align_internal::WritePadding(dest, padding / 2, options_.fill());\n    AbslStringify(dest, chain_writer.dest());\n    align_internal::WritePadding(dest, padding - padding / 2, options_.fill());\n  }\n}\n\ntemplate <typename... T>\ninline void AsciiCenterType<T...>::WriteTo(Writer& dest) const& {\n  if constexpr (HasStringifiedSize<T...>::value) {\n    const Position padding =\n        SaturatingSub(options_.width(), riegeli::StringifiedSize(values_));\n    dest.Write(ByteFill(padding / 2, options_.fill()));\n    dest.Write(values_);\n    dest.Write(ByteFill(padding - padding / 2, options_.fill()));\n  } else {\n    RestrictedChainWriter chain_writer;\n    chain_writer.Write(values_);\n    if (ABSL_PREDICT_FALSE(!chain_writer.Close())) {\n      dest.Fail(chain_writer.status());\n      return;\n    }\n    const Position padding =\n        SaturatingSub(options_.width(), chain_writer.dest().size());\n    dest.Write(ByteFill(padding / 2, options_.fill()));\n    dest.Write(std::move(chain_writer.dest()));\n    dest.Write(ByteFill(padding - padding / 2, options_.fill()));\n  }\n}\n\ntemplate <typename... T>\ninline void AsciiCenterType<T...>::WriteTo(Writer& dest) && {\n  if constexpr (HasStringifiedSize<T...>::value) {\n    const Position padding =\n        SaturatingSub(options_.width(), riegeli::StringifiedSize(values_));\n    dest.Write(ByteFill(padding / 2, options_.fill()));\n    dest.Write(std::move(values_));\n    dest.Write(ByteFill(padding - padding / 2, options_.fill()));\n  } else {\n    RestrictedChainWriter chain_writer;\n    chain_writer.Write(std::move(values_));\n    if (ABSL_PREDICT_FALSE(!chain_writer.Close())) {\n      dest.Fail(chain_writer.status());\n      return;\n    }\n    const Position padding =\n        SaturatingSub(options_.width(), chain_writer.dest().size());\n    dest.Write(ByteFill(padding / 2, options_.fill()));\n    dest.Write(std::move(chain_writer.dest()));\n    dest.Write(ByteFill(padding - padding / 2, options_.fill()));\n  }\n}\n\ntemplate <typename... T>\ntemplate <typename Sink>\ninline void AsciiRightType<T...>::Stringify(Sink& dest) const& {\n  if constexpr (HasStringifiedSize<T...>::value) {\n    align_internal::WritePadding(\n        dest,\n        SaturatingSub(options_.width(), riegeli::StringifiedSize(values_)),\n        options_.fill());\n    StringifyWriter writer(&dest);\n    writer.Write(values_);\n    writer.Close();\n  } else {\n    RestrictedChainWriter chain_writer;\n    chain_writer.Write(values_);\n    if (ABSL_PREDICT_FALSE(!chain_writer.Close())) return;\n    const Position padding =\n        SaturatingSub(options_.width(), chain_writer.dest().size());\n    align_internal::WritePadding(dest, padding, options_.fill());\n    AbslStringify(dest, chain_writer.dest());\n  }\n}\n\ntemplate <typename... T>\ntemplate <typename Sink>\ninline void AsciiRightType<T...>::Stringify(Sink& dest) && {\n  if constexpr (HasStringifiedSize<T...>::value) {\n    align_internal::WritePadding(\n        dest,\n        SaturatingSub(options_.width(), riegeli::StringifiedSize(values_)),\n        options_.fill());\n    StringifyWriter writer(&dest);\n    writer.Write(std::move(values_));\n    writer.Close();\n  } else {\n    RestrictedChainWriter chain_writer;\n    chain_writer.Write(std::move(values_));\n    if (ABSL_PREDICT_FALSE(!chain_writer.Close())) return;\n    const Position padding =\n        SaturatingSub(options_.width(), chain_writer.dest().size());\n    align_internal::WritePadding(dest, padding, options_.fill());\n    AbslStringify(dest, chain_writer.dest());\n  }\n}\n\ntemplate <typename... T>\ninline void AsciiRightType<T...>::WriteTo(Writer& dest) const& {\n  if constexpr (HasStringifiedSize<T...>::value) {\n    dest.Write(ByteFill(\n        SaturatingSub(options_.width(), riegeli::StringifiedSize(values_)),\n        options_.fill()));\n    dest.Write(values_);\n  } else {\n    RestrictedChainWriter chain_writer;\n    chain_writer.Write(values_);\n    if (ABSL_PREDICT_FALSE(!chain_writer.Close())) {\n      dest.Fail(chain_writer.status());\n      return;\n    }\n    dest.Write(\n        ByteFill(SaturatingSub(options_.width(), chain_writer.dest().size()),\n                 options_.fill()));\n    dest.Write(std::move(chain_writer.dest()));\n  }\n}\n\ntemplate <typename... T>\ninline void AsciiRightType<T...>::WriteTo(Writer& dest) && {\n  if constexpr (HasStringifiedSize<T...>::value) {\n    dest.Write(ByteFill(\n        SaturatingSub(options_.width(), riegeli::StringifiedSize(values_)),\n        options_.fill()));\n    dest.Write(std::move(values_));\n  } else {\n    RestrictedChainWriter chain_writer;\n    chain_writer.Write(std::move(values_));\n    if (ABSL_PREDICT_FALSE(!chain_writer.Close())) {\n      dest.Fail(chain_writer.status());\n      return;\n    }\n    dest.Write(\n        ByteFill(SaturatingSub(options_.width(), chain_writer.dest().size()),\n                 options_.fill()));\n    dest.Write(std::move(chain_writer.dest()));\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_TEXT_ASCII_ALIGN_H_\n"
  },
  {
    "path": "riegeli/text/concat.h",
    "content": "// Copyright 2025 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_TEXT_CONCAT_H_\n#define RIEGELI_TEXT_CONCAT_H_\n\n#include <ostream>\n#include <tuple>\n#include <type_traits>  // IWYU pragma: keep\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/bytes/ostream_writer.h\"\n#include \"riegeli/bytes/stringify.h\"\n#include \"riegeli/bytes/stringify_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// The type returned by `riegeli::Concat()` and `riegeli::OwningConcat()`.\ntemplate <typename... T>\nclass ConcatType {\n public:\n  explicit ConcatType(std::tuple<Initializer<T>...> values)\n      : values_(std::move(values)) {}\n\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const ConcatType& src) {\n    src.Stringify(dest);\n  }\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, ConcatType&& src) {\n    std::move(src).Stringify(dest);\n  }\n\n  friend std::ostream& operator<<(std::ostream& dest, const ConcatType& src) {\n    OStreamWriter<> writer(&dest);\n    src.WriteTo(writer);\n    writer.Close();\n    return dest;\n  }\n  friend std::ostream& operator<<(std::ostream& dest, ConcatType&& src) {\n    OStreamWriter<> writer(&dest);\n    std::move(src).WriteTo(writer);\n    writer.Close();\n    return dest;\n  }\n\n  friend auto RiegeliStringifiedSize(const ConcatType& src) {\n    return std::apply(\n        [](const T&... values) { return riegeli::StringifiedSize(values...); },\n        src.values_);\n  }\n\n private:\n  template <typename Sink>\n  void Stringify(Sink& dest) const& {\n    StringifyWriter writer(&dest);\n    WriteTo(writer);\n    writer.Close();\n  }\n  template <typename Sink>\n  void Stringify(Sink& dest) && {\n    StringifyWriter writer(&dest);\n    std::move(*this).WriteTo(writer);\n    writer.Close();\n  }\n\n  // Faster implementation if `Sink` is `WriterStringifySink`.\n  void Stringify(WriterStringifySink& dest) const& { WriteTo(*dest.dest()); }\n  void Stringify(WriterStringifySink& dest) && {\n    std::move(*this).WriteTo(*dest.dest());\n  }\n\n  void WriteTo(Writer& dest) const& {\n    std::apply([&](const T&... values) { dest.Write(values...); }, values_);\n  }\n  void WriteTo(Writer& dest) && {\n    std::apply([&](T&&... values) { dest.Write(std::forward<T>(values)...); },\n               std::move(values_));\n  }\n\n  ABSL_ATTRIBUTE_NO_UNIQUE_ADDRESS std::tuple<T...> values_;\n};\n\n// Wraps a sequence of values such that its stringified representation is the\n// concatenation of stringified representations of the values.\n//\n// `riegeli::Concat()` does not own the values, even if they involve\n// temporaries, hence it should be stringified by the same expression which\n// constructed it, so that the temporaries outlive its usage. For storing\n// a `ConcatType` in a variable or returning it from a function, use\n// `riegeli::OwningConcat()` or construct `ConcatType` directly.\ntemplate <typename... Srcs\n#if !__cpp_concepts\n          ,\n          std::enable_if_t<IsStringifiable<TargetRefT<Srcs>...>::value, int> = 0\n#endif\n          >\ninline ConcatType<TargetRefT<Srcs>...> Concat(\n    Srcs&&... srcs ABSL_ATTRIBUTE_LIFETIME_BOUND)\n#if __cpp_concepts\n    // For conjunctions, `requires` gives better error messages than\n    // `std::enable_if_t`, indicating the relevant argument.\n  requires(IsStringifiable<TargetRefT<Srcs>>::value && ...)\n#endif\n{\n  return ConcatType<TargetRefT<Srcs>...>(\n      std::forward_as_tuple(std::forward<Srcs>(srcs)...));\n}\n\n// `riegeli::OwningConcat()` is like `riegeli::Concat()`, but the arguments are\n// stored by value instead of by reference. This is useful for storing the\n// `ConcatType` in a variable or returning it from a function.\n//\n// If a particular argument is heavy and its lifetime is sufficient for storing\n// it by reference, convert `const std::string&` to `absl::string_view` or wrap\n// the argument in `std::cref()`.\ntemplate <typename... Srcs\n#if !__cpp_concepts\n          ,\n          std::enable_if_t<IsStringifiable<TargetT<Srcs>...>::value, int> = 0\n#endif\n          >\ninline ConcatType<TargetT<Srcs>...> OwningConcat(Srcs&&... srcs)\n#if __cpp_concepts\n    // For conjunctions, `requires` gives better error messages than\n    // `std::enable_if_t`, indicating the relevant argument.\n  requires(IsStringifiable<TargetT<Srcs>>::value && ...)\n#endif\n{\n  return ConcatType<TargetT<Srcs>...>(\n      std::forward_as_tuple(std::forward<Srcs>(srcs)...));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_TEXT_CONCAT_H_\n"
  },
  {
    "path": "riegeli/text/join.h",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_TEXT_JOIN_H_\n#define RIEGELI_TEXT_JOIN_H_\n\n#include <algorithm>\n#include <functional>\n#include <initializer_list>\n#include <iterator>\n#include <ostream>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/iterable.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/ostream_writer.h\"\n#include \"riegeli/bytes/stringify.h\"\n#include \"riegeli/bytes/stringify_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// `FormatterHasStringifiedSize<Formatter, Value>::value` is `true` if\n// `Formatter` supports `StringifiedSize()` when called with a `Value`.\n//\n// `formatter.StringifiedSize(value)` returns the size of the formatted value as\n// `Position` if easily known. A function returning `void` is treated as absent.\n\ntemplate <typename Formatter, typename Value, typename Enable = void>\nstruct FormatterHasStringifiedSize : std::false_type {};\n\ntemplate <typename Formatter, typename Value>\nstruct FormatterHasStringifiedSize<\n    Formatter, Value,\n    std::enable_if_t<std::is_convertible_v<\n        decltype(std::declval<const Formatter&>().StringifiedSize(\n            std::declval<const Value&>())),\n        Position>>> : std::true_type {};\n\n// The default formatter for `Join()` which formats a value by using\n// `Writer::Write()`.\nstruct DefaultFormatter {\n  template <typename Value>\n  void operator()(const Value& src, Writer& dest) const {\n    dest.Write(src);\n  }\n\n  template <typename Value>\n  auto StringifiedSize(const Value& src) const {\n    return riegeli::StringifiedSize(src);\n  }\n};\n\n// A formatter for `Join()` which formats a value by invoking a function and\n// using `Writer::Write()` on the result.\n//\n// The function should be cheap enough that invoking it twice to compute\n// `StringifiedSize()` is acceptable. If it is expensive, use a lambda as the\n// formatter: `[](Value src, Writer& dest) { dest.Write(function(src)); }`\ntemplate <typename Function>\nclass InvokingFormatter {\n public:\n  constexpr InvokingFormatter() : function_() {}\n\n  explicit InvokingFormatter(Initializer<Function> function)\n      : function_(std::move(function)) {}\n\n  template <typename Value>\n  void operator()(const Value& src, Writer& dest) const {\n    dest.Write(std::invoke(function_, src));\n  }\n\n  template <typename Value>\n  auto StringifiedSize(const Value& src) const {\n    return riegeli::StringifiedSize(std::invoke(function_, src));\n  }\n\n private:\n  Function function_;\n};\n\ntemplate <typename Function>\nexplicit InvokingFormatter(Function&& function)\n    -> InvokingFormatter<TargetT<Function>>;\n\n// A formatter for `Join()` which decorates the value with a string before\n// and/or a string after formatting it with another formatter.\ntemplate <typename ValueFormatter = DefaultFormatter>\nclass DecoratingFormatter {\n public:\n  explicit constexpr DecoratingFormatter(\n      absl::string_view after ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : value_formatter_(), after_(after) {}\n\n  explicit DecoratingFormatter(Initializer<ValueFormatter> value_formatter,\n                               absl::string_view after\n                                   ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : value_formatter_(std::move(value_formatter)), after_(after) {}\n\n  explicit constexpr DecoratingFormatter(\n      absl::string_view before ABSL_ATTRIBUTE_LIFETIME_BOUND,\n      absl::string_view after ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : before_(before), value_formatter_(), after_(after) {}\n\n  explicit DecoratingFormatter(\n      absl::string_view before ABSL_ATTRIBUTE_LIFETIME_BOUND,\n      Initializer<ValueFormatter> value_formatter,\n      absl::string_view after ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : before_(before),\n        value_formatter_(std::move(value_formatter)),\n        after_(after) {}\n\n  template <typename Value>\n  void operator()(const Value& src, Writer& dest) const {\n    dest.Write(before_);\n    value_formatter_(src, dest);\n    dest.Write(after_);\n  }\n\n  template <typename Value>\n  auto StringifiedSize(const Value& src) const {\n    if constexpr (FormatterHasStringifiedSize<ValueFormatter, Value>::value) {\n      return before_.size() + value_formatter_.StringifiedSize(src) +\n             after_.size();\n    }\n  }\n\n private:\n  absl::string_view before_;\n  ValueFormatter value_formatter_;\n  absl::string_view after_;\n};\n\nexplicit DecoratingFormatter(absl::string_view after)\n    -> DecoratingFormatter<DefaultFormatter>;\ntemplate <\n    typename ValueFormatter = DefaultFormatter,\n    std::enable_if_t<!std::is_convertible_v<ValueFormatter, absl::string_view>,\n                     int> = 0>\nexplicit DecoratingFormatter(ValueFormatter&& value_formatter,\n                             absl::string_view after)\n    -> DecoratingFormatter<TargetT<ValueFormatter>>;\nexplicit DecoratingFormatter(absl::string_view before, absl::string_view after)\n    -> DecoratingFormatter<DefaultFormatter>;\ntemplate <typename ValueFormatter = DefaultFormatter>\nexplicit DecoratingFormatter(absl::string_view before,\n                             ValueFormatter&& value_formatter,\n                             absl::string_view after)\n    -> DecoratingFormatter<TargetT<ValueFormatter>>;\n\n// A formatter for `Join()` which formats a pair with a separator between the\n// elements.\ntemplate <typename FirstFormatter = DefaultFormatter,\n          typename SecondFormatter = DefaultFormatter>\nclass PairFormatter {\n public:\n  explicit constexpr PairFormatter(\n      absl::string_view separator ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : first_formatter_(), separator_(separator), second_formatter_() {}\n\n  explicit PairFormatter(absl::string_view separator\n                             ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                         Initializer<SecondFormatter> second_formatter)\n      : first_formatter_(),\n        separator_(separator),\n        second_formatter_(std::move(second_formatter)) {}\n\n  explicit PairFormatter(Initializer<FirstFormatter> first_formatter,\n                         absl::string_view separator\n                             ABSL_ATTRIBUTE_LIFETIME_BOUND)\n      : first_formatter_(std::move(first_formatter)),\n        separator_(separator),\n        second_formatter_() {}\n\n  explicit PairFormatter(Initializer<FirstFormatter> first_formatter,\n                         absl::string_view separator\n                             ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                         Initializer<SecondFormatter> second_formatter)\n      : first_formatter_(std::move(first_formatter)),\n        separator_(separator),\n        second_formatter_(std::move(second_formatter)) {}\n\n  template <typename First, typename Second>\n  void operator()(const std::pair<First, Second>& src, Writer& dest) const {\n    first_formatter_(src.first, dest);\n    dest.Write(separator_);\n    second_formatter_(src.second, dest);\n  }\n\n  template <typename First, typename Second>\n  auto StringifiedSize(const std::pair<First, Second>& src) const {\n    if constexpr (std::conjunction_v<\n                      FormatterHasStringifiedSize<FirstFormatter, First>,\n                      FormatterHasStringifiedSize<SecondFormatter, Second>>) {\n      return first_formatter_.StringifiedSize(src.first) + separator_.size() +\n             second_formatter_.StringifiedSize(src.second);\n    }\n  }\n\n private:\n  FirstFormatter first_formatter_;\n  absl::string_view separator_;\n  SecondFormatter second_formatter_;\n};\n\ntemplate <typename SecondFormatter = DefaultFormatter>\nexplicit PairFormatter(absl::string_view separator,\n                       SecondFormatter&& second_formatter = {})\n    -> PairFormatter<DefaultFormatter, TargetT<SecondFormatter>>;\ntemplate <typename FirstFormatter = DefaultFormatter,\n          typename SecondFormatter = DefaultFormatter>\nexplicit PairFormatter(FirstFormatter&& first_formatter,\n                       absl::string_view separator,\n                       SecondFormatter&& second_formatter = {})\n    -> PairFormatter<TargetT<FirstFormatter>, TargetT<SecondFormatter>>;\n\n// The type returned by `riegeli::Join()` and `riegeli::OwningJoin()`.\ntemplate <typename Src, typename Formatter = DefaultFormatter>\nclass JoinType {\n public:\n  explicit JoinType(Initializer<Src> src, Initializer<Formatter> formatter = {})\n      : src_(std::move(src)), formatter_(std::move(formatter)) {}\n\n  explicit JoinType(Initializer<Src> src,\n                    absl::string_view separator ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                    Initializer<Formatter> formatter = {})\n      : src_(std::move(src)),\n        separator_(separator),\n        formatter_(std::move(formatter)) {}\n\n  JoinType(const JoinType& that) = default;\n  JoinType& operator=(const JoinType& that) = default;\n\n  JoinType(JoinType&& that) = default;\n  JoinType& operator=(JoinType&& that) = default;\n\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const JoinType& src) {\n    src.Stringify(dest);\n  }\n\n  friend std::ostream& operator<<(std::ostream& dest, const JoinType& src) {\n    OStreamWriter<> writer(&dest);\n    src.WriteTo(writer);\n    writer.Close();\n    return dest;\n  }\n\n  friend auto RiegeliStringifiedSize(const JoinType& src) {\n    return src.StringifiedSize();\n  }\n\n private:\n  template <typename Sink>\n  void Stringify(Sink& dest) const {\n    StringifyWriter writer(&dest);\n    WriteTo(writer);\n    writer.Close();\n  }\n\n  // Faster implementation if `Sink` is `WriterStringifySink`.\n  void Stringify(WriterStringifySink& dest) const { WriteTo(*dest.dest()); }\n\n  void WriteTo(Writer& dest) const;\n\n  auto StringifiedSize() const;\n\n  Src src_;\n  absl::string_view separator_;\n  Formatter formatter_;\n};\n\n// `riegeli::Join()` wraps a collection such that its stringified representation\n// joins elements with a separator. Each element is formatted with the given\n// formatter.\n//\n// `riegeli::Join()` does not own the collection nor the formatter, even if they\n// involve temporaries, hence it should be stringified by the same expression\n// which constructed it, so that the temporaries outlive its usage. For storing\n// a `JoinType` in a variable or returning it from a function, use\n// `riegeli::OwningJoin()` or construct `JoinType` directly.\n\ntemplate <typename Src, typename Formatter = DefaultFormatter,\n          std::enable_if_t<\n              !std::is_convertible_v<Formatter&&, absl::string_view>, int> = 0>\ninline JoinType<TargetRefT<Src>, TargetRefT<Formatter>> Join(\n    Src&& src ABSL_ATTRIBUTE_LIFETIME_BOUND,\n    Formatter&& formatter ABSL_ATTRIBUTE_LIFETIME_BOUND = Formatter()) {\n  return JoinType<TargetRefT<Src>, TargetRefT<Formatter>>(\n      std::forward<Src>(src), std::forward<Formatter>(formatter));\n}\n\ntemplate <typename Value = absl::string_view,\n          typename Formatter = DefaultFormatter,\n          std::enable_if_t<\n              !std::is_convertible_v<Formatter&&, absl::string_view>, int> = 0>\ninline JoinType<std::initializer_list<Value>, TargetRefT<Formatter>> Join(\n    std::initializer_list<Value> src ABSL_ATTRIBUTE_LIFETIME_BOUND,\n    Formatter&& formatter ABSL_ATTRIBUTE_LIFETIME_BOUND = Formatter()) {\n  return JoinType<std::initializer_list<Value>, TargetRefT<Formatter>>(\n      src, std::forward<Formatter>(formatter));\n}\n\ntemplate <typename Src, typename Formatter = DefaultFormatter>\ninline JoinType<TargetRefT<Src>, TargetRefT<Formatter>> Join(\n    Src&& src ABSL_ATTRIBUTE_LIFETIME_BOUND,\n    absl::string_view separator ABSL_ATTRIBUTE_LIFETIME_BOUND,\n    Formatter&& formatter ABSL_ATTRIBUTE_LIFETIME_BOUND = Formatter()) {\n  return JoinType<TargetRefT<Src>, TargetRefT<Formatter>>(\n      std::forward<Src>(src), separator, std::forward<Formatter>(formatter));\n}\n\ntemplate <typename Value = absl::string_view,\n          typename Formatter = DefaultFormatter>\ninline JoinType<std::initializer_list<Value>, TargetRefT<Formatter>> Join(\n    std::initializer_list<Value> src ABSL_ATTRIBUTE_LIFETIME_BOUND,\n    absl::string_view separator ABSL_ATTRIBUTE_LIFETIME_BOUND,\n    Formatter&& formatter ABSL_ATTRIBUTE_LIFETIME_BOUND = Formatter()) {\n  return JoinType<std::initializer_list<Value>, TargetRefT<Formatter>>(\n      src, separator, std::forward<Formatter>(formatter));\n}\n\n// `riegeli::OwningJoin()` is like `riegeli::Join()`, but the arguments are\n// stored by value instead of by reference. This is useful for storing the\n// `JoinType` in a variable or returning it from a function.\n//\n// If the lifetime of the collection is sufficient for storing it by reference,\n// wrap it in `std::cref()`.\n\ntemplate <typename Src, typename Formatter = DefaultFormatter,\n          std::enable_if_t<\n              !std::is_convertible_v<Formatter&&, absl::string_view>, int> = 0>\ninline JoinType<TargetT<Src>, TargetT<Formatter>> OwningJoin(\n    Src&& src, Formatter&& formatter = Formatter()) {\n  return JoinType<TargetT<Src>, TargetT<Formatter>>(\n      std::forward<Src>(src), std::forward<Formatter>(formatter));\n}\n\ntemplate <typename Value = absl::string_view,\n          typename Formatter = DefaultFormatter,\n          std::enable_if_t<\n              !std::is_convertible_v<Formatter&&, absl::string_view>, int> = 0>\ninline JoinType<std::initializer_list<Value>, TargetT<Formatter>> OwningJoin(\n    std::initializer_list<Value> src, Formatter&& formatter = Formatter()) {\n  return JoinType<std::initializer_list<Value>, TargetT<Formatter>>(\n      src, std::forward<Formatter>(formatter));\n}\n\ntemplate <typename Src, typename Formatter = DefaultFormatter>\ninline JoinType<TargetT<Src>, TargetT<Formatter>> OwningJoin(\n    Src&& src, absl::string_view separator ABSL_ATTRIBUTE_LIFETIME_BOUND,\n    Formatter&& formatter = Formatter()) {\n  return JoinType<TargetT<Src>, TargetT<Formatter>>(\n      std::forward<Src>(src), separator, std::forward<Formatter>(formatter));\n}\n\ntemplate <typename Value = absl::string_view,\n          typename Formatter = DefaultFormatter>\ninline JoinType<std::initializer_list<Value>, TargetT<Formatter>> OwningJoin(\n    std::initializer_list<Value> src,\n    absl::string_view separator ABSL_ATTRIBUTE_LIFETIME_BOUND,\n    Formatter&& formatter = Formatter()) {\n  return JoinType<std::initializer_list<Value>, TargetT<Formatter>>(\n      src, separator, std::forward<Formatter>(formatter));\n}\n\n// Implementation details follow.\n\ntemplate <typename Src, typename Formatter>\nvoid JoinType<Src, Formatter>::WriteTo(Writer& dest) const {\n  using std::begin;\n  using std::end;\n  auto iter = begin(src_);\n  auto end_iter = end(src_);\n  if (iter == end_iter) return;\n  for (;;) {\n    formatter_(*iter, dest);\n    ++iter;\n    if (iter == end_iter) break;\n    dest.Write(separator_);\n  }\n}\n\ntemplate <typename Src, typename Formatter>\nauto JoinType<Src, Formatter>::StringifiedSize() const {\n  if constexpr (std::conjunction_v<IsForwardIterable<Src>,\n                                   FormatterHasStringifiedSize<\n                                       Formatter, ElementTypeT<const Src>>>) {\n    using std::begin;\n    using std::end;\n    auto iter = begin(src_);\n    auto end_iter = end(src_);\n    Position stringified_size = 0;\n    if (iter == end_iter) return stringified_size;\n    for (;;) {\n      stringified_size += formatter_.StringifiedSize(*iter);\n      ++iter;\n      if (iter == end_iter) break;\n      stringified_size += separator_.size();\n    }\n    return stringified_size;\n  }\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_TEXT_JOIN_H_\n"
  },
  {
    "path": "riegeli/text/write_int.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/text/write_int.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#ifdef __SSSE3__\n#include <emmintrin.h>\n#include <tmmintrin.h>\n#endif\n\n#include <cstring>\n#include <limits>  // IWYU pragma: keep\n\n#include \"absl/base/attributes.h\"\n#include \"absl/numeric/bits.h\"  // IWYU pragma: keep\n#include \"absl/numeric/int128.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/endian/endian_writing.h\"\n\nnamespace riegeli::write_int_internal {\n\nnamespace {\n\n#ifdef __SSSE3__\n\ntemplate <DigitCase digit_case>\n__m128i HexDigits();\n\ntemplate <>\ninline __m128i HexDigits<DigitCase::kLower>() {\n  return _mm_setr_epi8('0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a',\n                       'b', 'c', 'd', 'e', 'f');\n}\n\ntemplate <>\ninline __m128i HexDigits<DigitCase::kUpper>() {\n  return _mm_setr_epi8('0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A',\n                       'B', 'C', 'D', 'E', 'F');\n}\n\ntemplate <DigitCase digit_case>\ninline __m128i WriteHex2Impl(uint8_t src) {\n  // Load 8-bit value to 128-bit register.\n  const __m128i value = _mm_cvtsi32_si128(src);\n  // Shift right by 4 bits.\n  const __m128i shifted = _mm_srli_epi16(value, 4);\n  // Interleave low and high nibbles into bytes.\n  const __m128i interleaved = _mm_unpacklo_epi8(shifted, value);\n  // Mask out high nibbles of bytes.\n  const __m128i masked = _mm_and_si128(interleaved, _mm_set1_epi8(0xf));\n  // Convert to characters.\n  return _mm_shuffle_epi8(HexDigits<digit_case>(), masked);\n}\n\ntemplate <DigitCase digit_case>\ninline __m128i WriteHex4Impl(uint16_t src) {\n  // Convert to Big Endian.\n  char encoded[2];\n  riegeli::WriteBigEndian<uint16_t>(src, encoded);\n  // Load 16-bit value to 128-bit register.\n  const __m128i value = _mm_loadu_si16(encoded);\n  // Shift right by 4 bits.\n  const __m128i shifted = _mm_srli_epi16(value, 4);\n  // Interleave low and high nibbles into bytes.\n  const __m128i interleaved = _mm_unpacklo_epi8(shifted, value);\n  // Mask out high nibbles of bytes.\n  const __m128i masked = _mm_and_si128(interleaved, _mm_set1_epi8(0xf));\n  // Convert to characters.\n  return _mm_shuffle_epi8(HexDigits<digit_case>(), masked);\n}\n\ntemplate <DigitCase digit_case>\ninline __m128i WriteHex8Impl(uint32_t src) {\n  // Convert to Big Endian.\n  char encoded[4];\n  riegeli::WriteBigEndian<uint32_t>(src, encoded);\n  // Load 32-bit value to 128-bit register.\n  const __m128i value = _mm_loadu_si32(encoded);\n  // Shift right by 4 bits.\n  const __m128i shifted = _mm_srli_epi32(value, 4);\n  // Interleave low and high nibbles into bytes.\n  const __m128i interleaved = _mm_unpacklo_epi8(shifted, value);\n  // Mask out high nibbles of bytes.\n  const __m128i masked = _mm_and_si128(interleaved, _mm_set1_epi8(0xf));\n  // Convert to characters.\n  return _mm_shuffle_epi8(HexDigits<digit_case>(), masked);\n}\n\ntemplate <DigitCase digit_case>\ninline __m128i WriteHex16Impl(uint64_t src) {\n  // Convert to Big Endian.\n  char encoded[8];\n  riegeli::WriteBigEndian<uint64_t>(src, encoded);\n  // Load 64-bit value to 128-bit register.\n  const __m128i value = _mm_loadu_si64(&encoded);\n  // Shift right by 4 bits.\n  const __m128i shifted = _mm_srli_epi64(value, 4);\n  // Interleave low and high nibbles into bytes.\n  const __m128i interleaved = _mm_unpacklo_epi8(shifted, value);\n  // Mask out high nibbles of bytes.\n  const __m128i masked = _mm_and_si128(interleaved, _mm_set1_epi8(0xf));\n  // Convert to characters.\n  return _mm_shuffle_epi8(HexDigits<digit_case>(), masked);\n}\n\n#endif\n\ntemplate <DigitCase digit_case, typename T>\ninline T DigitCaseDependent(T for_lower, T for_upper) {\n  switch (digit_case) {\n    case DigitCase::kLower:\n      return for_lower;\n    case DigitCase::kUpper:\n      return for_upper;\n  }\n}\n\n// `WriteHex{1,2,4,8,16,32}Impl()` write a fixed number of digits.\n\ntemplate <DigitCase digit_case>\ninline char* WriteHex1Impl(uint8_t src, char* dest) {\n  RIEGELI_ASSERT_LT(src, 0x10)\n      << \"Failed precondition of WriteHex1Impl(): value too large\";\n  *dest = static_cast<char>(src) +\n          (src < 10 ? '0' : DigitCaseDependent<digit_case>('a', 'A') - 10);\n  return dest + 1;\n}\n\ntemplate <DigitCase digit_case>\ninline char* WriteHex2Impl(uint8_t src, char* dest) {\n#ifdef __SSSE3__\n  _mm_storeu_si16(dest, WriteHex2Impl<digit_case>(src));\n#else\n  uint16_t out = src;\n  // Spread out nibbles to bytes (00AB -> 0AXB -> 0A0B).\n  out = (out | (out << 4)) & 0x0f0f;\n  // Convert each byte [0..9] to [6..15], and [10..15] to [16..21].\n  out += 0x0606;\n  // Keep bytes [6..15] unchanged. Convert each byte [16..21] to [55..60]\n  // for `DigitCase::kLower`, or [23..28] for `DigitCase::kUpper`.\n  out += DigitCaseDependent<digit_case>(39, 7) * ((out & 0x1010) >> 4);\n  // Convert each byte [6..15] to ['0'..'9'], and [55..60] to ['a'..'f'] for\n  // `DigitCase::kLower`, or [23..28] to ['A'..'F'] for `DigitCase::kUpper`.\n  out += 0x2a2a;\n  // Write the result, swapping the bytes.\n  riegeli::WriteBigEndian<uint16_t>(out, dest);\n#endif\n  return dest + 2;\n}\n\ntemplate <DigitCase digit_case>\ninline char* WriteHex4Impl(uint16_t src, char* dest) {\n#ifdef __SSSE3__\n  _mm_storeu_si32(dest, WriteHex4Impl<digit_case>(src));\n#else\n  uint32_t out = src;\n  // Spread out nibbles to bytes, swapping the middle ones\n  // (0000ABCD -> 0ABCXBCD -> 0A0C0B0D).\n  out = (out | (out << 12)) & 0x0f0f0f0f;\n  // Convert each byte [0..9] to [6..15], and [10..15] to [16..21].\n  out += 0x06060606;\n  // Keep bytes [6..15] unchanged. Convert each byte [16..21] to [55..60]\n  // for `DigitCase::kLower`, or [23..28] for `DigitCase::kUpper`.\n  out += DigitCaseDependent<digit_case>(39, 7) * ((out & 0x10101010) >> 4);\n  // Convert each byte [6..15] to ['0'..'9'], and [55..60] to ['a'..'f'] for\n  // `DigitCase::kLower`, or [23..28] to ['A'..'F'] for `DigitCase::kUpper`.\n  out += 0x2a2a2a2a;\n  // Swap the first and the last byte.\n  out = (out << 24) | (out >> 24) | (out & 0x00ffff00);\n  // Write the result.\n  riegeli::WriteLittleEndian<uint32_t>(out, dest);\n#endif\n  return dest + 4;\n}\n\ntemplate <DigitCase digit_case>\ninline char* WriteHex8Impl(uint32_t src, char* dest) {\n#ifdef __SSSE3__\n  _mm_storeu_si64(dest, WriteHex8Impl<digit_case>(src));\n  return dest + 8;\n#else\n  dest = WriteHex4Impl<digit_case>(IntCast<uint16_t>(src >> 16), dest);\n  return WriteHex4Impl<digit_case>(static_cast<uint16_t>(src), dest);\n#endif\n}\n\ntemplate <DigitCase digit_case>\ninline char* WriteHex16Impl(uint64_t src, char* dest) {\n#ifdef __SSSE3__\n  _mm_storeu_si128(reinterpret_cast<__m128i*>(dest),\n                   WriteHex16Impl<digit_case>(src));\n  return dest + 16;\n#else\n  dest = WriteHex8Impl<digit_case>(IntCast<uint32_t>(src >> 32), dest);\n  return WriteHex8Impl<digit_case>(static_cast<uint32_t>(src), dest);\n#endif\n}\n\ntemplate <DigitCase digit_case>\ninline char* WriteHex32Impl(absl::uint128 src, char* dest) {\n  dest = WriteHex16Impl<digit_case>(absl::Uint128High64(src), dest);\n  return WriteHex16Impl<digit_case>(absl::Uint128Low64(src), dest);\n}\n\n// `WriteHexImpl()` writes at least `width` digits.\n\n// Inline to optimize for a constant `width`.\ntemplate <DigitCase digit_case>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline char* WriteHexImpl(uint8_t src, char* dest,\n                                                       size_t width) {\n  if (src < uint8_t{1} << 4 && width <= 1) {\n    return WriteHex1Impl<digit_case>(src, dest);\n  }\n  if (width > 2) {\n    // Redundant condition suppresses gcc warning `-Wstringop-overflow`.\n    std::memset(dest, '0', width > 2 ? width - 2 : 0);\n    dest += width - 2;\n  }\n  return WriteHex2Impl<digit_case>(src, dest);\n}\n\n// Inline to optimize for a constant `width`.\ntemplate <DigitCase digit_case>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline char* WriteHexImpl(uint16_t src, char* dest,\n                                                       size_t width) {\n#ifdef __SSSE3__\n  const __m128i out = WriteHex4Impl<digit_case>(src);\n  if (src >= uint32_t{1} << 12 || width >= 4) {\n    if (width > 4) {\n      // Redundant condition suppresses gcc warning `-Wstringop-overflow`.\n      std::memset(dest, '0', width > 4 ? width - 4 : 0);\n      dest += width - 4;\n    }\n    _mm_storeu_si32(dest, out);\n    return dest + 4;\n  }\n  char str[4];\n  _mm_storeu_si32(str, out);\n  width = UnsignedMax(\n      width,\n      (IntCast<size_t>(absl::bit_width(IntCast<uint16_t>(src | 1))) + 3) / 4);\n  std::memcpy(dest, str + 4 - width, width);\n  return dest + width;\n#else\n  if (src <= std::numeric_limits<uint8_t>::max()) {\n    return WriteHexImpl<digit_case>(IntCast<uint8_t>(src), dest, width);\n  }\n  dest = WriteHexImpl<digit_case>(IntCast<uint8_t>(src >> 8), dest,\n                                  SaturatingSub(width, size_t{2}));\n  return WriteHex2Impl<digit_case>(static_cast<uint8_t>(src), dest);\n#endif\n}\n\n// Inline to optimize for a constant `width`.\ntemplate <DigitCase digit_case>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline char* WriteHexImpl(uint32_t src, char* dest,\n                                                       size_t width) {\n#ifdef __SSSE3__\n  const __m128i out = WriteHex8Impl<digit_case>(src);\n  if (src >= uint32_t{1} << 28 || width >= 8) {\n    if (width > 8) {\n      // Redundant condition suppresses gcc warning `-Wstringop-overflow`.\n      std::memset(dest, '0', width > 0 ? width - 8 : 0);\n      dest += width - 8;\n    }\n    _mm_storeu_si64(dest, out);\n    return dest + 8;\n  }\n  char str[8];\n  _mm_storeu_si64(str, out);\n  width =\n      UnsignedMax(width, (IntCast<size_t>(absl::bit_width(src | 1)) + 3) / 4);\n  std::memcpy(dest, str + 8 - width, width);\n  return dest + width;\n#else\n  if (src <= std::numeric_limits<uint16_t>::max()) {\n    return WriteHexImpl<digit_case>(IntCast<uint16_t>(src), dest, width);\n  }\n  dest = WriteHexImpl<digit_case>(IntCast<uint16_t>(src >> 16), dest,\n                                  SaturatingSub(width, size_t{4}));\n  return WriteHex4Impl<digit_case>(static_cast<uint16_t>(src), dest);\n#endif\n}\n\n// Inline to optimize for a constant `width`.\ntemplate <DigitCase digit_case>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline char* WriteHexImpl(uint64_t src, char* dest,\n                                                       size_t width) {\n#ifdef __SSSE3__\n  const __m128i out = WriteHex16Impl<digit_case>(src);\n  if (src >= uint64_t{1} << 60 || width >= 16) {\n    if (width > 16) {\n      // Redundant condition suppresses gcc warning `-Wstringop-overflow`.\n      std::memset(dest, '0', width > 16 ? width - 16 : 0);\n      dest += width - 16;\n    }\n    _mm_storeu_si128(reinterpret_cast<__m128i*>(dest), out);\n    return dest + 16;\n  }\n  alignas(16) char str[16];\n  _mm_store_si128(reinterpret_cast<__m128i*>(str), out);\n  width =\n      UnsignedMax(width, (IntCast<size_t>(absl::bit_width(src | 1)) + 3) / 4);\n  std::memcpy(dest, str + 16 - width, width);\n  return dest + width;\n#else\n  if (src <= std::numeric_limits<uint32_t>::max()) {\n    return WriteHexImpl<digit_case>(IntCast<uint32_t>(src), dest, width);\n  }\n  dest = WriteHexImpl<digit_case>(IntCast<uint32_t>(src >> 32), dest,\n                                  SaturatingSub(width, size_t{8}));\n  return WriteHex8Impl<digit_case>(static_cast<uint32_t>(src), dest);\n#endif\n}\n\n// Inline to optimize for a constant `width`.\ntemplate <DigitCase digit_case>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline char* WriteHexImpl(absl::uint128 src,\n                                                       char* dest,\n                                                       size_t width) {\n  if (absl::Uint128High64(src) == 0) {\n    return WriteHexImpl<digit_case>(absl::Uint128Low64(src), dest, width);\n  }\n  dest = WriteHexImpl<digit_case>(absl::Uint128High64(src), dest,\n                                  SaturatingSub(width, size_t{16}));\n  return WriteHex16Impl<digit_case>(absl::Uint128Low64(src), dest);\n}\n\n}  // namespace\n\ntemplate <>\nchar* WriteHex2<DigitCase::kLower>(uint8_t src, char* dest) {\n  return WriteHex2Impl<DigitCase::kLower>(src, dest);\n}\ntemplate <>\nchar* WriteHex4<DigitCase::kLower>(uint16_t src, char* dest) {\n  return WriteHex4Impl<DigitCase::kLower>(src, dest);\n}\ntemplate <>\nchar* WriteHex8<DigitCase::kLower>(uint32_t src, char* dest) {\n  return WriteHex8Impl<DigitCase::kLower>(src, dest);\n}\ntemplate <>\nchar* WriteHex16<DigitCase::kLower>(uint64_t src, char* dest) {\n  return WriteHex16Impl<DigitCase::kLower>(src, dest);\n}\ntemplate <>\nchar* WriteHex32<DigitCase::kLower>(absl::uint128 src, char* dest) {\n  return WriteHex32Impl<DigitCase::kLower>(src, dest);\n}\n\ntemplate <>\nchar* WriteHex2<DigitCase::kUpper>(uint8_t src, char* dest) {\n  return WriteHex2Impl<DigitCase::kUpper>(src, dest);\n}\ntemplate <>\nchar* WriteHex4<DigitCase::kUpper>(uint16_t src, char* dest) {\n  return WriteHex4Impl<DigitCase::kUpper>(src, dest);\n}\ntemplate <>\nchar* WriteHex8<DigitCase::kUpper>(uint32_t src, char* dest) {\n  return WriteHex8Impl<DigitCase::kUpper>(src, dest);\n}\ntemplate <>\nchar* WriteHex16<DigitCase::kUpper>(uint64_t src, char* dest) {\n  return WriteHex16Impl<DigitCase::kUpper>(src, dest);\n}\ntemplate <>\nchar* WriteHex32<DigitCase::kUpper>(absl::uint128 src, char* dest) {\n  return WriteHex32Impl<DigitCase::kUpper>(src, dest);\n}\n\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint8_t src, char* dest) {\n  return WriteHexImpl<DigitCase::kLower>(src, dest, 0);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint16_t src, char* dest) {\n  return WriteHexImpl<DigitCase::kLower>(src, dest, 0);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint32_t src, char* dest) {\n  return WriteHexImpl<DigitCase::kLower>(src, dest, 0);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint64_t src, char* dest) {\n  return WriteHexImpl<DigitCase::kLower>(src, dest, 0);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(absl::uint128 src, char* dest) {\n  return WriteHexImpl<DigitCase::kLower>(src, dest, 0);\n}\n\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint8_t src, char* dest) {\n  return WriteHexImpl<DigitCase::kUpper>(src, dest, 0);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint16_t src, char* dest) {\n  return WriteHexImpl<DigitCase::kUpper>(src, dest, 0);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint32_t src, char* dest) {\n  return WriteHexImpl<DigitCase::kUpper>(src, dest, 0);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint64_t src, char* dest) {\n  return WriteHexImpl<DigitCase::kUpper>(src, dest, 0);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(absl::uint128 src, char* dest) {\n  return WriteHexImpl<DigitCase::kUpper>(src, dest, 0);\n}\n\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint8_t src, char* dest, size_t width) {\n  return WriteHexImpl<DigitCase::kLower>(src, dest, width);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint16_t src, char* dest, size_t width) {\n  return WriteHexImpl<DigitCase::kLower>(src, dest, width);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint32_t src, char* dest, size_t width) {\n  return WriteHexImpl<DigitCase::kLower>(src, dest, width);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint64_t src, char* dest, size_t width) {\n  return WriteHexImpl<DigitCase::kLower>(src, dest, width);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(absl::uint128 src, char* dest, size_t width) {\n  return WriteHexImpl<DigitCase::kLower>(src, dest, width);\n}\n\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint8_t src, char* dest, size_t width) {\n  return WriteHexImpl<DigitCase::kUpper>(src, dest, width);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint16_t src, char* dest, size_t width) {\n  return WriteHexImpl<DigitCase::kUpper>(src, dest, width);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint32_t src, char* dest, size_t width) {\n  return WriteHexImpl<DigitCase::kUpper>(src, dest, width);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint64_t src, char* dest, size_t width) {\n  return WriteHexImpl<DigitCase::kUpper>(src, dest, width);\n}\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(absl::uint128 src, char* dest, size_t width) {\n  return WriteHexImpl<DigitCase::kUpper>(src, dest, width);\n}\n\ntemplate <>\nvoid WriteHexBackward2<DigitCase::kLower>(uint8_t src, char* dest) {\n  WriteHex2Impl<DigitCase::kLower>(src, dest - 2);\n}\ntemplate <>\nvoid WriteHexBackward4<DigitCase::kLower>(uint16_t src, char* dest) {\n  WriteHex4Impl<DigitCase::kLower>(src, dest - 4);\n}\ntemplate <>\nvoid WriteHexBackward8<DigitCase::kLower>(uint32_t src, char* dest) {\n  WriteHex8Impl<DigitCase::kLower>(src, dest - 8);\n}\ntemplate <>\nvoid WriteHexBackward16<DigitCase::kLower>(uint64_t src, char* dest) {\n  WriteHex16Impl<DigitCase::kLower>(src, dest - 16);\n}\ntemplate <>\nvoid WriteHexBackward32<DigitCase::kLower>(absl::uint128 src, char* dest) {\n  WriteHex32Impl<DigitCase::kLower>(src, dest - 32);\n}\n\ntemplate <>\nvoid WriteHexBackward2<DigitCase::kUpper>(uint8_t src, char* dest) {\n  WriteHex2Impl<DigitCase::kUpper>(src, dest - 2);\n}\ntemplate <>\nvoid WriteHexBackward4<DigitCase::kUpper>(uint16_t src, char* dest) {\n  WriteHex4Impl<DigitCase::kUpper>(src, dest - 4);\n}\ntemplate <>\nvoid WriteHexBackward8<DigitCase::kUpper>(uint32_t src, char* dest) {\n  WriteHex8Impl<DigitCase::kUpper>(src, dest - 8);\n}\ntemplate <>\nvoid WriteHexBackward16<DigitCase::kUpper>(uint64_t src, char* dest) {\n  WriteHex16Impl<DigitCase::kUpper>(src, dest - 16);\n}\ntemplate <>\nvoid WriteHexBackward32<DigitCase::kUpper>(absl::uint128 src, char* dest) {\n  WriteHex32Impl<DigitCase::kUpper>(src, dest - 32);\n}\n\n}  // namespace riegeli::write_int_internal\n"
  },
  {
    "path": "riegeli/text/write_int.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_TEXT_WRITE_INT_H_\n#define RIEGELI_TEXT_WRITE_INT_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <ostream>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/bits.h\"\n#include \"absl/numeric/int128.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/ostream_writer.h\"\n#include \"riegeli/bytes/write_int_internal.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\n// The type returned by `riegeli::Dec()`.\ntemplate <typename T>\nclass DecType {\n public:\n  explicit DecType(T value, size_t width)\n      : value_(std::move(value)), width_(width) {}\n\n  const T& value() const { return value_; }\n  size_t width() const { return width_; }\n\n  template <typename Sink, typename DependentT = T,\n            std::enable_if_t<IsInt<DependentT>::value, int> = 0>\n  friend void AbslStringify(Sink& dest, const DecType& src) {\n    src.Stringify(dest);\n  }\n\n  template <typename DependentT = T,\n            std::enable_if_t<IsInt<DependentT>::value, int> = 0>\n  friend std::ostream& operator<<(std::ostream& dest, const DecType& src) {\n    OStreamWriter writer(&dest);\n    src.WriteTo(writer);\n    writer.Close();\n    return dest;\n  }\n\n  template <typename DependentT = T,\n            std::enable_if_t<IsInt<DependentT>::value, int> = 0>\n  friend Position RiegeliStringifiedSize(const DecType& src) {\n    return src.StringifiedSize();\n  }\n\n private:\n  template <typename Sink, typename DependentT = T,\n            std::enable_if_t<IsUnsignedInt<DependentT>::value, int> = 0>\n  void Stringify(Sink& dest) const;\n  template <typename Sink, typename DependentT = T,\n            std::enable_if_t<IsSignedInt<DependentT>::value, int> = 0>\n  void Stringify(Sink& dest) const;\n\n  // Faster implementation if `Sink` is `WriterStringifySink`.\n  void Stringify(WriterStringifySink& dest) const { WriteTo(*dest.dest()); }\n\n  template <typename DependentT = T,\n            std::enable_if_t<IsUnsignedInt<DependentT>::value, int> = 0>\n  void WriteTo(Writer& dest) const;\n  template <typename DependentT = T,\n            std::enable_if_t<IsSignedInt<DependentT>::value, int> = 0>\n  void WriteTo(Writer& dest) const;\n\n  template <typename DependentT = T,\n            std::enable_if_t<IsUnsignedInt<DependentT>::value, int> = 0>\n  Position StringifiedSize() const;\n  template <typename DependentT = T,\n            std::enable_if_t<IsSignedInt<DependentT>::value, int> = 0>\n  Position StringifiedSize() const;\n\n  T value_;\n  size_t width_;\n};\n\n// Specialization of `DecType` for `char` which is written as unsigned.\ntemplate <>\nclass DecType<char> {\n public:\n  explicit DecType(char value, size_t width) : value_(value), width_(width) {}\n\n  const char& value() const { return value_; }\n  size_t width() const { return width_; }\n\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const DecType& src) {\n    AbslStringify(\n        dest, DecType<unsigned char>(static_cast<unsigned char>(src.value()),\n                                     src.width()));\n  }\n\n  friend std::ostream& operator<<(std::ostream& dest, const DecType& src) {\n    return dest << DecType<unsigned char>(\n               static_cast<unsigned char>(src.value()), src.width());\n  }\n\n  friend Position RiegeliStringifiedSize(const DecType& src) {\n    return RiegeliStringifiedSize(DecType<unsigned char>(\n        static_cast<unsigned char>(src.value()), src.width()));\n  }\n\n private:\n  char value_;\n  size_t width_;\n};\n\n// Wraps an integer such that its stringified representation is padded with\n// zeros to at least the given width.\n//\n// For negative numbers the width includes the minus sign.\n//\n// `char` is stringified as unsigned.\n//\n// If no minimum width is needed, integers can be written to `riegeli::Writer`\n// and `riegeli::BackwardWriter` directly, without wrapping, except that `bool`,\n// `wchar_t`, `char16_t`, nor `char32_t` cannot be written directly, and for\n// `char` and `char8_t` written directly the character itself is written, not\n// its decimal representation.\ntemplate <typename T>\ninline DecType<T> Dec(T value, size_t width = 0) {\n  return DecType<T>(value, width);\n}\n\nenum class DigitCase {\n  kLower,  // ['0'..'9'], ['a'..'f']\n  kUpper,  // ['0'..'9'], ['A'..'F']\n};\n\n// The type returned by `riegeli::Hex()`.\ntemplate <typename T, DigitCase digit_case = DigitCase::kLower>\nclass HexType {\n public:\n  explicit HexType(T value, size_t width)\n      : value_(std::move(value)), width_(width) {}\n\n  const T& value() const { return value_; }\n  size_t width() const { return width_; }\n\n  template <typename Sink, typename DependentT = T,\n            std::enable_if_t<IsInt<DependentT>::value, int> = 0>\n  friend void AbslStringify(Sink& dest, const HexType& src) {\n    src.Stringify(dest);\n  }\n\n  template <typename DependentT = T,\n            std::enable_if_t<IsInt<DependentT>::value, int> = 0>\n  friend std::ostream& operator<<(std::ostream& dest, const HexType& src) {\n    OStreamWriter writer(&dest);\n    src.WriteTo(writer);\n    writer.Close();\n    return dest;\n  }\n\n  template <typename DependentT = T,\n            std::enable_if_t<IsInt<DependentT>::value, int> = 0>\n  friend Position RiegeliStringifiedSize(const HexType& src) {\n    return src.StringifiedSize();\n  }\n\n private:\n  template <typename Sink, typename DependentT = T,\n            std::enable_if_t<IsUnsignedInt<DependentT>::value, int> = 0>\n  void Stringify(Sink& dest) const;\n  template <typename Sink, typename DependentT = T,\n            std::enable_if_t<IsSignedInt<DependentT>::value, int> = 0>\n  void Stringify(Sink& dest) const;\n\n  // Faster implementation if `Sink` is `WriterStringifySink`.\n  void Stringify(WriterStringifySink& dest) const { WriteTo(*dest.dest()); }\n\n  template <typename DependentT = T,\n            std::enable_if_t<IsUnsignedInt<DependentT>::value, int> = 0>\n  void WriteTo(Writer& dest) const;\n  template <typename DependentT = T,\n            std::enable_if_t<IsSignedInt<DependentT>::value, int> = 0>\n  void WriteTo(Writer& dest) const;\n\n  template <typename DependentT = T,\n            std::enable_if_t<IsUnsignedInt<DependentT>::value, int> = 0>\n  Position StringifiedSize() const;\n  template <typename DependentT = T,\n            std::enable_if_t<IsSignedInt<DependentT>::value, int> = 0>\n  Position StringifiedSize() const;\n\n  T value_;\n  size_t width_;\n};\n\n// Specialization of `HexType` for `char` which is written as unsigned.\ntemplate <DigitCase digit_case>\nclass HexType<char, digit_case> {\n public:\n  explicit HexType(char value, size_t width) : value_(value), width_(width) {}\n\n  const char& value() const { return value_; }\n  size_t width() const { return width_; }\n\n  template <typename Sink>\n  friend void AbslStringify(Sink& dest, const HexType& src) {\n    AbslStringify(dest,\n                  HexType<unsigned char, digit_case>(\n                      static_cast<unsigned char>(src.value()), src.width()));\n  }\n\n  friend std::ostream& operator<<(std::ostream& dest, const HexType& src) {\n    return dest << HexType<unsigned char, digit_case>(\n               static_cast<unsigned char>(src.value()), src.width());\n  }\n\n  friend Position RiegeliStringifiedSize(const HexType& src) {\n    return RiegeliStringifiedSize(HexType<unsigned char, digit_case>(\n        static_cast<unsigned char>(src.value()), src.width()));\n  }\n\n private:\n  char value_;\n  size_t width_;\n};\n\n// Wraps an integer such that its stringified representation is hexadecimal,\n// with lower case digits, padded with zeros to at least the given width.\n//\n// For negative numbers the width includes the minus sign.\n//\n// `char` is written as unsigned.\ntemplate <typename T>\ninline HexType<T> Hex(T value, size_t width = 0) {\n  return HexType<T>(value, width);\n}\n\n// Wraps an integer such that its stringified representation is hexadecimal,\n// with upper case digits, padded with zeros to at least the given width.\n//\n// For negative numbers the width includes the minus sign.\n//\n// `char` is written as unsigned.\ntemplate <typename T>\ninline HexType<T, DigitCase::kUpper> HexUpperCase(T value, size_t width = 0) {\n  return HexType<T, DigitCase::kUpper>(value, width);\n}\n\n// Implementation details follow.\n\nnamespace write_int_internal {\n\ntemplate <typename T, std::enable_if_t<FitsIn<T, uint8_t>::value, int> = 0>\ninline size_t HexStringifiedSizeUnsigned(T src) {\n  return IntCast<uint8_t>(src) < 0x10 ? size_t{1} : size_t{2};\n}\n\ntemplate <typename T,\n          std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, uint8_t>>,\n                                              FitsIn<T, uint16_t>>,\n                           int> = 0>\ninline size_t HexStringifiedSizeUnsigned(T src) {\n  return (IntCast<size_t>(absl::bit_width(IntCast<uint16_t>(src | 1))) + 3) / 4;\n}\n\ntemplate <typename T, std::enable_if_t<\n                          std::conjunction_v<std::negation<FitsIn<T, uint16_t>>,\n                                             FitsIn<T, uint32_t>>,\n                          int> = 0>\ninline size_t HexStringifiedSizeUnsigned(T src) {\n  return (IntCast<size_t>(absl::bit_width(IntCast<uint32_t>(src | 1))) + 3) / 4;\n}\n\ntemplate <typename T, std::enable_if_t<\n                          std::conjunction_v<std::negation<FitsIn<T, uint32_t>>,\n                                             FitsIn<T, uint64_t>>,\n                          int> = 0>\ninline size_t HexStringifiedSizeUnsigned(T src) {\n  return (IntCast<size_t>(absl::bit_width(IntCast<uint64_t>(src | 1))) + 3) / 4;\n}\n\ntemplate <typename T, std::enable_if_t<\n                          std::conjunction_v<std::negation<FitsIn<T, uint64_t>>,\n                                             FitsIn<T, absl::uint128>>,\n                          int> = 0>\ninline size_t HexStringifiedSizeUnsigned(T src) {\n  return (IntCast<size_t>(absl::Uint128High64(src) == 0\n                              ? absl::bit_width(absl::Uint128Low64(src) | 1)\n                              : absl::bit_width(absl::Uint128High64(src)) +\n                                    64) +\n          3) /\n         4;\n}\n\ntemplate <typename T>\ninline size_t HexStringifiedSizeSigned(T src) {\n  if (src >= 0) {\n    return HexStringifiedSizeUnsigned(UnsignedCast(src));\n  } else {\n    return HexStringifiedSizeUnsigned(NegatingUnsignedCast(src)) + 1;\n  }\n}\n\n// `WriteHex{2,4,8,16,32}()` writes a fixed number of digits.\ntemplate <DigitCase digit_case>\nchar* WriteHex2(uint8_t src, char* dest);\ntemplate <DigitCase digit_case>\nchar* WriteHex4(uint16_t src, char* dest);\ntemplate <DigitCase digit_case>\nchar* WriteHex8(uint32_t src, char* dest);\ntemplate <DigitCase digit_case>\nchar* WriteHex16(uint64_t src, char* dest);\ntemplate <DigitCase digit_case>\nchar* WriteHex32(absl::uint128 src, char* dest);\n\ntemplate <>\nchar* WriteHex2<DigitCase::kLower>(uint8_t src, char* dest);\ntemplate <>\nchar* WriteHex4<DigitCase::kLower>(uint16_t src, char* dest);\ntemplate <>\nchar* WriteHex8<DigitCase::kLower>(uint32_t src, char* dest);\ntemplate <>\nchar* WriteHex16<DigitCase::kLower>(uint64_t src, char* dest);\ntemplate <>\nchar* WriteHex32<DigitCase::kLower>(absl::uint128 src, char* dest);\n\ntemplate <>\nchar* WriteHex2<DigitCase::kUpper>(uint8_t src, char* dest);\ntemplate <>\nchar* WriteHex4<DigitCase::kUpper>(uint16_t src, char* dest);\ntemplate <>\nchar* WriteHex8<DigitCase::kUpper>(uint32_t src, char* dest);\ntemplate <>\nchar* WriteHex16<DigitCase::kUpper>(uint64_t src, char* dest);\ntemplate <>\nchar* WriteHex32<DigitCase::kUpper>(absl::uint128 src, char* dest);\n\n// `WriteHex()` with no width parameter writes no leading zeros, except for 0\n// itself.\ntemplate <DigitCase digit_case>\nchar* WriteHex(uint8_t src, char* dest);\ntemplate <DigitCase digit_case>\nchar* WriteHex(uint16_t src, char* dest);\ntemplate <DigitCase digit_case>\nchar* WriteHex(uint32_t src, char* dest);\ntemplate <DigitCase digit_case>\nchar* WriteHex(uint64_t src, char* dest);\ntemplate <DigitCase digit_case>\nchar* WriteHex(absl::uint128 src, char* dest);\n\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint8_t src, char* dest);\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint16_t src, char* dest);\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint32_t src, char* dest);\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint64_t src, char* dest);\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(absl::uint128 src, char* dest);\n\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint8_t src, char* dest);\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint16_t src, char* dest);\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint32_t src, char* dest);\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint64_t src, char* dest);\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(absl::uint128 src, char* dest);\n\n// `WriteHex()` with a width parameter writes at least `width` digits.\ntemplate <DigitCase digit_case>\nchar* WriteHex(uint8_t src, char* dest, size_t width);\ntemplate <DigitCase digit_case>\nchar* WriteHex(uint16_t src, char* dest, size_t width);\ntemplate <DigitCase digit_case>\nchar* WriteHex(uint32_t src, char* dest, size_t width);\ntemplate <DigitCase digit_case>\nchar* WriteHex(uint64_t src, char* dest, size_t width);\ntemplate <DigitCase digit_case>\nchar* WriteHex(absl::uint128 src, char* dest, size_t width);\n\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint8_t src, char* dest, size_t width);\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint16_t src, char* dest, size_t width);\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint32_t src, char* dest, size_t width);\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(uint64_t src, char* dest, size_t width);\ntemplate <>\nchar* WriteHex<DigitCase::kLower>(absl::uint128 src, char* dest, size_t width);\n\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint8_t src, char* dest, size_t width);\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint16_t src, char* dest, size_t width);\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint32_t src, char* dest, size_t width);\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(uint64_t src, char* dest, size_t width);\ntemplate <>\nchar* WriteHex<DigitCase::kUpper>(absl::uint128 src, char* dest, size_t width);\n\n// `WriteHexUnsigned()` writes at least `width` digits.\n\ntemplate <DigitCase digit_case, typename T,\n          std::enable_if_t<FitsIn<T, uint8_t>::value, int> = 0>\ninline char* WriteHexUnsigned(T src, char* dest, size_t width) {\n  return width == 2 ? WriteHex2<digit_case>(IntCast<uint8_t>(src), dest)\n         : width <= 1\n             ? WriteHex<digit_case>(IntCast<uint8_t>(src), dest)\n             : WriteHex<digit_case>(IntCast<uint8_t>(src), dest, width);\n}\n\ntemplate <DigitCase digit_case, typename T,\n          std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, uint8_t>>,\n                                              FitsIn<T, uint16_t>>,\n                           int> = 0>\ninline char* WriteHexUnsigned(T src, char* dest, size_t width) {\n  return width == 4 ? WriteHex4<digit_case>(IntCast<uint16_t>(src), dest)\n         : width <= 1\n             ? WriteHex<digit_case>(IntCast<uint16_t>(src), dest)\n             : WriteHex<digit_case>(IntCast<uint16_t>(src), dest, width);\n}\n\ntemplate <\n    DigitCase digit_case, typename T,\n    std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, uint16_t>>,\n                                        FitsIn<T, uint32_t>>,\n                     int> = 0>\ninline char* WriteHexUnsigned(T src, char* dest, size_t width) {\n  return width == 8 ? WriteHex8<digit_case>(IntCast<uint32_t>(src), dest)\n         : width <= 1\n             ? WriteHex<digit_case>(IntCast<uint32_t>(src), dest)\n             : WriteHex<digit_case>(IntCast<uint32_t>(src), dest, width);\n}\n\ntemplate <\n    DigitCase digit_case, typename T,\n    std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, uint32_t>>,\n                                        FitsIn<T, uint64_t>>,\n                     int> = 0>\ninline char* WriteHexUnsigned(T src, char* dest, size_t width) {\n  return width == 16 ? WriteHex16<digit_case>(IntCast<uint64_t>(src), dest)\n         : width <= 1\n             ? WriteHex<digit_case>(IntCast<uint64_t>(src), dest)\n             : WriteHex<digit_case>(IntCast<uint64_t>(src), dest, width);\n}\n\ntemplate <\n    DigitCase digit_case, typename T,\n    std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, uint64_t>>,\n                                        FitsIn<T, absl::uint128>>,\n                     int> = 0>\ninline char* WriteHexUnsigned(T src, char* dest, size_t width) {\n  return width == 32 ? WriteHex32<digit_case>(IntCast<absl::uint128>(src), dest)\n         : width <= 1\n             ? WriteHex<digit_case>(IntCast<absl::uint128>(src), dest)\n             : WriteHex<digit_case>(IntCast<absl::uint128>(src), dest, width);\n}\n\n// `WriteHexBackward{2,4,8,16,32}()` writes a fixed number of digits.\ntemplate <DigitCase digit_case>\nvoid WriteHexBackward2(uint8_t src, char* dest);\ntemplate <DigitCase digit_case>\nvoid WriteHexBackward4(uint16_t src, char* dest);\ntemplate <DigitCase digit_case>\nvoid WriteHexBackward8(uint32_t src, char* dest);\ntemplate <DigitCase digit_case>\nvoid WriteHexBackward16(uint64_t src, char* dest);\ntemplate <DigitCase digit_case>\nvoid WriteHexBackward32(absl::uint128 src, char* dest);\n\ntemplate <>\nvoid WriteHexBackward2<DigitCase::kLower>(uint8_t src, char* dest);\ntemplate <>\nvoid WriteHexBackward4<DigitCase::kLower>(uint16_t src, char* dest);\ntemplate <>\nvoid WriteHexBackward8<DigitCase::kLower>(uint32_t src, char* dest);\ntemplate <>\nvoid WriteHexBackward16<DigitCase::kLower>(uint64_t src, char* dest);\ntemplate <>\nvoid WriteHexBackward32<DigitCase::kLower>(absl::uint128 src, char* dest);\n\ntemplate <>\nvoid WriteHexBackward2<DigitCase::kUpper>(uint8_t src, char* dest);\ntemplate <>\nvoid WriteHexBackward4<DigitCase::kUpper>(uint16_t src, char* dest);\ntemplate <>\nvoid WriteHexBackward8<DigitCase::kUpper>(uint32_t src, char* dest);\ntemplate <>\nvoid WriteHexBackward16<DigitCase::kUpper>(uint64_t src, char* dest);\ntemplate <>\nvoid WriteHexBackward32<DigitCase::kUpper>(absl::uint128 src, char* dest);\n\ntemplate <typename T>\nconstexpr size_t MaxLengthWriteHexUnsignedBackward() {\n  return FitsIn<T, uint8_t>::value    ? 2\n         : FitsIn<T, uint16_t>::value ? 4\n         : FitsIn<T, uint32_t>::value ? 8\n         : FitsIn<T, uint64_t>::value ? 16\n                                      : 32;\n}\n\n// `WriteHexUnsignedBackward<T>()` writes at least `width` digits.\n//\n// `width` must be at most `MaxLengthWriteHexUnsignedBackward<T>()`, and that\n// much space must be available before `dest`.\n\ntemplate <DigitCase digit_case, typename T,\n          std::enable_if_t<FitsIn<T, uint8_t>::value, int> = 0>\ninline char* WriteHexUnsignedBackward(T src, char* dest, size_t width) {\n  RIEGELI_ASSERT_LE(width, 2u)\n      << \"Failed precondition of WriteHexUnsignedBackward(): width too large\";\n  WriteHexBackward2<digit_case>(IntCast<uint8_t>(src), dest);\n  return dest - UnsignedMax(width, HexStringifiedSizeUnsigned(src));\n}\n\ntemplate <DigitCase digit_case, typename T,\n          std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, uint8_t>>,\n                                              FitsIn<T, uint16_t>>,\n                           int> = 0>\ninline char* WriteHexUnsignedBackward(T src, char* dest, size_t width) {\n  RIEGELI_ASSERT_LE(width, 4u)\n      << \"Failed precondition of WriteHexUnsignedBackward(): width too large\";\n  WriteHexBackward4<digit_case>(IntCast<uint16_t>(src), dest);\n  return dest - UnsignedMax(width, HexStringifiedSizeUnsigned(src));\n}\n\ntemplate <\n    DigitCase digit_case, typename T,\n    std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, uint16_t>>,\n                                        FitsIn<T, uint32_t>>,\n                     int> = 0>\ninline char* WriteHexUnsignedBackward(T src, char* dest, size_t width) {\n  RIEGELI_ASSERT_LE(width, 8u)\n      << \"Failed precondition of WriteHexUnsignedBackward(): width too large\";\n  WriteHexBackward8<digit_case>(IntCast<uint32_t>(src), dest);\n  return dest - UnsignedMax(width, HexStringifiedSizeUnsigned(src));\n}\n\ntemplate <\n    DigitCase digit_case, typename T,\n    std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, uint32_t>>,\n                                        FitsIn<T, uint64_t>>,\n                     int> = 0>\ninline char* WriteHexUnsignedBackward(T src, char* dest, size_t width) {\n  RIEGELI_ASSERT_LE(width, 16u)\n      << \"Failed precondition of WriteHexUnsignedBackward(): width too large\";\n  WriteHexBackward16<digit_case>(IntCast<uint64_t>(src), dest);\n  return dest - UnsignedMax(width, HexStringifiedSizeUnsigned(src));\n}\n\ntemplate <\n    DigitCase digit_case, typename T,\n    std::enable_if_t<std::conjunction_v<std::negation<FitsIn<T, uint64_t>>,\n                                        FitsIn<T, absl::uint128>>,\n                     int> = 0>\ninline char* WriteHexUnsignedBackward(T src, char* dest, size_t width) {\n  RIEGELI_ASSERT_LE(width, 32u)\n      << \"Failed precondition of WriteHexUnsignedBackward(): width too large\";\n  WriteHexBackward32<digit_case>(IntCast<absl::uint128>(src), dest);\n  return dest - UnsignedMax(width, HexStringifiedSizeUnsigned(src));\n}\n\n}  // namespace write_int_internal\n\ntemplate <typename T>\ntemplate <typename Sink, typename DependentT,\n          std::enable_if_t<IsUnsignedInt<DependentT>::value, int>>\ninline void DecType<T>::Stringify(Sink& dest) const {\n  // `digits10` is rounded down, `kMaxNumDigits` is rounded up, hence `+ 1`.\n  constexpr size_t kMaxNumDigits = std::numeric_limits<T>::digits10 + 1;\n  char str[kMaxNumDigits];\n  char* const begin =\n      write_int_internal::WriteDecUnsignedBackward(value_, str + kMaxNumDigits);\n  const size_t length = PtrDistance(begin, str + kMaxNumDigits);\n  if (width_ > length) dest.Append(width_ - length, '0');\n  dest.Append(absl::string_view(begin, length));\n}\n\ntemplate <typename T>\ntemplate <typename Sink, typename DependentT,\n          std::enable_if_t<IsSignedInt<DependentT>::value, int>>\ninline void DecType<T>::Stringify(Sink& dest) const {\n  // `digits10` is rounded down, `kMaxNumDigits` is rounded up, hence `+ 1`.\n  constexpr size_t kMaxNumDigits = std::numeric_limits<T>::digits10 + 1;\n  // `+ 1` for the minus sign.\n  char str[kMaxNumDigits + 1];\n  char* begin;\n  size_t length;\n  if (value_ >= 0) {\n    begin = write_int_internal::WriteDecUnsignedBackward(\n        UnsignedCast(value_), str + (kMaxNumDigits + 1));\n    length = PtrDistance(begin, str + (kMaxNumDigits + 1));\n    if (width_ > length) dest.Append(width_ - length, '0');\n  } else {\n    // Leave space for the minus sign.\n    begin = write_int_internal::WriteDecUnsignedBackward(\n                NegatingUnsignedCast(value_), str + (kMaxNumDigits + 1)) -\n            1;\n    length = PtrDistance(begin, str + (kMaxNumDigits + 1));\n    if (width_ > length) {\n      dest.Append(\"-\");\n      dest.Append(width_ - length, '0');\n      ++begin;\n      --length;\n    } else {\n      *begin = '-';\n    }\n  }\n  dest.Append(absl::string_view(begin, length));\n}\n\ntemplate <typename T>\ntemplate <typename DependentT,\n          std::enable_if_t<IsUnsignedInt<DependentT>::value, int>>\ninline void DecType<T>::WriteTo(Writer& dest) const {\n  // `digits10` is rounded down, `kMaxNumDigits` is rounded up, hence `+ 1`.\n  constexpr size_t kMaxNumDigits = std::numeric_limits<T>::digits10 + 1;\n  if (ABSL_PREDICT_FALSE(!dest.Push(UnsignedMax(width_, kMaxNumDigits)))) {\n    return;\n  }\n  dest.set_cursor(\n      write_int_internal::WriteDecUnsigned(value_, dest.cursor(), width_));\n}\n\ntemplate <typename T>\ntemplate <typename DependentT,\n          std::enable_if_t<IsSignedInt<DependentT>::value, int>>\ninline void DecType<T>::WriteTo(Writer& dest) const {\n  // `digits10` is rounded down, `kMaxNumDigits` is rounded up, hence `+ 1`.\n  constexpr size_t kMaxNumDigits = std::numeric_limits<T>::digits10 + 1;\n  // `+ 1` for the minus sign.\n  if (ABSL_PREDICT_FALSE(!dest.Push(UnsignedMax(width_, kMaxNumDigits + 1)))) {\n    return;\n  }\n  dest.set_cursor(\n      write_int_internal::WriteDecSigned(value_, dest.cursor(), width_));\n}\n\ntemplate <typename T>\ntemplate <typename DependentT,\n          std::enable_if_t<IsUnsignedInt<DependentT>::value, int>>\ninline Position DecType<T>::StringifiedSize() const {\n  return UnsignedMax(width_,\n                     write_int_internal::StringifiedSizeUnsigned(value_));\n}\n\ntemplate <typename T>\ntemplate <typename DependentT,\n          std::enable_if_t<IsSignedInt<DependentT>::value, int>>\ninline Position DecType<T>::StringifiedSize() const {\n  return UnsignedMax(width_, write_int_internal::StringifiedSizeSigned(value_));\n}\n\ntemplate <typename T, DigitCase digit_case>\ntemplate <typename Sink, typename DependentT,\n          std::enable_if_t<IsUnsignedInt<DependentT>::value, int>>\ninline void HexType<T, digit_case>::Stringify(Sink& dest) const {\n  constexpr size_t kMaxNumDigits =\n      write_int_internal::MaxLengthWriteHexUnsignedBackward<T>();\n  size_t width = width_;\n  if (width > kMaxNumDigits) {\n    dest.Append(width - kMaxNumDigits, '0');\n    width = kMaxNumDigits;\n  }\n  char str[kMaxNumDigits];\n  char* const begin = write_int_internal::WriteHexUnsignedBackward<digit_case>(\n      value_, str + kMaxNumDigits, width);\n  dest.Append(\n      absl::string_view(begin, PtrDistance(begin, str + kMaxNumDigits)));\n}\n\ntemplate <typename T, DigitCase digit_case>\ntemplate <typename Sink, typename DependentT,\n          std::enable_if_t<IsSignedInt<DependentT>::value, int>>\ninline void HexType<T, digit_case>::Stringify(Sink& dest) const {\n  constexpr size_t kMaxNumDigits =\n      write_int_internal::MaxLengthWriteHexUnsignedBackward<MakeUnsignedT<T>>();\n  size_t width = width_;\n  // `+ 1` for the minus sign.\n  char str[kMaxNumDigits + 1];\n  char* begin;\n  if (value_ >= 0) {\n    if (width > kMaxNumDigits) {\n      dest.Append(width - kMaxNumDigits, '0');\n      width = kMaxNumDigits;\n    }\n    begin = write_int_internal::WriteHexUnsignedBackward<digit_case>(\n        UnsignedCast(value_), str + (kMaxNumDigits + 1), width);\n  } else if (width > kMaxNumDigits + 1) {\n    dest.Append(\"-\");\n    dest.Append(width - (kMaxNumDigits + 1), '0');\n    begin = write_int_internal::WriteHexUnsignedBackward<digit_case>(\n        NegatingUnsignedCast(value_), str + (kMaxNumDigits + 1), kMaxNumDigits);\n  } else {\n    begin = write_int_internal::WriteHexUnsignedBackward<digit_case>(\n        NegatingUnsignedCast(value_), str + (kMaxNumDigits + 1),\n        SaturatingSub(width, size_t{1}));\n    --begin;\n    *begin = '-';\n  }\n  dest.Append(\n      absl::string_view(begin, PtrDistance(begin, str + (kMaxNumDigits + 1))));\n}\n\ntemplate <typename T, DigitCase digit_case>\ntemplate <typename DependentT,\n          std::enable_if_t<IsUnsignedInt<DependentT>::value, int>>\ninline void HexType<T, digit_case>::WriteTo(Writer& dest) const {\n  constexpr size_t kMaxNumDigits = (std::numeric_limits<T>::digits + 3) / 4;\n  if (ABSL_PREDICT_FALSE(!dest.Push(UnsignedMax(width_, kMaxNumDigits)))) {\n    return;\n  }\n  dest.set_cursor(write_int_internal::WriteHexUnsigned<digit_case>(\n      value_, dest.cursor(), width_));\n}\n\ntemplate <typename T, DigitCase digit_case>\ntemplate <typename DependentT,\n          std::enable_if_t<IsSignedInt<DependentT>::value, int>>\ninline void HexType<T, digit_case>::WriteTo(Writer& dest) const {\n  constexpr size_t kMaxNumDigits = (std::numeric_limits<T>::digits + 3) / 4;\n  // `+ 1` for the minus sign.\n  if (ABSL_PREDICT_FALSE(!dest.Push(UnsignedMax(width_, kMaxNumDigits + 1)))) {\n    return;\n  }\n  MakeUnsignedT<T> abs_value;\n  char* cursor = dest.cursor();\n  size_t width = width_;\n  if (value_ >= 0) {\n    abs_value = UnsignedCast(value_);\n  } else {\n    *cursor = '-';\n    ++cursor;\n    abs_value = NegatingUnsignedCast(value_);\n    width = SaturatingSub(width, size_t{1});\n  }\n  dest.set_cursor(write_int_internal::WriteHexUnsigned<digit_case>(\n      abs_value, cursor, width));\n}\n\ntemplate <typename T, DigitCase digit_case>\ntemplate <typename DependentT,\n          std::enable_if_t<IsUnsignedInt<DependentT>::value, int>>\ninline Position HexType<T, digit_case>::StringifiedSize() const {\n  constexpr size_t kMaxNumDigits = (std::numeric_limits<T>::digits + 3) / 4;\n  if (width_ >= kMaxNumDigits) return width_;\n  return UnsignedMax(width_,\n                     write_int_internal::HexStringifiedSizeUnsigned(value_));\n}\n\ntemplate <typename T, DigitCase digit_case>\ntemplate <typename DependentT,\n          std::enable_if_t<IsSignedInt<DependentT>::value, int>>\ninline Position HexType<T, digit_case>::StringifiedSize() const {\n  return UnsignedMax(width_,\n                     write_int_internal::HexStringifiedSizeSigned(value_));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_TEXT_WRITE_INT_H_\n"
  },
  {
    "path": "riegeli/varint/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"varint_reading\",\n    srcs = [\n        \"varint_internal.h\",\n        \"varint_reading.cc\",\n    ],\n    hdrs = [\"varint_reading.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/bytes:reader\",\n        \"@com_google_absl//absl/base:config\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:bits\",\n        \"@com_google_absl//absl/strings:cord\",\n        \"@com_google_absl//absl/strings:string_view\",\n    ],\n)\n\ncc_library(\n    name = \"varint_writing\",\n    srcs = [\"varint_internal.h\"],\n    hdrs = [\"varint_writing.h\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:constexpr\",\n        \"//riegeli/bytes:backward_writer\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/numeric:bits\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/varint/varint_internal.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_VARINT_VARINT_INTERNAL_H_\n#define RIEGELI_VARINT_VARINT_INTERNAL_H_\n\n// IWYU pragma: private, include \"riegeli/varint/varint_reading.h\"\n// IWYU pragma: private, include \"riegeli/varint/varint_writing.h\"\n\n#include <stddef.h>\n\nnamespace riegeli {\n\ninline constexpr size_t kMaxLengthVarint32 = 5;\ninline constexpr size_t kMaxLengthVarint64 = 10;\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_VARINT_VARINT_INTERNAL_H_\n"
  },
  {
    "path": "riegeli/varint/varint_reading.cc",
    "content": "// Copyright 2019 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/varint/varint_reading.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <cstring>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/config.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/bits.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\nnamespace {\n\ntemplate <typename T>\nsize_t kMaxLengthVarint;\n\ntemplate <>\nconstexpr size_t kMaxLengthVarint<uint32_t> = kMaxLengthVarint32;\ntemplate <>\nconstexpr size_t kMaxLengthVarint<uint64_t> = kMaxLengthVarint64;\n\ntemplate <typename T, size_t initial_index, size_t length>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline T ReadVarintValue(const char* src, T acc) {\n  if constexpr (initial_index < length) {\n    const T byte = T{static_cast<uint8_t>(src[initial_index])};\n    acc += (byte - 1) << (initial_index * 7);\n    return ReadVarintValue<T, initial_index + 1, length>(src, acc);\n  } else {\n    return acc;\n  }\n}\n\ntemplate <typename T, bool canonical, size_t initial_index, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool ReadVarintFromReaderBufferLoop(\n    Reader& src, const char* cursor, T acc, T& dest) {\n  const T byte = T{static_cast<uint8_t>(cursor[index])};\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= T{1} << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return false;\n    }\n  } else if (byte >= 0x80) {\n    return ReadVarintFromReaderBufferLoop<T, canonical, initial_index,\n                                          index + 1>(src, cursor, acc, dest);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return false;\n  }\n  acc = ReadVarintValue<T, initial_index, index + 1>(cursor, acc);\n  src.move_cursor(index + 1);\n  dest = acc;\n  return true;\n}\n\ntemplate <typename T, bool canonical, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool ReadVarintFromReaderLoop(Reader& src,\n                                                                  T acc,\n                                                                  T& dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(index + 1, kMaxLengthVarint<T>))) {\n    return false;\n  }\n  const T byte = T{static_cast<uint8_t>(src.cursor()[index])};\n  acc += (byte - 1) << (index * 7);\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= T{1} << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return false;\n    }\n  } else if (byte >= 0x80) {\n    return ReadVarintFromReaderLoop<T, canonical, index + 1>(src, acc, dest);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return false;\n  }\n  src.move_cursor(index + 1);\n  dest = acc;\n  return true;\n}\n\ntemplate <typename T, bool canonical, size_t initial_index, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool ReadVarintFromCordBufferLoop(\n    absl::Cord::CharIterator& src, const char* cursor, T acc, T& dest) {\n  const T byte = T{static_cast<uint8_t>(cursor[index])};\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= T{1} << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return false;\n    }\n  } else if (byte >= 0x80) {\n    return ReadVarintFromCordBufferLoop<T, canonical, initial_index, index + 1>(\n        src, cursor, acc, dest);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return false;\n  }\n  acc = ReadVarintValue<T, initial_index, index + 1>(cursor, acc);\n  absl::Cord::Advance(&src, index + 1);\n  dest = acc;\n  return true;\n}\n\ntemplate <typename T, bool canonical, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool ReadVarintFromCordLoop(\n    absl::Cord::CharIterator& src, size_t available, T acc, T& dest) {\n  RIEGELI_ASSERT_GT(available, index)\n      << \"Failed precondition of ReadVarintFromCordLoop(): not enough data\";\n  const T byte = T{static_cast<uint8_t>(*src)};\n  acc += (byte - 1) << (index * 7);\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= T{1} << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return false;\n    }\n  } else if (byte >= 0x80) {\n    if (ABSL_PREDICT_FALSE(available == index + 1)) return false;\n    ++src;\n    return ReadVarintFromCordLoop<T, canonical, index + 1>(src, available, acc,\n                                                           dest);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return false;\n  }\n  ++src;\n  dest = acc;\n  return true;\n}\n\ntemplate <typename T, bool canonical, size_t initial_index, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline size_t ReadVarintFromArrayLoop(\n    const char* src, size_t available, T acc, T& dest) {\n  if (ABSL_PREDICT_FALSE(available == index)) return 0;\n  const T byte = T{static_cast<uint8_t>(src[index])};\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= T{1} << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return 0;\n    }\n  } else if (byte >= 0x80) {\n    return ReadVarintFromArrayLoop<T, canonical, initial_index, index + 1>(\n        src, available, acc, dest);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return false;\n  }\n  dest = ReadVarintValue<T, initial_index, index + 1>(src, acc);\n  return index + 1;\n}\n\ntemplate <size_t initial_index, size_t length>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline void CopyVarintValue(const char* src,\n                                                         char* dest) {\n  std::memcpy(dest + initial_index, src + initial_index,\n              length - initial_index);\n}\n\ntemplate <typename T, bool canonical, size_t initial_index, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline size_t CopyVarintFromReaderBufferLoop(\n    Reader& src, const char* cursor, char* dest) {\n  const uint8_t byte = static_cast<uint8_t>(cursor[index]);\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= T{1} << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return 0;\n    }\n  } else if (byte >= 0x80) {\n    return CopyVarintFromReaderBufferLoop<T, canonical, initial_index,\n                                          index + 1>(src, cursor, dest);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return 0;\n  }\n  CopyVarintValue<initial_index, index + 1>(cursor, dest);\n  src.move_cursor(index + 1);\n  return index + 1;\n}\n\ntemplate <typename T, bool canonical, size_t initial_index, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline size_t CopyVarintFromReaderLoop(\n    Reader& src, char* dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(index + 1, kMaxLengthVarint<T>))) return 0;\n  const uint8_t byte = static_cast<uint8_t>(src.cursor()[index]);\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= 1u << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return 0;\n    }\n  } else if (byte >= 0x80) {\n    return CopyVarintFromReaderLoop<T, canonical, initial_index, index + 1>(\n        src, dest);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return 0;\n  }\n  CopyVarintValue<initial_index, index + 1>(src.cursor(), dest);\n  src.move_cursor(index + 1);\n  return index + 1;\n}\n\ntemplate <typename T, bool canonical, size_t initial_index, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline size_t CopyVarintFromCordBufferLoop(\n    absl::Cord::CharIterator& src, const char* cursor, char* dest) {\n  const uint8_t byte = static_cast<uint8_t>(cursor[index]);\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= 1u << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return 0;\n    }\n  } else if (byte >= 0x80) {\n    return CopyVarintFromCordBufferLoop<T, canonical, initial_index, index + 1>(\n        src, cursor, dest);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return 0;\n  }\n  CopyVarintValue<initial_index, index + 1>(cursor, dest);\n  absl::Cord::Advance(&src, index + 1);\n  return index + 1;\n}\n\ntemplate <typename T, bool canonical, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline size_t CopyVarintFromCordLoop(\n    absl::Cord::CharIterator& src, size_t available, char* dest) {\n  RIEGELI_ASSERT_GT(available, index)\n      << \"Failed precondition of CopyVarintFromCordLoop(): not enough data\";\n  const uint8_t byte = static_cast<uint8_t>(*src);\n  dest[index] = static_cast<char>(byte);\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= 1u << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return 0;\n    }\n  } else if (byte >= 0x80) {\n    if (ABSL_PREDICT_FALSE(available == index + 1)) return 0;\n    ++src;\n    return CopyVarintFromCordLoop<T, canonical, index + 1>(src, available,\n                                                           dest);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return 0;\n  }\n  ++src;\n  return index + 1;\n}\n\ntemplate <typename T, bool canonical, size_t initial_index, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline size_t CopyVarintFromArrayLoop(\n    const char* src, size_t available, char* dest) {\n  if (ABSL_PREDICT_FALSE(available == index)) return 0;\n  const uint8_t byte = static_cast<uint8_t>(src[index]);\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= 1u << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return 0;\n    }\n  } else if (byte >= 0x80) {\n    return CopyVarintFromArrayLoop<T, canonical, initial_index, index + 1>(\n        src, available, dest);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return 0;\n  }\n  CopyVarintValue<initial_index, index + 1>(src, dest);\n  return index + 1;\n}\n\ntemplate <typename T, bool canonical, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool SkipVarintFromReaderBufferLoop(\n    Reader& src, const char* cursor) {\n  const uint8_t byte = static_cast<uint8_t>(cursor[index]);\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= T{1} << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return false;\n    }\n  } else if (byte >= 0x80) {\n    return SkipVarintFromReaderBufferLoop<T, canonical, index + 1>(src, cursor);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return false;\n  }\n  src.move_cursor(index + 1);\n  return true;\n}\n\ntemplate <typename T, bool canonical, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool SkipVarintFromReaderLoop(Reader& src) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(index + 1, kMaxLengthVarint<T>))) {\n    return false;\n  }\n  const uint8_t byte = static_cast<uint8_t>(src.cursor()[index]);\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= 1u << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return false;\n    }\n  } else if (byte >= 0x80) {\n    return SkipVarintFromReaderLoop<T, canonical, index + 1>(src);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return false;\n  }\n  src.move_cursor(index + 1);\n  return true;\n}\n\ntemplate <typename T, bool canonical, size_t initial_index, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool SkipVarintFromCordBufferLoop(\n    absl::Cord::CharIterator& src, const char* cursor) {\n  const T byte = T{static_cast<uint8_t>(cursor[index])};\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= T{1} << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return false;\n    }\n  } else if (byte >= 0x80) {\n    return SkipVarintFromCordBufferLoop<T, canonical, initial_index, index + 1>(\n        src, cursor);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return false;\n  }\n  absl::Cord::Advance(&src, index + 1);\n  return true;\n}\n\ntemplate <typename T, bool canonical, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline bool SkipVarintFromCordLoop(\n    absl::Cord::CharIterator& src, size_t available) {\n  RIEGELI_ASSERT_GT(available, index)\n      << \"Failed precondition of SkipVarintFromCordLoop(): not enough data\";\n  const T byte = T{static_cast<uint8_t>(*src)};\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= T{1} << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return false;\n    }\n  } else if (byte >= 0x80) {\n    if (ABSL_PREDICT_FALSE(available == index + 1)) return false;\n    ++src;\n    return SkipVarintFromCordLoop<T, canonical, index + 1>(src, available);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return false;\n  }\n  ++src;\n  return true;\n}\n\ntemplate <typename T, bool canonical, size_t index>\nABSL_ATTRIBUTE_ALWAYS_INLINE inline size_t SkipVarintFromArrayLoop(\n    const char* src, size_t available) {\n  if (ABSL_PREDICT_FALSE(available == index)) return 0;\n  const uint8_t byte = static_cast<uint8_t>(src[index]);\n  if constexpr (index == kMaxLengthVarint<T> - 1) {\n    // Last possible byte.\n    if (ABSL_PREDICT_FALSE(byte >= 1u << (sizeof(T) * 8 - index * 7))) {\n      // The representation is longer than `kMaxLengthVarint<T>`\n      // or the represented value does not fit in `T`.\n      return 0;\n    }\n  } else if (byte >= 0x80) {\n    return SkipVarintFromArrayLoop<T, canonical, index + 1>(src, available);\n  }\n  if constexpr (canonical) {\n    if (ABSL_PREDICT_FALSE(byte == 0)) return 0;\n  }\n  return index + 1;\n}\n\ninline uint64_t ReadNativeEndian(const char* src) {\n  uint64_t dest;\n  std::memcpy(&dest, src, sizeof(dest));\n  return dest;\n}\n\n}  // namespace\n\nnamespace varint_internal {\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool ReadVarintFromReaderBuffer(Reader& src, const char* cursor, T acc,\n                                T& dest) {\n  RIEGELI_ASSERT_GE(src.available(), initial_index)\n      << \"Failed precondition of ReadVarintFromReaderBuffer(): \"\n         \"not enough buffered data\";\n  if (ABSL_PREDICT_TRUE(src.available() >= kMaxLengthVarint<T>) ||\n      static_cast<uint8_t>(src.limit()[-1]) < 0x80) {\n    return ReadVarintFromReaderBufferLoop<T, canonical, initial_index,\n                                          initial_index>(src, cursor, acc,\n                                                         dest);\n  }\n  // Do not inline this call to avoid a frame pointer.\n  return ReadVarintFromReader<T, canonical, initial_index>(src, acc, dest);\n}\n\ntemplate bool ReadVarintFromReaderBuffer<uint32_t, false, 2>(Reader& src,\n                                                             const char* cursor,\n                                                             uint32_t acc,\n                                                             uint32_t& dest);\ntemplate bool ReadVarintFromReaderBuffer<uint64_t, false, 2>(Reader& src,\n                                                             const char* cursor,\n                                                             uint64_t acc,\n                                                             uint64_t& dest);\ntemplate bool ReadVarintFromReaderBuffer<uint32_t, true, 2>(Reader& src,\n                                                            const char* cursor,\n                                                            uint32_t acc,\n                                                            uint32_t& dest);\ntemplate bool ReadVarintFromReaderBuffer<uint64_t, true, 2>(Reader& src,\n                                                            const char* cursor,\n                                                            uint64_t acc,\n                                                            uint64_t& dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool ReadVarintFromReader(Reader& src, T acc, T& dest) {\n  RIEGELI_ASSERT_GE(src.available(), initial_index)\n      << \"Failed precondition of ReadVarintFromReader(): \"\n         \"not enough buffered data\";\n  return ReadVarintFromReaderLoop<T, canonical, initial_index>(src, acc, dest);\n}\n\ntemplate bool ReadVarintFromReader<uint32_t, false, 1>(Reader& src,\n                                                       uint32_t acc,\n                                                       uint32_t& dest);\ntemplate bool ReadVarintFromReader<uint64_t, false, 1>(Reader& src,\n                                                       uint64_t acc,\n                                                       uint64_t& dest);\ntemplate bool ReadVarintFromReader<uint32_t, true, 1>(Reader& src, uint32_t acc,\n                                                      uint32_t& dest);\ntemplate bool ReadVarintFromReader<uint64_t, true, 1>(Reader& src, uint64_t acc,\n                                                      uint64_t& dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool ReadVarintFromCordBuffer(absl::Cord::CharIterator& src, size_t available,\n                              T acc, T& dest) {\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  RIEGELI_ASSERT_GE(chunk.size(), initial_index)\n      << \"Failed precondition of ReadVarintFromCordBuffer(): \"\n         \"not enough buffered data\";\n  const size_t available_in_buffer = UnsignedMin(available, chunk.size());\n  if (ABSL_PREDICT_TRUE(available_in_buffer >= kMaxLengthVarint<T>) ||\n      static_cast<uint8_t>(chunk[available_in_buffer - 1]) < 0x80) {\n    return ReadVarintFromCordBufferLoop<T, canonical, initial_index,\n                                        initial_index>(src, chunk.data(), acc,\n                                                       dest);\n  }\n  if (ABSL_PREDICT_FALSE(available == initial_index)) return false;\n  // Do not inline this call to avoid a frame pointer.\n  return ReadVarintFromCord<T, canonical, initial_index>(src, available, acc,\n                                                         dest);\n}\n\ntemplate bool ReadVarintFromCordBuffer<uint32_t, false, 2>(\n    absl::Cord::CharIterator& src, size_t available, uint32_t acc,\n    uint32_t& dest);\ntemplate bool ReadVarintFromCordBuffer<uint64_t, false, 2>(\n    absl::Cord::CharIterator& src, size_t available, uint64_t acc,\n    uint64_t& dest);\ntemplate bool ReadVarintFromCordBuffer<uint32_t, true, 2>(\n    absl::Cord::CharIterator& src, size_t available, uint32_t acc,\n    uint32_t& dest);\ntemplate bool ReadVarintFromCordBuffer<uint64_t, true, 2>(\n    absl::Cord::CharIterator& src, size_t available, uint64_t acc,\n    uint64_t& dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool ReadVarintFromCord(absl::Cord::CharIterator& src, size_t available, T acc,\n                        T& dest) {\n  RIEGELI_ASSERT_GT(available, initial_index)\n      << \"Failed precondition of ReadVarintFromCord(): not enough data\";\n  absl::Cord::Advance(&src, initial_index);\n  return ReadVarintFromCordLoop<T, canonical, initial_index>(src, available,\n                                                             acc, dest);\n}\n\ntemplate bool ReadVarintFromCord<uint32_t, false, 1>(\n    absl::Cord::CharIterator& src, size_t available, uint32_t acc,\n    uint32_t& dest);\ntemplate bool ReadVarintFromCord<uint64_t, false, 1>(\n    absl::Cord::CharIterator& src, size_t available, uint64_t acc,\n    uint64_t& dest);\ntemplate bool ReadVarintFromCord<uint32_t, true, 1>(\n    absl::Cord::CharIterator& src, size_t available, uint32_t acc,\n    uint32_t& dest);\ntemplate bool ReadVarintFromCord<uint64_t, true, 1>(\n    absl::Cord::CharIterator& src, size_t available, uint64_t acc,\n    uint64_t& dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t ReadVarintFromArray(const char* src, size_t available, T acc, T& dest) {\n  RIEGELI_ASSERT_GE(available, initial_index)\n      << \"Failed precondition of ReadVarintFromArray(): not enough data\";\n  return ReadVarintFromArrayLoop<T, canonical, initial_index, initial_index>(\n      src, available, acc, dest);\n}\n\ntemplate size_t ReadVarintFromArray<uint32_t, false, 2>(const char* src,\n                                                        size_t available,\n                                                        uint32_t acc,\n                                                        uint32_t& dest);\ntemplate size_t ReadVarintFromArray<uint64_t, false, 2>(const char* src,\n                                                        size_t available,\n                                                        uint64_t acc,\n                                                        uint64_t& dest);\ntemplate size_t ReadVarintFromArray<uint32_t, true, 2>(const char* src,\n                                                       size_t available,\n                                                       uint32_t acc,\n                                                       uint32_t& dest);\ntemplate size_t ReadVarintFromArray<uint64_t, true, 2>(const char* src,\n                                                       size_t available,\n                                                       uint64_t acc,\n                                                       uint64_t& dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t CopyVarintFromReaderBuffer(Reader& src, const char* cursor, char* dest) {\n  RIEGELI_ASSERT_GE(src.available(), initial_index)\n      << \"Failed precondition of CopyVarintFromReaderBuffer(): \"\n         \"not enough buffered data\";\n  if (ABSL_PREDICT_TRUE(src.available() >= kMaxLengthVarint<T>) ||\n      static_cast<uint8_t>(src.limit()[-1]) < 0x80) {\n    return CopyVarintFromReaderBufferLoop<T, canonical, initial_index,\n                                          initial_index>(src, cursor, dest);\n  }\n  // Do not inline this call to avoid a frame pointer.\n  return CopyVarintFromReader<T, canonical, initial_index>(src, dest);\n}\n\ntemplate size_t CopyVarintFromReaderBuffer<uint32_t, false, 2>(\n    Reader& src, const char* cursor, char* dest);\ntemplate size_t CopyVarintFromReaderBuffer<uint64_t, false, 2>(\n    Reader& src, const char* cursor, char* dest);\ntemplate size_t CopyVarintFromReaderBuffer<uint32_t, true, 2>(\n    Reader& src, const char* cursor, char* dest);\ntemplate size_t CopyVarintFromReaderBuffer<uint64_t, true, 2>(\n    Reader& src, const char* cursor, char* dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t CopyVarintFromReader(Reader& src, char* dest) {\n  RIEGELI_ASSERT_GE(src.available(), initial_index)\n      << \"Failed precondition of CopyVarintFromReader(): \"\n         \"not enough buffered data\";\n  return CopyVarintFromReaderLoop<T, canonical, initial_index, initial_index>(\n      src, dest);\n}\n\ntemplate size_t CopyVarintFromReader<uint32_t, false, 1>(Reader& src,\n                                                         char* dest);\ntemplate size_t CopyVarintFromReader<uint64_t, false, 1>(Reader& src,\n                                                         char* dest);\ntemplate size_t CopyVarintFromReader<uint32_t, true, 1>(Reader& src,\n                                                        char* dest);\ntemplate size_t CopyVarintFromReader<uint64_t, true, 1>(Reader& src,\n                                                        char* dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t CopyVarintFromCordBuffer(absl::Cord::CharIterator& src, size_t available,\n                                char* dest) {\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  RIEGELI_ASSERT_GE(chunk.size(), initial_index)\n      << \"Failed precondition of CopyVarintFromCordBuffer(): \"\n         \"not enough buffered data\";\n  const size_t available_in_buffer = UnsignedMin(available, chunk.size());\n  if (ABSL_PREDICT_TRUE(available_in_buffer >= kMaxLengthVarint<T>) ||\n      static_cast<uint8_t>(chunk[available_in_buffer - 1]) < 0x80) {\n    return CopyVarintFromCordBufferLoop<T, canonical, initial_index,\n                                        initial_index>(src, chunk.data(), dest);\n  }\n  if (ABSL_PREDICT_FALSE(available == initial_index)) return 0;\n  // Do not inline this call to avoid a frame pointer.\n  return CopyVarintFromCord<T, canonical, initial_index>(src, available, dest);\n}\n\ntemplate size_t CopyVarintFromCordBuffer<uint32_t, false, 2>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\ntemplate size_t CopyVarintFromCordBuffer<uint64_t, false, 2>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\ntemplate size_t CopyVarintFromCordBuffer<uint32_t, true, 2>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\ntemplate size_t CopyVarintFromCordBuffer<uint64_t, true, 2>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t CopyVarintFromCord(absl::Cord::CharIterator& src, size_t available,\n                          char* dest) {\n  RIEGELI_ASSERT_GT(available, initial_index)\n      << \"Failed precondition of CopyVarintFromCord(): not enough data\";\n  absl::Cord::Advance(&src, initial_index);\n  return CopyVarintFromCordLoop<T, canonical, initial_index>(src, available,\n                                                             dest);\n}\n\ntemplate size_t CopyVarintFromCord<uint32_t, false, 1>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\ntemplate size_t CopyVarintFromCord<uint64_t, false, 1>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\ntemplate size_t CopyVarintFromCord<uint32_t, true, 1>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\ntemplate size_t CopyVarintFromCord<uint64_t, true, 1>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t CopyVarintFromArray(const char* src, size_t available, char* dest) {\n  RIEGELI_ASSERT_GE(available, initial_index)\n      << \"Failed precondition of CopyVarintFromArray(): not enough data\";\n  return CopyVarintFromArrayLoop<T, canonical, initial_index, initial_index>(\n      src, available, dest);\n}\n\ntemplate size_t CopyVarintFromArray<uint32_t, false, 2>(const char* src,\n                                                        size_t available,\n                                                        char* dest);\ntemplate size_t CopyVarintFromArray<uint64_t, false, 2>(const char* src,\n                                                        size_t available,\n                                                        char* dest);\ntemplate size_t CopyVarintFromArray<uint32_t, true, 2>(const char* src,\n                                                       size_t available,\n                                                       char* dest);\ntemplate size_t CopyVarintFromArray<uint64_t, true, 2>(const char* src,\n                                                       size_t available,\n                                                       char* dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool SkipVarintFromReaderBuffer(Reader& src, const char* cursor) {\n  RIEGELI_ASSERT_GE(src.available(), initial_index)\n      << \"Failed precondition of SkipVarintFromReaderBuffer(): \"\n         \"not enough buffered data\";\n  if (ABSL_PREDICT_TRUE(src.available() >= kMaxLengthVarint<T>) ||\n      static_cast<uint8_t>(src.limit()[-1]) < 0x80) {\n    return SkipVarintFromReaderBufferLoop<T, canonical, initial_index>(src,\n                                                                       cursor);\n  }\n  // Do not inline this call to avoid a frame pointer.\n  return SkipVarintFromReader<T, canonical, initial_index>(src);\n}\n\ntemplate bool SkipVarintFromReaderBuffer<uint32_t, false, 2>(\n    Reader& src, const char* cursor);\ntemplate bool SkipVarintFromReaderBuffer<uint64_t, false, 2>(\n    Reader& src, const char* cursor);\ntemplate bool SkipVarintFromReaderBuffer<uint32_t, true, 2>(Reader& src,\n                                                            const char* cursor);\ntemplate bool SkipVarintFromReaderBuffer<uint64_t, true, 2>(Reader& src,\n                                                            const char* cursor);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool SkipVarintFromReader(Reader& src) {\n  RIEGELI_ASSERT_GE(src.available(), initial_index)\n      << \"Failed precondition of SkipVarintFromReader(): \"\n         \"not enough buffered data\";\n  return SkipVarintFromReaderLoop<T, canonical, initial_index>(src);\n}\n\ntemplate bool SkipVarintFromReader<uint32_t, false, 1>(Reader& src);\ntemplate bool SkipVarintFromReader<uint64_t, false, 1>(Reader& src);\ntemplate bool SkipVarintFromReader<uint32_t, true, 1>(Reader& src);\ntemplate bool SkipVarintFromReader<uint64_t, true, 1>(Reader& src);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool SkipVarintFromCordBuffer(absl::Cord::CharIterator& src, size_t available) {\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  RIEGELI_ASSERT_GE(chunk.size(), initial_index)\n      << \"Failed precondition of SkipVarintFromCordBuffer(): \"\n         \"not enough buffered data\";\n  const size_t available_in_buffer = UnsignedMin(available, chunk.size());\n  if (ABSL_PREDICT_TRUE(available_in_buffer >= kMaxLengthVarint<T>) ||\n      static_cast<uint8_t>(chunk[available_in_buffer - 1]) < 0x80) {\n    return SkipVarintFromCordBufferLoop<T, canonical, initial_index,\n                                        initial_index>(src, chunk.data());\n  }\n  if (ABSL_PREDICT_FALSE(available == initial_index)) return false;\n  // Do not inline this call to avoid a frame pointer.\n  return SkipVarintFromCord<T, canonical, initial_index>(src, available);\n}\n\ntemplate bool SkipVarintFromCordBuffer<uint32_t, false, 2>(\n    absl::Cord::CharIterator& src, size_t available);\ntemplate bool SkipVarintFromCordBuffer<uint64_t, false, 2>(\n    absl::Cord::CharIterator& src, size_t available);\ntemplate bool SkipVarintFromCordBuffer<uint32_t, true, 2>(\n    absl::Cord::CharIterator& src, size_t available);\ntemplate bool SkipVarintFromCordBuffer<uint64_t, true, 2>(\n    absl::Cord::CharIterator& src, size_t available);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool SkipVarintFromCord(absl::Cord::CharIterator& src, size_t available) {\n  RIEGELI_ASSERT_GT(available, initial_index)\n      << \"Failed precondition of SkipVarintFromCord(): not enough data\";\n  absl::Cord::Advance(&src, initial_index);\n  return SkipVarintFromCordLoop<T, canonical, initial_index>(src, available);\n}\n\ntemplate bool SkipVarintFromCord<uint32_t, false, 1>(\n    absl::Cord::CharIterator& src, size_t available);\ntemplate bool SkipVarintFromCord<uint64_t, false, 1>(\n    absl::Cord::CharIterator& src, size_t available);\ntemplate bool SkipVarintFromCord<uint32_t, true, 1>(\n    absl::Cord::CharIterator& src, size_t available);\ntemplate bool SkipVarintFromCord<uint64_t, true, 1>(\n    absl::Cord::CharIterator& src, size_t available);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t SkipVarintFromArray(const char* src, size_t available) {\n  RIEGELI_ASSERT_GE(available, initial_index)\n      << \"Failed precondition of SkipVarintFromArray(): not enough data\";\n  return SkipVarintFromArrayLoop<T, canonical, initial_index>(src, available);\n}\n\ntemplate size_t SkipVarintFromArray<uint32_t, false, 2>(const char* src,\n                                                        size_t available);\ntemplate size_t SkipVarintFromArray<uint64_t, false, 2>(const char* src,\n                                                        size_t available);\ntemplate size_t SkipVarintFromArray<uint32_t, true, 2>(const char* src,\n                                                       size_t available);\ntemplate size_t SkipVarintFromArray<uint64_t, true, 2>(const char* src,\n                                                       size_t available);\n\n}  // namespace varint_internal\n\nsize_t CountVarints(absl::string_view value) {\n  // The number of varints is the number of bytes with the highest bit clear.\n  // This is easier to compute as the total number of bytes, minus the number\n  // of bytes with the highest bit set.\n  size_t num_varints = value.size();\n  if (value.size() < sizeof(uint64_t)) {\n    // Count byte by byte.\n    for (const char byte : value) {\n      num_varints -= static_cast<uint8_t>(byte) >> 7;\n    }\n    return num_varints;\n  }\n\n  // Count in whole blocks, except for the last one.\n  const char* const limit = value.data() + value.size() - sizeof(uint64_t);\n  const char* cursor = value.data();\n  while (cursor < limit) {\n    const uint64_t block = ReadNativeEndian(cursor);\n    num_varints -=\n        IntCast<size_t>(absl::popcount(block & uint64_t{0x8080808080808080}));\n    cursor += 8;\n  }\n\n  // Count in the last, possibly incomplete block.\n  const uint64_t block = ReadNativeEndian(limit);\n  uint64_t mask = uint64_t{0x8080808080808080};\n#if ABSL_IS_LITTLE_ENDIAN\n  mask <<= PtrDistance(limit, cursor) * 8;\n#elif ABSL_IS_BIG_ENDIAN\n  mask >>= PtrDistance(limit, cursor) * 8;\n#else\n#error Unknown endianness\n#endif\n  num_varints -= IntCast<size_t>(absl::popcount(block & mask));\n\n  return num_varints;\n}\n\nbool VerifyBools(absl::string_view value) {\n  uint64_t bit_or = 0;\n  if (value.size() < sizeof(uint64_t)) {\n    // Verify byte by byte.\n    for (const char byte : value) {\n      bit_or |= static_cast<uint8_t>(byte);\n    }\n    return bit_or <= 1;\n  }\n\n  // Verify whole blocks, except for the last one.\n  const char* const limit = value.data() + value.size() - sizeof(uint64_t);\n  const char* cursor = value.data();\n  while (cursor < limit) {\n    bit_or |= ReadNativeEndian(cursor);\n    cursor += 8;\n  }\n  // Verify the last, possibly incomplete block.\n  bit_or |= ReadNativeEndian(limit);\n\n  return (bit_or & ~uint64_t{0x0101010101010101}) == 0;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/varint/varint_reading.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_VARINT_VARINT_READING_H_\n#define RIEGELI_VARINT_VARINT_READING_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <optional>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/strings/cord.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/varint/varint_internal.h\"  // IWYU pragma: export\n\nnamespace riegeli {\n\n// Unless stated otherwise, reading a varint tolerates representations\n// which are not the shortest, but rejects representations longer than\n// `kMaxLengthVarint{32,64}` bytes or with bits set outside the range of\n// possible values.\n\n// Reads a varint. This corresponds to protobuf types `{int,uint}{32,64}`\n// (with a cast needed in the case of `int{32,64}`).\n//\n// Warning: protobuf writes values of type `int32` by casting them to `uint64`,\n// not `uint32` (negative values take 10 bytes, not 5), hence they must be read\n// with `ReadVarint64()`, not `ReadVarint32()`, if negative values are possible.\n//\n// Return values:\n//  * `true`                     - success (`dest` is set)\n//  * `false` (when `src.ok()`)  - source ends too early or varint is invalid\n//                                 (`src` position is unchanged,\n//                                 `dest` is undefined)\n//  * `false` (when `!src.ok()`) - failure\n//                                 (`src` position is unchanged,\n//                                 `dest` is undefined)\nbool ReadVarint32(Reader& src, uint32_t& dest);\nbool ReadVarint64(Reader& src, uint64_t& dest);\n\n// Reads a signed varint (zigzag-encoded). This corresponds to protobuf types\n// `sint{32,64}`.\n//\n// Return values:\n//  * `true`                     - success (`dest` is set)\n//  * `false` (when `src.ok()`)  - source ends too early or varint is invalid\n//                                 (`src` position is unchanged,\n//                                 `dest` is undefined)\n//  * `false` (when `!src.ok()`) - failure\n//                                 (`src` position is unchanged,\n//                                 `dest` is undefined)\nbool ReadVarintSigned32(Reader& src, uint32_t& dest);\nbool ReadVarintSigned64(Reader& src, uint64_t& dest);\n\n// Reads a varint. This corresponds to protobuf types `{int,uint}{32,64}`\n// (with a cast needed in the case of `int{32,64}`).\n//\n// Accepts only the canonical representation, i.e. the shortest: rejecting a\n// trailing zero byte, except for 0 itself.\n//\n// Warning: protobuf writes values of type `int32` by casting them to `uint64`,\n// not `uint32` (negative values take 10 bytes, not 5), hence they must be read\n// with `ReadCanonicalVarint64()`, not `ReadCanonicalVarint32()`, if negative\n// values are possible.\n//\n// Return values:\n//  * `true`                     - success (`dest` is set)\n//  * `false` (when `src.ok()`)  - source ends too early or varint is invalid\n//                                 (`src` position is unchanged,\n//                                 `dest` is undefined)\n//  * `false` (when `!src.ok()`) - failure\n//                                 (`src` position is unchanged,\n//                                 `dest` is undefined)\nbool ReadCanonicalVarint32(Reader& src, uint32_t& dest);\nbool ReadCanonicalVarint64(Reader& src, uint64_t& dest);\n\n// Reads a varint, at most `available` bytes long. This corresponds to protobuf\n// types `{int,uint}{32,64}` (with a cast needed in the case of `int{32,64}`).\n//\n// Warning: protobuf writes values of type `int32` by casting them to `uint64`,\n// not `uint32` (negative values take 10 bytes, not 5), hence they must be read\n// with `ReadVarint64()`, not `ReadVarint32()`, if negative values are possible.\n//\n// Return values:\n//  * `true`  - success (`dest` is set)\n//  * `false` - source ends too early or varint is invalid\n//              (`src` moved less than `available` unless `available == 0`,\n//              `dest` is undefined)\nbool ReadVarint32(absl::Cord::CharIterator& src, size_t available,\n                  uint32_t& dest);\nbool ReadVarint64(absl::Cord::CharIterator& src, size_t available,\n                  uint64_t& dest);\n\n// Reads a signed varint (zigzag-encoded), at most `available` bytes long.\n// This corresponds to protobuf types `sint{32,64}`.\n//\n// Return values:\n//  * `true`  - success (`dest` is set)\n//  * `false` - source ends too early or varint is invalid\n//              (`src` moved less than `available` unless `available == 0`,\n//              `dest` is undefined)\nbool ReadVarintSigned32(absl::Cord::CharIterator& src, size_t available,\n                        uint32_t& dest);\nbool ReadVarintSigned64(absl::Cord::CharIterator& src, size_t available,\n                        uint64_t& dest);\n\n// Reads a varint, at most `available` bytes long. This corresponds to protobuf\n// types `{int,uint}{32,64}` (with a cast needed in the case of `int{32,64}`).\n//\n// Accepts only the canonical representation, i.e. the shortest: rejecting a\n// trailing zero byte, except for 0 itself.\n//\n// Warning: protobuf writes values of type `int32` by casting them to `uint64`,\n// not `uint32` (negative values take 10 bytes, not 5), hence they must be read\n// with `ReadCanonicalVarint64()`, not `ReadCanonicalVarint32()`, if negative\n// values are possible.\n//\n// Return values:\n//  * `true`  - success (`dest` is set)\n//  * `false` - source ends too early or varint is invalid\n//              (`src` moved less than `available` unless `available == 0`,\n//              `dest` is undefined)\nbool ReadCanonicalVarint32(absl::Cord::CharIterator& src, size_t available,\n                           uint32_t& dest);\nbool ReadCanonicalVarint64(absl::Cord::CharIterator& src, size_t available,\n                           uint64_t& dest);\n\n// Reads a varint from an array. This corresponds to protobuf types\n// `{int,uint}{32,64}` (with a cast needed in the case of `int{32,64}`).\n//\n// Warning: protobuf writes values of type `int32` by casting them to `uint64`,\n// not `uint32` (negative values take 10 bytes, not 5), hence they must be read\n// with `ReadVarint64()`, not `ReadVarint32()`, if negative values are possible.\n//\n// Return values:\n//  * positive `length` - success, `length` bytes read (`dest` is set)\n//  * 0                 - source ends too early or varint is invalid\n//                        (`dest` is undefined)\nsize_t ReadVarint32(const char* src, size_t available, uint32_t& dest);\nsize_t ReadVarint64(const char* src, size_t available, uint64_t& dest);\n\n// Reads a signed varint (zigzag-encoded) from an array. This corresponds to\n// protobuf types `sint{32,64}`.\n//\n// Return values:\n//  * positive `length` - success, `length` bytes read (`dest` is set)\n//  * 0                 - source ends too early or varint is invalid\n//                        (`dest` is undefined)\nsize_t ReadVarintSigned32(const char* src, size_t available, int32_t& dest);\nsize_t ReadVarintSigned64(const char* src, size_t available, int64_t& dest);\n\n// Reads a varint from an array. This corresponds to protobuf types\n// `{int,uint}{32,64}` (with a cast needed in the case of `int{32,64}`).\n//\n// Accepts only the canonical representation, i.e. the shortest: rejecting a\n// trailing zero byte, except for 0 itself.\n//\n// Warning: protobuf writes values of type `int32` by casting them to `uint64`,\n// not `uint32` (negative values take 10 bytes, not 5), hence they must be read\n// with `ReadVarint64()`, not `ReadVarint32()`, if negative values are possible.\n//\n// Return values:\n//  * positive `length` - success, `length` bytes read (`dest` is set)\n//  * 0                 - source ends too early or varint is invalid\n//                        (`dest` is undefined)\nsize_t ReadCanonicalVarint32(const char* src, size_t available, uint32_t& dest);\nsize_t ReadCanonicalVarint64(const char* src, size_t available, uint64_t& dest);\n\n// Copies a varint to an array, without decoding and encoding but with\n// validation.\n//\n// Writes up to `kMaxLengthVarint{32,64}` bytes to `dest[]`.\n//\n// Return values:\n//  * positive `length`    - success, `length` bytes copied (`dest[]` is filled)\n//  * 0 (when `src.ok()`)  - source ends too early or varint is invalid\n//                           (`src` position is unchanged,\n//                           `dest[]` is undefined)\n//  * 0 (when `!src.ok()`) - failure\n//                           (`src` position is unchanged,\n//                           `dest[]` is undefined)\nsize_t CopyVarint32(Reader& src, char* dest);\nsize_t CopyVarint64(Reader& src, char* dest);\n\n// Copies a varint to an array, without decoding and encoding but with\n// validation.\n//\n// Accepts only the canonical representation, i.e. the shortest: rejecting a\n// trailing zero byte, except for 0 itself.\n//\n// Writes up to `kMaxLengthVarint{32,64}` bytes to `dest[]`.\n//\n// Return values:\n//  * positive `length`    - success, `length` bytes copied (`dest[]` is filled)\n//  * 0 (when `src.ok()`)  - source ends too early or varint is invalid\n//                           (`src` position is unchanged,\n//                           `dest[]` is undefined)\n//  * 0 (when `!src.ok()`) - failure\n//                           (`src` position is unchanged,\n//                           `dest[]` is undefined)\nsize_t CopyCanonicalVarint32(Reader& src, char* dest);\nsize_t CopyCanonicalVarint64(Reader& src, char* dest);\n\n// Copies a varint to an array, at most `available` bytes long, without decoding\n// and encoding but with validation.\n//\n// Writes up to `kMaxLengthVarint{32,64}` bytes to `dest[]`.\n//\n// Return values:\n//  * positive `length` - success, `length` bytes copied (`dest[]` is filled)\n//  * 0                 - source ends too early or varint is invalid\n//                        (`src` moved less than `available`\n//                        unless `available == 0`,\n//                        `dest[]` is undefined)\nsize_t CopyVarint32(absl::Cord::CharIterator& src, size_t available,\n                    char* dest);\nsize_t CopyVarint64(absl::Cord::CharIterator& src, size_t available,\n                    char* dest);\n\n// Copies a varint to an array, at most `available` bytes long, without decoding\n// and encoding but with validation.\n//\n// Accepts only the canonical representation, i.e. the shortest: rejecting a\n// trailing zero byte, except for 0 itself.\n//\n// Writes up to `kMaxLengthVarint{32,64}` bytes to `dest[]`.\n//\n// Return values:\n//  * positive `length` - success, `length` bytes copied (`dest[]` is filled)\n//  * 0                 - source ends too early or varint is invalid\n//                        (`src` moved less than `available`\n//                        unless `available == 0`,\n//                        `dest[]` is undefined)\nsize_t CopyCanonicalVarint32(absl::Cord::CharIterator& src, size_t available,\n                             char* dest);\nsize_t CopyCanonicalVarint64(absl::Cord::CharIterator& src, size_t available,\n                             char* dest);\n\n// Copies a varint from an array to an array, without decoding and encoding but\n// with validation.\n//\n// Writes up to `kMaxLengthVarint{32,64}` bytes to `dest[]`.\n//\n// Return values:\n//  * positive `length` - success, `length` bytes copied (`dest[]` is filled)\n//  * 0                 - source ends too early or varint is invalid\n//                        (`dest[]` is undefined)\nsize_t CopyVarint32(const char* src, size_t available, char* dest);\nsize_t CopyVarint64(const char* src, size_t available, char* dest);\n\n// Copies a varint from an array to an array, without decoding and encoding but\n// with validation.\n//\n// Accepts only the canonical representation, i.e. the shortest: rejecting a\n// trailing zero byte, except for 0 itself.\n//\n// Writes up to `kMaxLengthVarint{32,64}` bytes to `dest[]`.\n//\n// Return values:\n//  * positive `length` - success, `length` bytes copied (`dest[]` is filled)\n//  * 0                 - source ends too early or varint is invalid\n//                        (`dest[]` is undefined)\nsize_t CopyCanonicalVarint32(const char* src, size_t available, char* dest);\nsize_t CopyCanonicalVarint64(const char* src, size_t available, char* dest);\n\n// Skips a varint, without decoding but with validation.\n//\n// Return values:\n//  * `true`                     - success\n//  * `false` (when `src.ok()`)  - source ends too early or varint is invalid\n//                                 (`src` position is unchanged)\n//  * `false` (when `!src.ok()`) - failure\n//                                 (`src` position is unchanged)\nbool SkipVarint32(Reader& src);\nbool SkipVarint64(Reader& src);\n\n// Skips a varint, without decoding but with validation.\n//\n// Accepts only the canonical representation, i.e. the shortest: rejecting a\n// trailing zero byte, except for 0 itself.\n//\n// Return values:\n//  * `true`                     - success\n//  * `false` (when `src.ok()`)  - source ends too early or varint is invalid\n//                                 (`src` position is unchanged)\n//  * `false` (when `!src.ok()`) - failure\n//                                 (`src` position is unchanged)\nbool SkipCanonicalVarint32(Reader& src);\nbool SkipCanonicalVarint64(Reader& src);\n\n// Skips a varint, at most `available` bytes long, without decoding but with\n// validation.\n//\n// Return values:\n//  * positive `length` - success\n//  * 0                 - source ends too early or varint is invalid\n//                        (`src` moved less than `available`\n//                        unless `available == 0`)\nbool SkipVarint32(absl::Cord::CharIterator& src, size_t available);\nbool SkipVarint64(absl::Cord::CharIterator& src, size_t available);\n\n// Skips a varint, at most `available` bytes long, without decoding but with\n// validation.\n//\n// Accepts only the canonical representation, i.e. the shortest: rejecting a\n// trailing zero byte, except for 0 itself.\n//\n// Return values:\n//  * positive `length` - success\n//  * 0                 - source ends too early or varint is invalid\n//                        (`src` moved less than `available`\n//                        unless `available == 0`)\nbool SkipCanonicalVarint32(absl::Cord::CharIterator& src, size_t available);\nbool SkipCanonicalVarint64(absl::Cord::CharIterator& src, size_t available);\n\n// Skips a varint from an array, without decoding but with validation.\n//\n// Return values:\n//  * positive `length` - success, `length` bytes can be skipped\n//  * 0                 - source ends too early or varint is invalid\nsize_t SkipVarint32(const char* src, size_t available);\nsize_t SkipVarint64(const char* src, size_t available);\n\n// Skips a varint from an array, without decoding but with validation.\n//\n// Accepts only the canonical representation, i.e. the shortest: rejecting a\n// trailing zero byte, except for 0 itself.\n//\n// Return values:\n//  * positive `length` - success, `length` bytes can be skipped\n//  * 0                 - source ends too early or varint is invalid\nsize_t SkipCanonicalVarint32(const char* src, size_t available);\nsize_t SkipCanonicalVarint64(const char* src, size_t available);\n\n// Decodes a signed varint (zigzag-decoding) from an unsigned value read as a\n// plain varint. This corresponds to protobuf types `sint{32,64}`.\nconstexpr int32_t DecodeVarintSigned32(uint32_t repr);\nconstexpr int64_t DecodeVarintSigned64(uint64_t repr);\n\n// Counts the number of varints in `value`.\n//\n// If varints are valid only up to some point, then returns at least the number\n// of valid varints. Returns `value.size()` only if each varint is valid and\n// takes one byte.\nsize_t CountVarints(absl::string_view value);\n\n// Checks if each byte of `value` is a valid representation for a `bool`,\n// i.e. 0 or 1. Optimized for the result being `true`.\nbool VerifyBools(absl::string_view value);\n\n// Implementation details follow.\n\nnamespace varint_internal {\n\ninline size_t Remaining(const absl::Cord::CharIterator& src) {\n  return IntCast<size_t>(absl::Cord::Distance(src, absl::Cord::CharIterator()));\n}\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool ReadVarintFromReaderBuffer(Reader& src, const char* cursor, T acc,\n                                T& dest);\n\nextern template bool ReadVarintFromReaderBuffer<uint32_t, false, 2>(\n    Reader& src, const char* cursor, uint32_t acc, uint32_t& dest);\nextern template bool ReadVarintFromReaderBuffer<uint64_t, false, 2>(\n    Reader& src, const char* cursor, uint64_t acc, uint64_t& dest);\nextern template bool ReadVarintFromReaderBuffer<uint32_t, true, 2>(\n    Reader& src, const char* cursor, uint32_t acc, uint32_t& dest);\nextern template bool ReadVarintFromReaderBuffer<uint64_t, true, 2>(\n    Reader& src, const char* cursor, uint64_t acc, uint64_t& dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool ReadVarintFromReader(Reader& src, T acc, T& dest);\n\nextern template bool ReadVarintFromReader<uint32_t, false, 1>(Reader& src,\n                                                              uint32_t acc,\n                                                              uint32_t& dest);\nextern template bool ReadVarintFromReader<uint64_t, false, 1>(Reader& src,\n                                                              uint64_t acc,\n                                                              uint64_t& dest);\nextern template bool ReadVarintFromReader<uint32_t, true, 1>(Reader& src,\n                                                             uint32_t acc,\n                                                             uint32_t& dest);\nextern template bool ReadVarintFromReader<uint64_t, true, 1>(Reader& src,\n                                                             uint64_t acc,\n                                                             uint64_t& dest);\n\n}  // namespace varint_internal\n\ninline bool ReadVarint32(Reader& src, uint32_t& dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthVarint32))) return false;\n  const uint32_t byte0 = uint32_t{static_cast<uint8_t>(src.cursor()[0])};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    src.move_cursor(1);\n    dest = byte0;\n    return true;\n  }\n  if (ABSL_PREDICT_TRUE(src.available() >= 2)) {\n    const uint32_t byte1 = uint32_t{static_cast<uint8_t>(src.cursor()[1])};\n    const uint32_t acc = byte0 + ((byte1 - 1) << 7);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      src.move_cursor(2);\n      dest = acc;\n      return true;\n    }\n    return varint_internal::ReadVarintFromReaderBuffer<uint32_t,\n                                                       /*canonical=*/false, 2>(\n        src, src.cursor(), acc, dest);\n  }\n  return varint_internal::ReadVarintFromReader<uint32_t,\n                                               /*canonical=*/false, 1>(\n      src, byte0, dest);\n}\n\ninline bool ReadVarint64(Reader& src, uint64_t& dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthVarint64))) return false;\n  const uint64_t byte0 = uint64_t{static_cast<uint8_t>(src.cursor()[0])};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    src.move_cursor(1);\n    dest = byte0;\n    return true;\n  }\n  if (ABSL_PREDICT_TRUE(src.available() >= 2)) {\n    const uint64_t byte1 = uint64_t{static_cast<uint8_t>(src.cursor()[1])};\n    const uint64_t acc = byte0 + ((byte1 - 1) << 7);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      src.move_cursor(2);\n      dest = acc;\n      return true;\n    }\n    return varint_internal::ReadVarintFromReaderBuffer<uint64_t,\n                                                       /*canonical=*/false, 2>(\n        src, src.cursor(), acc, dest);\n  }\n  return varint_internal::ReadVarintFromReader<uint64_t,\n                                               /*canonical=*/false, 1>(\n      src, byte0, dest);\n}\n\ninline bool ReadVarintSigned32(Reader& src, int32_t& dest) {\n  uint32_t unsigned_dest;\n  if (ABSL_PREDICT_FALSE(!ReadVarint32(src, unsigned_dest))) return false;\n  dest = DecodeVarintSigned32(unsigned_dest);\n  return true;\n}\n\ninline bool ReadVarintSigned64(Reader& src, int64_t& dest) {\n  uint64_t unsigned_dest;\n  if (ABSL_PREDICT_FALSE(!ReadVarint64(src, unsigned_dest))) return false;\n  dest = DecodeVarintSigned64(unsigned_dest);\n  return true;\n}\n\ninline bool ReadCanonicalVarint32(Reader& src, uint32_t& dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthVarint32))) return false;\n  const uint32_t byte0 = uint32_t{static_cast<uint8_t>(src.cursor()[0])};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    src.move_cursor(1);\n    dest = byte0;\n    return true;\n  }\n  if (ABSL_PREDICT_TRUE(src.available() >= 2)) {\n    const uint32_t byte1 = uint32_t{static_cast<uint8_t>(src.cursor()[1])};\n    const uint32_t acc = byte0 + ((byte1 - 1) << 7);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      if (ABSL_PREDICT_FALSE(byte1 == 0)) return false;\n      src.move_cursor(2);\n      dest = acc;\n      return true;\n    }\n    return varint_internal::ReadVarintFromReaderBuffer<uint32_t,\n                                                       /*canonical=*/true, 2>(\n        src, src.cursor(), acc, dest);\n  }\n  return varint_internal::ReadVarintFromReader<uint32_t,\n                                               /*canonical=*/true, 1>(\n      src, byte0, dest);\n}\n\ninline bool ReadCanonicalVarint64(Reader& src, uint64_t& dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthVarint64))) return false;\n  const uint64_t byte0 = uint64_t{static_cast<uint8_t>(src.cursor()[0])};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    src.move_cursor(1);\n    dest = byte0;\n    return true;\n  }\n  if (ABSL_PREDICT_TRUE(src.available() >= 2)) {\n    const uint64_t byte1 = uint64_t{static_cast<uint8_t>(src.cursor()[1])};\n    const uint64_t acc = byte0 + ((byte1 - 1) << 7);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      if (ABSL_PREDICT_FALSE(byte1 == 0)) return false;\n      src.move_cursor(2);\n      dest = acc;\n      return true;\n    }\n    return varint_internal::ReadVarintFromReaderBuffer<uint64_t,\n                                                       /*canonical=*/true, 2>(\n        src, src.cursor(), acc, dest);\n  }\n  return varint_internal::ReadVarintFromReader<uint64_t,\n                                               /*canonical=*/true, 1>(\n      src, byte0, dest);\n}\n\nnamespace varint_internal {\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool ReadVarintFromCordBuffer(absl::Cord::CharIterator& src, size_t available,\n                              T acc, T& dest);\n\nextern template bool ReadVarintFromCordBuffer<uint32_t, false, 2>(\n    absl::Cord::CharIterator& src, size_t available, uint32_t acc,\n    uint32_t& dest);\nextern template bool ReadVarintFromCordBuffer<uint64_t, false, 2>(\n    absl::Cord::CharIterator& src, size_t available, uint64_t acc,\n    uint64_t& dest);\nextern template bool ReadVarintFromCordBuffer<uint32_t, true, 2>(\n    absl::Cord::CharIterator& src, size_t available, uint32_t acc,\n    uint32_t& dest);\nextern template bool ReadVarintFromCordBuffer<uint64_t, true, 2>(\n    absl::Cord::CharIterator& src, size_t available, uint64_t acc,\n    uint64_t& dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool ReadVarintFromCord(absl::Cord::CharIterator& src, size_t available, T acc,\n                        T& dest);\n\nextern template bool ReadVarintFromCord<uint32_t, false, 1>(\n    absl::Cord::CharIterator& src, size_t available, uint32_t acc,\n    uint32_t& dest);\nextern template bool ReadVarintFromCord<uint64_t, false, 1>(\n    absl::Cord::CharIterator& src, size_t available, uint64_t acc,\n    uint64_t& dest);\nextern template bool ReadVarintFromCord<uint32_t, true, 1>(\n    absl::Cord::CharIterator& src, size_t available, uint32_t acc,\n    uint32_t& dest);\nextern template bool ReadVarintFromCord<uint64_t, true, 1>(\n    absl::Cord::CharIterator& src, size_t available, uint64_t acc,\n    uint64_t& dest);\n\n}  // namespace varint_internal\n\ninline bool ReadVarint32(absl::Cord::CharIterator& src, size_t available,\n                         uint32_t& dest) {\n  RIEGELI_ASSERT_LE(available, varint_internal::Remaining(src))\n      << \"Failed precondition of ReadVarint32(): not enough remaining data\";\n  if (ABSL_PREDICT_FALSE(available == 0)) return false;\n  const uint32_t byte0 = uint32_t{static_cast<uint8_t>(*src)};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    ++src;\n    dest = byte0;\n    return true;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return false;\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  if (ABSL_PREDICT_TRUE(chunk.size() >= 2)) {\n    const uint32_t byte1 = uint32_t{static_cast<uint8_t>(chunk[1])};\n    const uint32_t acc = byte0 + ((byte1 - 1) << 7);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      absl::Cord::Advance(&src, 2);\n      dest = acc;\n      return true;\n    }\n    return varint_internal::ReadVarintFromCordBuffer<uint32_t,\n                                                     /*canonical=*/false, 2>(\n        src, available, acc, dest);\n  }\n  return varint_internal::ReadVarintFromCord<uint32_t,\n                                             /*canonical=*/false, 1>(\n      src, available, byte0, dest);\n}\n\ninline bool ReadVarint64(absl::Cord::CharIterator& src, size_t available,\n                         uint64_t& dest) {\n  RIEGELI_ASSERT_LE(available, varint_internal::Remaining(src))\n      << \"Failed precondition of ReadVarint64(): not enough remaining data\";\n  if (ABSL_PREDICT_FALSE(available == 0)) return false;\n  const uint64_t byte0 = uint64_t{static_cast<uint8_t>(*src)};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    ++src;\n    dest = byte0;\n    return true;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return false;\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  if (ABSL_PREDICT_TRUE(chunk.size() >= 2)) {\n    const uint64_t byte1 = uint64_t{static_cast<uint8_t>(chunk[1])};\n    const uint64_t acc = byte0 + ((byte1 - 1) << 7);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      absl::Cord::Advance(&src, 2);\n      dest = acc;\n      return true;\n    }\n    return varint_internal::ReadVarintFromCordBuffer<uint64_t,\n                                                     /*canonical=*/false, 2>(\n        src, available, acc, dest);\n  }\n  return varint_internal::ReadVarintFromCord<uint64_t,\n                                             /*canonical=*/false, 1>(\n      src, available, byte0, dest);\n}\n\ninline bool ReadVarintSigned32(absl::Cord::CharIterator& src, size_t available,\n                               int32_t& dest) {\n  uint32_t unsigned_dest;\n  if (ABSL_PREDICT_FALSE(!ReadVarint32(src, available, unsigned_dest))) {\n    return false;\n  }\n  dest = DecodeVarintSigned32(unsigned_dest);\n  return true;\n}\n\ninline bool ReadVarintSigned64(absl::Cord::CharIterator& src, size_t available,\n                               int64_t& dest) {\n  uint64_t unsigned_dest;\n  if (ABSL_PREDICT_FALSE(!ReadVarint64(src, available, unsigned_dest))) {\n    return false;\n  }\n  dest = DecodeVarintSigned64(unsigned_dest);\n  return true;\n}\n\ninline bool ReadCanonicalVarint32(absl::Cord::CharIterator& src,\n                                  size_t available, uint32_t& dest) {\n  RIEGELI_ASSERT_LE(available, varint_internal::Remaining(src))\n      << \"Failed precondition of ReadCanonicalVarint32(): \"\n         \"not enough remaining data\";\n  if (ABSL_PREDICT_FALSE(available == 0)) return false;\n  const uint32_t byte0 = uint32_t{static_cast<uint8_t>(*src)};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    ++src;\n    dest = byte0;\n    return true;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return false;\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  if (ABSL_PREDICT_TRUE(chunk.size() >= 2)) {\n    const uint32_t byte1 = uint32_t{static_cast<uint8_t>(chunk[1])};\n    const uint32_t acc = byte0 + ((byte1 - 1) << 7);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      if (ABSL_PREDICT_FALSE(byte1 == 0)) return false;\n      absl::Cord::Advance(&src, 2);\n      dest = acc;\n      return true;\n    }\n    return varint_internal::ReadVarintFromCordBuffer<uint32_t,\n                                                     /*canonical=*/true, 2>(\n        src, available, acc, dest);\n  }\n  return varint_internal::ReadVarintFromCord<uint32_t,\n                                             /*canonical=*/true, 1>(\n      src, available, byte0, dest);\n}\n\ninline bool ReadCanonicalVarint64(absl::Cord::CharIterator& src,\n                                  size_t available, uint64_t& dest) {\n  RIEGELI_ASSERT_LE(available, varint_internal::Remaining(src))\n      << \"Failed precondition of ReadCanonicalVarint64(): \"\n         \"not enough remaining data\";\n  if (ABSL_PREDICT_FALSE(available == 0)) return false;\n  const uint64_t byte0 = uint64_t{static_cast<uint8_t>(*src)};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    ++src;\n    dest = byte0;\n    return true;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return false;\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  if (ABSL_PREDICT_TRUE(chunk.size() >= 2)) {\n    const uint64_t byte1 = uint64_t{static_cast<uint8_t>(chunk[1])};\n    const uint64_t acc = byte0 + ((byte1 - 1) << 7);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      if (ABSL_PREDICT_FALSE(byte1 == 0)) return false;\n      absl::Cord::Advance(&src, 2);\n      dest = acc;\n      return true;\n    }\n    return varint_internal::ReadVarintFromCordBuffer<uint64_t,\n                                                     /*canonical=*/true, 2>(\n        src, available, acc, dest);\n  }\n  return varint_internal::ReadVarintFromCord<uint64_t,\n                                             /*canonical=*/true, 1>(\n      src, available, byte0, dest);\n}\n\nnamespace varint_internal {\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t ReadVarintFromArray(const char* src, size_t available, T acc, T& dest);\n\nextern template size_t ReadVarintFromArray<uint32_t, false, 2>(const char* src,\n                                                               size_t available,\n                                                               uint32_t acc,\n                                                               uint32_t& dest);\nextern template size_t ReadVarintFromArray<uint64_t, false, 2>(const char* src,\n                                                               size_t available,\n                                                               uint64_t acc,\n                                                               uint64_t& dest);\nextern template size_t ReadVarintFromArray<uint32_t, true, 2>(const char* src,\n                                                              size_t available,\n                                                              uint32_t acc,\n                                                              uint32_t& dest);\nextern template size_t ReadVarintFromArray<uint64_t, true, 2>(const char* src,\n                                                              size_t available,\n                                                              uint64_t acc,\n                                                              uint64_t& dest);\n\n}  // namespace varint_internal\n\ninline size_t ReadVarint32(const char* src, size_t available, uint32_t& dest) {\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint32_t byte0 = uint32_t{static_cast<uint8_t>(src[0])};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    dest = byte0;\n    return 1;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const uint32_t byte1 = uint32_t{static_cast<uint8_t>(src[1])};\n  const uint32_t acc = byte0 + ((byte1 - 1) << 7);\n  if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n    dest = acc;\n    return 2;\n  }\n  return varint_internal::ReadVarintFromArray<uint32_t, /*canonical=*/false, 2>(\n      src, available, acc, dest);\n}\n\ninline size_t ReadVarint64(const char* src, size_t available, uint64_t& dest) {\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint64_t byte0 = uint64_t{static_cast<uint8_t>(src[0])};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    dest = byte0;\n    return 1;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const uint64_t byte1 = uint64_t{static_cast<uint8_t>(src[1])};\n  const uint64_t acc = byte0 + ((byte1 - 1) << 7);\n  if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n    dest = acc;\n    return 2;\n  }\n  return varint_internal::ReadVarintFromArray<uint64_t, /*canonical=*/false, 2>(\n      src, available, acc, dest);\n}\n\ninline size_t ReadVarintSigned32(const char* src, size_t available,\n                                 int32_t& dest) {\n  uint32_t unsigned_dest;\n  const size_t length = ReadVarint32(src, available, unsigned_dest);\n  if (ABSL_PREDICT_FALSE(length == 0)) return 0;\n  dest = DecodeVarintSigned32(unsigned_dest);\n  return length;\n}\n\ninline size_t ReadVarintSigned64(const char* src, size_t available,\n                                 int64_t& dest) {\n  uint64_t unsigned_dest;\n  const size_t length = ReadVarint64(src, available, unsigned_dest);\n  if (ABSL_PREDICT_FALSE(length == 0)) return 0;\n  dest = DecodeVarintSigned64(unsigned_dest);\n  return length;\n}\n\ninline size_t ReadCanonicalVarint32(const char* src, size_t available,\n                                    uint32_t& dest) {\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint32_t byte0 = uint32_t{static_cast<uint8_t>(src[0])};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    dest = byte0;\n    return 1;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const uint32_t byte1 = uint32_t{static_cast<uint8_t>(src[1])};\n  const uint32_t acc = byte0 + ((byte1 - 1) << 7);\n  if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n    if (ABSL_PREDICT_FALSE(byte1 == 0)) return 0;\n    dest = acc;\n    return 2;\n  }\n  return varint_internal::ReadVarintFromArray<uint32_t, /*canonical=*/true, 2>(\n      src, available, acc, dest);\n}\n\ninline size_t ReadCanonicalVarint64(const char* src, size_t available,\n                                    uint64_t& dest) {\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint64_t byte0 = uint64_t{static_cast<uint8_t>(src[0])};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    dest = byte0;\n    return 1;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const uint64_t byte1 = uint64_t{static_cast<uint8_t>(src[1])};\n  const uint64_t acc = byte0 + ((byte1 - 1) << 7);\n  if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n    if (ABSL_PREDICT_FALSE(byte1 == 0)) return 0;\n    dest = acc;\n    return 2;\n  }\n  return varint_internal::ReadVarintFromArray<uint64_t, /*canonical=*/true, 2>(\n      src, available, acc, dest);\n}\n\ninline std::optional<const char*> ReadVarint32(const char* src,\n                                               const char* limit,\n                                               uint32_t& dest) {\n  const size_t length = ReadVarint32(src, PtrDistance(src, limit), dest);\n  if (ABSL_PREDICT_FALSE(length == 0)) return std::nullopt;\n  return src + length;\n}\n\ninline std::optional<const char*> ReadVarint64(const char* src,\n                                               const char* limit,\n                                               uint64_t& dest) {\n  const size_t length = ReadVarint64(src, PtrDistance(src, limit), dest);\n  if (ABSL_PREDICT_FALSE(length == 0)) return std::nullopt;\n  return src + length;\n}\n\ninline std::optional<const char*> ReadVarintSigned32(const char* src,\n                                                     const char* limit,\n                                                     int32_t& dest) {\n  const size_t length = ReadVarintSigned32(src, PtrDistance(src, limit), dest);\n  if (ABSL_PREDICT_FALSE(length == 0)) return std::nullopt;\n  return src + length;\n}\n\ninline std::optional<const char*> ReadVarintSigned64(const char* src,\n                                                     const char* limit,\n                                                     int64_t& dest) {\n  const size_t length = ReadVarintSigned64(src, PtrDistance(src, limit), dest);\n  if (ABSL_PREDICT_FALSE(length == 0)) return std::nullopt;\n  return src + length;\n}\n\nnamespace varint_internal {\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t CopyVarintFromReaderBuffer(Reader& src, const char* cursor, char* dest);\n\nextern template size_t CopyVarintFromReaderBuffer<uint32_t, false, 2>(\n    Reader& src, const char* cursor, char* dest);\nextern template size_t CopyVarintFromReaderBuffer<uint64_t, false, 2>(\n    Reader& src, const char* cursor, char* dest);\nextern template size_t CopyVarintFromReaderBuffer<uint32_t, true, 2>(\n    Reader& src, const char* cursor, char* dest);\nextern template size_t CopyVarintFromReaderBuffer<uint64_t, true, 2>(\n    Reader& src, const char* cursor, char* dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t CopyVarintFromReader(Reader& src, char* dest);\n\nextern template size_t CopyVarintFromReader<uint32_t, false, 1>(Reader& src,\n                                                                char* dest);\nextern template size_t CopyVarintFromReader<uint64_t, false, 1>(Reader& src,\n                                                                char* dest);\n\nextern template size_t CopyVarintFromReader<uint32_t, true, 1>(Reader& src,\n                                                               char* dest);\nextern template size_t CopyVarintFromReader<uint64_t, true, 1>(Reader& src,\n                                                               char* dest);\n\n}  // namespace varint_internal\n\ninline size_t CopyVarint32(Reader& src, char* dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthVarint32))) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(src.cursor()[0]);\n  dest[0] = static_cast<char>(byte0);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    src.move_cursor(1);\n    return 1;\n  }\n  if (ABSL_PREDICT_TRUE(src.available() >= 2)) {\n    const uint8_t byte1 = static_cast<uint8_t>(src.cursor()[1]);\n    dest[1] = static_cast<char>(byte1);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      src.move_cursor(2);\n      return 2;\n    }\n    return varint_internal::CopyVarintFromReaderBuffer<uint32_t,\n                                                       /*canonical=*/false, 2>(\n        src, src.cursor(), dest);\n  }\n  return varint_internal::CopyVarintFromReader<uint32_t, /*canonical=*/false,\n                                               1>(src, dest);\n}\n\ninline size_t CopyVarint64(Reader& src, char* dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthVarint64))) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(src.cursor()[0]);\n  dest[0] = static_cast<char>(byte0);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    src.move_cursor(1);\n    return 1;\n  }\n  if (ABSL_PREDICT_TRUE(src.available() >= 2)) {\n    const uint8_t byte1 = static_cast<uint8_t>(src.cursor()[1]);\n    dest[1] = static_cast<char>(byte1);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      src.move_cursor(2);\n      return 2;\n    }\n    return varint_internal::CopyVarintFromReaderBuffer<uint64_t,\n                                                       /*canonical=*/false, 2>(\n        src, src.cursor(), dest);\n  }\n  return varint_internal::CopyVarintFromReader<uint64_t, /*canonical=*/false,\n                                               1>(src, dest);\n}\n\ninline size_t CopyCanonicalVarint32(Reader& src, char* dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthVarint32))) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(src.cursor()[0]);\n  dest[0] = static_cast<char>(byte0);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    src.move_cursor(1);\n    return 1;\n  }\n  if (ABSL_PREDICT_TRUE(src.available() >= 2)) {\n    const uint8_t byte1 = static_cast<uint8_t>(src.cursor()[1]);\n    dest[1] = static_cast<char>(byte1);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      if (ABSL_PREDICT_FALSE(byte1 == 0)) return 0;\n      src.move_cursor(2);\n      return 2;\n    }\n    return varint_internal::CopyVarintFromReaderBuffer<uint32_t,\n                                                       /*canonical=*/true, 2>(\n        src, src.cursor(), dest);\n  }\n  return varint_internal::CopyVarintFromReader<uint32_t, /*canonical=*/true, 1>(\n      src, dest);\n}\n\ninline size_t CopyCanonicalVarint64(Reader& src, char* dest) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthVarint64))) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(src.cursor()[0]);\n  dest[0] = static_cast<char>(byte0);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    src.move_cursor(1);\n    return 1;\n  }\n  if (ABSL_PREDICT_TRUE(src.available() >= 2)) {\n    const uint8_t byte1 = static_cast<uint8_t>(src.cursor()[1]);\n    dest[1] = static_cast<char>(byte1);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      if (ABSL_PREDICT_FALSE(byte1 == 0)) return 0;\n      src.move_cursor(2);\n      return 2;\n    }\n    return varint_internal::CopyVarintFromReaderBuffer<uint64_t,\n                                                       /*canonical=*/true, 2>(\n        src, src.cursor(), dest);\n  }\n  return varint_internal::CopyVarintFromReader<uint64_t, /*canonical=*/true, 1>(\n      src, dest);\n}\n\nnamespace varint_internal {\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t CopyVarintFromCordBuffer(absl::Cord::CharIterator& src, size_t available,\n                                char* dest);\n\nextern template size_t CopyVarintFromCordBuffer<uint32_t, false, 2>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\nextern template size_t CopyVarintFromCordBuffer<uint64_t, false, 2>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\nextern template size_t CopyVarintFromCordBuffer<uint32_t, true, 2>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\nextern template size_t CopyVarintFromCordBuffer<uint64_t, true, 2>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t CopyVarintFromCord(absl::Cord::CharIterator& src, size_t available,\n                          char* dest);\n\nextern template size_t CopyVarintFromCord<uint32_t, false, 1>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\nextern template size_t CopyVarintFromCord<uint64_t, false, 1>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\nextern template size_t CopyVarintFromCord<uint32_t, true, 1>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\nextern template size_t CopyVarintFromCord<uint64_t, true, 1>(\n    absl::Cord::CharIterator& src, size_t available, char* dest);\n\n}  // namespace varint_internal\n\ninline size_t CopyVarint32(absl::Cord::CharIterator& src, size_t available,\n                           char* dest) {\n  RIEGELI_ASSERT_LE(available, varint_internal::Remaining(src))\n      << \"Failed precondition of CopyVarint32(): not enough remaining data\";\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(*src);\n  dest[0] = static_cast<char>(byte0);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    ++src;\n    return 1;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  if (ABSL_PREDICT_TRUE(chunk.size() >= 2)) {\n    const uint8_t byte1 = static_cast<uint8_t>(chunk[1]);\n    dest[1] = static_cast<char>(byte1);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      absl::Cord::Advance(&src, 2);\n      return 2;\n    }\n    return varint_internal::CopyVarintFromCordBuffer<uint32_t,\n                                                     /*canonical=*/false, 2>(\n        src, available, dest);\n  }\n  return varint_internal::CopyVarintFromCord<uint32_t,\n                                             /*canonical=*/false, 1>(\n      src, available, dest);\n}\n\ninline size_t CopyVarint64(absl::Cord::CharIterator& src, size_t available,\n                           char* dest) {\n  RIEGELI_ASSERT_LE(available, varint_internal::Remaining(src))\n      << \"Failed precondition of CopyVarint64(): not enough remaining data\";\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(*src);\n  dest[0] = static_cast<char>(byte0);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    ++src;\n    return 1;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  if (ABSL_PREDICT_TRUE(chunk.size() >= 2)) {\n    const uint8_t byte1 = static_cast<uint8_t>(chunk[1]);\n    dest[1] = static_cast<char>(byte1);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      absl::Cord::Advance(&src, 2);\n      return 2;\n    }\n    return varint_internal::CopyVarintFromCordBuffer<uint64_t,\n                                                     /*canonical=*/false, 2>(\n        src, available, dest);\n  }\n  return varint_internal::CopyVarintFromCord<uint64_t,\n                                             /*canonical=*/false, 1>(\n      src, available, dest);\n}\n\ninline size_t CopyCanonicalVarint32(absl::Cord::CharIterator& src,\n                                    size_t available, char* dest) {\n  RIEGELI_ASSERT_LE(available, varint_internal::Remaining(src))\n      << \"Failed precondition of CopyCanonicalVarint32(): \"\n         \"not enough remaining data\";\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(*src);\n  dest[0] = static_cast<char>(byte0);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    ++src;\n    return 1;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  if (ABSL_PREDICT_TRUE(chunk.size() >= 2)) {\n    const uint8_t byte1 = static_cast<uint8_t>(chunk[1]);\n    dest[1] = static_cast<char>(byte1);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      if (ABSL_PREDICT_FALSE(byte1 == 0)) return 0;\n      absl::Cord::Advance(&src, 2);\n      return 2;\n    }\n    return varint_internal::CopyVarintFromCordBuffer<uint32_t,\n                                                     /*canonical=*/true, 2>(\n        src, available, dest);\n  }\n  return varint_internal::CopyVarintFromCord<uint32_t,\n                                             /*canonical=*/true, 1>(\n      src, available, dest);\n}\n\ninline size_t CopyCanonicalVarint64(absl::Cord::CharIterator& src,\n                                    size_t available, char* dest) {\n  RIEGELI_ASSERT_LE(available, varint_internal::Remaining(src))\n      << \"Failed precondition of CopyCanonicalVarint64(): \"\n         \"not enough remaining data\";\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(*src);\n  dest[0] = static_cast<char>(byte0);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    ++src;\n    return 1;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  if (ABSL_PREDICT_TRUE(chunk.size() >= 2)) {\n    const uint8_t byte1 = static_cast<uint8_t>(chunk[1]);\n    dest[1] = static_cast<char>(byte1);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      if (ABSL_PREDICT_FALSE(byte1 == 0)) return 0;\n      absl::Cord::Advance(&src, 2);\n      return 2;\n    }\n    return varint_internal::CopyVarintFromCordBuffer<uint64_t,\n                                                     /*canonical=*/true, 2>(\n        src, available, dest);\n  }\n  return varint_internal::CopyVarintFromCord<uint64_t,\n                                             /*canonical=*/true, 1>(\n      src, available, dest);\n}\n\nnamespace varint_internal {\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t CopyVarintFromArray(const char* src, size_t available, char* dest);\n\nextern template size_t CopyVarintFromArray<uint32_t, false, 2>(const char* src,\n                                                               size_t available,\n                                                               char* dest);\nextern template size_t CopyVarintFromArray<uint64_t, false, 2>(const char* src,\n                                                               size_t available,\n                                                               char* dest);\nextern template size_t CopyVarintFromArray<uint32_t, true, 2>(const char* src,\n                                                              size_t available,\n                                                              char* dest);\nextern template size_t CopyVarintFromArray<uint64_t, true, 2>(const char* src,\n                                                              size_t available,\n                                                              char* dest);\n\n}  // namespace varint_internal\n\ninline size_t CopyVarint32(const char* src, size_t available, char* dest) {\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(src[0]);\n  dest[0] = static_cast<char>(byte0);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) return 1;\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const uint8_t byte1 = static_cast<uint8_t>(src[1]);\n  dest[1] = static_cast<char>(byte1);\n  if (ABSL_PREDICT_TRUE(byte1 < 0x80)) return 2;\n  return varint_internal::CopyVarintFromArray<uint32_t, /*canonical=*/false, 2>(\n      src, available, dest);\n}\n\ninline size_t CopyVarint64(const char* src, size_t available, char* dest) {\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(src[0]);\n  dest[0] = static_cast<char>(byte0);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) return 1;\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const uint8_t byte1 = static_cast<uint8_t>(src[1]);\n  dest[1] = static_cast<char>(byte1);\n  if (ABSL_PREDICT_TRUE(byte1 < 0x80)) return 2;\n  return varint_internal::CopyVarintFromArray<uint64_t, /*canonical=*/false, 2>(\n      src, available, dest);\n}\n\ninline std::optional<size_t> CopyVarint32(const char* src, const char* limit,\n                                          char* dest) {\n  const size_t length = CopyVarint32(src, PtrDistance(src, limit), dest);\n  if (ABSL_PREDICT_FALSE(length == 0)) return std::nullopt;\n  return length;\n}\n\ninline std::optional<size_t> CopyVarint64(const char* src, const char* limit,\n                                          char* dest) {\n  const size_t length = CopyVarint64(src, PtrDistance(src, limit), dest);\n  if (ABSL_PREDICT_FALSE(length == 0)) return std::nullopt;\n  return length;\n}\n\ninline size_t CopyCanonicalVarint32(const char* src, size_t available,\n                                    char* dest) {\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(src[0]);\n  dest[0] = static_cast<char>(byte0);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) return 1;\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const uint8_t byte1 = static_cast<uint8_t>(src[1]);\n  dest[1] = static_cast<char>(byte1);\n  if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n    if (ABSL_PREDICT_FALSE(byte1 == 0)) return 0;\n    return 2;\n  }\n  return varint_internal::CopyVarintFromArray<uint32_t, /*canonical=*/true, 2>(\n      src, available, dest);\n}\n\ninline size_t CopyCanonicalVarint64(const char* src, size_t available,\n                                    char* dest) {\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(src[0]);\n  dest[0] = static_cast<char>(byte0);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) return 1;\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const uint8_t byte1 = static_cast<uint8_t>(src[1]);\n  dest[1] = static_cast<char>(byte1);\n  if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n    if (ABSL_PREDICT_FALSE(byte1 == 0)) return 0;\n    return 2;\n  }\n  return varint_internal::CopyVarintFromArray<uint64_t, /*canonical=*/true, 2>(\n      src, available, dest);\n}\n\nnamespace varint_internal {\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool SkipVarintFromReaderBuffer(Reader& src, const char* cursor);\n\nextern template bool SkipVarintFromReaderBuffer<uint32_t, false, 2>(\n    Reader& src, const char* cursor);\nextern template bool SkipVarintFromReaderBuffer<uint64_t, false, 2>(\n    Reader& src, const char* cursor);\nextern template bool SkipVarintFromReaderBuffer<uint32_t, true, 2>(\n    Reader& src, const char* cursor);\nextern template bool SkipVarintFromReaderBuffer<uint64_t, true, 2>(\n    Reader& src, const char* cursor);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool SkipVarintFromReader(Reader& src);\n\nextern template bool SkipVarintFromReader<uint32_t, false, 1>(Reader& src);\nextern template bool SkipVarintFromReader<uint64_t, false, 1>(Reader& src);\nextern template bool SkipVarintFromReader<uint32_t, true, 1>(Reader& src);\nextern template bool SkipVarintFromReader<uint64_t, true, 1>(Reader& src);\n\n}  // namespace varint_internal\n\ninline bool SkipVarint32(Reader& src) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthVarint32))) return false;\n  const uint8_t byte0 = static_cast<uint8_t>(src.cursor()[0]);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    src.move_cursor(1);\n    return true;\n  }\n  if (ABSL_PREDICT_TRUE(src.available() >= 2)) {\n    const uint8_t byte1 = static_cast<uint8_t>(src.cursor()[1]);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      src.move_cursor(2);\n      return true;\n    }\n    return varint_internal::SkipVarintFromReaderBuffer<uint32_t,\n                                                       /*canonical=*/false, 2>(\n        src, src.cursor());\n  }\n  return varint_internal::SkipVarintFromReader<uint32_t, /*canonical=*/false,\n                                               1>(src);\n}\n\ninline bool SkipVarint64(Reader& src) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthVarint64))) return false;\n  const uint8_t byte0 = static_cast<uint8_t>(src.cursor()[0]);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    src.move_cursor(1);\n    return true;\n  }\n  if (ABSL_PREDICT_TRUE(src.available() >= 2)) {\n    const uint8_t byte1 = static_cast<uint8_t>(src.cursor()[1]);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      src.move_cursor(2);\n      return true;\n    }\n    return varint_internal::SkipVarintFromReaderBuffer<uint64_t,\n                                                       /*canonical=*/false, 2>(\n        src, src.cursor());\n  }\n  return varint_internal::SkipVarintFromReader<uint64_t, /*canonical=*/false,\n                                               1>(src);\n}\n\ninline bool SkipCanonicalVarint32(Reader& src) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthVarint32))) return false;\n  const uint8_t byte0 = static_cast<uint8_t>(src.cursor()[0]);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    src.move_cursor(1);\n    return true;\n  }\n  if (ABSL_PREDICT_TRUE(src.available() >= 2)) {\n    const uint8_t byte1 = static_cast<uint8_t>(src.cursor()[1]);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      if (ABSL_PREDICT_FALSE(byte1 == 0)) return false;\n      src.move_cursor(2);\n      return true;\n    }\n    return varint_internal::SkipVarintFromReaderBuffer<uint32_t,\n                                                       /*canonical=*/true, 2>(\n        src, src.cursor());\n  }\n  return varint_internal::SkipVarintFromReader<uint32_t, /*canonical=*/true, 1>(\n      src);\n}\n\ninline bool SkipCanonicalVarint64(Reader& src) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(1, kMaxLengthVarint64))) return false;\n  const uint8_t byte0 = static_cast<uint8_t>(src.cursor()[0]);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    src.move_cursor(1);\n    return true;\n  }\n  if (ABSL_PREDICT_TRUE(src.available() >= 2)) {\n    const uint8_t byte1 = static_cast<uint8_t>(src.cursor()[1]);\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      if (ABSL_PREDICT_FALSE(byte1 == 0)) return false;\n      src.move_cursor(2);\n      return true;\n    }\n    return varint_internal::SkipVarintFromReaderBuffer<uint64_t,\n                                                       /*canonical=*/true, 2>(\n        src, src.cursor());\n  }\n  return varint_internal::SkipVarintFromReader<uint64_t, /*canonical=*/true, 1>(\n      src);\n}\n\nnamespace varint_internal {\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool SkipVarintFromCordBuffer(absl::Cord::CharIterator& src, size_t available);\n\nextern template bool SkipVarintFromCordBuffer<uint32_t, false, 2>(\n    absl::Cord::CharIterator& src, size_t available);\nextern template bool SkipVarintFromCordBuffer<uint64_t, false, 2>(\n    absl::Cord::CharIterator& src, size_t available);\nextern template bool SkipVarintFromCordBuffer<uint32_t, true, 2>(\n    absl::Cord::CharIterator& src, size_t available);\nextern template bool SkipVarintFromCordBuffer<uint64_t, true, 2>(\n    absl::Cord::CharIterator& src, size_t available);\n\ntemplate <typename T, bool canonical, size_t initial_index>\nbool SkipVarintFromCord(absl::Cord::CharIterator& src, size_t available);\n\nextern template bool SkipVarintFromCord<uint32_t, false, 1>(\n    absl::Cord::CharIterator& src, size_t available);\nextern template bool SkipVarintFromCord<uint64_t, false, 1>(\n    absl::Cord::CharIterator& src, size_t available);\nextern template bool SkipVarintFromCord<uint32_t, true, 1>(\n    absl::Cord::CharIterator& src, size_t available);\nextern template bool SkipVarintFromCord<uint64_t, true, 1>(\n    absl::Cord::CharIterator& src, size_t available);\n\n}  // namespace varint_internal\n\ninline bool SkipVarint32(absl::Cord::CharIterator& src, size_t available) {\n  RIEGELI_ASSERT_LE(available, varint_internal::Remaining(src))\n      << \"Failed precondition of SkipVarint32(): not enough remaining data\";\n  if (ABSL_PREDICT_FALSE(available == 0)) return false;\n  const uint32_t byte0 = uint32_t{static_cast<uint8_t>(*src)};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    ++src;\n    return true;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return false;\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  if (ABSL_PREDICT_TRUE(chunk.size() >= 2)) {\n    const uint32_t byte1 = uint32_t{static_cast<uint8_t>(chunk[1])};\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      absl::Cord::Advance(&src, 2);\n      return true;\n    }\n    return varint_internal::SkipVarintFromCordBuffer<uint32_t,\n                                                     /*canonical=*/false, 2>(\n        src, available);\n  }\n  return varint_internal::SkipVarintFromCord<uint32_t,\n                                             /*canonical=*/false, 1>(src,\n                                                                     available);\n}\n\ninline bool SkipVarint64(absl::Cord::CharIterator& src, size_t available) {\n  RIEGELI_ASSERT_LE(available, varint_internal::Remaining(src))\n      << \"Failed precondition of SkipVarint64(): not enough remaining data\";\n  if (ABSL_PREDICT_FALSE(available == 0)) return false;\n  const uint64_t byte0 = uint64_t{static_cast<uint8_t>(*src)};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    ++src;\n    return true;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return false;\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  if (ABSL_PREDICT_TRUE(chunk.size() >= 2)) {\n    const uint64_t byte1 = uint64_t{static_cast<uint8_t>(chunk[1])};\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      absl::Cord::Advance(&src, 2);\n      return true;\n    }\n    return varint_internal::SkipVarintFromCordBuffer<uint64_t,\n                                                     /*canonical=*/false, 2>(\n        src, available);\n  }\n  return varint_internal::SkipVarintFromCord<uint64_t,\n                                             /*canonical=*/false, 1>(src,\n                                                                     available);\n}\n\ninline bool SkipCanonicalVarint32(absl::Cord::CharIterator& src,\n                                  size_t available) {\n  RIEGELI_ASSERT_LE(available, varint_internal::Remaining(src))\n      << \"Failed precondition of SkipCanonicalVarint32(): \"\n         \"not enough remaining data\";\n  if (ABSL_PREDICT_FALSE(available == 0)) return false;\n  const uint32_t byte0 = uint32_t{static_cast<uint8_t>(*src)};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    ++src;\n    return true;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return false;\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  if (ABSL_PREDICT_TRUE(chunk.size() >= 2)) {\n    const uint32_t byte1 = uint32_t{static_cast<uint8_t>(chunk[1])};\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      if (ABSL_PREDICT_FALSE(byte1 == 0)) return false;\n      absl::Cord::Advance(&src, 2);\n      return true;\n    }\n    return varint_internal::SkipVarintFromCordBuffer<uint32_t,\n                                                     /*canonical=*/true, 2>(\n        src, available);\n  }\n  return varint_internal::SkipVarintFromCord<uint32_t,\n                                             /*canonical=*/true, 1>(src,\n                                                                    available);\n}\n\ninline bool SkipCanonicalVarint64(absl::Cord::CharIterator& src,\n                                  size_t available) {\n  RIEGELI_ASSERT_LE(available, varint_internal::Remaining(src))\n      << \"Failed precondition of SkipCanonicalVarint64(): \"\n         \"not enough remaining data\";\n  if (ABSL_PREDICT_FALSE(available == 0)) return false;\n  const uint64_t byte0 = uint64_t{static_cast<uint8_t>(*src)};\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) {\n    ++src;\n    return true;\n  }\n  if (ABSL_PREDICT_FALSE(available == 1)) return false;\n  const absl::string_view chunk = absl::Cord::ChunkRemaining(src);\n  if (ABSL_PREDICT_TRUE(chunk.size() >= 2)) {\n    const uint64_t byte1 = uint64_t{static_cast<uint8_t>(chunk[1])};\n    if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n      if (ABSL_PREDICT_FALSE(byte1 == 0)) return false;\n      absl::Cord::Advance(&src, 2);\n      return true;\n    }\n    return varint_internal::SkipVarintFromCordBuffer<uint64_t,\n                                                     /*canonical=*/true, 2>(\n        src, available);\n  }\n  return varint_internal::SkipVarintFromCord<uint64_t,\n                                             /*canonical=*/true, 1>(src,\n                                                                    available);\n}\n\nnamespace varint_internal {\n\ntemplate <typename T, bool canonical, size_t initial_index>\nsize_t SkipVarintFromArray(const char* src, size_t available);\n\nextern template size_t SkipVarintFromArray<uint32_t, false, 2>(\n    const char* src, size_t available);\nextern template size_t SkipVarintFromArray<uint64_t, false, 2>(\n    const char* src, size_t available);\nextern template size_t SkipVarintFromArray<uint32_t, true, 2>(const char* src,\n                                                              size_t available);\nextern template size_t SkipVarintFromArray<uint64_t, true, 2>(const char* src,\n                                                              size_t available);\n\n}  // namespace varint_internal\n\ninline size_t SkipVarint32(const char* src, size_t available) {\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(src[0]);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) return 1;\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const uint8_t byte1 = static_cast<uint8_t>(src[1]);\n  if (ABSL_PREDICT_TRUE(byte1 < 0x80)) return 2;\n  return varint_internal::SkipVarintFromArray<uint32_t, /*canonical=*/false, 2>(\n      src, available);\n}\n\ninline size_t SkipVarint64(const char* src, size_t available) {\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(src[0]);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) return 1;\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const uint8_t byte1 = static_cast<uint8_t>(src[1]);\n  if (ABSL_PREDICT_TRUE(byte1 < 0x80)) return 2;\n  return varint_internal::SkipVarintFromArray<uint64_t, /*canonical=*/false, 2>(\n      src, available);\n}\n\ninline size_t SkipCanonicalVarint32(const char* src, size_t available) {\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(src[0]);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) return 1;\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const uint8_t byte1 = static_cast<uint8_t>(src[1]);\n  if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n    if (ABSL_PREDICT_FALSE(byte1 == 0)) return 0;\n    return 2;\n  }\n  return varint_internal::SkipVarintFromArray<uint32_t, /*canonical=*/true, 2>(\n      src, available);\n}\n\ninline size_t SkipCanonicalVarint64(const char* src, size_t available) {\n  if (ABSL_PREDICT_FALSE(available == 0)) return 0;\n  const uint8_t byte0 = static_cast<uint8_t>(src[0]);\n  if (ABSL_PREDICT_TRUE(byte0 < 0x80)) return 1;\n  if (ABSL_PREDICT_FALSE(available == 1)) return 0;\n  const uint8_t byte1 = static_cast<uint8_t>(src[1]);\n  if (ABSL_PREDICT_TRUE(byte1 < 0x80)) {\n    if (ABSL_PREDICT_FALSE(byte1 == 0)) return 0;\n    return 2;\n  }\n  return varint_internal::SkipVarintFromArray<uint64_t, /*canonical=*/true, 2>(\n      src, available);\n}\n\nconstexpr int32_t DecodeVarintSigned32(uint32_t repr) {\n  return static_cast<int32_t>((repr >> 1) ^ (~(repr & 1) + 1));\n}\n\nconstexpr int64_t DecodeVarintSigned64(uint64_t repr) {\n  return static_cast<int64_t>((repr >> 1) ^ (~(repr & 1) + 1));\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_VARINT_VARINT_READING_H_\n"
  },
  {
    "path": "riegeli/varint/varint_writing.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_VARINT_VARINT_WRITING_H_\n#define RIEGELI_VARINT_VARINT_WRITING_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/numeric/bits.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/constexpr.h\"\n#include \"riegeli/bytes/backward_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/varint/varint_internal.h\"  // IWYU pragma: export\n\nnamespace riegeli {\n\n// Writes a varint. This corresponds to protobuf types `{int,uint}{32,64}`\n// (with a cast needed in the case of `int{32,64}`).\n//\n// Warning: protobuf writes values of type `int32` by casting them to `uint64`,\n// not `uint32` (negative values take 10 bytes, not 5), hence they must be\n// written with `WriteVarint64()`, not `WriteVarint32()`, if negative values are\n// possible.\n//\n// Return values:\n//  * `true`  - success (`dest.ok()`)\n//  * `false` - failure (`!dest.ok()`)\nbool WriteVarint32(uint32_t data, Writer& dest);\nbool WriteVarint64(uint64_t data, Writer& dest);\nbool WriteVarint32(uint32_t data, BackwardWriter& dest);\nbool WriteVarint64(uint64_t data, BackwardWriter& dest);\n\n// Writes a signed varint (zigzag-encoded). This corresponds to protobuf types\n// `sint{32,64}`.\n//\n// Return values:\n//  * `true`  - success (`dest.ok()`)\n//  * `false` - failure (`!dest.ok()`)\nbool WriteVarintSigned32(int32_t data, Writer& dest);\nbool WriteVarintSigned64(int64_t data, Writer& dest);\nbool WriteVarintSigned32(int32_t data, BackwardWriter& dest);\nbool WriteVarintSigned64(int64_t data, BackwardWriter& dest);\n\n// Returns the length needed to write a given value as a varint.\n// This corresponds to protobuf types `{int,uint}{32,64}` (with a cast needed in\n// the case of `int{32,64}`).\n//\n// Warning: protobuf writes values of type `int32` by casting them to `uint64`,\n// not `uint32` (negative values take 10 bytes, not 5), hence they must be\n// measured with `LengthVarint64()`, not `LengthVarint32()`, if negative values\n// are possible.\n//\n// The result is at most `kMaxLengthVarint{32,64}`.\nconstexpr size_t LengthVarint32(uint32_t data);\nconstexpr size_t LengthVarint64(uint64_t data);\n\n// Returns the length needed to write a given value as a signed varint\n// (zigzag-encoded). This corresponds to protobuf types `sint{32,64}`.\n//\n// The result is at most `kMaxLengthVarint{32,64}`.\nconstexpr size_t LengthVarintSigned32(int32_t data);\nconstexpr size_t LengthVarintSigned64(int64_t data);\n\n// Writes a varint to an array. This corresponds to protobuf types\n// `{int,uint}{32,64}` (with a cast needed in the case of `int{32,64}`).\n//\n// Warning: protobuf writes values of type `int32` by casting them to `uint64`,\n// not `uint32` (negative values take 10 bytes, not 5), hence they must be\n// written with `WriteVarint64()`, not `WriteVarint32()`, if negative values are\n// possible.\n//\n// Writes at most `LengthVarint{32,64}(data)` bytes to `dest[]`. Returns the\n// updated `dest` after the written value.\nchar* WriteVarint32(uint32_t data, char* dest);\nchar* WriteVarint64(uint64_t data, char* dest);\n\n// Writes a signed varint (zigzag-encoded) to an array. This corresponds to\n// protobuf types `sint{32,64}`.\n//\n// Writes at most `LengthVarintSigned{32,64}(data)` bytes to `dest[]`. Returns\n// the updated `dest` after the written value.\nchar* WriteVarintSigned32(int32_t data, char* dest);\nchar* WriteVarintSigned64(int64_t data, char* dest);\n\n// Encodes a signed varint (zigzag-encoding) as an unsigned value to be written\n// as a plain varint. This corresponds to protobuf types `sint{32,64}`.\nconstexpr uint32_t EncodeVarintSigned32(int32_t value);\nconstexpr uint64_t EncodeVarintSigned64(int64_t value);\n\n// Implementation details follow.\n\ninline bool WriteVarint32(uint32_t data, Writer& dest) {\n  if (ABSL_PREDICT_FALSE(!dest.Push(RIEGELI_IS_CONSTANT(data) ||\n                                            RIEGELI_IS_CONSTANT(data < 0x80)\n                                        ? LengthVarint32(data)\n                                        : kMaxLengthVarint32))) {\n    return false;\n  }\n  dest.set_cursor(WriteVarint32(data, dest.cursor()));\n  return true;\n}\n\ninline bool WriteVarint64(uint64_t data, Writer& dest) {\n  if (ABSL_PREDICT_FALSE(!dest.Push(RIEGELI_IS_CONSTANT(data) ||\n                                            RIEGELI_IS_CONSTANT(data < 0x80)\n                                        ? LengthVarint64(data)\n                                        : kMaxLengthVarint64))) {\n    return false;\n  }\n  dest.set_cursor(WriteVarint64(data, dest.cursor()));\n  return true;\n}\n\ninline bool WriteVarint32(uint32_t data, BackwardWriter& dest) {\n  const size_t length = LengthVarint32(data);\n  if (ABSL_PREDICT_FALSE(!dest.Push(length))) return false;\n  dest.move_cursor(length);\n  WriteVarint32(data, dest.cursor());\n  return true;\n}\n\ninline bool WriteVarint64(uint64_t data, BackwardWriter& dest) {\n  const size_t length = LengthVarint64(data);\n  if (ABSL_PREDICT_FALSE(!dest.Push(length))) return false;\n  dest.move_cursor(length);\n  WriteVarint64(data, dest.cursor());\n  return true;\n}\n\ninline bool WriteVarintSigned32(int32_t data, Writer& dest) {\n  return WriteVarint32(EncodeVarintSigned32(data), dest);\n}\n\ninline bool WriteVarintSigned64(int64_t data, Writer& dest) {\n  return WriteVarint64(EncodeVarintSigned64(data), dest);\n}\n\ninline bool WriteVarintSigned32(int32_t data, BackwardWriter& dest) {\n  return WriteVarint32(EncodeVarintSigned32(data), dest);\n}\n\ninline bool WriteVarintSigned64(int64_t data, BackwardWriter& dest) {\n  return WriteVarint64(EncodeVarintSigned64(data), dest);\n}\n\nconstexpr size_t LengthVarint32(uint32_t data) {\n  const size_t width = IntCast<size_t>(absl::bit_width(data | 1));\n  // This is the same as `(width + 6) / 7` for `width` in [1..32],\n  // but performs division by a power of 2.\n  return (width * 9 + 64) / 64;\n}\n\nconstexpr size_t LengthVarint64(uint64_t data) {\n  const size_t width = IntCast<size_t>(absl::bit_width(data | 1));\n  // This is the same as `(width + 6) / 7` for `width` in [1..64],\n  // but performs division by a power of 2.\n  return (width * 9 + 64) / 64;\n}\n\nconstexpr size_t LengthVarintSigned32(int32_t data) {\n  return LengthVarint32(EncodeVarintSigned32(data));\n}\n\nconstexpr size_t LengthVarintSigned64(int64_t data) {\n  return LengthVarint64(EncodeVarintSigned64(data));\n}\n\ninline char* WriteVarint32(uint32_t data, char* dest) {\n  if (ABSL_PREDICT_TRUE(data < 0x80)) {\n    *dest++ = static_cast<char>(data);\n    return dest;\n  }\n  do {\n    *dest++ = static_cast<char>(data | 0x80);\n    data >>= 7;\n  } while (data >= 0x80);\n  *dest++ = static_cast<char>(data);\n  return dest;\n}\n\ninline char* WriteVarint64(uint64_t data, char* dest) {\n  if (ABSL_PREDICT_TRUE(data < 0x80)) {\n    *dest++ = static_cast<char>(data);\n    return dest;\n  }\n  do {\n    *dest++ = static_cast<char>(data | 0x80);\n    data >>= 7;\n  } while (data >= 0x80);\n  *dest++ = static_cast<char>(data);\n  return dest;\n}\n\ninline char* WriteVarintSigned32(int32_t data, char* dest) {\n  return WriteVarint32(EncodeVarintSigned32(data), dest);\n}\n\ninline char* WriteVarintSigned64(int64_t data, char* dest) {\n  return WriteVarint64(EncodeVarintSigned64(data), dest);\n}\n\nconstexpr uint32_t EncodeVarintSigned32(int32_t value) {\n  return (static_cast<uint32_t>(value) << 1) ^\n         static_cast<uint32_t>(value >> 31);\n}\n\nconstexpr uint64_t EncodeVarintSigned64(int64_t value) {\n  return (static_cast<uint64_t>(value) << 1) ^\n         static_cast<uint64_t>(value >> 63);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_VARINT_VARINT_WRITING_H_\n"
  },
  {
    "path": "riegeli/xz/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"xz_reader\",\n    srcs = [\"xz_reader.cc\"],\n    hdrs = [\"xz_reader.h\"],\n    deps = [\n        \":xz_error\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_reader\",\n        \"//riegeli/bytes:reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@xz//:lzma\",\n    ],\n)\n\ncc_library(\n    name = \"xz_writer\",\n    srcs = [\"xz_writer.cc\"],\n    hdrs = [\"xz_writer.h\"],\n    deps = [\n        \":xz_error\",\n        \":xz_reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_writer\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@xz//:lzma\",\n    ],\n)\n\ncc_library(\n    name = \"xz_error\",\n    srcs = [\"xz_error.cc\"],\n    hdrs = [\"xz_error.h\"],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \"//riegeli/base:assert\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@xz//:lzma\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/xz/xz_error.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/xz/xz_error.h\"\n\n#include <string>\n\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"lzma.h\"\n#include \"riegeli/base/assert.h\"\n\nnamespace riegeli::xz_internal {\n\nabsl::Status XzErrorToStatus(absl::string_view operation,\n                             lzma_ret liblzma_code) {\n  absl::StatusCode code;\n  switch (liblzma_code) {\n    case LZMA_OK:\n      return absl::OkStatus();\n    case LZMA_NO_CHECK:\n    case LZMA_UNSUPPORTED_CHECK:\n    case LZMA_DATA_ERROR:\n      code = absl::StatusCode::kInvalidArgument;\n      break;\n    case LZMA_MEM_ERROR:\n    case LZMA_MEMLIMIT_ERROR:\n      code = absl::StatusCode::kResourceExhausted;\n      break;\n    default:\n      // Should not happen.\n      code = absl::StatusCode::kInternal;\n      break;\n  }\n  std::string message = absl::StrCat(operation, \" failed\");\n  absl::string_view details;\n  switch (liblzma_code) {\n    case LZMA_OK:\n      RIEGELI_ASSUME_UNREACHABLE() << \"Handled before switch\";\n    case LZMA_STREAM_END:\n      details = \"End of stream was reached\";\n      break;\n    case LZMA_NO_CHECK:\n      details = \"Input stream has no integrity check\";\n      break;\n    case LZMA_UNSUPPORTED_CHECK:\n      details = \"Cannot calculate the integrity check\";\n      break;\n    case LZMA_GET_CHECK:\n      details = \"Integrity check type is now available\";\n      break;\n    case LZMA_MEM_ERROR:\n      details = \"Cannot allocate memory\";\n      break;\n    case LZMA_MEMLIMIT_ERROR:\n      details = \"Memory usage limit was reached\";\n      break;\n    case LZMA_FORMAT_ERROR:\n      details = \"File format not recognized\";\n      break;\n    case LZMA_OPTIONS_ERROR:\n      details = \"Invalid or unsupported options\";\n      break;\n    case LZMA_DATA_ERROR:\n      details = \"Data is corrupt\";\n      break;\n    case LZMA_BUF_ERROR:\n      details = \"No progress is possible\";\n      break;\n    case LZMA_PROG_ERROR:\n      details = \"Programming error\";\n      break;\n    default:\n      absl::StrAppend(&message, \": unknown liblzma error code: \", liblzma_code);\n      break;\n  }\n  if (!details.empty()) absl::StrAppend(&message, \": \", details);\n  return absl::Status(code, message);\n}\n\n}  // namespace riegeli::xz_internal\n"
  },
  {
    "path": "riegeli/xz/xz_error.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_XZ_XZ_ERROR_H_\n#define RIEGELI_XZ_XZ_ERROR_H_\n\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"lzma.h\"\n\nnamespace riegeli::xz_internal {\n\nabsl::Status XzErrorToStatus(absl::string_view operation,\n                             lzma_ret liblzma_code);\n\n}  // namespace riegeli::xz_internal\n\n#endif  // RIEGELI_XZ_XZ_ERROR_H_\n"
  },
  {
    "path": "riegeli/xz/xz_reader.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/xz/xz_reader.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"lzma.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/xz/xz_error.h\"\n\nnamespace riegeli {\n\nvoid XzReaderBase::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of XzReader: null Reader pointer\";\n  if (ABSL_PREDICT_FALSE(!src->ok()) && src->available() == 0) {\n    FailWithoutAnnotation(AnnotateOverSrc(src->status()));\n    return;\n  }\n  initial_compressed_pos_ = src->pos();\n  InitializeDecompressor();\n}\n\ninline void XzReaderBase::InitializeDecompressor() {\n  decompressor_ =\n      KeyedRecyclingPool<lzma_stream, LzmaStreamKey, LzmaStreamDeleter>::global(\n          recycling_pool_options_)\n          .Get(LzmaStreamKey(container_),\n               [] { return riegeli::Maker<lzma_stream>(); });\n  switch (container_) {\n    case Container::kXz: {\n      const lzma_ret liblzma_code = lzma_stream_decoder(\n          decompressor_.get(), std::numeric_limits<uint64_t>::max(), flags_);\n      if (ABSL_PREDICT_FALSE(liblzma_code != LZMA_OK)) {\n        FailOperation(\"lzma_stream_decoder()\", liblzma_code);\n      }\n      return;\n    }\n    case Container::kLzma: {\n      const lzma_ret liblzma_code = lzma_alone_decoder(\n          decompressor_.get(), std::numeric_limits<uint64_t>::max());\n      if (ABSL_PREDICT_FALSE(liblzma_code != LZMA_OK)) {\n        FailOperation(\"lzma_alone_decoder()\", liblzma_code);\n      }\n      return;\n    }\n    case Container::kXzOrLzma: {\n      const lzma_ret liblzma_code = lzma_auto_decoder(\n          decompressor_.get(), std::numeric_limits<uint64_t>::max(), flags_);\n      if (ABSL_PREDICT_FALSE(liblzma_code != LZMA_OK)) {\n        FailOperation(\"lzma_auto_decoder()\", liblzma_code);\n      }\n      return;\n    }\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown container format: \" << static_cast<int>(container_);\n}\n\nvoid XzReaderBase::Done() {\n  if (ABSL_PREDICT_FALSE(TruncatedAtClose())) {\n    Reader& src = *SrcReader();\n    FailWithoutAnnotation(AnnotateOverSrc(src.AnnotateStatus(\n        absl::InvalidArgumentError(\"Truncated Xz-compressed stream\"))));\n  }\n  BufferedReader::Done();\n  decompressor_.reset();\n}\n\ninline bool XzReaderBase::FailOperation(absl::string_view operation,\n                                        lzma_ret liblzma_code) {\n  RIEGELI_ASSERT_NE(liblzma_code, LZMA_OK)\n      << \"Failed precondition of XzReaderBase::FailOperation(): \"\n         \"liblzma error code not failed\";\n  RIEGELI_ASSERT(is_open())\n      << \"Failed precondition of XzReaderBase::FailOperation(): \"\n         \"Object closed\";\n  return Fail(xz_internal::XzErrorToStatus(operation, liblzma_code));\n}\n\nabsl::Status XzReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    if (ABSL_PREDICT_FALSE(truncated_ && (flags_ & LZMA_CONCATENATED) == 0)) {\n      status = Annotate(status, \"reading truncated Xz-compressed stream\");\n    }\n    Reader& src = *SrcReader();\n    status = src.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `src` with the compressed position.\n  // Clarify that the current position is the uncompressed position instead of\n  // delegating to `BufferedReader::AnnotateStatusImpl()`.\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status XzReaderBase::AnnotateOverSrc(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\ninline bool XzReaderBase::TruncatedAtClose() {\n  if (!truncated_) return false;\n  if ((flags_ & LZMA_CONCATENATED) == 0) return true;\n  Reader& src = *SrcReader();\n  if (src.pos() == initial_compressed_pos_) {\n    // Empty concatenated stream.\n    return false;\n  }\n  // Check if the stream ends cleanly.\n  decompressor_->next_out = nullptr;\n  decompressor_->avail_out = 0;\n  decompressor_->next_in = nullptr;\n  decompressor_->avail_in = 0;\n  const lzma_ret liblzma_code = lzma_code(decompressor_.get(), LZMA_FINISH);\n  switch (liblzma_code) {\n    case LZMA_OK:\n      RIEGELI_ASSUME_UNREACHABLE()\n          << \"lzma_code(LZMA_FINISH) with no buffer returned LZMA_OK\";\n    case LZMA_BUF_ERROR:\n      return true;\n    case LZMA_STREAM_END:\n      return false;\n    default:\n      FailOperation(\"lzma_code()\", liblzma_code);\n      return false;\n  }\n}\n\nbool XzReaderBase::ReadInternal(size_t min_length, size_t max_length,\n                                char* dest) {\n  RIEGELI_ASSERT_GT(min_length, 0u)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"nothing to read\";\n  RIEGELI_ASSERT_GE(max_length, min_length)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"max_length < min_length\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::ReadInternal()\";\n  Reader& src = *SrcReader();\n  truncated_ = false;\n  max_length = UnsignedMin(max_length,\n                           std::numeric_limits<Position>::max() - limit_pos());\n  decompressor_->next_out = reinterpret_cast<uint8_t*>(dest);\n  for (;;) {\n    decompressor_->avail_out = PtrDistance(\n        reinterpret_cast<char*>(decompressor_->next_out), dest + max_length);\n    decompressor_->next_in = reinterpret_cast<const uint8_t*>(src.cursor());\n    decompressor_->avail_in = src.available();\n    const lzma_ret liblzma_code = lzma_code(decompressor_.get(), LZMA_RUN);\n    src.set_cursor(reinterpret_cast<const char*>(decompressor_->next_in));\n    const size_t length_read =\n        PtrDistance(dest, reinterpret_cast<char*>(decompressor_->next_out));\n    switch (liblzma_code) {\n      case LZMA_OK:\n        if (length_read >= min_length) break;\n        ABSL_FALLTHROUGH_INTENDED;\n      case LZMA_BUF_ERROR:\n        if (ABSL_PREDICT_FALSE(decompressor_->avail_in > 0)) {\n          RIEGELI_ASSERT_EQ(decompressor_->avail_out, 0u)\n              << \"lzma_code() returned but there are still input data \"\n                 \"and output space\";\n          RIEGELI_ASSERT_EQ(length_read,\n                            std::numeric_limits<Position>::max() - limit_pos())\n              << \"The position does not overflow but the output buffer is \"\n                 \"full, while less than min_length was output, which is \"\n                 \"impossible because the buffer has size max_length which is \"\n                 \"at least min_length if the position does not overflow\";\n          move_limit_pos(length_read);\n          return FailOverflow();\n        }\n        if (ABSL_PREDICT_FALSE(!src.Pull())) {\n          move_limit_pos(length_read);\n          if (ABSL_PREDICT_FALSE(!src.ok())) {\n            return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n          }\n          truncated_ = true;\n          return false;\n        }\n        continue;\n      case LZMA_STREAM_END:\n        decompressor_.reset();\n        move_limit_pos(length_read);\n        // Avoid `BufferedReader` allocating another buffer.\n        set_exact_size(limit_pos());\n        return length_read >= min_length;\n      default:\n        FailOperation(\"lzma_code()\", liblzma_code);\n        break;\n    }\n    move_limit_pos(length_read);\n    return length_read >= min_length;\n  }\n}\n\nvoid XzReaderBase::ExactSizeReached() {\n  if (decompressor_ == nullptr) return;\n  char buffer[1];\n  if (ABSL_PREDICT_FALSE(XzReaderBase::ReadInternal(1, 1, buffer))) {\n    decompressor_.reset();\n    Fail(absl::FailedPreconditionError(\n        \"Uncompressed size reached but more data can be decompressed, \"\n        \"which implies that seeking back and reading again encountered \"\n        \"changed Xz-compressed data\"));\n  }\n}\n\nbool XzReaderBase::ToleratesReadingAhead() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->ToleratesReadingAhead();\n}\n\nbool XzReaderBase::SupportsRewind() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRewind();\n}\n\nbool XzReaderBase::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (new_pos <= limit_pos()) {\n    // Seeking backwards.\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    Reader& src = *SrcReader();\n    truncated_ = false;\n    set_buffer();\n    set_limit_pos(0);\n    decompressor_.reset();\n    if (ABSL_PREDICT_FALSE(!src.Seek(initial_compressed_pos_))) {\n      return FailWithoutAnnotation(AnnotateOverSrc(src.StatusOrAnnotate(\n          absl::DataLossError(\"Xz-compressed stream got truncated\"))));\n    }\n    InitializeDecompressor();\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    if (new_pos == 0) return true;\n  }\n  return BufferedReader::SeekBehindBuffer(new_pos);\n}\n\nbool XzReaderBase::SupportsNewReader() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsNewReader();\n}\n\nstd::unique_ptr<Reader> XzReaderBase::NewReaderImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point\n  // if `SrcReader()->SupportsNewReader()`.\n  Reader& src = *SrcReader();\n  std::unique_ptr<Reader> compressed_reader =\n      src.NewReader(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    return nullptr;\n  }\n  std::unique_ptr<Reader> reader =\n      std::make_unique<XzReader<std::unique_ptr<Reader>>>(\n          std::move(compressed_reader),\n          XzReaderBase::Options()\n              .set_container(container_)\n              .set_concatenate((flags_ & LZMA_CONCATENATED) != 0)\n              .set_buffer_options(buffer_options())\n              .set_recycling_pool_options(recycling_pool_options_));\n  reader->Seek(initial_pos);\n  return reader;\n}\n\nbool RecognizeXz(Reader& src) {\n  static constexpr char kMagic[] = {'\\xfd', '7', 'z', 'X', 'Z', '\\x00'};\n  return src.Pull(sizeof(kMagic)) &&\n         absl::string_view(src.cursor(), sizeof(kMagic)) ==\n             absl::string_view(kMagic, sizeof(kMagic));\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/xz/xz_reader.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_XZ_XZ_READER_H_\n#define RIEGELI_XZ_XZ_READER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <memory>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"lzma.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `XzReader`.\nclass XzReaderBase : public BufferedReader {\n public:\n  // Specifies what container format to expect.\n  enum class Container {\n    kXz,        // Xz container (recommended).\n    kLzma,      // Lzma container (legacy file format).\n    kXzOrLzma,  // Xz or Lzma container.\n  };\n\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // What container format to expect.\n    //\n    // Default: `Container::kXzOrLzma`.\n    static constexpr Container kDefaultContainer = Container::kXzOrLzma;\n    Options& set_container(Container container) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      container_ = container;\n      return *this;\n    }\n    Options&& set_container(Container container) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_container(container));\n    }\n    Container container() const { return container_; }\n\n    // If `true`, concatenated compressed streams are decoded to concatenation\n    // of their decompressed contents. An empty compressed stream is decoded to\n    // empty decompressed contents.\n    //\n    // If `false`, exactly one compressed stream is consumed.\n    //\n    // `concatenate()` is supported only for `Container::kXz` and\n    // `Container::kXzOrLzma` (if the actual format is `kXz`)\n    //\n    // Default: `false`.\n    Options& set_concatenate(bool concatenate) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      concatenate_ = concatenate;\n      return *this;\n    }\n    Options&& set_concatenate(bool concatenate) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_concatenate(concatenate));\n    }\n    bool concatenate() const { return concatenate_; }\n\n    // Options for a global `RecyclingPool` of decompression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // decompression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    Options& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    Options&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    Container container_ = kDefaultContainer;\n    bool concatenate_ = false;\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Returns the compressed `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns `true` if the source is truncated (without a clean end of the\n  // compressed stream) at the current position. In such case, if the source\n  // does not grow, `Close()` will fail.\n  //\n  // Precondition: `Options::concatenate()` was `false`.\n  bool truncated() const {\n    RIEGELI_ASSERT_EQ(flags_ & LZMA_CONCATENATED, 0u)\n        << \"Failed precondition of XzReaderBase::truncated(): \"\n           \"Options::concatenate() is true\";\n    return truncated_ && available() == 0;\n  }\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRewind() override;\n  bool SupportsNewReader() override;\n\n protected:\n  explicit XzReaderBase(Closed) noexcept : BufferedReader(kClosed) {}\n\n  explicit XzReaderBase(BufferOptions buffer_options, Container container,\n                        uint32_t flags,\n                        const RecyclingPoolOptions& recycling_pool_options);\n\n  XzReaderBase(XzReaderBase&& that) noexcept;\n  XzReaderBase& operator=(XzReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, Container container, uint32_t flags,\n             const RecyclingPoolOptions& recycling_pool_options);\n  static int GetWindowBits(const Options& options);\n  void Initialize(Reader* src);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool ReadInternal(size_t min_length, size_t max_length, char* dest) override;\n  void ExactSizeReached() override;\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  struct LzmaStreamDeleter {\n    void operator()(lzma_stream* ptr) const {\n      lzma_end(ptr);\n      delete ptr;\n    }\n  };\n\n  struct LzmaStreamKey : WithEqual<LzmaStreamKey> {\n    LzmaStreamKey() = default;\n    explicit LzmaStreamKey(Container container) : container(container) {}\n\n    friend bool operator==(LzmaStreamKey a, LzmaStreamKey b) {\n      return a.container == b.container;\n    }\n    template <typename HashState>\n    friend HashState AbslHashValue(HashState hash_state, LzmaStreamKey self) {\n      return HashState::combine(std::move(hash_state), self.container);\n    }\n\n    Container container;\n  };\n\n  void InitializeDecompressor();\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation,\n                                         lzma_ret liblzma_code);\n  bool TruncatedAtClose();\n\n  Container container_ = Options::kDefaultContainer;\n  uint32_t flags_ = 0;\n  Position initial_compressed_pos_ = 0;\n  // If `true`, the source is truncated (without a clean end of the compressed\n  // stream) at the current position. If the source does not grow, `Close()`\n  // will fail.\n  bool truncated_ = false;\n  RecyclingPoolOptions recycling_pool_options_;\n  // If `ok()` but `decompressor_ == nullptr` then all data have been\n  // decompressed, `exact_size() == limit_pos()`, and `ReadInternal()` must not\n  // be called again.\n  KeyedRecyclingPool<lzma_stream, LzmaStreamKey, LzmaStreamDeleter>::Handle\n      decompressor_;\n};\n\n// A `Reader` which decompresses data with Xz (LZMA) after getting it from\n// another `Reader`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Reader` must not be accessed until the `XzReader` is closed\n// or no longer used.\ntemplate <typename Src = Reader*>\nclass XzReader : public XzReaderBase {\n public:\n  // Creates a closed `XzReader`.\n  explicit XzReader(Closed) noexcept : XzReaderBase(kClosed) {}\n\n  // Will read from the compressed `Reader` provided by `src`.\n  explicit XzReader(Initializer<Src> src, Options options = Options());\n\n  XzReader(XzReader&&) = default;\n  XzReader& operator=(XzReader&&) = default;\n\n  // Makes `*this` equivalent to a newly constructed `XzReader`. This avoids\n  // constructing a temporary `XzReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  void VerifyEndImpl() override;\n\n private:\n  // The object providing and possibly owning the compressed `Reader`.\n  Dependency<Reader*, Src> src_;\n};\n\nexplicit XzReader(Closed) -> XzReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit XzReader(Src&& src,\n                  XzReaderBase::Options options = XzReaderBase::Options())\n    -> XzReader<TargetT<Src>>;\n\n// Returns `true` if the data look like they have been Xz-compressed with\n// `Container::kXz`.\n//\n// The current position of `src` is unchanged.\nbool RecognizeXz(Reader& src);\n\n// Implementation details follow.\n\ninline XzReaderBase::XzReaderBase(\n    BufferOptions buffer_options, Container container, uint32_t flags,\n    const RecyclingPoolOptions& recycling_pool_options)\n    : BufferedReader(buffer_options),\n      container_(container),\n      flags_(flags),\n      recycling_pool_options_(recycling_pool_options) {}\n\ninline XzReaderBase::XzReaderBase(XzReaderBase&& that) noexcept\n    : BufferedReader(static_cast<BufferedReader&&>(that)),\n      container_(that.container_),\n      flags_(that.flags_),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      truncated_(that.truncated_),\n      recycling_pool_options_(that.recycling_pool_options_),\n      decompressor_(std::move(that.decompressor_)) {}\n\ninline XzReaderBase& XzReaderBase::operator=(XzReaderBase&& that) noexcept {\n  BufferedReader::operator=(static_cast<BufferedReader&&>(that));\n  container_ = that.container_;\n  flags_ = that.flags_;\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  truncated_ = that.truncated_;\n  recycling_pool_options_ = that.recycling_pool_options_;\n  decompressor_ = std::move(that.decompressor_);\n  return *this;\n}\n\ninline void XzReaderBase::Reset(Closed) {\n  BufferedReader::Reset(kClosed);\n  container_ = Options::kDefaultContainer;\n  flags_ = 0;\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  recycling_pool_options_ = RecyclingPoolOptions();\n  decompressor_.reset();\n}\n\ninline void XzReaderBase::Reset(\n    BufferOptions buffer_options, Container container, uint32_t flags,\n    const RecyclingPoolOptions& recycling_pool_options) {\n  BufferedReader::Reset(buffer_options);\n  container_ = container;\n  flags_ = flags;\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  recycling_pool_options_ = recycling_pool_options;\n  decompressor_.reset();\n}\n\ntemplate <typename Src>\ninline XzReader<Src>::XzReader(Initializer<Src> src, Options options)\n    : XzReaderBase(options.buffer_options(), options.container(),\n                   options.concatenate() ? LZMA_CONCATENATED : 0,\n                   options.recycling_pool_options()),\n      src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void XzReader<Src>::Reset(Closed) {\n  XzReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void XzReader<Src>::Reset(Initializer<Src> src, Options options) {\n  XzReaderBase::Reset(options.buffer_options(), options.container(),\n                      options.concatenate() ? LZMA_CONCATENATED : 0,\n                      options.recycling_pool_options());\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\nvoid XzReader<Src>::Done() {\n  XzReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(AnnotateOverSrc(src_->status()));\n    }\n  }\n}\n\ntemplate <typename Src>\nvoid XzReader<Src>::SetReadAllHintImpl(bool read_all_hint) {\n  XzReaderBase::SetReadAllHintImpl(read_all_hint);\n  if (src_.IsOwning()) src_->SetReadAllHint(read_all_hint);\n}\n\ntemplate <typename Src>\nvoid XzReader<Src>::VerifyEndImpl() {\n  XzReaderBase::VerifyEndImpl();\n  if (src_.IsOwning() && ABSL_PREDICT_TRUE(ok())) src_->VerifyEnd();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_XZ_XZ_READER_H_\n"
  },
  {
    "path": "riegeli/xz/xz_writer.cc",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/xz/xz_writer.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"lzma.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/xz/xz_error.h\"\n#include \"riegeli/xz/xz_reader.h\"\n\nnamespace riegeli {\n\nstatic_assert(static_cast<int>(XzWriterBase::Container::kXz) ==\n                  static_cast<int>(XzReaderBase::Container::kXz),\n              \"Mismatched Container enums\");\nstatic_assert(static_cast<int>(XzWriterBase::Container::kLzma) ==\n                  static_cast<int>(XzReaderBase::Container::kLzma),\n              \"Mismatched Container enums\");\n\nvoid XzWriterBase::Initialize(Writer* dest, uint32_t preset, Check check,\n                              int parallelism) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of XzWriter: null Writer pointer\";\n  if (ABSL_PREDICT_FALSE(!dest->ok())) {\n    FailWithoutAnnotation(AnnotateOverDest(dest->status()));\n    return;\n  }\n  initial_compressed_pos_ = dest->pos();\n  compressor_ =\n      KeyedRecyclingPool<lzma_stream, LzmaStreamKey, LzmaStreamDeleter>::global(\n          recycling_pool_options_)\n          .Get(LzmaStreamKey(container_,\n                             container_ == Container::kXz && parallelism > 0,\n                             preset),\n               [] { return riegeli::Maker<lzma_stream>(); });\n  switch (container_) {\n    case Container::kXz: {\n      if (parallelism == 0) {\n        flush_action_ = LZMA_SYNC_FLUSH;\n        const lzma_ret liblzma_code = lzma_easy_encoder(\n            compressor_.get(), preset, static_cast<lzma_check>(check));\n        if (ABSL_PREDICT_FALSE(liblzma_code != LZMA_OK)) {\n          FailOperation(\"lzma_easy_encoder()\", liblzma_code);\n        }\n      } else {\n        // `lzma_stream_encoder_mt()` does not support `LZMA_SYNC_FLUSH`.\n        flush_action_ = LZMA_FULL_FLUSH;\n        lzma_mt mt_options{};\n        mt_options.threads = SaturatingIntCast<uint32_t>(parallelism);\n        mt_options.preset = preset;\n        mt_options.check = static_cast<lzma_check>(check);\n        const lzma_ret liblzma_code =\n            lzma_stream_encoder_mt(compressor_.get(), &mt_options);\n        if (ABSL_PREDICT_FALSE(liblzma_code != LZMA_OK)) {\n          FailOperation(\"lzma_stream_encoder_mt()\", liblzma_code);\n        }\n      }\n      return;\n    }\n    case Container::kLzma: {\n      // `lzma_alone_encoder()` does not support `LZMA_SYNC_FLUSH` nor\n      // `LZMA_FULL_FLUSH`.\n      flush_action_ = LZMA_RUN;\n      lzma_options_lzma options;\n      if (ABSL_PREDICT_FALSE(lzma_lzma_preset(&options, preset))) {\n        FailOperation(\"lzma_lzma_preset() failed\", LZMA_OPTIONS_ERROR);\n        return;\n      }\n      const lzma_ret liblzma_code =\n          lzma_alone_encoder(compressor_.get(), &options);\n      if (ABSL_PREDICT_FALSE(liblzma_code != LZMA_OK)) {\n        FailOperation(\"lzma_alone_encoder()\", liblzma_code);\n      }\n      return;\n    }\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown container format: \" << static_cast<int>(container_);\n}\n\nvoid XzWriterBase::DoneBehindBuffer(absl::string_view src) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::DoneBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  Writer& dest = *DestWriter();\n  WriteInternal(src, dest, LZMA_FINISH);\n}\n\nvoid XzWriterBase::Done() {\n  BufferedWriter::Done();\n  compressor_.reset();\n  associated_reader_.Reset();\n}\n\ninline bool XzWriterBase::FailOperation(absl::string_view operation,\n                                        lzma_ret liblzma_code) {\n  RIEGELI_ASSERT_NE(liblzma_code, LZMA_OK)\n      << \"Failed precondition of XzWriterBase::FailOperation(): \"\n         \"liblzma error code not failed\";\n  RIEGELI_ASSERT(is_open())\n      << \"Failed precondition of XzWriterBase::FailOperation(): \"\n         \"Object closed\";\n  return Fail(xz_internal::XzErrorToStatus(operation, liblzma_code));\n}\n\nabsl::Status XzWriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    status = dest.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `dest` with the compressed\n  // position. Clarify that the current position is the uncompressed position\n  // instead of delegating to `BufferedWriter::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status XzWriterBase::AnnotateOverDest(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nbool XzWriterBase::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of BufferedWriter::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedWriter::WriteInternal()\";\n  Writer& dest = *DestWriter();\n  return WriteInternal(src, dest, LZMA_RUN);\n}\n\ninline bool XzWriterBase::WriteInternal(absl::string_view src, Writer& dest,\n                                        lzma_action flush) {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of XzWriterBase::WriteInternal()\";\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  compressor_->next_in = reinterpret_cast<const uint8_t*>(src.data());\n  for (;;) {\n    compressor_->avail_in =\n        PtrDistance(reinterpret_cast<const char*>(compressor_->next_in),\n                    src.data() + src.size());\n    compressor_->next_out = reinterpret_cast<uint8_t*>(dest.cursor());\n    compressor_->avail_out = dest.available();\n    const lzma_ret liblzma_code = lzma_code(compressor_.get(), flush);\n    dest.set_cursor(reinterpret_cast<char*>(compressor_->next_out));\n    const size_t length_written = PtrDistance(\n        src.data(), reinterpret_cast<const char*>(compressor_->next_in));\n    switch (liblzma_code) {\n      case LZMA_OK:\n      case LZMA_BUF_ERROR:\n        if (compressor_->avail_out == 0) {\n          if (ABSL_PREDICT_FALSE(!dest.Push())) {\n            return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n          }\n          continue;\n        }\n        RIEGELI_ASSERT_EQ(compressor_->avail_in, 0u)\n            << \"lzma_code() returned but there are still input data \"\n               \"and output space\";\n        break;\n      case LZMA_STREAM_END:\n        break;\n      default:\n        return FailOperation(\"lzma_code()\", liblzma_code);\n    }\n    RIEGELI_ASSERT_EQ(length_written, src.size())\n        << \"lzma_code() returned but there are still input data\";\n    move_start_pos(length_written);\n    return true;\n  }\n}\n\nbool XzWriterBase::FlushBehindBuffer(absl::string_view src,\n                                     FlushType flush_type) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::FlushBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  if (src.empty() && flush_action_ == LZMA_RUN) return true;\n  Writer& dest = *DestWriter();\n  return WriteInternal(src, dest, flush_action_);\n}\n\nbool XzWriterBase::SupportsReadMode() {\n  switch (container_) {\n    case Container::kXz: {\n      Writer* const dest = DestWriter();\n      return dest != nullptr && dest->SupportsReadMode();\n    }\n    case Container::kLzma:\n      return false;\n  }\n  RIEGELI_ASSUME_UNREACHABLE()\n      << \"Unknown container format: \" << static_cast<int>(container_);\n}\n\nReader* XzWriterBase::ReadModeBehindBuffer(Position initial_pos) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::ReadModeBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!XzWriterBase::FlushBehindBuffer(\n          absl::string_view(), FlushType::kFromObject))) {\n    return nullptr;\n  }\n  Writer& dest = *DestWriter();\n  Reader* const compressed_reader = dest.ReadMode(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    return nullptr;\n  }\n  XzReader<>* const reader = associated_reader_.ResetReader(\n      compressed_reader,\n      XzReaderBase::Options()\n          .set_container(static_cast<XzReaderBase::Container>(container_))\n          .set_buffer_options(buffer_options())\n          .set_recycling_pool_options(recycling_pool_options_));\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/xz/xz_writer.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_XZ_XZ_WRITER_H_\n#define RIEGELI_XZ_XZ_WRITER_H_\n\n#include <stdint.h>\n\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"lzma.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n\nnamespace riegeli {\n\nclass Reader;\ntemplate <typename Src>\nclass XzReader;\n\n// Template parameter independent part of `XzWriter`.\nclass XzWriterBase : public BufferedWriter {\n public:\n  // Specifies what container format to write.\n  enum class Container {\n    kXz,    // Xz container (recommended).\n    kLzma,  // Lzma container (legacy file format).\n  };\n\n  // Specifies what integrity check to use.\n  enum class Check {\n    kNone = LZMA_CHECK_NONE,      // No check.\n    kCrc32 = LZMA_CHECK_CRC32,    // CRC32 (IEEE 802.3)\n    kCrc64 = LZMA_CHECK_CRC64,    // CRC64 (ECMA-182; default)\n    kSha256 = LZMA_CHECK_SHA256,  // SHA-256\n  };\n\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // What container format to write.\n    //\n    // `Flush()` is effective and `ReadMode()` is supported only with\n    // `Container::kXz`.\n    //\n    // Default: `Container::kXz`.\n    static constexpr Container kDefaultContainer = Container::kXz;\n    Options& set_container(Container container) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      container_ = container;\n      return *this;\n    }\n    Options&& set_container(Container container) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_container(container));\n    }\n    Container container() const { return container_; }\n\n    // Tunes the tradeoff between compression density and compression speed\n    // (higher = better density but slower and requires more memory for\n    // compression and decompression).\n    //\n    // `compression_level` must be between `kMinCompressionLevel` (0) and\n    // `kMaxCompressionLevel` (9). Default: `kDefaultCompressionLevel` (6).\n    static constexpr int kMinCompressionLevel = 0;\n    static constexpr int kMaxCompressionLevel = 9;\n    static constexpr int kDefaultCompressionLevel = 6;\n    Options& set_compression_level(int compression_level) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(compression_level, kMinCompressionLevel)\n          << \"Failed precondition of \"\n             \"XzWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      RIEGELI_ASSERT_LE(compression_level, kMaxCompressionLevel)\n          << \"Failed precondition of \"\n             \"XzWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      preset_ = (preset_ & ~LZMA_PRESET_LEVEL_MASK) |\n                IntCast<uint32_t>(compression_level);\n      return *this;\n    }\n    Options&& set_compression_level(int compression_level) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_compression_level(compression_level));\n    }\n    int compression_level() const {\n      return IntCast<int>(preset_ & LZMA_PRESET_LEVEL_MASK);\n    }\n\n    // Within a given compression level, further tunes the tradeoff between\n    // compression density and compression speed (`true` = better density but\n    // slower), without affecting memory requirements (only compression requires\n    // slightly more memory with compression levels <= 3).\n    Options& set_extreme(bool extreme) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      preset_ = (preset_ & LZMA_PRESET_LEVEL_MASK) |\n                (extreme ? LZMA_PRESET_EXTREME : 0);\n      return *this;\n    }\n    Options&& set_extreme(bool extreme) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_extreme(extreme));\n    }\n    bool extreme() const { return (preset_ & LZMA_PRESET_EXTREME) != 0; }\n\n    // Integrity check to use.\n    //\n    // This is effective only with `Container::kXz`.\n    //\n    // Default: `Check::kCrc64`.\n    static constexpr Check kDefaultCheck = Check::kCrc64;\n    Options& set_check(Check check) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      check_ = check;\n      return *this;\n    }\n    Options&& set_check(Check check) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_check(check));\n    }\n    Check check() const { return check_; }\n\n    // Number of background threads to use. Larger parallelism can increase\n    // throughput, up to a point where it no longer matters; smaller parallelism\n    // reduces memory usage. `parallelism() == 0` disables background threads.\n    //\n    // `parallelism() > 0` is effective only with `Container::kXz`.\n    //\n    // `parallelism() > 0` has a side effect of forcing `Flush()` to finish the\n    // current block, which degrades compression density.\n    //\n    // Default: 0.\n    Options& set_parallelism(int parallelism) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(parallelism, 0)\n          << \"Failed precondition of XzWriterBase::Options::set_parallelism(): \"\n             \"negative parallelism\";\n      parallelism_ = parallelism;\n      return *this;\n    }\n    Options&& set_parallelism(int parallelism) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_parallelism(parallelism));\n    }\n    int parallelism() const { return parallelism_; }\n\n    // Options for a global `KeyedRecyclingPool` of compression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // compression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    Options& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    Options&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    template <typename Dest>\n    friend class XzWriter;  // For `preset_`.\n\n    Container container_ = kDefaultContainer;\n    uint32_t preset_ = kDefaultCompressionLevel;\n    Check check_ = kDefaultCheck;\n    int parallelism_ = 0;\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Returns the compressed `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsReadMode() override;\n\n protected:\n  explicit XzWriterBase(Closed) noexcept : BufferedWriter(kClosed) {}\n\n  explicit XzWriterBase(BufferOptions buffer_options, Container container,\n                        const RecyclingPoolOptions& recycling_pool_options);\n\n  XzWriterBase(XzWriterBase&& that) noexcept;\n  XzWriterBase& operator=(XzWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, Container container,\n             const RecyclingPoolOptions& recycling_pool_options);\n  void Initialize(Writer* dest, uint32_t preset, Check check, int parallelism);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  void DoneBehindBuffer(absl::string_view src) override;\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool WriteInternal(absl::string_view src) override;\n  bool FlushBehindBuffer(absl::string_view src, FlushType flush_type) override;\n  Reader* ReadModeBehindBuffer(Position initial_pos) override;\n\n private:\n  struct LzmaStreamDeleter {\n    void operator()(lzma_stream* ptr) const {\n      lzma_end(ptr);\n      delete ptr;\n    }\n  };\n\n  struct LzmaStreamKey : WithEqual<LzmaStreamKey> {\n    LzmaStreamKey() = default;\n    explicit LzmaStreamKey(Container container, bool with_parallelism,\n                           uint32_t preset)\n        : container(container),\n          with_parallelism(with_parallelism),\n          preset(preset) {}\n\n    friend bool operator==(LzmaStreamKey a, LzmaStreamKey b) {\n      return a.container == b.container &&\n             a.with_parallelism == b.with_parallelism && a.preset == b.preset;\n    }\n    template <typename HashState>\n    friend HashState AbslHashValue(HashState hash_state, LzmaStreamKey self) {\n      return HashState::combine(std::move(hash_state), self.container,\n                                self.with_parallelism, self.preset);\n    }\n\n    Container container;\n    bool with_parallelism;\n    uint32_t preset;\n  };\n\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation,\n                                         lzma_ret liblzma_code);\n  bool WriteInternal(absl::string_view src, Writer& dest, lzma_action flush);\n\n  Container container_ = Container::kXz;\n  lzma_action flush_action_ = LZMA_SYNC_FLUSH;\n  RecyclingPoolOptions recycling_pool_options_;\n  Position initial_compressed_pos_ = 0;\n  KeyedRecyclingPool<lzma_stream, LzmaStreamKey, LzmaStreamDeleter>::Handle\n      compressor_;\n\n  AssociatedReader<XzReader<Reader*>> associated_reader_;\n};\n\n// A `Writer` which compresses data with Xz (LZMA) before passing it to another\n// `Writer`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Writer` must not be accessed until the `XzWriter` is closed\n// or no longer used, except that it is allowed to read the destination of the\n// compressed `Writer` immediately after `Flush()`. `Flush()` is effective only\n// with `Container::kXz`.\ntemplate <typename Dest = Writer*>\nclass XzWriter : public XzWriterBase {\n public:\n  // Creates a closed `XzWriter`.\n  explicit XzWriter(Closed) noexcept : XzWriterBase(kClosed) {}\n\n  // Will write to the compressed `Writer` provided by `dest`.\n  explicit XzWriter(Initializer<Dest> dest, Options options = Options());\n\n  XzWriter(XzWriter&& that) = default;\n  XzWriter& operator=(XzWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `XzWriter`. This avoids\n  // constructing a temporary `XzWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  // The object providing and possibly owning the compressed `Writer`.\n  Dependency<Writer*, Dest> dest_;\n};\n\nexplicit XzWriter(Closed) -> XzWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit XzWriter(Dest&& dest,\n                  XzWriterBase::Options options = XzWriterBase::Options())\n    -> XzWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline XzWriterBase::XzWriterBase(\n    BufferOptions buffer_options, Container container,\n    const RecyclingPoolOptions& recycling_pool_options)\n    : BufferedWriter(buffer_options),\n      container_(container),\n      recycling_pool_options_(recycling_pool_options) {}\n\ninline XzWriterBase::XzWriterBase(XzWriterBase&& that) noexcept\n    : BufferedWriter(static_cast<BufferedWriter&&>(that)),\n      container_(that.container_),\n      flush_action_(that.flush_action_),\n      recycling_pool_options_(that.recycling_pool_options_),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      compressor_(std::move(that.compressor_)),\n      associated_reader_(std::move(that.associated_reader_)) {}\n\ninline XzWriterBase& XzWriterBase::operator=(XzWriterBase&& that) noexcept {\n  BufferedWriter::operator=(static_cast<BufferedWriter&&>(that));\n  container_ = that.container_;\n  flush_action_ = that.flush_action_;\n  recycling_pool_options_ = that.recycling_pool_options_;\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  compressor_ = std::move(that.compressor_);\n  associated_reader_ = std::move(that.associated_reader_);\n  return *this;\n}\n\ninline void XzWriterBase::Reset(Closed) {\n  BufferedWriter::Reset(kClosed);\n  container_ = Options::kDefaultContainer;\n  flush_action_ = LZMA_SYNC_FLUSH;\n  recycling_pool_options_ = RecyclingPoolOptions();\n  initial_compressed_pos_ = 0;\n  compressor_.reset();\n  associated_reader_.Reset();\n}\n\ninline void XzWriterBase::Reset(\n    BufferOptions buffer_options, Container container,\n    const RecyclingPoolOptions& recycling_pool_options) {\n  BufferedWriter::Reset(buffer_options);\n  container_ = container;\n  flush_action_ = LZMA_SYNC_FLUSH;\n  recycling_pool_options_ = recycling_pool_options;\n  initial_compressed_pos_ = 0;\n  compressor_.reset();\n  associated_reader_.Reset();\n}\n\ntemplate <typename Dest>\ninline XzWriter<Dest>::XzWriter(Initializer<Dest> dest, Options options)\n    : XzWriterBase(options.buffer_options(), options.container(),\n                   options.recycling_pool_options()),\n      dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.preset_, options.check(),\n             options.parallelism());\n}\n\ntemplate <typename Dest>\ninline void XzWriter<Dest>::Reset(Closed) {\n  XzWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void XzWriter<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  XzWriterBase::Reset(options.buffer_options(), options.container(),\n                      options.recycling_pool_options());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.preset_, options.check(),\n             options.parallelism());\n}\n\ntemplate <typename Dest>\nvoid XzWriter<Dest>::Done() {\n  XzWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <typename Dest>\nbool XzWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!XzWriterBase::FlushImpl(flush_type))) return false;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Flush(flush_type))) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n  return true;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_XZ_XZ_WRITER_H_\n"
  },
  {
    "path": "riegeli/zlib/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"zlib_reader\",\n    srcs = [\n        \"zlib_dictionary.h\",\n        \"zlib_reader.cc\",\n    ],\n    hdrs = [\"zlib_reader.h\"],\n    deps = [\n        \":zlib_error\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:shared_ptr\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_reader\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/endian:endian_reading\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@zlib\",\n    ],\n)\n\ncc_library(\n    name = \"zlib_writer\",\n    srcs = [\n        \"zlib_dictionary.h\",\n        \"zlib_writer.cc\",\n    ],\n    hdrs = [\"zlib_writer.h\"],\n    deps = [\n        \":zlib_error\",\n        \":zlib_reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:compare\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:shared_ptr\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_writer\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@zlib\",\n    ],\n)\n\ncc_library(\n    name = \"zlib_error\",\n    srcs = [\"zlib_error.cc\"],\n    hdrs = [\"zlib_error.h\"],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \"//riegeli/base:assert\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@zlib\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/zlib/zlib_dictionary.h",
    "content": "// Copyright 2020 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_ZLIB_ZLIB_DICTIONARY_H_\n#define RIEGELI_ZLIB_ZLIB_DICTIONARY_H_\n\n// IWYU pragma: private, include \"riegeli/zlib/zlib_reader.h\"\n// IWYU pragma: private, include \"riegeli/zlib/zlib_writer.h\"\n\n#include <string>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/shared_ptr.h\"\n\nnamespace riegeli {\n\n// Stores an optional Zlib dictionary for compression and decompression.\n//\n// An empty dictionary is equivalent to having no dictionary.\n//\n// A `ZlibDictionary` object can own the dictionary data, or can hold a pointer\n// to unowned dictionary data which must not be changed until the last\n// `ZlibReader` and `ZlibWriter` using this dictionary is closed or no longer\n// used. If the same dictionary is needed for multiple compression or\n// decompression sessions, the `ZlibDictionary` object can be reused.\n//\n// Copying a `ZlibDictionary` object is cheap, sharing the actual dictionary.\nclass ZlibDictionary {\n public:\n  // Creates an empty `ZlibDictionary`.\n  ZlibDictionary() = default;\n\n  ZlibDictionary(const ZlibDictionary& that) = default;\n  ZlibDictionary& operator=(const ZlibDictionary& that) = default;\n\n  ZlibDictionary(ZlibDictionary&& that) = default;\n  ZlibDictionary& operator=(ZlibDictionary&& that) = default;\n\n  // Resets the `ZlibDictionary` to the empty state.\n  ZlibDictionary& Reset() & ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  ZlibDictionary&& Reset() && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(Reset());\n  }\n\n  // Sets a dictionary (data which should contain sequences that are commonly\n  // seen in the data being compressed).\n  ZlibDictionary& set_data(BytesInitializer data) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  ZlibDictionary&& set_data(BytesInitializer data) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_data(std::move(data)));\n  }\n\n  // Like `set_data()`, but does not take ownership of `data`, which must not be\n  // changed until the last `ZlibReader` and `ZlibWriter` using this dictionary\n  // is closed or no longer used.\n  ZlibDictionary& set_data_unowned(\n      absl::string_view data ABSL_ATTRIBUTE_LIFETIME_BOUND) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  ZlibDictionary&& set_data_unowned(\n      absl::string_view data ABSL_ATTRIBUTE_LIFETIME_BOUND) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_data_unowned(data));\n  }\n\n  // Returns `true` if no dictionary is present.\n  bool empty() const { return data_.empty(); }\n\n  // Returns the dictionary data.\n  absl::string_view data() const ABSL_ATTRIBUTE_LIFETIME_BOUND { return data_; }\n\n private:\n  SharedPtr<const std::string> owned_data_;\n  absl::string_view data_;\n};\n\n// Implementation details follow.\n\ninline ZlibDictionary& ZlibDictionary::Reset() & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  owned_data_.Reset();\n  data_ = absl::string_view();\n  return *this;\n}\n\ninline ZlibDictionary& ZlibDictionary::set_data(BytesInitializer data) &\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  owned_data_.Reset(std::move(data));\n  data_ = owned_data_->data();\n  return *this;\n}\n\ninline ZlibDictionary& ZlibDictionary::set_data_unowned(\n    absl::string_view data ABSL_ATTRIBUTE_LIFETIME_BOUND) &\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  owned_data_.Reset();\n  data_ = data;\n  return *this;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_ZLIB_ZLIB_DICTIONARY_H_\n"
  },
  {
    "path": "riegeli/zlib/zlib_error.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/zlib/zlib_error.h\"\n\n#include <string>\n\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"zlib.h\"\n\nnamespace riegeli::zlib_internal {\n\nabsl::Status ZlibErrorToStatus(absl::string_view operation, int zlib_code,\n                               const char* details) {\n  absl::StatusCode code;\n  switch (zlib_code) {\n    case Z_OK:\n      return absl::OkStatus();\n    case Z_NEED_DICT:\n    case Z_DATA_ERROR:\n      code = absl::StatusCode::kInvalidArgument;\n      break;\n    case Z_MEM_ERROR:\n      code = absl::StatusCode::kResourceExhausted;\n      break;\n    default:\n      // Should not happen.\n      code = absl::StatusCode::kInternal;\n      break;\n  }\n  std::string message = absl::StrCat(operation, \" failed\");\n  if (details == nullptr) {\n    switch (zlib_code) {\n      case Z_OK:\n        RIEGELI_ASSUME_UNREACHABLE() << \"Handled before switch\";\n      case Z_STREAM_END:\n        details = \"stream end\";\n        break;\n      case Z_NEED_DICT:\n        details = \"need dictionary\";\n        break;\n      case Z_ERRNO:\n        details = \"file error\";\n        break;\n      case Z_STREAM_ERROR:\n        details = \"stream error\";\n        break;\n      case Z_DATA_ERROR:\n        details = \"data error\";\n        break;\n      case Z_MEM_ERROR:\n        details = \"insufficient memory\";\n        break;\n      case Z_BUF_ERROR:\n        details = \"buffer error\";\n        break;\n      case Z_VERSION_ERROR:\n        details = \"incompatible version\";\n        break;\n      default:\n        absl::StrAppend(&message, \": unknown zlib error code: \", zlib_code);\n        break;\n    }\n  }\n  if (details != nullptr) absl::StrAppend(&message, \": \", details);\n  return absl::Status(code, message);\n}\n\n}  // namespace riegeli::zlib_internal\n"
  },
  {
    "path": "riegeli/zlib/zlib_error.h",
    "content": "// Copyright 2023 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_ZLIB_ZLIB_ERROR_H_\n#define RIEGELI_ZLIB_ZLIB_ERROR_H_\n\n#include \"absl/base/attributes.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n\nnamespace riegeli::zlib_internal {\n\nABSL_ATTRIBUTE_COLD absl::Status ZlibErrorToStatus(absl::string_view operation,\n                                                   int zlib_code,\n                                                   const char* details);\n\n}  // namespace riegeli::zlib_internal\n\n#endif  // RIEGELI_ZLIB_ZLIB_ERROR_H_\n"
  },
  {
    "path": "riegeli/zlib/zlib_reader.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/zlib/zlib_reader.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/endian/endian_reading.h\"\n#include \"riegeli/zlib/zlib_error.h\"\n#include \"zconf.h\"\n#include \"zlib.h\"\n\nnamespace riegeli {\n\nstatic_assert(ZlibReaderBase::Options::kMaxWindowLog == MAX_WBITS,\n              \"Mismatched constant\");\nstatic_assert(ZlibReaderBase::Options::kDefaultWindowLog == MAX_WBITS,\n              \"Mismatched constant\");\n\nvoid ZlibReaderBase::ZStreamDeleter::operator()(void* ptr) const {\n  z_stream* const z_stream_ptr = static_cast<z_stream*>(ptr);\n  const int zlib_code = inflateEnd(z_stream_ptr);\n  RIEGELI_ASSERT_EQ(zlib_code, Z_OK) << \"inflateEnd() failed\";\n  delete z_stream_ptr;\n}\n\nvoid ZlibReaderBase::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of ZlibReader: null Reader pointer\";\n  if (ABSL_PREDICT_FALSE(!src->ok()) && src->available() == 0) {\n    FailWithoutAnnotation(AnnotateOverSrc(src->status()));\n    return;\n  }\n  initial_compressed_pos_ = src->pos();\n  InitializeDecompressor();\n}\n\ninline void ZlibReaderBase::InitializeDecompressor() {\n  decompressor_ =\n      RecyclingPool<void, ZStreamDeleter>::global(recycling_pool_options_)\n          .Get(\n              [&] {\n                auto ptr =\n                    riegeli::Maker<z_stream>().UniquePtr<ZStreamDeleter>();\n                const int zlib_code = inflateInit2(ptr.get(), window_bits_);\n                if (ABSL_PREDICT_FALSE(zlib_code != Z_OK)) {\n                  FailOperation(\"inflateInit2()\", zlib_code);\n                }\n                return ptr;\n              },\n              [&](void* ptr) {\n                z_stream* const z_stream_ptr = static_cast<z_stream*>(ptr);\n                const int zlib_code = inflateReset2(z_stream_ptr, window_bits_);\n                if (ABSL_PREDICT_FALSE(zlib_code != Z_OK)) {\n                  FailOperation(\"inflateReset2()\", zlib_code);\n                }\n              });\n}\n\nvoid ZlibReaderBase::Done() {\n  if (ABSL_PREDICT_FALSE(truncated_)) {\n    Reader& src = *SrcReader();\n    FailWithoutAnnotation(AnnotateOverSrc(src.AnnotateStatus(\n        absl::InvalidArgumentError(\"Truncated Zlib-compressed stream\"))));\n  }\n  BufferedReader::Done();\n  decompressor_.reset();\n  dictionary_ = ZlibDictionary();\n}\n\ninline bool ZlibReaderBase::FailOperation(absl::string_view operation,\n                                          int zlib_code) {\n  RIEGELI_ASSERT_NE(zlib_code, Z_OK)\n      << \"Failed precondition of ZlibReaderBase::FailOperation(): \"\n         \"zlib error code not failed\";\n  RIEGELI_ASSERT(is_open())\n      << \"Failed precondition of ZlibReaderBase::FailOperation(): \"\n         \"Object closed\";\n  z_stream* const z_stream_ptr = static_cast<z_stream*>(decompressor_.get());\n  return Fail(zlib_internal::ZlibErrorToStatus(operation, zlib_code,\n                                               z_stream_ptr->msg));\n}\n\nabsl::Status ZlibReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    if (ABSL_PREDICT_FALSE(truncated_)) {\n      status = Annotate(status, \"reading truncated Zlib-compressed stream\");\n    }\n    Reader& src = *SrcReader();\n    status = src.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `src` with the compressed position.\n  // Clarify that the current position is the uncompressed position instead of\n  // delegating to `BufferedReader::AnnotateStatusImpl()`.\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status ZlibReaderBase::AnnotateOverSrc(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nbool ZlibReaderBase::ReadInternal(size_t min_length, size_t max_length,\n                                  char* dest) {\n  RIEGELI_ASSERT_GT(min_length, 0u)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"nothing to read\";\n  RIEGELI_ASSERT_GE(max_length, min_length)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"max_length < min_length\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::ReadInternal()\";\n  Reader& src = *SrcReader();\n  truncated_ = false;\n  max_length = UnsignedMin(max_length,\n                           std::numeric_limits<Position>::max() - limit_pos());\n  z_stream* const z_stream_ptr = static_cast<z_stream*>(decompressor_.get());\n  z_stream_ptr->next_out = reinterpret_cast<Bytef*>(dest);\n  for (;;) {\n    z_stream_ptr->avail_out = SaturatingIntCast<uInt>(PtrDistance(\n        reinterpret_cast<char*>(z_stream_ptr->next_out), dest + max_length));\n    z_stream_ptr->next_in = const_cast<z_const Bytef*>(\n        reinterpret_cast<const Bytef*>(src.cursor()));\n    z_stream_ptr->avail_in = SaturatingIntCast<uInt>(src.available());\n    if (z_stream_ptr->avail_in > 0) stream_had_data_ = true;\n    int zlib_code = inflate(z_stream_ptr, Z_NO_FLUSH);\n    src.set_cursor(reinterpret_cast<const char*>(z_stream_ptr->next_in));\n    const size_t length_read =\n        PtrDistance(dest, reinterpret_cast<char*>(z_stream_ptr->next_out));\n    switch (zlib_code) {\n      case Z_OK:\n        if (length_read >= min_length) break;\n        ABSL_FALLTHROUGH_INTENDED;\n      case Z_BUF_ERROR:\n        if (ABSL_PREDICT_FALSE(z_stream_ptr->avail_in > 0)) {\n          RIEGELI_ASSERT_EQ(z_stream_ptr->avail_out, 0u)\n              << \"inflate() returned but there are still input data \"\n                 \"and output space\";\n          RIEGELI_ASSERT_EQ(length_read,\n                            std::numeric_limits<Position>::max() - limit_pos())\n              << \"The position does not overflow but the output buffer is \"\n                 \"full, while less than min_length was output, which is \"\n                 \"impossible because the buffer has size max_length which is \"\n                 \"at least min_length if the position does not overflow\";\n          move_limit_pos(length_read);\n          return FailOverflow();\n        }\n        if (ABSL_PREDICT_FALSE(!src.Pull())) {\n          move_limit_pos(length_read);\n          if (ABSL_PREDICT_FALSE(!src.ok())) {\n            return FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n          }\n          if (ABSL_PREDICT_FALSE(!concatenate_ || stream_had_data_)) {\n            truncated_ = true;\n          }\n          return false;\n        }\n        continue;\n      case Z_STREAM_END:\n        if (concatenate_) {\n          const int zlib_code = inflateReset(z_stream_ptr);\n          if (ABSL_PREDICT_FALSE(zlib_code != Z_OK)) {\n            FailOperation(\"inflateReset()\", zlib_code);\n            break;\n          }\n          stream_had_data_ = false;\n          if (length_read >= min_length) break;\n          continue;\n        }\n        decompressor_.reset();\n        move_limit_pos(length_read);\n        // Avoid `BufferedReader` allocating another buffer.\n        set_exact_size(limit_pos());\n        return length_read >= min_length;\n      case Z_NEED_DICT:\n        if (ABSL_PREDICT_TRUE(!dictionary_.empty())) {\n          zlib_code = inflateSetDictionary(\n              z_stream_ptr,\n              const_cast<z_const Bytef*>(\n                  reinterpret_cast<const Bytef*>(dictionary_.data().data())),\n              SaturatingIntCast<uInt>(dictionary_.data().size()));\n          if (ABSL_PREDICT_FALSE(zlib_code != Z_OK)) {\n            FailOperation(\"inflateSetDictionary()\", zlib_code);\n            break;\n          }\n          continue;\n        }\n        ABSL_FALLTHROUGH_INTENDED;\n      default:\n        FailOperation(\"inflate()\", zlib_code);\n        break;\n    }\n    move_limit_pos(length_read);\n    return length_read >= min_length;\n  }\n}\n\nvoid ZlibReaderBase::ExactSizeReached() {\n  if (decompressor_ == nullptr) return;\n  char buffer[1];\n  if (ABSL_PREDICT_FALSE(ZlibReaderBase::ReadInternal(1, 1, buffer))) {\n    decompressor_.reset();\n    Fail(absl::FailedPreconditionError(\n        \"Uncompressed size reached but more data can be decompressed, \"\n        \"which implies that seeking back and reading again encountered \"\n        \"changed Zlib-compressed data\"));\n  }\n}\n\nbool ZlibReaderBase::ToleratesReadingAhead() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->ToleratesReadingAhead();\n}\n\nbool ZlibReaderBase::SupportsRewind() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRewind();\n}\n\nbool ZlibReaderBase::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (new_pos <= limit_pos()) {\n    // Seeking backwards.\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    Reader& src = *SrcReader();\n    truncated_ = false;\n    stream_had_data_ = false;\n    set_buffer();\n    set_limit_pos(0);\n    decompressor_.reset();\n    if (ABSL_PREDICT_FALSE(!src.Seek(initial_compressed_pos_))) {\n      return FailWithoutAnnotation(AnnotateOverSrc(src.StatusOrAnnotate(\n          absl::DataLossError(\"Zlib-compressed stream got truncated\"))));\n    }\n    InitializeDecompressor();\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    if (new_pos == 0) return true;\n  }\n  return BufferedReader::SeekBehindBuffer(new_pos);\n}\n\nbool ZlibReaderBase::SupportsNewReader() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsNewReader();\n}\n\nstd::unique_ptr<Reader> ZlibReaderBase::NewReaderImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point\n  // if `SrcReader()->SupportsNewReader()`.\n  Reader& src = *SrcReader();\n  std::unique_ptr<Reader> compressed_reader =\n      src.NewReader(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    return nullptr;\n  }\n  std::unique_ptr<Reader> reader =\n      std::make_unique<ZlibReader<std::unique_ptr<Reader>>>(\n          std::move(compressed_reader),\n          ZlibReaderBase::Options()\n              .set_header(window_bits_ < 0\n                              ? Header::kRaw\n                              : static_cast<Header>(window_bits_ & ~15))\n              .set_window_log(window_bits_ < 0 ? -window_bits_\n                                               : window_bits_ & 15)\n              .set_concatenate(concatenate_)\n              .set_dictionary(dictionary_)\n              .set_buffer_options(buffer_options())\n              .set_recycling_pool_options(recycling_pool_options_));\n  reader->Seek(initial_pos);\n  return reader;\n}\n\nbool RecognizeZlib(Reader& src, ZlibReaderBase::Header header,\n                   const RecyclingPoolOptions& recycling_pool_options) {\n  RIEGELI_ASSERT_NE(header, ZlibReaderBase::Header::kRaw)\n      << \"Failed precondition of RecognizeZlib(): \"\n         \"Header::kRaw cannot be reliably detected\";\n  using ZStreamDeleter = ZlibReaderBase::ZStreamDeleter;\n  // If `header == Header::kRaw` then `window_bits == -1`, which causes\n  // `inflateInit2()` or `inflateReset2()` to fail.\n  const int window_bits = static_cast<int>(header);\n  int zlib_code;\n  const RecyclingPool<z_stream, ZStreamDeleter>::Handle decompressor =\n      RecyclingPool<z_stream, ZStreamDeleter>::global(recycling_pool_options)\n          .Get(\n              [&] {\n                auto ptr =\n                    riegeli::Maker<z_stream>().UniquePtr<ZStreamDeleter>();\n                zlib_code = inflateInit2(ptr.get(), window_bits);\n                return ptr;\n              },\n              [&](z_stream* ptr) {\n                zlib_code = inflateReset2(ptr, window_bits);\n              });\n  if (ABSL_PREDICT_FALSE(zlib_code != Z_OK)) return false;\n\n  char dest[1];\n  size_t cursor_index = 0;\n  decompressor->next_out = reinterpret_cast<Bytef*>(dest);\n  decompressor->avail_out = 1;\n  for (;;) {\n    decompressor->next_in = const_cast<z_const Bytef*>(\n        reinterpret_cast<const Bytef*>(src.cursor() + cursor_index));\n    decompressor->avail_in =\n        SaturatingIntCast<uInt>(src.available() - cursor_index);\n    // `Z_BLOCK` stops after decoding the header.\n    switch (inflate(decompressor.get(), Z_BLOCK)) {\n      case Z_OK:\n        if (\n            // Decoded the header.\n            (decompressor->data_type & 128) != 0 ||\n            // Output a byte. This is impossible if `header != Header::kRaw`;\n            // kept for robustness.\n            decompressor->avail_out < 1) {\n          return true;\n        }\n        ABSL_FALLTHROUGH_INTENDED;\n      case Z_BUF_ERROR:\n        RIEGELI_ASSERT_EQ(decompressor->avail_in, 0u)\n            << \"inflate() returned but there are still input data\";\n        cursor_index = src.available();\n        if (ABSL_PREDICT_FALSE(!src.Pull(cursor_index + 1))) return false;\n        continue;\n      case Z_STREAM_END:  // This is impossible if `header != Header::kRaw`;\n                          // kept for robustness.\n      case Z_NEED_DICT:\n        return true;\n      default:\n        return false;\n    }\n  }\n}\n\nstd::optional<uint32_t> GzipUncompressedSizeModulo4G(Reader& src) {\n  RIEGELI_ASSERT(src.SupportsRandomAccess())\n      << \"Failed precondition of GzipUncompressedSizeModulo4G(): \"\n         \"Reader does not support random access\";\n  const std::optional<Position> compressed_size = src.Size();\n  if (ABSL_PREDICT_FALSE(compressed_size == std::nullopt ||\n                         *compressed_size < 20)) {\n    return std::nullopt;\n  }\n  const Position pos_before = src.pos();\n  uint32_t uncompressed_size;\n  if (ABSL_PREDICT_FALSE(!src.Seek(*compressed_size - sizeof(uint32_t)) ||\n                         !ReadLittleEndian<uint32_t>(src, uncompressed_size) ||\n                         !src.Seek(pos_before))) {\n    return std::nullopt;\n  }\n  return uncompressed_size;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/zlib/zlib_reader.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_ZLIB_ZLIB_READER_H_\n#define RIEGELI_ZLIB_ZLIB_READER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/zlib/zlib_dictionary.h\"  // IWYU pragma: export\n\nnamespace riegeli {\n\n// Template parameter independent part of `ZlibReader`.\nclass ZlibReaderBase : public BufferedReader {\n public:\n  // Specifies what format of header to expect.\n  enum class Header {\n    kZlib = 0,         // Zlib header.\n    kGzip = 16,        // Gzip header.\n    kZlibOrGzip = 32,  // Zlib or Gzip header.\n    kRaw = -1,         // No header; compressor must write no header too.\n  };\n\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // What format of header to expect.\n    //\n    // Default: `Header::kZlibOrGzip`.\n    static constexpr Header kDefaultHeader = Header::kZlibOrGzip;\n    Options& set_header(Header header) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      header_ = header;\n      return *this;\n    }\n    Options&& set_header(Header header) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_header(header));\n    }\n    Header header() const { return header_; }\n\n    // Maximum acceptable logarithm of the LZ77 sliding window size.\n    //\n    // `window_log` must be between `kMinWindowLog` (9) and\n    // `kMaxWindowLog` (15). Default: `kDefaultWindowLog` (15).\n    static constexpr int kMinWindowLog = 9;\n    static constexpr int kMaxWindowLog = 15;      // `MAX_WBITS`\n    static constexpr int kDefaultWindowLog = 15;  // `MAX_WBITS`\n    Options& set_window_log(int window_log) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(window_log, kMinWindowLog)\n          << \"Failed precondition of \"\n             \"ZlibReaderBase::Options::set_window_log(): \"\n             \"window log out of range\";\n      RIEGELI_ASSERT_LE(window_log, kMaxWindowLog)\n          << \"Failed precondition of \"\n             \"ZlibReaderBase::Options::set_window_log(): \"\n             \"window log out of range\";\n      window_log_ = window_log;\n      return *this;\n    }\n    Options&& set_window_log(int window_log) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_window_log(window_log));\n    }\n    int window_log() const { return window_log_; }\n\n    // If `true`, concatenated compressed streams are decoded to concatenation\n    // of their decompressed contents. An empty compressed stream is decoded to\n    // empty decompressed contents.\n    //\n    // If `false`, exactly one compressed stream is consumed.\n    //\n    // Default: `false`.\n    Options& set_concatenate(bool concatenate) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      concatenate_ = concatenate;\n      return *this;\n    }\n    Options&& set_concatenate(bool concatenate) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_concatenate(concatenate));\n    }\n    bool concatenate() const { return concatenate_; }\n\n    // Zlib dictionary. The same dictionary must have been used for compression,\n    // except that it is allowed to supply a dictionary for decompression even\n    // if no dictionary was used for compression.\n    //\n    // Default: `ZlibDictionary()`.\n    Options& set_dictionary(ZlibDictionary dictionary) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      dictionary_ = std::move(dictionary);\n      return *this;\n    }\n    Options&& set_dictionary(ZlibDictionary dictionary) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_dictionary(std::move(dictionary)));\n    }\n    ZlibDictionary& dictionary() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n    const ZlibDictionary& dictionary() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n\n    // Options for a global `RecyclingPool` of decompression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // decompression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    Options& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    Options&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    Header header_ = kDefaultHeader;\n    int window_log_ = kDefaultWindowLog;\n    bool concatenate_ = false;\n    ZlibDictionary dictionary_;\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Returns the compressed `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns `true` if the source is truncated (without a clean end of the\n  // compressed stream) at the current position. In such case, if the source\n  // does not grow, `Close()` will fail.\n  //\n  // Precondition: `Options::concatenate()` was `false`.\n  bool truncated() const {\n    RIEGELI_ASSERT(!concatenate_)\n        << \"Failed precondition of ZlibReaderBase::truncated(): \"\n           \"Options::concatenate() is true\";\n    return truncated_ && available() == 0;\n  }\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRewind() override;\n  bool SupportsNewReader() override;\n\n protected:\n  explicit ZlibReaderBase(Closed) noexcept : BufferedReader(kClosed) {}\n\n  explicit ZlibReaderBase(BufferOptions buffer_options, int window_bits,\n                          bool concatenate, ZlibDictionary&& dictionary,\n                          const RecyclingPoolOptions& recycling_pool_options);\n\n  ZlibReaderBase(ZlibReaderBase&& that) noexcept;\n  ZlibReaderBase& operator=(ZlibReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, int window_bits, bool concatenate,\n             ZlibDictionary&& dictionary,\n             const RecyclingPoolOptions& recycling_pool_options);\n  static int GetWindowBits(const Options& options);\n  void Initialize(Reader* src);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool ReadInternal(size_t min_length, size_t max_length, char* dest) override;\n  void ExactSizeReached() override;\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  // For `ZStreamDeleter`.\n  friend bool RecognizeZlib(Reader& src, ZlibReaderBase::Header header,\n                            const RecyclingPoolOptions& recycling_pool_options);\n\n  struct ZStreamDeleter {\n    // `void*` is `z_stream*`. Avoid including `zlib.h` in the header.\n    void operator()(void* ptr) const;\n  };\n\n  void InitializeDecompressor();\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation,\n                                         int zlib_code);\n\n  int window_bits_ = 0;\n  bool concatenate_ = false;\n  Position initial_compressed_pos_ = 0;\n  // If `true`, the source is truncated (without a clean end of the compressed\n  // stream) at the current position. If the source does not grow, `Close()`\n  // will fail.\n  bool truncated_ = false;\n  // If `true`, some compressed data from the current stream were processed.\n  // If `concatenate_` and `!stream_had_data_`, an end of the source is\n  // legitimate, it does not imply that the source is truncated.\n  bool stream_had_data_ = false;\n  ZlibDictionary dictionary_;\n  RecyclingPoolOptions recycling_pool_options_;\n  // If `ok()` but `decompressor_ == nullptr` then all data have been\n  // decompressed, `exact_size() == limit_pos()`, and `ReadInternal()` must not\n  // be called again.\n  //\n  // `void` is `z_stream`. Avoid including `zlib.h` in the header.\n  RecyclingPool<void, ZStreamDeleter>::Handle decompressor_;\n};\n\n// A `Reader` which decompresses data with Zlib after getting it from another\n// `Reader`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Reader` must not be accessed until the `ZlibReader` is closed\n// or no longer used.\ntemplate <typename Src = Reader*>\nclass ZlibReader : public ZlibReaderBase {\n public:\n  // Creates a closed `ZlibReader`.\n  explicit ZlibReader(Closed) noexcept : ZlibReaderBase(kClosed) {}\n\n  // Will read from the compressed `Reader` provided by `src`.\n  explicit ZlibReader(Initializer<Src> src, Options options = Options());\n\n  ZlibReader(ZlibReader&&) = default;\n  ZlibReader& operator=(ZlibReader&&) = default;\n\n  // Makes `*this` equivalent to a newly constructed `ZlibReader`. This avoids\n  // constructing a temporary `ZlibReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  void VerifyEndImpl() override;\n\n private:\n  // The object providing and possibly owning the compressed `Reader`.\n  Dependency<Reader*, Src> src_;\n};\n\nexplicit ZlibReader(Closed) -> ZlibReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit ZlibReader(Src&& src,\n                    ZlibReaderBase::Options options = ZlibReaderBase::Options())\n    -> ZlibReader<TargetT<Src>>;\n\n// Returns `true` if the data look like they have been Zlib-compressed.\n//\n// The current position of `src` is unchanged.\n//\n// Precondition: `header != ZlibReaderBase::Header::kRaw`\nbool RecognizeZlib(\n    Reader& src,\n    ZlibReaderBase::Header header = ZlibReaderBase::Header::kZlibOrGzip,\n    const RecyclingPoolOptions& recycling_pool_options =\n        RecyclingPoolOptions());\nbool RecognizeZlib(Reader& src,\n                   const RecyclingPoolOptions& recycling_pool_options);\n\n// Returns the claimed uncompressed size of Gzip-compressed data (with\n// `ZlibWriterBase::Header::kGzip`) modulo 4G. The compressed stream must not\n// have anything appended.\n//\n// If the data consists of multiple streams, only the last stream is considered.\n//\n// Returns `std::nullopt` on failure. If the data are not Gzip-compressed, or\n// have something appended, then this is generally not detected and the returned\n// value will be meaningless. If the data were longer than 4G, then only the\n// lowest 32 bits are returned.\n//\n// The current position of `src` is unchanged.\n//\n// Precondition: `src.SupportsRandomAccess()`\nstd::optional<uint32_t> GzipUncompressedSizeModulo4G(Reader& src);\n\n// Implementation details follow.\n\ninline ZlibReaderBase::ZlibReaderBase(\n    BufferOptions buffer_options, int window_bits, bool concatenate,\n    ZlibDictionary&& dictionary,\n    const RecyclingPoolOptions& recycling_pool_options)\n    : BufferedReader(buffer_options),\n      window_bits_(window_bits),\n      concatenate_(concatenate),\n      dictionary_(std::move(dictionary)),\n      recycling_pool_options_(recycling_pool_options) {}\n\ninline ZlibReaderBase::ZlibReaderBase(ZlibReaderBase&& that) noexcept\n    : BufferedReader(static_cast<BufferedReader&&>(that)),\n      window_bits_(that.window_bits_),\n      concatenate_(that.concatenate_),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      truncated_(that.truncated_),\n      stream_had_data_(that.stream_had_data_),\n      dictionary_(std::move(that.dictionary_)),\n      recycling_pool_options_(that.recycling_pool_options_),\n      decompressor_(std::move(that.decompressor_)) {}\n\ninline ZlibReaderBase& ZlibReaderBase::operator=(\n    ZlibReaderBase&& that) noexcept {\n  BufferedReader::operator=(static_cast<BufferedReader&&>(that));\n  window_bits_ = that.window_bits_;\n  concatenate_ = that.concatenate_;\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  truncated_ = that.truncated_;\n  stream_had_data_ = that.stream_had_data_;\n  dictionary_ = std::move(that.dictionary_);\n  recycling_pool_options_ = that.recycling_pool_options_;\n  decompressor_ = std::move(that.decompressor_);\n  return *this;\n}\n\ninline void ZlibReaderBase::Reset(Closed) {\n  BufferedReader::Reset(kClosed);\n  window_bits_ = 0;\n  concatenate_ = false;\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  stream_had_data_ = false;\n  recycling_pool_options_ = RecyclingPoolOptions();\n  decompressor_.reset();\n  // Must be destroyed after `decompressor_`.\n  dictionary_ = ZlibDictionary();\n}\n\ninline void ZlibReaderBase::Reset(\n    BufferOptions buffer_options, int window_bits, bool concatenate,\n    ZlibDictionary&& dictionary,\n    const RecyclingPoolOptions& recycling_pool_options) {\n  BufferedReader::Reset(buffer_options);\n  window_bits_ = window_bits;\n  concatenate_ = concatenate;\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  stream_had_data_ = false;\n  recycling_pool_options_ = recycling_pool_options;\n  decompressor_.reset();\n  // Must be destroyed after `decompressor_`.\n  dictionary_ = std::move(dictionary);\n}\n\ninline int ZlibReaderBase::GetWindowBits(const Options& options) {\n  return options.header() == Header::kRaw\n             ? -options.window_log()\n             : options.window_log() + static_cast<int>(options.header());\n}\n\ntemplate <typename Src>\ninline ZlibReader<Src>::ZlibReader(Initializer<Src> src, Options options)\n    : ZlibReaderBase(options.buffer_options(), GetWindowBits(options),\n                     options.concatenate(), std::move(options.dictionary()),\n                     options.recycling_pool_options()),\n      src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void ZlibReader<Src>::Reset(Closed) {\n  ZlibReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void ZlibReader<Src>::Reset(Initializer<Src> src, Options options) {\n  ZlibReaderBase::Reset(options.buffer_options(), GetWindowBits(options),\n                        options.concatenate(), std::move(options.dictionary()),\n                        options.recycling_pool_options());\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\nvoid ZlibReader<Src>::Done() {\n  ZlibReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(AnnotateOverSrc(src_->status()));\n    }\n  }\n}\n\ntemplate <typename Src>\nvoid ZlibReader<Src>::SetReadAllHintImpl(bool read_all_hint) {\n  ZlibReaderBase::SetReadAllHintImpl(read_all_hint);\n  if (src_.IsOwning()) src_->SetReadAllHint(read_all_hint);\n}\n\ntemplate <typename Src>\nvoid ZlibReader<Src>::VerifyEndImpl() {\n  ZlibReaderBase::VerifyEndImpl();\n  if (src_.IsOwning() && ABSL_PREDICT_TRUE(ok())) src_->VerifyEnd();\n}\n\ninline bool RecognizeZlib(Reader& src,\n                          const RecyclingPoolOptions& recycling_pool_options) {\n  return RecognizeZlib(src, ZlibReaderBase::Header::kZlibOrGzip,\n                       recycling_pool_options);\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_ZLIB_ZLIB_READER_H_\n"
  },
  {
    "path": "riegeli/zlib/zlib_writer.cc",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"riegeli/zlib/zlib_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/zlib/zlib_error.h\"\n#include \"riegeli/zlib/zlib_reader.h\"\n#include \"zconf.h\"\n#include \"zlib.h\"\n\nnamespace riegeli {\n\nstatic_assert(ZlibWriterBase::Options::kMinCompressionLevel == Z_NO_COMPRESSION,\n              \"Mismatched constant\");\nstatic_assert(ZlibWriterBase::Options::kMaxCompressionLevel ==\n                  Z_BEST_COMPRESSION,\n              \"Mismatched constant\");\nstatic_assert(ZlibWriterBase::Options::kMaxWindowLog == MAX_WBITS,\n              \"Mismatched constant\");\nstatic_assert(ZlibWriterBase::Options::kDefaultWindowLog == MAX_WBITS,\n              \"Mismatched constant\");\n\nvoid ZlibWriterBase::ZStreamDeleter::operator()(void* ptr) const {\n  z_stream* const z_stream_ptr = static_cast<z_stream*>(ptr);\n  const int zlib_code = deflateEnd(z_stream_ptr);\n  RIEGELI_ASSERT(zlib_code == Z_OK || zlib_code == Z_DATA_ERROR)\n      << \"deflateEnd() failed: \" << zlib_code;\n  delete z_stream_ptr;\n}\n\nvoid ZlibWriterBase::Initialize(Writer* dest, int compression_level) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of ZlibWriter: null Writer pointer\";\n  if (ABSL_PREDICT_FALSE(!dest->ok())) {\n    FailWithoutAnnotation(AnnotateOverDest(dest->status()));\n    return;\n  }\n  initial_compressed_pos_ = dest->pos();\n  compressor_ =\n      KeyedRecyclingPool<void, ZStreamKey, ZStreamDeleter>::global(\n          recycling_pool_options_)\n          .Get(\n              ZStreamKey(compression_level, window_bits_),\n              [&] {\n                auto ptr =\n                    riegeli::Maker<z_stream>().UniquePtr<ZStreamDeleter>();\n                const int zlib_code =\n                    deflateInit2(ptr.get(), compression_level, Z_DEFLATED,\n                                 window_bits_, 8, Z_DEFAULT_STRATEGY);\n                if (ABSL_PREDICT_FALSE(zlib_code != Z_OK)) {\n                  FailOperation(\"deflateInit2()\", zlib_code);\n                }\n                return ptr;\n              },\n              [&](void* ptr) {\n                z_stream* const z_stream_ptr = static_cast<z_stream*>(ptr);\n                const int zlib_code = deflateReset(z_stream_ptr);\n                if (ABSL_PREDICT_FALSE(zlib_code != Z_OK)) {\n                  FailOperation(\"deflateReset()\", zlib_code);\n                }\n              });\n  if (!dictionary_.empty()) {\n    z_stream* const z_stream_ptr = static_cast<z_stream*>(compressor_.get());\n    const int zlib_code = deflateSetDictionary(\n        z_stream_ptr,\n        const_cast<z_const Bytef*>(\n            reinterpret_cast<const Bytef*>(dictionary_.data().data())),\n        SaturatingIntCast<uInt>(dictionary_.data().size()));\n    if (ABSL_PREDICT_FALSE(zlib_code != Z_OK)) {\n      FailOperation(\"deflateSetDictionary()\", zlib_code);\n    }\n  }\n}\n\nvoid ZlibWriterBase::DoneBehindBuffer(absl::string_view src) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::DoneBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  Writer& dest = *DestWriter();\n  WriteInternal(src, dest, Z_FINISH);\n}\n\nvoid ZlibWriterBase::Done() {\n  BufferedWriter::Done();\n  compressor_.reset();\n  dictionary_ = ZlibDictionary();\n  associated_reader_.Reset();\n}\n\ninline bool ZlibWriterBase::FailOperation(absl::string_view operation,\n                                          int zlib_code) {\n  RIEGELI_ASSERT_NE(zlib_code, Z_OK)\n      << \"Failed precondition of ZlibWriterBase::FailOperation(): \"\n         \"zlib error code not failed\";\n  RIEGELI_ASSERT(is_open())\n      << \"Failed precondition of ZlibWriterBase::FailOperation(): \"\n         \"Object closed\";\n  z_stream* const z_stream_ptr = static_cast<z_stream*>(compressor_.get());\n  return Fail(zlib_internal::ZlibErrorToStatus(operation, zlib_code,\n                                               z_stream_ptr->msg));\n}\n\nabsl::Status ZlibWriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    status = dest.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `dest` with the compressed\n  // position. Clarify that the current position is the uncompressed position\n  // instead of delegating to `BufferedWriter::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status ZlibWriterBase::AnnotateOverDest(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nbool ZlibWriterBase::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of BufferedWriter::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedWriter::WriteInternal()\";\n  Writer& dest = *DestWriter();\n  return WriteInternal(src, dest, Z_NO_FLUSH);\n}\n\ninline bool ZlibWriterBase::WriteInternal(absl::string_view src, Writer& dest,\n                                          int flush) {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of ZlibWriterBase::WriteInternal()\";\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  z_stream* const z_stream_ptr = static_cast<z_stream*>(compressor_.get());\n  z_stream_ptr->next_in =\n      const_cast<z_const Bytef*>(reinterpret_cast<const Bytef*>(src.data()));\n  for (;;) {\n    // If `z_stream_ptr->avail_out == 0` then `deflate()` returns `Z_BUF_ERROR`,\n    // so `dest.Push()` first.\n    if (ABSL_PREDICT_FALSE(!dest.Push())) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    }\n    size_t avail_in =\n        PtrDistance(reinterpret_cast<const char*>(z_stream_ptr->next_in),\n                    src.data() + src.size());\n    int op = flush;\n    if (ABSL_PREDICT_FALSE(avail_in > std::numeric_limits<uInt>::max())) {\n      avail_in = size_t{std::numeric_limits<uInt>::max()};\n      op = Z_NO_FLUSH;\n    }\n    z_stream_ptr->avail_in = IntCast<uInt>(avail_in);\n    z_stream_ptr->next_out = reinterpret_cast<Bytef*>(dest.cursor());\n    z_stream_ptr->avail_out = SaturatingIntCast<uInt>(dest.available());\n    const int zlib_code = deflate(z_stream_ptr, op);\n    dest.set_cursor(reinterpret_cast<char*>(z_stream_ptr->next_out));\n    const size_t length_written = PtrDistance(\n        src.data(), reinterpret_cast<const char*>(z_stream_ptr->next_in));\n    switch (zlib_code) {\n      case Z_OK:\n        if (z_stream_ptr->avail_out == 0) continue;\n        RIEGELI_ASSERT_EQ(z_stream_ptr->avail_in, 0u)\n            << \"deflate() returned but there are still input data \"\n               \"and output space\";\n        if (ABSL_PREDICT_FALSE(length_written < src.size())) continue;\n        break;\n      case Z_STREAM_END:\n        break;\n      case Z_BUF_ERROR:\n        RIEGELI_ASSERT_EQ(op, Z_SYNC_FLUSH)\n            << \"deflate() returned an unexpected Z_BUF_ERROR\";\n        break;\n      default:\n        return FailOperation(\"deflate()\", zlib_code);\n    }\n    RIEGELI_ASSERT_EQ(length_written, src.size())\n        << \"deflate() returned but there are still input data\";\n    move_start_pos(length_written);\n    return true;\n  }\n}\n\nbool ZlibWriterBase::FlushBehindBuffer(absl::string_view src,\n                                       FlushType flush_type) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::FlushBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  return WriteInternal(src, dest, Z_SYNC_FLUSH);\n}\n\nbool ZlibWriterBase::SupportsReadMode() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsReadMode();\n}\n\nReader* ZlibWriterBase::ReadModeBehindBuffer(Position initial_pos) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::ReadModeBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ZlibWriterBase::FlushBehindBuffer(\n          absl::string_view(), FlushType::kFromObject))) {\n    return nullptr;\n  }\n  Writer& dest = *DestWriter();\n  Reader* const compressed_reader = dest.ReadMode(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    return nullptr;\n  }\n  ZlibReader<>* const reader = associated_reader_.ResetReader(\n      compressed_reader,\n      ZlibReaderBase::Options()\n          .set_header(window_bits_ < 0 ? ZlibReaderBase::Header::kRaw\n                                       : static_cast<ZlibReaderBase::Header>(\n                                             window_bits_ & ~15))\n          .set_window_log(window_bits_ < 0 ? -window_bits_ : window_bits_ & 15)\n          .set_dictionary(dictionary_)\n          .set_buffer_options(buffer_options())\n          .set_recycling_pool_options(recycling_pool_options_));\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/zlib/zlib_writer.h",
    "content": "// Copyright 2018 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_ZLIB_ZLIB_WRITER_H_\n#define RIEGELI_ZLIB_ZLIB_WRITER_H_\n\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/compare.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/zlib/zlib_dictionary.h\"  // IWYU pragma: export\n\nnamespace riegeli {\n\nclass Reader;\ntemplate <typename Src>\nclass ZlibReader;\n\n// Template parameter independent part of `ZlibWriter`.\nclass ZlibWriterBase : public BufferedWriter {\n public:\n  // Specifies what format of header to write.\n  enum class Header {\n    kZlib = 0,   // Zlib header.\n    kGzip = 16,  // Gzip header.\n    kRaw = -1,   // No header; decompressor must expect no header too.\n  };\n\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // What format of header to write.\n    //\n    // Default: `Header::kZlib`.\n    static constexpr Header kDefaultHeader = Header::kZlib;\n    Options& set_header(Header header) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      header_ = header;\n      return *this;\n    }\n    Options&& set_header(Header header) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_header(header));\n    }\n    Header header() const { return header_; }\n\n    // Tunes the tradeoff between compression density and compression speed\n    // (higher = better density but slower).\n    //\n    // `compression_level` must be between `kMinCompressionLevel` (0) and\n    // `kMaxCompressionLevel` (9). Default: `kDefaultCompressionLevel` (6).\n    static constexpr int kMinCompressionLevel = 0;  // `Z_NO_COMPRESSION`\n    static constexpr int kMaxCompressionLevel = 9;  // `Z_BEST_COMPRESSION`\n    static constexpr int kDefaultCompressionLevel = 6;\n    Options& set_compression_level(int compression_level) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(compression_level, kMinCompressionLevel)\n          << \"Failed precondition of \"\n             \"ZlibWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      RIEGELI_ASSERT_LE(compression_level, kMaxCompressionLevel)\n          << \"Failed precondition of \"\n             \"ZlibWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      compression_level_ = compression_level;\n      return *this;\n    }\n    Options&& set_compression_level(int compression_level) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_compression_level(compression_level));\n    }\n    int compression_level() const { return compression_level_; }\n\n    // Logarithm of the LZ77 sliding window size. This tunes the tradeoff\n    // between compression density and memory usage (higher = better density but\n    // more memory).\n    //\n    // `window_log` must be between `kMinWindowLog` (9) and\n    // `kMaxWindowLog` (15). Default: `kDefaultWindowLog` (15).\n    static constexpr int kMinWindowLog = 9;\n    static constexpr int kMaxWindowLog = 15;      // `MAX_WBITS`\n    static constexpr int kDefaultWindowLog = 15;  // `MAX_WBITS`\n    Options& set_window_log(int window_log) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(window_log, kMinWindowLog)\n          << \"Failed precondition of \"\n             \"ZlibWriterBase::Options::set_window_log(): \"\n             \"window log out of range\";\n      RIEGELI_ASSERT_LE(window_log, kMaxWindowLog)\n          << \"Failed precondition of \"\n             \"ZlibWriterBase::Options::set_window_log(): \"\n             \"window log out of range\";\n      window_log_ = window_log;\n      return *this;\n    }\n    Options&& set_window_log(int window_log) && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_window_log(window_log));\n    }\n    int window_log() const { return window_log_; }\n\n    // Zlib dictionary. The same dictionary must be used for decompression.\n    //\n    // Default: `ZlibDictionary()`.\n    Options& set_dictionary(ZlibDictionary dictionary) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      dictionary_ = std::move(dictionary);\n      return *this;\n    }\n    Options&& set_dictionary(ZlibDictionary dictionary) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_dictionary(std::move(dictionary)));\n    }\n    ZlibDictionary& dictionary() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n    const ZlibDictionary& dictionary() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n\n    // Options for a global `KeyedRecyclingPool` of compression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // compression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    Options& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    Options&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    Header header_ = kDefaultHeader;\n    int compression_level_ = kDefaultCompressionLevel;\n    int window_log_ = kDefaultWindowLog;\n    ZlibDictionary dictionary_;\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Returns the compressed `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsReadMode() override;\n\n protected:\n  explicit ZlibWriterBase(Closed) noexcept : BufferedWriter(kClosed) {}\n\n  explicit ZlibWriterBase(BufferOptions buffer_options, int window_bits,\n                          ZlibDictionary&& dictionary,\n                          const RecyclingPoolOptions& recycling_pool_options);\n\n  ZlibWriterBase(ZlibWriterBase&& that) noexcept;\n  ZlibWriterBase& operator=(ZlibWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, int window_bits,\n             ZlibDictionary&& dictionary,\n             const RecyclingPoolOptions& recycling_pool_options);\n  static int GetWindowBits(const Options& options);\n  void Initialize(Writer* dest, int compression_level);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  void DoneBehindBuffer(absl::string_view src) override;\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool WriteInternal(absl::string_view src) override;\n  bool FlushBehindBuffer(absl::string_view src, FlushType flush_type) override;\n  Reader* ReadModeBehindBuffer(Position initial_pos) override;\n\n private:\n  struct ZStreamDeleter {\n    // `void*` is `z_stream*`. Avoid including `zlib.h` in the header.\n    void operator()(void* ptr) const;\n  };\n\n  struct ZStreamKey : WithEqual<ZStreamKey> {\n    ZStreamKey() = default;\n    explicit ZStreamKey(int compression_level, int window_bits)\n        : compression_level(compression_level), window_bits(window_bits) {}\n\n    friend bool operator==(ZStreamKey a, ZStreamKey b) {\n      return a.compression_level == b.compression_level &&\n             a.window_bits == b.window_bits;\n    }\n    template <typename HashState>\n    friend HashState AbslHashValue(HashState hash_state, ZStreamKey self) {\n      return HashState::combine(std::move(hash_state), self.compression_level,\n                                self.window_bits);\n    }\n\n    int compression_level;\n    int window_bits;\n  };\n\n  ABSL_ATTRIBUTE_COLD bool FailOperation(absl::string_view operation,\n                                         int zlib_code);\n  bool WriteInternal(absl::string_view src, Writer& dest, int flush);\n\n  int window_bits_ = 0;\n  ZlibDictionary dictionary_;\n  RecyclingPoolOptions recycling_pool_options_;\n  Position initial_compressed_pos_ = 0;\n  // `void` is `z_stream`. Avoid including `zlib.h` in the header.\n  KeyedRecyclingPool<void, ZStreamKey, ZStreamDeleter>::Handle compressor_;\n\n  AssociatedReader<ZlibReader<Reader*>> associated_reader_;\n};\n\n// A `Writer` which compresses data with Zlib before passing it to another\n// `Writer`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Writer` must not be accessed until the `ZlibWriter` is closed\n// or no longer used, except that it is allowed to read the destination of the\n// compressed `Writer` immediately after `Flush()`.\ntemplate <typename Dest = Writer*>\nclass ZlibWriter : public ZlibWriterBase {\n public:\n  // Creates a closed `ZlibWriter`.\n  explicit ZlibWriter(Closed) noexcept : ZlibWriterBase(kClosed) {}\n\n  // Will write to the compressed `Writer` provided by `dest`.\n  explicit ZlibWriter(Initializer<Dest> dest, Options options = Options());\n\n  ZlibWriter(ZlibWriter&& that) = default;\n  ZlibWriter& operator=(ZlibWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `ZlibWriter`. This avoids\n  // constructing a temporary `ZlibWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  // The object providing and possibly owning the compressed `Writer`.\n  Dependency<Writer*, Dest> dest_;\n};\n\nexplicit ZlibWriter(Closed) -> ZlibWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit ZlibWriter(Dest&& dest,\n                    ZlibWriterBase::Options options = ZlibWriterBase::Options())\n    -> ZlibWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline ZlibWriterBase::ZlibWriterBase(\n    BufferOptions buffer_options, int window_bits, ZlibDictionary&& dictionary,\n    const RecyclingPoolOptions& recycling_pool_options)\n    : BufferedWriter(buffer_options),\n      window_bits_(window_bits),\n      dictionary_(std::move(dictionary)),\n      recycling_pool_options_(recycling_pool_options) {}\n\ninline ZlibWriterBase::ZlibWriterBase(ZlibWriterBase&& that) noexcept\n    : BufferedWriter(static_cast<BufferedWriter&&>(that)),\n      window_bits_(that.window_bits_),\n      dictionary_(std::move(that.dictionary_)),\n      recycling_pool_options_(that.recycling_pool_options_),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      compressor_(std::move(that.compressor_)),\n      associated_reader_(std::move(that.associated_reader_)) {}\n\ninline ZlibWriterBase& ZlibWriterBase::operator=(\n    ZlibWriterBase&& that) noexcept {\n  BufferedWriter::operator=(static_cast<BufferedWriter&&>(that));\n  window_bits_ = that.window_bits_;\n  dictionary_ = std::move(that.dictionary_);\n  recycling_pool_options_ = that.recycling_pool_options_;\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  compressor_ = std::move(that.compressor_);\n  associated_reader_ = std::move(that.associated_reader_);\n  return *this;\n}\n\ninline void ZlibWriterBase::Reset(Closed) {\n  BufferedWriter::Reset(kClosed);\n  window_bits_ = 0;\n  recycling_pool_options_ = RecyclingPoolOptions();\n  initial_compressed_pos_ = 0;\n  compressor_.reset();\n  // Must be destroyed after `compressor_`.\n  dictionary_ = ZlibDictionary();\n  associated_reader_.Reset();\n}\n\ninline void ZlibWriterBase::Reset(\n    BufferOptions buffer_options, int window_bits, ZlibDictionary&& dictionary,\n    const RecyclingPoolOptions& recycling_pool_options) {\n  BufferedWriter::Reset(buffer_options);\n  window_bits_ = window_bits;\n  recycling_pool_options_ = recycling_pool_options;\n  initial_compressed_pos_ = 0;\n  compressor_.reset();\n  // Must be destroyed after `compressor_`.\n  dictionary_ = std::move(dictionary);\n  associated_reader_.Reset();\n}\n\ninline int ZlibWriterBase::GetWindowBits(const Options& options) {\n  return options.header() == Header::kRaw\n             ? -options.window_log()\n             : options.window_log() + static_cast<int>(options.header());\n}\n\ntemplate <typename Dest>\ninline ZlibWriter<Dest>::ZlibWriter(Initializer<Dest> dest, Options options)\n    : ZlibWriterBase(options.buffer_options(), GetWindowBits(options),\n                     std::move(options.dictionary()),\n                     options.recycling_pool_options()),\n      dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.compression_level());\n}\n\ntemplate <typename Dest>\ninline void ZlibWriter<Dest>::Reset(Closed) {\n  ZlibWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void ZlibWriter<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  ZlibWriterBase::Reset(options.buffer_options(), GetWindowBits(options),\n                        std::move(options.dictionary()),\n                        options.recycling_pool_options());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.compression_level());\n}\n\ntemplate <typename Dest>\nvoid ZlibWriter<Dest>::Done() {\n  ZlibWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <typename Dest>\nbool ZlibWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ZlibWriterBase::FlushImpl(flush_type))) return false;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Flush(flush_type))) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n  return true;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_ZLIB_ZLIB_WRITER_H_\n"
  },
  {
    "path": "riegeli/zstd/BUILD",
    "content": "load(\"@rules_cc//cc:defs.bzl\", \"cc_library\")\n\npackage(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\nlicenses([\"notice\"])\n\ncc_library(\n    name = \"zstd_reader\",\n    srcs = [\"zstd_reader.cc\"],\n    hdrs = [\"zstd_reader.h\"],\n    # zstd_reader.cc has #define before #include to influence what the included\n    # files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \":zstd_dictionary\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_reader\",\n        \"//riegeli/bytes:reader\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@net_zstd//:zstd\",\n    ],\n)\n\ncc_library(\n    name = \"zstd_writer\",\n    srcs = [\"zstd_writer.cc\"],\n    hdrs = [\"zstd_writer.h\"],\n    # zstd_writer.cc has #define before #include to influence what the included\n    # files provide.\n    features = [\"-use_header_modules\"],\n    deps = [\n        \":zstd_dictionary\",\n        \":zstd_reader\",\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:assert\",\n        \"//riegeli/base:dependency\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:object\",\n        \"//riegeli/base:recycling_pool\",\n        \"//riegeli/base:status\",\n        \"//riegeli/base:types\",\n        \"//riegeli/bytes:buffer_options\",\n        \"//riegeli/bytes:buffered_writer\",\n        \"//riegeli/bytes:reader\",\n        \"//riegeli/bytes:writer\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/status\",\n        \"@com_google_absl//absl/strings\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@net_zstd//:zstd\",\n    ],\n)\n\ncc_library(\n    name = \"zstd_dictionary\",\n    srcs = [\"zstd_dictionary.cc\"],\n    hdrs = [\"zstd_dictionary.h\"],\n    # zstd_dictionary.cc has #define before #include to influence what the\n    # included files provide.\n    features = [\"-use_header_modules\"],\n    visibility = [\"//visibility:private\"],\n    deps = [\n        \"//riegeli/base:arithmetic\",\n        \"//riegeli/base:bytes_ref\",\n        \"//riegeli/base:initializer\",\n        \"//riegeli/base:shared_ptr\",\n        \"@com_google_absl//absl/base\",\n        \"@com_google_absl//absl/base:core_headers\",\n        \"@com_google_absl//absl/strings:string_view\",\n        \"@com_google_absl//absl/synchronization\",\n        \"@net_zstd//:zstd\",\n    ],\n)\n"
  },
  {
    "path": "riegeli/zstd/zstd_dictionary.cc",
    "content": "#include \"absl/base/attributes.h\"\n// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// Enables the experimental zstd API:\n//  * `ZSTD_createCDict_advanced()`\n//  * `ZSTD_createDDict_advanced()`\n//  * `ZSTD_dictLoadMethod_e`\n//  * `ZSTD_dictContentType_e`\n#define ZSTD_STATIC_LINKING_ONLY\n\n#include <stdint.h>\n\n#include <memory>\n#include <utility>\n\n#include \"absl/base/call_once.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/synchronization/mutex.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/shared_ptr.h\"\n#include \"riegeli/zstd/zstd_dictionary.h\"\n#include \"zstd.h\"\n\nnamespace riegeli {\n\n// Constants are defined as integer literals in zstd_dictionary.h and asserted\n// here to avoid depending on `ZSTD_STATIC_LINKING_ONLY` in zstd_dictionary.h.\nstatic_assert(\n    static_cast<ZSTD_dictContentType_e>(ZstdDictionary::Type::kAuto) ==\n            ZSTD_dct_auto &&\n        static_cast<ZSTD_dictContentType_e>(ZstdDictionary::Type::kRaw) ==\n            ZSTD_dct_rawContent &&\n        static_cast<ZSTD_dictContentType_e>(\n            ZstdDictionary::Type::kSerialized) == ZSTD_dct_fullDict,\n    \"Enum values of ZstdDictionary::Type disagree with ZSTD_dct \"\n    \"constants\");\n\ninline ZstdDictionary::ZSTD_CDictHandle\nZstdDictionary::Repr::PrepareCompressionDictionary(\n    int compression_level) const {\n  SharedPtr<const ZSTD_CDictCache> compression_cache;\n  {\n    absl::MutexLock lock(compression_cache_mutex_);\n    if (compression_cache_ == nullptr ||\n        compression_cache_->compression_level != compression_level) {\n      compression_cache_.Reset(riegeli::Maker(compression_level));\n    }\n    compression_cache = compression_cache_;\n  }\n  absl::call_once(compression_cache->compression_once, [&] {\n    compression_cache->compression_dictionary.reset(ZSTD_createCDict_advanced(\n        data_.data(), data_.size(), ZSTD_dlm_byRef,\n        static_cast<ZSTD_dictContentType_e>(type_),\n        ZSTD_getCParams(compression_level, 0, data_.size()), ZSTD_defaultCMem));\n  });\n  ZSTD_CDict* const ptr = compression_cache->compression_dictionary.get();\n  return ZSTD_CDictHandle(ptr,\n                          ZSTD_CDictReleaser{std::move(compression_cache)});\n}\n\ninline const ZSTD_DDict* ZstdDictionary::Repr::PrepareDecompressionDictionary()\n    const {\n  absl::call_once(decompression_once_, [&] {\n    decompression_dictionary_.reset(ZSTD_createDDict_advanced(\n        data_.data(), data_.size(), ZSTD_dlm_byRef,\n        static_cast<ZSTD_dictContentType_e>(type_), ZSTD_defaultCMem));\n  });\n  return decompression_dictionary_.get();\n}\n\nZstdDictionary::ZSTD_CDictHandle ZstdDictionary::PrepareCompressionDictionary(\n    int compression_level) const {\n  if (repr_ == nullptr) return nullptr;\n  return repr_->PrepareCompressionDictionary(compression_level);\n}\n\nconst ZSTD_DDict* ZstdDictionary::PrepareDecompressionDictionary() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  if (repr_ == nullptr) return nullptr;\n  return repr_->PrepareDecompressionDictionary();\n}\n\nuint32_t ZstdDictionary::DictId() const {\n  if (repr_ == nullptr) return 0;\n  return IntCast<uint32_t>(\n      ZSTD_getDictID_fromDict(repr_->data().data(), repr_->data().size()));\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/zstd/zstd_dictionary.h",
    "content": "// Copyright 2021 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_ZSTD_ZSTD_DICTIONARY_H_\n#define RIEGELI_ZSTD_ZSTD_DICTIONARY_H_\n\n// IWYU pragma: private, include \"riegeli/zstd/zstd_reader.h\"\n// IWYU pragma: private, include \"riegeli/zstd/zstd_writer.h\"\n\n#include <stdint.h>\n\n#include <memory>\n#include <string>\n#include <type_traits>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/call_once.h\"\n#include \"absl/base/thread_annotations.h\"\n#include \"absl/strings/string_view.h\"\n#include \"absl/synchronization/mutex.h\"\n#include \"riegeli/base/bytes_ref.h\"\n#include \"riegeli/base/maker.h\"\n#include \"riegeli/base/shared_ptr.h\"\n#include \"zstd.h\"\n\nnamespace riegeli {\n\n// Stores an optional Zstd dictionary for compression and decompression.\n//\n// An empty dictionary is equivalent to having no dictionary.\n//\n// A `ZstdDictionary` object can own the dictionary data, or can hold a pointer\n// to unowned dictionary data which must not be changed until the last\n// `ZstdReader` or `ZstdWriter` using this dictionary is closed or no longer\n// used. A `ZstdDictionary` object also holds prepared structures derived from\n// dictionary data. If the same dictionary is needed for multiple compression\n// or decompression sessions, the `ZstdDictionary` object can be reused to avoid\n// preparing them again.\n//\n// The prepared dictionary for compression depends on the compression level. At\n// most one prepared dictionary is cached, corresponding to the last compression\n// level used.\n//\n// Copying a `ZstdDictionary` object is cheap, sharing the actual dictionary.\nclass ZstdDictionary {\n private:\n  struct ZSTD_CDictReleaser;\n\n public:\n  // Interpretation of dictionary data.\n  enum class Type {\n    // If dictionary data begin with `ZSTD_MAGIC_DICTIONARY`, then like\n    // `kSerialized`, otherwise like `kRaw`.\n    kAuto = 0,\n    // Dictionary data should contain sequences that are commonly seen in the\n    // data being compressed.\n    kRaw = 1,\n    // Shared with the dictBuilder library.\n    kSerialized = 2,\n  };\n\n  // Owning handle to a compression dictionary in the prepared form.\n  using ZSTD_CDictHandle = std::unique_ptr<ZSTD_CDict, ZSTD_CDictReleaser>;\n\n  // Creates an empty `ZstdDictionary`.\n  ZstdDictionary() = default;\n\n  ZstdDictionary(const ZstdDictionary& that) = default;\n  ZstdDictionary& operator=(const ZstdDictionary& that) = default;\n\n  ZstdDictionary(ZstdDictionary&& that) = default;\n  ZstdDictionary& operator=(ZstdDictionary&& that) = default;\n\n  // Resets the `ZstdDictionary` to the empty state.\n  ZstdDictionary& Reset() & ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  ZstdDictionary&& Reset() && ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(Reset());\n  }\n\n  // Sets a dictionary.\n  ZstdDictionary& set_data(BytesInitializer data, Type type = Type::kAuto) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  ZstdDictionary&& set_data(BytesInitializer data, Type type = Type::kAuto) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_data(std::move(data), type));\n  }\n\n  // Like `set_data()`, but does not take ownership of `data`, which must not\n  // be changed until the last `ZstdReader` or `ZstdWriter` using this\n  // dictionary is closed or no longer used.\n  ZstdDictionary& set_data_unowned(absl::string_view data\n                                       ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                                   Type type = Type::kAuto) &\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n  ZstdDictionary&& set_data_unowned(absl::string_view data\n                                        ABSL_ATTRIBUTE_LIFETIME_BOUND,\n                                    Type type = Type::kAuto) &&\n      ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return std::move(set_data_unowned(data, type));\n  }\n\n  // Returns `true` if no dictionary is present.\n  bool empty() const;\n\n  // Returns the dictionary data.\n  absl::string_view data() const ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns the compression dictionary in the prepared form, or `nullptr` if\n  // no dictionary is present or `ZSTD_createCDict_advanced()` failed.\n  ZSTD_CDictHandle PrepareCompressionDictionary(int compression_level) const;\n\n  // Returns the decompression dictionary in the prepared form, or `nullptr` if\n  // no dictionary is present or `ZSTD_createDDict_advanced()` failed.\n  //\n  // The dictionary is owned by `*this`.\n  const ZSTD_DDict* PrepareDecompressionDictionary() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  // Returns the dictionary ID, or 0 is no dictionary is present.\n  uint32_t DictId() const;\n\n private:\n  enum class Ownership { kCopied, kUnowned };\n\n  struct ZSTD_CDictDeleter {\n    void operator()(ZSTD_CDict* ptr) const { ZSTD_freeCDict(ptr); }\n  };\n\n  struct ZSTD_CDictCache;\n\n  struct ZSTD_CDictReleaser {\n    void operator()(ABSL_ATTRIBUTE_UNUSED ZSTD_CDict* ptr) {\n      // `*ptr` is owned by `*compression_cache`.\n      compression_cache.Reset();\n    }\n    SharedPtr<const ZSTD_CDictCache> compression_cache;\n  };\n\n  class Repr;\n\n  SharedPtr<const Repr> repr_;\n};\n\n// Implementation details follow.\n\nclass ZstdDictionary::Repr {\n public:\n  // Owns a copy of `data`.\n  explicit Repr(Type type, BytesInitializer data,\n                std::integral_constant<Ownership, Ownership::kCopied>)\n      : type_(type), owned_data_(std::move(data)), data_(owned_data_) {}\n\n  // Does not take ownership of `data`, which must not be changed until the\n  // last `ZstdWriter` or `ZstdReader` using this dictionary is closed or no\n  // longer used.\n  explicit Repr(Type type, absl::string_view data,\n                std::integral_constant<Ownership, Ownership::kUnowned>)\n      : type_(type), data_(data) {}\n\n  Repr(const Repr&) = delete;\n  Repr& operator=(const Repr&) = delete;\n\n  // Returns the compression dictionary in the prepared form, or `nullptr` if\n  // no dictionary is present or `ZSTD_createCDict_advanced()` failed.\n  ZSTD_CDictHandle PrepareCompressionDictionary(int compression_level) const;\n\n  // Returns the decompression dictionary in the prepared form, or `nullptr`\n  // if no dictionary is present or `ZSTD_createDDict_advanced()` failed.\n  //\n  // The dictionary is owned by `*this`.\n  const ZSTD_DDict* PrepareDecompressionDictionary() const\n      ABSL_ATTRIBUTE_LIFETIME_BOUND;\n\n  absl::string_view data() const { return data_; }\n\n private:\n  struct ZSTD_DDictDeleter {\n    void operator()(ZSTD_DDict* ptr) const { ZSTD_freeDDict(ptr); }\n  };\n\n  Type type_;\n  std::string owned_data_;\n  absl::string_view data_;\n\n  mutable absl::Mutex compression_cache_mutex_;\n  mutable SharedPtr<const ZSTD_CDictCache> compression_cache_\n      ABSL_GUARDED_BY(compression_cache_mutex_);\n\n  mutable absl::once_flag decompression_once_;\n  mutable std::unique_ptr<ZSTD_DDict, ZSTD_DDictDeleter>\n      decompression_dictionary_;\n};\n\n// Holds a compression dictionary prepared for a particular compression level.\n//\n// If several callers of `ZstdDictionary` need a prepared dictionary with the\n// same compression level at the same time, they wait for the first one to\n// prepare it, and they share it.\n//\n// If the callers need it with different compression levels, they do not wait.\n// The dictionary will be prepared again if varying compression levels later\n// repeat, because the cache holds at most one entry.\nstruct ZstdDictionary::ZSTD_CDictCache {\n  explicit ZSTD_CDictCache(int compression_level)\n      : compression_level(compression_level) {}\n\n  int compression_level;\n  mutable absl::once_flag compression_once;\n  mutable std::unique_ptr<ZSTD_CDict, ZSTD_CDictDeleter> compression_dictionary;\n};\n\ninline ZstdDictionary& ZstdDictionary::Reset() & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  repr_.Reset();\n  return *this;\n}\n\ninline ZstdDictionary& ZstdDictionary::set_data(BytesInitializer data,\n                                                Type type) &\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  repr_.Reset(\n      riegeli::Maker(type, std::move(data),\n                     std::integral_constant<Ownership, Ownership::kCopied>()));\n  return *this;\n}\n\ninline ZstdDictionary& ZstdDictionary::set_data_unowned(\n    absl::string_view data ABSL_ATTRIBUTE_LIFETIME_BOUND, Type type) &\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  repr_.Reset(riegeli::Maker(\n      type, data, std::integral_constant<Ownership, Ownership::kUnowned>()));\n  return *this;\n}\n\ninline bool ZstdDictionary::empty() const {\n  return repr_ == nullptr || repr_->data().empty();\n}\n\ninline absl::string_view ZstdDictionary::data() const\n    ABSL_ATTRIBUTE_LIFETIME_BOUND {\n  if (repr_ == nullptr) return absl::string_view();\n  return repr_->data();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_ZSTD_ZSTD_DICTIONARY_H_\n"
  },
  {
    "path": "riegeli/zstd/zstd_reader.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// Enables the experimental zstd API:\n//  * `ZSTD_FRAMEHEADERSIZE_PREFIX()`\n//  * `ZSTD_FRAMEHEADERSIZE_MAX`\n//  * `ZSTD_f_zstd1`\n//  * `ZSTD_d_stableOutBuffer`\n//  * `ZSTD_skippableFrame`\n//  * `ZSTD_frameHeader`\n//  * `ZSTD_getFrameHeader()`\n#define ZSTD_STATIC_LINKING_ONLY\n\n#include \"riegeli/zstd/zstd_reader.h\"\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"zstd.h\"\n\nnamespace riegeli {\n\nvoid ZstdReaderBase::Initialize(Reader* src) {\n  RIEGELI_ASSERT_NE(src, nullptr)\n      << \"Failed precondition of ZstdReader: null Reader pointer\";\n  if (ABSL_PREDICT_FALSE(!src->ok()) && src->available() == 0) {\n    FailWithoutAnnotation(AnnotateOverSrc(src->status()));\n    return;\n  }\n  initial_compressed_pos_ = src->pos();\n  InitializeDecompressor(*src);\n}\n\ninline void ZstdReaderBase::InitializeDecompressor(Reader& src) {\n  decompressor_ =\n      RecyclingPool<ZSTD_DCtx, ZSTD_DCtxDeleter>::global(\n          recycling_pool_options_)\n          .Get(\n              [] {\n                return std::unique_ptr<ZSTD_DCtx, ZSTD_DCtxDeleter>(\n                    ZSTD_createDCtx());\n              },\n              [](ZSTD_DCtx* decompressor) {\n                {\n                  const size_t result = ZSTD_DCtx_reset(\n                      decompressor, ZSTD_reset_session_and_parameters);\n                  RIEGELI_ASSERT(!ZSTD_isError(result))\n                      << \"ZSTD_DCtx_reset() failed: \"\n                      << ZSTD_getErrorName(result);\n                }\n#if ZSTD_VERSION_NUMBER <= 10405\n                // Workaround for https://github.com/facebook/zstd/issues/2331\n                {\n                  const size_t result = ZSTD_DCtx_setParameter(\n                      decompressor, ZSTD_d_stableOutBuffer, 0);\n                  RIEGELI_ASSERT(!ZSTD_isError(result))\n                      << \"ZSTD_DCtx_setParameter(ZSTD_d_stableOutBuffer) \"\n                         \"failed: \"\n                      << ZSTD_getErrorName(result);\n                }\n#endif\n              });\n  if (ABSL_PREDICT_FALSE(decompressor_ == nullptr)) {\n    Fail(absl::InternalError(\"ZSTD_createDCtx() failed\"));\n    return;\n  }\n  {\n    // Maximum window size could also be found with\n    // `ZSTD_dParam_getBounds(ZSTD_d_windowLogMax)`.\n    const size_t result =\n        ZSTD_DCtx_setParameter(decompressor_.get(), ZSTD_d_windowLogMax,\n                               sizeof(size_t) == 4 ? 30 : 31);\n    if (ABSL_PREDICT_FALSE(ZSTD_isError(result))) {\n      Fail(absl::InternalError(\n          absl::StrCat(\"ZSTD_DCtx_setParameter(ZSTD_d_windowLogMax) failed: \",\n                       ZSTD_getErrorName(result))));\n      return;\n    }\n  }\n  if (!dictionary_.empty()) {\n    const ZSTD_DDict* const decompression_dictionary =\n        dictionary_.PrepareDecompressionDictionary();\n    if (ABSL_PREDICT_FALSE(decompression_dictionary == nullptr)) {\n      Fail(absl::InternalError(\"ZSTD_createDDict_advanced() failed\"));\n      return;\n    }\n    const size_t result =\n        ZSTD_DCtx_refDDict(decompressor_.get(), decompression_dictionary);\n    if (ABSL_PREDICT_FALSE(ZSTD_isError(result))) {\n      Fail(absl::InternalError(absl::StrCat(\"ZSTD_DCtx_refDDict() failed: \",\n                                            ZSTD_getErrorName(result))));\n      return;\n    }\n  }\n  if (!concatenate_ && exact_size() == std::nullopt) {\n    set_exact_size(ZstdUncompressedSize(src));\n  }\n  just_initialized_ = true;\n}\n\nvoid ZstdReaderBase::Done() {\n  if (ABSL_PREDICT_FALSE(truncated_) && growing_source_) {\n    Reader& src = *SrcReader();\n    FailWithoutAnnotation(AnnotateOverSrc(src.AnnotateStatus(\n        absl::InvalidArgumentError(\"Truncated Zstd-compressed stream\"))));\n  }\n  BufferedReader::Done();\n  decompressor_.reset();\n  dictionary_ = ZstdDictionary();\n}\n\nabsl::Status ZstdReaderBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    if (ABSL_PREDICT_FALSE(truncated_)) {\n      status = Annotate(status, \"reading truncated Zstd-compressed stream\");\n    }\n    Reader& src = *SrcReader();\n    status = src.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `src` with the compressed position.\n  // Clarify that the current position is the uncompressed position instead of\n  // delegating to `BufferedReader::AnnotateStatusImpl()`.\n  return AnnotateOverSrc(std::move(status));\n}\n\nabsl::Status ZstdReaderBase::AnnotateOverSrc(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nbool ZstdReaderBase::ReadInternal(size_t min_length, size_t max_length,\n                                  char* dest) {\n  RIEGELI_ASSERT_GT(min_length, 0u)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"nothing to read\";\n  RIEGELI_ASSERT_GE(max_length, min_length)\n      << \"Failed precondition of BufferedReader::ReadInternal(): \"\n         \"max_length < min_length\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedReader::ReadInternal()\";\n  Reader& src = *SrcReader();\n  truncated_ = false;\n  if (just_initialized_ && !concatenate_ && exact_size() == std::nullopt) {\n    // Try again in case the source has grown.\n    set_exact_size(ZstdUncompressedSize(src));\n  }\n  size_t effective_min_length = min_length;\n  if (just_initialized_ && !growing_source_ && exact_size() != std::nullopt &&\n      max_length >= *exact_size()) {\n    // Avoid a memory copy from an internal buffer of the Zstd engine to `dest`\n    // by promising to decompress all remaining data to `dest`.\n    const size_t result =\n        ZSTD_DCtx_setParameter(decompressor_.get(), ZSTD_d_stableOutBuffer, 1);\n    if (ABSL_PREDICT_FALSE(ZSTD_isError(result))) {\n      return Fail(absl::InternalError(absl::StrCat(\n          \"ZSTD_DCtx_setParameter(ZSTD_d_stableOutBuffer) failed: \",\n          ZSTD_getErrorName(result))));\n    }\n    effective_min_length = std::numeric_limits<size_t>::max();\n  }\n  just_initialized_ = false;\n  max_length = UnsignedMin(max_length,\n                           std::numeric_limits<Position>::max() - limit_pos());\n  ZSTD_outBuffer output = {dest, max_length, 0};\n  for (;;) {\n    ZSTD_inBuffer input = {src.cursor(), src.available(), 0};\n    const size_t result =\n        ZSTD_decompressStream(decompressor_.get(), &output, &input);\n    src.set_cursor(static_cast<const char*>(input.src) + input.pos);\n    if (ABSL_PREDICT_FALSE(result == 0)) {\n      if (concatenate_) {\n        if (output.pos >= min_length) {\n          move_limit_pos(output.pos);\n          return true;\n        }\n        continue;\n      }\n      decompressor_.reset();\n      move_limit_pos(output.pos);\n      // Avoid `BufferedReader` allocating another buffer.\n      set_exact_size(limit_pos());\n      return output.pos >= min_length;\n    }\n    if (ABSL_PREDICT_FALSE(ZSTD_isError(result))) {\n      Fail(absl::InvalidArgumentError(absl::StrCat(\n          \"ZSTD_decompressStream() failed: \", ZSTD_getErrorName(result))));\n      move_limit_pos(output.pos);\n      return output.pos >= min_length;\n    }\n    if (output.pos >= effective_min_length) {\n      move_limit_pos(output.pos);\n      return true;\n    }\n    if (ABSL_PREDICT_FALSE(input.pos < input.size)) {\n      RIEGELI_ASSERT_EQ(output.pos, output.size)\n          << \"ZSTD_decompressStream() returned but there are still \"\n             \"input data and output space\";\n      RIEGELI_ASSERT_EQ(output.pos,\n                        std::numeric_limits<Position>::max() - limit_pos())\n          << \"The position does not overflow but the output buffer is full, \"\n             \"while less than min_length was output, which implies that \"\n             \"ZSTD_decompressStream() wants to output more than the \"\n             \"expected decompressed size to a flat buffer\";\n      move_limit_pos(output.pos);\n      return FailOverflow();\n    }\n    if (ABSL_PREDICT_FALSE(!src.Pull(1, result))) {\n      move_limit_pos(output.pos);\n      if (ABSL_PREDICT_FALSE(!src.ok())) {\n        FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n      } else if (ABSL_PREDICT_FALSE(!concatenate_ || input.pos > 0)) {\n        if (!growing_source_) {\n          Fail(absl::InvalidArgumentError(\"Truncated Zstd-compressed stream\"));\n        }\n        truncated_ = true;\n      }\n      return output.pos >= min_length;\n    }\n  }\n}\n\nvoid ZstdReaderBase::ExactSizeReached() {\n  if (decompressor_ == nullptr) return;\n  // This is especially important if `exact_size() == 0` because in that case\n  // `ReadInternal()` was never called and the compressed stream was not\n  // consumed.\n  char buffer[1];\n  if (ABSL_PREDICT_FALSE(ZstdReaderBase::ReadInternal(1, 1, buffer))) {\n    decompressor_.reset();\n    Fail(absl::FailedPreconditionError(\n        \"Uncompressed size reached but more data can be decompressed, \"\n        \"which implies that seeking back and reading again encountered \"\n        \"changed Zstd-compressed data\"));\n  }\n}\n\nbool ZstdReaderBase::ToleratesReadingAhead() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->ToleratesReadingAhead();\n}\n\nbool ZstdReaderBase::SupportsRewind() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsRewind();\n}\n\nbool ZstdReaderBase::SeekBehindBuffer(Position new_pos) {\n  RIEGELI_ASSERT(new_pos < start_pos() || new_pos > limit_pos())\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"position in the buffer, use Seek() instead\";\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedReader::SeekBehindBuffer(): \"\n         \"buffer not empty\";\n  if (new_pos <= limit_pos()) {\n    // Seeking backwards.\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    Reader& src = *SrcReader();\n    truncated_ = false;\n    set_buffer();\n    set_limit_pos(0);\n    decompressor_.reset();\n    if (ABSL_PREDICT_FALSE(!src.Seek(initial_compressed_pos_))) {\n      return FailWithoutAnnotation(AnnotateOverSrc(src.StatusOrAnnotate(\n          absl::DataLossError(\"Zstd-compressed stream got truncated\"))));\n    }\n    InitializeDecompressor(src);\n    if (ABSL_PREDICT_FALSE(!ok())) return false;\n    if (new_pos == 0) return true;\n  }\n  return BufferedReader::SeekBehindBuffer(new_pos);\n}\n\nbool ZstdReaderBase::SupportsNewReader() {\n  Reader* const src = SrcReader();\n  return src != nullptr && src->SupportsNewReader();\n}\n\nstd::unique_ptr<Reader> ZstdReaderBase::NewReaderImpl(Position initial_pos) {\n  if (ABSL_PREDICT_FALSE(!ok())) return nullptr;\n  // `NewReaderImpl()` is thread-safe from this point\n  // if `SrcReader()->SupportsNewReader()`.\n  Reader& src = *SrcReader();\n  std::unique_ptr<Reader> compressed_reader =\n      src.NewReader(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverSrc(src.status()));\n    return nullptr;\n  }\n  std::unique_ptr<Reader> reader =\n      std::make_unique<ZstdReader<std::unique_ptr<Reader>>>(\n          std::move(compressed_reader),\n          ZstdReaderBase::Options()\n              .set_growing_source(growing_source_)\n              .set_concatenate(concatenate_)\n              .set_dictionary(dictionary_)\n              .set_buffer_options(buffer_options())\n              .set_recycling_pool_options(recycling_pool_options_));\n  reader->Seek(initial_pos);\n  return reader;\n}\n\nnamespace {\n\ninline bool GetFrameHeader(Reader& src, ZSTD_frameHeader& header) {\n  if (ABSL_PREDICT_FALSE(!src.Pull(ZSTD_FRAMEHEADERSIZE_PREFIX(ZSTD_f_zstd1),\n                                   ZSTD_FRAMEHEADERSIZE_MAX))) {\n    return false;\n  }\n  for (;;) {\n    const size_t result =\n        ZSTD_getFrameHeader(&header, src.cursor(), src.available());\n    if (ABSL_PREDICT_TRUE(result == 0)) return true;\n    if (ABSL_PREDICT_FALSE(ZSTD_isError(result))) return false;\n    if (ABSL_PREDICT_FALSE(!src.Pull(result))) return false;\n  }\n}\n\n}  // namespace\n\nbool RecognizeZstd(Reader& src) {\n  ZSTD_frameHeader header;\n  return GetFrameHeader(src, header);\n}\n\nstd::optional<Position> ZstdUncompressedSize(Reader& src) {\n  ZSTD_frameHeader header;\n  if (ABSL_PREDICT_FALSE(!GetFrameHeader(src, header))) return std::nullopt;\n  if (header.frameType == ZSTD_skippableFrame) return 0;\n  if (header.frameContentSize == ZSTD_CONTENTSIZE_UNKNOWN) return std::nullopt;\n  return IntCast<Position>(header.frameContentSize);\n}\n\nstd::optional<uint32_t> ZstdDictId(Reader& src) {\n  ZSTD_frameHeader header;\n  if (ABSL_PREDICT_FALSE(!GetFrameHeader(src, header))) return std::nullopt;\n  return IntCast<uint32_t>(header.dictID);\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/zstd/zstd_reader.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_ZSTD_ZSTD_READER_H_\n#define RIEGELI_ZSTD_ZSTD_READER_H_\n\n#include <stddef.h>\n#include <stdint.h>\n\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_reader.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/zstd/zstd_dictionary.h\"  // IWYU pragma: export\n#include \"zstd.h\"\n\nnamespace riegeli {\n\n// Template parameter independent part of `ZstdReader`.\nclass ZstdReaderBase : public BufferedReader {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // If `true`, supports decompressing as much as possible from a truncated\n    // source, then retrying when the source has grown. This has a small\n    // performance penalty.\n    //\n    // Default: `false`.\n    Options& set_growing_source(bool growing_source) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      growing_source_ = growing_source;\n      return *this;\n    }\n    Options&& set_growing_source(bool growing_source) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_growing_source(growing_source));\n    }\n    bool growing_source() const { return growing_source_; }\n\n    // If `true`, concatenated compressed frames are decoded to concatenation\n    // of their decompressed contents. An empty compressed stream is decoded to\n    // empty decompressed contents.\n    //\n    // If `false`, exactly one compressed frame is consumed.\n    //\n    // Default: `false`.\n    Options& set_concatenate(bool concatenate) & ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      concatenate_ = concatenate;\n      return *this;\n    }\n    Options&& set_concatenate(bool concatenate) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_concatenate(concatenate));\n    }\n    bool concatenate() const { return concatenate_; }\n\n    // Zstd dictionary. The same dictionary must have been used for compression,\n    // except that it is allowed to supply a dictionary for decompression even\n    // if no dictionary was used for compression.\n    //\n    // Default: `ZstdDictionary()`.\n    Options& set_dictionary(ZstdDictionary dictionary) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      dictionary_ = std::move(dictionary);\n      return *this;\n    }\n    Options&& set_dictionary(ZstdDictionary dictionary) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_dictionary(std::move(dictionary)));\n    }\n    ZstdDictionary& dictionary() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n    const ZstdDictionary& dictionary() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n\n    // Options for a global `RecyclingPool` of decompression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // decompression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    Options& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    Options&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    bool growing_source_ = false;\n    bool concatenate_ = false;\n    ZstdDictionary dictionary_;\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Returns the compressed `Reader`. Unchanged by `Close()`.\n  virtual Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  // Returns `true` if the source is truncated (without a clean end of the\n  // compressed stream) at the current position. In such case, if the source\n  // does not grow, `Close()` will fail.\n  //\n  // Precondition: `Options::concatenate()` was `false`.\n  bool truncated() const {\n    RIEGELI_ASSERT(!concatenate_)\n        << \"Failed precondition of ZstdReaderBase::truncated(): \"\n           \"Options::concatenate() is true\";\n    return truncated_ && available() == 0;\n  }\n\n  bool ToleratesReadingAhead() override;\n  bool SupportsRewind() override;\n  bool SupportsNewReader() override;\n\n protected:\n  explicit ZstdReaderBase(Closed) noexcept : BufferedReader(kClosed) {}\n\n  explicit ZstdReaderBase(BufferOptions buffer_options, bool growing_source,\n                          bool concatenate, ZstdDictionary&& dictionary,\n                          const RecyclingPoolOptions& recycling_pool_options);\n\n  ZstdReaderBase(ZstdReaderBase&& that) noexcept;\n  ZstdReaderBase& operator=(ZstdReaderBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, bool growing_source,\n             bool concatenate, ZstdDictionary&& dictionary,\n             const RecyclingPoolOptions& recycling_pool_options);\n  void Initialize(Reader* src);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverSrc(absl::Status status);\n\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  bool ReadInternal(size_t min_length, size_t max_length, char* dest) override;\n  void ExactSizeReached() override;\n  bool SeekBehindBuffer(Position new_pos) override;\n  std::unique_ptr<Reader> NewReaderImpl(Position initial_pos) override;\n\n private:\n  struct ZSTD_DCtxDeleter {\n    void operator()(ZSTD_DCtx* ptr) const { ZSTD_freeDCtx(ptr); }\n  };\n\n  void InitializeDecompressor(Reader& src);\n\n  // If `true`, supports decompressing as much as possible from a truncated\n  // source, then retrying when the source has grown.\n  bool growing_source_ = false;\n  bool concatenate_ = false;\n  Position initial_compressed_pos_ = 0;\n  // If `true`, the source is truncated (without a clean end of the compressed\n  // stream) at the current position. If the source does not grow, `Close()`\n  // will fail.\n  bool truncated_ = false;\n  // If `true`, calling `ZSTD_DCtx_setParameter()` is valid.\n  bool just_initialized_ = false;\n  ZstdDictionary dictionary_;\n  RecyclingPoolOptions recycling_pool_options_;\n  // If `ok()` but `decompressor_ == nullptr` then all data have been\n  // decompressed, `exact_size() == limit_pos()`, and `ReadInternal()` must not\n  // be called again.\n  RecyclingPool<ZSTD_DCtx, ZSTD_DCtxDeleter>::Handle decompressor_;\n};\n\n// A `Reader` which decompresses data with Zstd after getting it from another\n// `Reader`.\n//\n// The `Src` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Reader`. `Src` must support\n// `Dependency<Reader*, Src>`, e.g. `Reader*` (not owned, default),\n// `ChainReader<>` (owned), `std::unique_ptr<Reader>` (owned),\n// `Any<Reader*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Reader` must not be accessed until the `ZstdReader` is closed\n// or no longer used.\ntemplate <typename Src = Reader*>\nclass ZstdReader : public ZstdReaderBase {\n public:\n  // Creates a closed `ZstdReader`.\n  explicit ZstdReader(Closed) noexcept : ZstdReaderBase(kClosed) {}\n\n  // Will read from the compressed `Reader` provided by `src`.\n  explicit ZstdReader(Initializer<Src> src, Options options = Options());\n\n  ZstdReader(ZstdReader&& that) = default;\n  ZstdReader& operator=(ZstdReader&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `ZstdReader`. This avoids\n  // constructing a temporary `ZstdReader` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Src> src,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Reader`.\n  // Unchanged by `Close()`.\n  Src& src() ABSL_ATTRIBUTE_LIFETIME_BOUND { return src_.manager(); }\n  const Src& src() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return src_.manager();\n  }\n  Reader* SrcReader() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return src_.get();\n  }\n\n protected:\n  void Done() override;\n  void SetReadAllHintImpl(bool read_all_hint) override;\n  void VerifyEndImpl() override;\n\n private:\n  // The object providing and possibly owning the compressed `Reader`.\n  Dependency<Reader*, Src> src_;\n};\n\nexplicit ZstdReader(Closed) -> ZstdReader<DeleteCtad<Closed>>;\ntemplate <typename Src>\nexplicit ZstdReader(Src&& src,\n                    ZstdReaderBase::Options options = ZstdReaderBase::Options())\n    -> ZstdReader<TargetT<Src>>;\n\n// Returns `true` if the data look like they have been Zstd-compressed.\n//\n// The current position of `src` is unchanged.\nbool RecognizeZstd(Reader& src);\n\n// Returns the claimed uncompressed size of Zstd-compressed data.\n//\n// If the data consists of multiple frames, only the first frame is considered.\n//\n// Returns `std::nullopt` if the size was not stored or on failure. The size is\n// stored if `ZstdWriterBase::Options::pledged_size() != std::nullopt`.\n//\n// The current position of `src` is unchanged.\nstd::optional<Position> ZstdUncompressedSize(Reader& src);\n\n// Returns the dictionary ID needed to read `src`, or 0 if none is needed.\n//\n// Returns `std::nullopt` on failure.\n//\n// The current position of `src` is unchanged.\nstd::optional<uint32_t> ZstdDictId(Reader& src);\n\n// Implementation details follow.\n\ninline ZstdReaderBase::ZstdReaderBase(\n    BufferOptions buffer_options, bool growing_source, bool concatenate,\n    ZstdDictionary&& dictionary,\n    const RecyclingPoolOptions& recycling_pool_options)\n    : BufferedReader(buffer_options),\n      growing_source_(growing_source),\n      concatenate_(concatenate),\n      dictionary_(std::move(dictionary)),\n      recycling_pool_options_(recycling_pool_options) {}\n\ninline ZstdReaderBase::ZstdReaderBase(ZstdReaderBase&& that) noexcept\n    : BufferedReader(static_cast<BufferedReader&&>(that)),\n      growing_source_(that.growing_source_),\n      concatenate_(that.concatenate_),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      truncated_(that.truncated_),\n      just_initialized_(that.just_initialized_),\n      dictionary_(std::move(that.dictionary_)),\n      recycling_pool_options_(that.recycling_pool_options_),\n      decompressor_(std::move(that.decompressor_)) {}\n\ninline ZstdReaderBase& ZstdReaderBase::operator=(\n    ZstdReaderBase&& that) noexcept {\n  BufferedReader::operator=(static_cast<BufferedReader&&>(that));\n  growing_source_ = that.growing_source_;\n  concatenate_ = that.concatenate_;\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  truncated_ = that.truncated_;\n  just_initialized_ = that.just_initialized_;\n  dictionary_ = std::move(that.dictionary_);\n  recycling_pool_options_ = that.recycling_pool_options_;\n  decompressor_ = std::move(that.decompressor_);\n  return *this;\n}\n\ninline void ZstdReaderBase::Reset(Closed) {\n  BufferedReader::Reset(kClosed);\n  growing_source_ = false;\n  concatenate_ = false;\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  just_initialized_ = false;\n  recycling_pool_options_ = RecyclingPoolOptions();\n  decompressor_.reset();\n  dictionary_ = ZstdDictionary();\n}\n\ninline void ZstdReaderBase::Reset(\n    BufferOptions buffer_options, bool growing_source, bool concatenate,\n    ZstdDictionary&& dictionary,\n    const RecyclingPoolOptions& recycling_pool_options) {\n  BufferedReader::Reset(buffer_options);\n  growing_source_ = growing_source;\n  concatenate_ = concatenate;\n  initial_compressed_pos_ = 0;\n  truncated_ = false;\n  just_initialized_ = false;\n  recycling_pool_options_ = recycling_pool_options;\n  decompressor_.reset();\n  dictionary_ = std::move(dictionary);\n}\n\ntemplate <typename Src>\ninline ZstdReader<Src>::ZstdReader(Initializer<Src> src, Options options)\n    : ZstdReaderBase(options.buffer_options(), options.growing_source(),\n                     options.concatenate(), std::move(options.dictionary()),\n                     options.recycling_pool_options()),\n      src_(std::move(src)) {\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\ninline void ZstdReader<Src>::Reset(Closed) {\n  ZstdReaderBase::Reset(kClosed);\n  src_.Reset();\n}\n\ntemplate <typename Src>\ninline void ZstdReader<Src>::Reset(Initializer<Src> src, Options options) {\n  ZstdReaderBase::Reset(options.buffer_options(), options.growing_source(),\n                        options.concatenate(), std::move(options.dictionary()),\n                        options.recycling_pool_options());\n  src_.Reset(std::move(src));\n  Initialize(src_.get());\n}\n\ntemplate <typename Src>\nvoid ZstdReader<Src>::Done() {\n  ZstdReaderBase::Done();\n  if (src_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!src_->Close())) {\n      FailWithoutAnnotation(AnnotateOverSrc(src_->status()));\n    }\n  }\n}\n\ntemplate <typename Src>\nvoid ZstdReader<Src>::SetReadAllHintImpl(bool read_all_hint) {\n  ZstdReaderBase::SetReadAllHintImpl(read_all_hint);\n  if (src_.IsOwning()) src_->SetReadAllHint(read_all_hint);\n}\n\ntemplate <typename Src>\nvoid ZstdReader<Src>::VerifyEndImpl() {\n  ZstdReaderBase::VerifyEndImpl();\n  if (src_.IsOwning() && ABSL_PREDICT_TRUE(ok())) src_->VerifyEnd();\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_ZSTD_ZSTD_READER_H_\n"
  },
  {
    "path": "riegeli/zstd/zstd_writer.cc",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// Enables the experimental zstd API:\n//  * `ZSTD_WINDOWLOG_MIN`\n//  * `ZSTD_WINDOWLOG_MAX`\n//  * `ZSTD_TARGETCBLOCKSIZE_MIN`\n//  * `ZSTD_TARGETCBLOCKSIZE_MAX`\n//  * `ZSTD_c_srcSizeHint`\n#define ZSTD_STATIC_LINKING_ONLY\n\n#include \"riegeli/zstd/zstd_writer.h\"\n\n#include <stddef.h>\n\n#include <limits>\n#include <memory>\n#include <optional>\n#include <utility>\n\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/str_cat.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/status.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/reader.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/zstd/zstd_reader.h\"\n#include \"zstd.h\"\n\nnamespace riegeli {\n\nstatic_assert(ZstdWriterBase::Options::kMinWindowLog == ZSTD_WINDOWLOG_MIN);\nstatic_assert(ZstdWriterBase::Options::kMaxWindowLog == ZSTD_WINDOWLOG_MAX);\nstatic_assert(ZstdWriterBase::Options::kMinTargetCBlockSize ==\n              ZSTD_TARGETCBLOCKSIZE_MIN);\nstatic_assert(ZstdWriterBase::Options::kMaxTargetCBlockSize ==\n              ZSTD_TARGETCBLOCKSIZE_MAX);\n\nvoid ZstdWriterBase::Initialize(Writer* dest, int compression_level,\n                                int window_log_or_0,\n                                int target_cblock_size_or_0,\n                                bool store_checksum) {\n  RIEGELI_ASSERT_NE(dest, nullptr)\n      << \"Failed precondition of ZstdWriter: null Writer pointer\";\n  if (ABSL_PREDICT_FALSE(!dest->ok())) {\n    FailWithoutAnnotation(AnnotateOverDest(dest->status()));\n    return;\n  }\n  initial_compressed_pos_ = dest->pos();\n  compressor_ = RecyclingPool<ZSTD_CCtx, ZSTD_CCtxDeleter>::global(\n                    recycling_pool_options_)\n                    .Get(\n                        [] {\n                          return std::unique_ptr<ZSTD_CCtx, ZSTD_CCtxDeleter>(\n                              ZSTD_createCCtx());\n                        },\n                        [](ZSTD_CCtx* compressor) {\n                          const size_t result = ZSTD_CCtx_reset(\n                              compressor, ZSTD_reset_session_and_parameters);\n                          RIEGELI_ASSERT(!ZSTD_isError(result))\n                              << \"ZSTD_CCtx_reset() failed: \"\n                              << ZSTD_getErrorName(result);\n                        });\n  if (ABSL_PREDICT_FALSE(compressor_ == nullptr)) {\n    Fail(absl::InternalError(\"ZSTD_createCCtx() failed\"));\n    return;\n  }\n  {\n    const size_t result = ZSTD_CCtx_setParameter(\n        compressor_.get(), ZSTD_c_compressionLevel, compression_level);\n    if (ABSL_PREDICT_FALSE(ZSTD_isError(result))) {\n      Fail(absl::InternalError(absl::StrCat(\n          \"ZSTD_CCtx_setParameter(ZSTD_c_compressionLevel) failed: \",\n          ZSTD_getErrorName(result))));\n      return;\n    }\n  }\n  if (window_log_or_0 != 0) {\n    const size_t result = ZSTD_CCtx_setParameter(\n        compressor_.get(), ZSTD_c_windowLog, window_log_or_0);\n    if (ABSL_PREDICT_FALSE(ZSTD_isError(result))) {\n      Fail(absl::InternalError(\n          absl::StrCat(\"ZSTD_CCtx_setParameter(ZSTD_c_windowLog) failed: \",\n                       ZSTD_getErrorName(result))));\n      return;\n    }\n  }\n  if (target_cblock_size_or_0 != 0) {\n    const size_t result = ZSTD_CCtx_setParameter(\n        compressor_.get(), ZSTD_c_targetCBlockSize, target_cblock_size_or_0);\n    if (ABSL_PREDICT_FALSE(ZSTD_isError(result))) {\n      Fail(absl::InternalError(absl::StrCat(\n          \"ZSTD_CCtx_setParameter(ZSTD_c_targetCBlockSize) failed: \",\n          ZSTD_getErrorName(result))));\n      return;\n    }\n  }\n  {\n    const size_t result = ZSTD_CCtx_setParameter(\n        compressor_.get(), ZSTD_c_checksumFlag, store_checksum ? 1 : 0);\n    if (ABSL_PREDICT_FALSE(ZSTD_isError(result))) {\n      Fail(absl::InternalError(\n          absl::StrCat(\"ZSTD_CCtx_setParameter(ZSTD_c_checksumFlag) failed: \",\n                       ZSTD_getErrorName(result))));\n      return;\n    }\n  }\n  if (pledged_size_ != std::nullopt) {\n    BufferedWriter::SetWriteSizeHintImpl(*pledged_size_);\n    const size_t result = ZSTD_CCtx_setPledgedSrcSize(\n        compressor_.get(), IntCast<unsigned long long>(*pledged_size_));\n    if (ABSL_PREDICT_FALSE(ZSTD_isError(result))) {\n      Fail(absl::InternalError(\n          absl::StrCat(\"ZSTD_CCtx_setPledgedSrcSize() failed: \",\n                       ZSTD_getErrorName(result))));\n      return;\n    }\n  }\n  if (!dictionary_.empty()) {\n    compression_dictionary_ =\n        dictionary_.PrepareCompressionDictionary(compression_level);\n    if (ABSL_PREDICT_FALSE(compression_dictionary_ == nullptr)) {\n      Fail(absl::InternalError(\"ZSTD_createCDict_advanced() failed\"));\n      return;\n    }\n    const size_t result =\n        ZSTD_CCtx_refCDict(compressor_.get(), compression_dictionary_.get());\n    if (ABSL_PREDICT_FALSE(ZSTD_isError(result))) {\n      Fail(absl::InternalError(absl::StrCat(\"ZSTD_CCtx_refCDict() failed: \",\n                                            ZSTD_getErrorName(result))));\n      return;\n    }\n  }\n}\n\nvoid ZstdWriterBase::DoneBehindBuffer(absl::string_view src) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::DoneBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return;\n  Writer& dest = *DestWriter();\n  WriteInternal(src, dest, ZSTD_e_end);\n}\n\nvoid ZstdWriterBase::Done() {\n  BufferedWriter::Done();\n  compressor_.reset();\n  dictionary_ = ZstdDictionary();\n  associated_reader_.Reset();\n}\n\nabsl::Status ZstdWriterBase::AnnotateStatusImpl(absl::Status status) {\n  if (is_open()) {\n    Writer& dest = *DestWriter();\n    status = dest.AnnotateStatus(std::move(status));\n  }\n  // The status might have been annotated by `dest` with the compressed\n  // position. Clarify that the current position is the uncompressed position\n  // instead of delegating to `BufferedWriter::AnnotateStatusImpl()`.\n  return AnnotateOverDest(std::move(status));\n}\n\nabsl::Status ZstdWriterBase::AnnotateOverDest(absl::Status status) {\n  if (is_open()) {\n    return Annotate(status, absl::StrCat(\"at uncompressed byte \", pos()));\n  }\n  return status;\n}\n\nvoid ZstdWriterBase::SetWriteSizeHintImpl(\n    std::optional<Position> write_size_hint) {\n  BufferedWriter::SetWriteSizeHintImpl(write_size_hint);\n  if (ABSL_PREDICT_FALSE(!ok()) || compressor_ == nullptr) return;\n  // Ignore failure if compression already started.\n  ZSTD_CCtx_setParameter(\n      compressor_.get(), ZSTD_c_srcSizeHint,\n      write_size_hint == std::nullopt\n          ? 0\n          : SaturatingIntCast<int>(SaturatingAdd(pos(), *write_size_hint)));\n}\n\nbool ZstdWriterBase::WriteInternal(absl::string_view src) {\n  RIEGELI_ASSERT(!src.empty())\n      << \"Failed precondition of BufferedWriter::WriteInternal(): \"\n         \"nothing to write\";\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of BufferedWriter::WriteInternal()\";\n  Writer& dest = *DestWriter();\n  return WriteInternal(src, dest, ZSTD_e_continue);\n}\n\ninline bool ZstdWriterBase::WriteInternal(absl::string_view src, Writer& dest,\n                                          ZSTD_EndDirective end_op) {\n  RIEGELI_ASSERT_OK(*this)\n      << \"Failed precondition of ZstdWriterBase::WriteInternal()\";\n  if (ABSL_PREDICT_FALSE(src.size() >\n                         std::numeric_limits<Position>::max() - start_pos())) {\n    return FailOverflow();\n  }\n  if (pledged_size_ != std::nullopt) {\n    const Position next_pos = start_pos() + src.size();\n    if (compressor_ == nullptr) {\n      if (ABSL_PREDICT_FALSE(!src.empty())) {\n        return Fail(absl::FailedPreconditionError(\n            absl::StrCat(\"Actual size does not match pledged size: \", next_pos,\n                         \" > \", *pledged_size_)));\n      }\n      return true;\n    }\n    if (next_pos >= *pledged_size_) {\n      // Notify `compressor_` that this is the last fragment. This enables\n      // optimizations (compressing directly to a long enough output buffer).\n      end_op = ZSTD_e_end;\n      if (reserve_max_size_ && start_pos() == 0) {\n        // Ensure that the output buffer is actually long enough.\n        dest.Push(ZSTD_compressBound(*pledged_size_));\n      }\n    }\n    if (end_op == ZSTD_e_end) {\n      if (ABSL_PREDICT_FALSE(next_pos != *pledged_size_)) {\n        return Fail(absl::FailedPreconditionError(absl::StrCat(\n            \"Actual size does not match pledged size: \", next_pos,\n            next_pos > *pledged_size_ ? \" > \" : \" < \", *pledged_size_)));\n      }\n    }\n  }\n  ZSTD_inBuffer input = {src.data(), src.size(), 0};\n  for (;;) {\n    ZSTD_outBuffer output = {dest.cursor(), dest.available(), 0};\n    const size_t result =\n        ZSTD_compressStream2(compressor_.get(), &output, &input, end_op);\n    dest.set_cursor(static_cast<char*>(output.dst) + output.pos);\n    if (result == 0) {\n      RIEGELI_ASSERT_EQ(input.pos, input.size)\n          << \"ZSTD_compressStream2() returned 0 but there are still input data\";\n      move_start_pos(input.pos);\n      if (end_op == ZSTD_e_end) compressor_.reset();\n      return true;\n    }\n    if (ABSL_PREDICT_FALSE(ZSTD_isError(result))) {\n      return Fail(absl::InternalError(absl::StrCat(\n          \"ZSTD_compressStream2() failed: \", ZSTD_getErrorName(result))));\n    }\n    if (output.pos < output.size) {\n      RIEGELI_ASSERT_EQ(input.pos, input.size)\n          << \"ZSTD_compressStream2() returned but there are still input data \"\n             \"and output space\";\n      move_start_pos(input.pos);\n      return true;\n    }\n    if (ABSL_PREDICT_FALSE(!dest.Push(1, result))) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    }\n  }\n}\n\nbool ZstdWriterBase::FlushBehindBuffer(absl::string_view src,\n                                       FlushType flush_type) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::FlushBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ok())) return false;\n  Writer& dest = *DestWriter();\n  return WriteInternal(src, dest, ZSTD_e_flush);\n}\n\nbool ZstdWriterBase::SupportsReadMode() {\n  Writer* const dest = DestWriter();\n  return dest != nullptr && dest->SupportsReadMode();\n}\n\nReader* ZstdWriterBase::ReadModeBehindBuffer(Position initial_pos) {\n  RIEGELI_ASSERT_EQ(start_to_limit(), 0u)\n      << \"Failed precondition of BufferedWriter::ReadModeBehindBuffer(): \"\n         \"buffer not empty\";\n  if (ABSL_PREDICT_FALSE(!ZstdWriterBase::FlushBehindBuffer(\n          absl::string_view(), FlushType::kFromObject))) {\n    return nullptr;\n  }\n  Writer& dest = *DestWriter();\n  Reader* const compressed_reader = dest.ReadMode(initial_compressed_pos_);\n  if (ABSL_PREDICT_FALSE(compressed_reader == nullptr)) {\n    FailWithoutAnnotation(AnnotateOverDest(dest.status()));\n    return nullptr;\n  }\n  ZstdReader<>* const reader = associated_reader_.ResetReader(\n      compressed_reader,\n      ZstdReaderBase::Options()\n          .set_dictionary(dictionary_)\n          .set_buffer_options(buffer_options())\n          .set_recycling_pool_options(recycling_pool_options_));\n  reader->Seek(initial_pos);\n  return reader;\n}\n\n}  // namespace riegeli\n"
  },
  {
    "path": "riegeli/zstd/zstd_writer.h",
    "content": "// Copyright 2017 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//      http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef RIEGELI_ZSTD_ZSTD_WRITER_H_\n#define RIEGELI_ZSTD_ZSTD_WRITER_H_\n\n#include <stddef.h>\n\n#include <optional>\n#include <utility>\n\n#include \"absl/base/attributes.h\"\n#include \"absl/base/optimization.h\"\n#include \"absl/status/status.h\"\n#include \"absl/strings/string_view.h\"\n#include \"riegeli/base/arithmetic.h\"\n#include \"riegeli/base/assert.h\"\n#include \"riegeli/base/dependency.h\"\n#include \"riegeli/base/initializer.h\"\n#include \"riegeli/base/object.h\"\n#include \"riegeli/base/recycling_pool.h\"\n#include \"riegeli/base/types.h\"\n#include \"riegeli/bytes/buffer_options.h\"\n#include \"riegeli/bytes/buffered_writer.h\"\n#include \"riegeli/bytes/writer.h\"\n#include \"riegeli/zstd/zstd_dictionary.h\"  // IWYU pragma: export\n#include \"zstd.h\"\n\nnamespace riegeli {\n\nclass Reader;\ntemplate <typename Src>\nclass ZstdReader;\n\n// Template parameter independent part of `ZstdWriter`.\nclass ZstdWriterBase : public BufferedWriter {\n public:\n  class Options : public BufferOptionsBase<Options> {\n   public:\n    Options() noexcept {}\n\n    // Tunes the tradeoff between compression density and compression speed\n    // (higher = better density but slower).\n    //\n    // `compression_level` must be between `kMinCompressionLevel` (-131072) and\n    // `kMaxCompressionLevel` (22). Level 0 is currently equivalent to 3.\n    // Default: `kDefaultCompressionLevel` (3).\n    static constexpr int kMinCompressionLevel =\n        -(1 << 17);                                  // `ZSTD_minCLevel()`\n    static constexpr int kMaxCompressionLevel = 22;  // `ZSTD_maxCLevel()`\n    static constexpr int kDefaultCompressionLevel = 3;\n    Options& set_compression_level(int compression_level) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      RIEGELI_ASSERT_GE(compression_level, kMinCompressionLevel)\n          << \"Failed precondition of \"\n             \"ZstdWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      RIEGELI_ASSERT_LE(compression_level, kMaxCompressionLevel)\n          << \"Failed precondition of \"\n             \"ZstdWriterBase::Options::set_compression_level(): \"\n             \"compression level out of range\";\n      compression_level_ = compression_level;\n      return *this;\n    }\n    Options&& set_compression_level(int compression_level) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_compression_level(compression_level));\n    }\n    int compression_level() const { return compression_level_; }\n\n    // Logarithm of the LZ77 sliding window size. This tunes the tradeoff\n    // between compression density and memory usage (higher = better density but\n    // more memory).\n    //\n    // Special value `std::nullopt` means to derive `window_log` from\n    // `compression_level` and `size_hint`.\n    //\n    // `window_log` must be `std::nullopt` or between `kMinWindowLog` (10) and\n    // `kMaxWindowLog` (30 in 32-bit build, 31 in 64-bit build). Default:\n    // `std::nullopt`.\n    static constexpr int kMinWindowLog = 10;  // `ZSTD_WINDOWLOG_MIN`\n    static constexpr int kMaxWindowLog =\n        sizeof(size_t) == 4 ? 30 : 31;  // `ZSTD_WINDOWLOG_MAX`\n    Options& set_window_log(std::optional<int> window_log) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      if (window_log != std::nullopt) {\n        RIEGELI_ASSERT_GE(*window_log, kMinWindowLog)\n            << \"Failed precondition of \"\n               \"ZstdWriterBase::Options::set_window_log(): \"\n               \"window log out of range\";\n        RIEGELI_ASSERT_LE(*window_log, kMaxWindowLog)\n            << \"Failed precondition of \"\n               \"ZstdWriterBase::Options::set_window_log(): \"\n               \"window log out of range\";\n      }\n      window_log_or_0_ = window_log.value_or(0);\n      return *this;\n    }\n    Options&& set_window_log(std::optional<int> window_log) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_window_log(window_log));\n    }\n    std::optional<int> window_log() const {\n      if (window_log_or_0_ == 0) return std::nullopt;\n      return window_log_or_0_;\n    }\n\n    // Attempts to fit compressed block size into approximately\n    // `target_cblock_size`. A lower value allows the decompressor to begin\n    // decompression sooner, having less data available, at the cost of\n    // reducing compression density.\n    //\n    // Special value `std::nullopt` means no target.\n    //\n    // `target_cblock_size` must be `std::nullopt` or between\n    // `kMinTargetCBlockSize` (1340) and `kMaxTargetCBlockSize` (131072).\n    // Default: `std::nullopt`.\n    static constexpr int kMinTargetCBlockSize =\n        1340;  // `ZSTD_TARGETCBLOCKSIZE_MIN`\n    static constexpr int kMaxTargetCBlockSize =\n        1 << 17;  // `ZSTD_TARGETCBLOCKSIZE_MAX`\n    Options& set_target_cblock_size(std::optional<int> target_cblock_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      if (target_cblock_size != std::nullopt) {\n        RIEGELI_ASSERT_GE(*target_cblock_size, kMinTargetCBlockSize)\n            << \"Failed precondition of \"\n               \"ZstdWriterBase::Options::set_target_cblock_size(): \"\n               \"target CBlock size out of range\";\n        RIEGELI_ASSERT_LE(*target_cblock_size, kMaxTargetCBlockSize)\n            << \"Failed precondition of \"\n               \"ZstdWriterBase::Options::set_target_cblock_size(): \"\n               \"target CBlock size out of range\";\n      }\n      target_cblock_size_or_0_ = target_cblock_size.value_or(0);\n      return *this;\n    }\n    Options&& set_target_cblock_size(std::optional<int> target_cblock_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_target_cblock_size(target_cblock_size));\n    }\n    std::optional<int> target_cblock_size() const {\n      if (target_cblock_size_or_0_ == 0) return std::nullopt;\n      return target_cblock_size_or_0_;\n    }\n\n    // Zstd dictionary. The same dictionary must be used for decompression.\n    //\n    // Default: `ZstdDictionary()`.\n    Options& set_dictionary(ZstdDictionary dictionary) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      dictionary_ = std::move(dictionary);\n      return *this;\n    }\n    Options&& set_dictionary(ZstdDictionary dictionary) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_dictionary(std::move(dictionary)));\n    }\n    ZstdDictionary& dictionary() ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n    const ZstdDictionary& dictionary() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return dictionary_;\n    }\n\n    // If `true`, computes checksum of uncompressed data and stores it in the\n    // compressed stream. This lets decompression verify the checksum.\n    //\n    // Default: `false`.\n    Options& set_store_checksum(bool store_checksum) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      store_checksum_ = store_checksum;\n      return *this;\n    }\n    Options&& set_store_checksum(bool store_checksum) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_store_checksum(store_checksum));\n    }\n    bool store_checksum() const { return store_checksum_; }\n\n    // Exact uncompressed size, or `std::nullopt` if unknown. This may improve\n    // compression density and performance, and causes the size to be stored in\n    // the compressed stream header.\n    //\n    // If the pledged size turns out to not match reality, compression fails.\n    //\n    // Default: `std::nullopt`.\n    Options& set_pledged_size(std::optional<Position> pledged_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      pledged_size_ = pledged_size;\n      return *this;\n    }\n    Options&& set_pledged_size(std::optional<Position> pledged_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_pledged_size(pledged_size));\n    }\n    std::optional<Position> pledged_size() const { return pledged_size_; }\n\n    // If `false`, `ZstdWriter` lets the destination choose buffer sizes.\n    //\n    // If `true`, `ZstdWriter` tries to compress all data in one step:\n    //\n    //  * Flattens uncompressed data if `pledged_size()` is not `std::nullopt`.\n    //\n    //  * Asks the destination for a flat buffer with the maximum possible\n    //    compressed size, as long as the uncompressed size is known before\n    //    compression begins, e.g. if `pledged_size()` is not `std::nullopt`.\n    //\n    // This makes compression slightly faster, but increases memory usage.\n    //\n    // Default: `false`.\n    Options& set_reserve_max_size(bool reserve_max_size) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      reserve_max_size_ = reserve_max_size;\n      return *this;\n    }\n    Options&& set_reserve_max_size(bool reserve_max_size) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_reserve_max_size(reserve_max_size));\n    }\n    bool reserve_max_size() const { return reserve_max_size_; }\n\n    // Returns effective `BufferOptions` as overridden by other options:\n    // If `reserve_max_size()` is `true` and `pledged_size()` is not\n    // `std::nullopt`, then `pledged_size()` overrides `buffer_size()`.\n    BufferOptions effective_buffer_options() const {\n      BufferOptions options = buffer_options();\n      if (reserve_max_size() && pledged_size() != std::nullopt) {\n        options.set_buffer_size(\n            UnsignedMax(SaturatingIntCast<size_t>(*pledged_size()), size_t{1}));\n      }\n      return options;\n    }\n\n    // Options for a global `RecyclingPool` of compression contexts.\n    //\n    // They tune the amount of memory which is kept to speed up creation of new\n    // compression sessions, and usage of a background thread to clean it.\n    //\n    // Default: `RecyclingPoolOptions()`.\n    Options& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      recycling_pool_options_ = recycling_pool_options;\n      return *this;\n    }\n    Options&& set_recycling_pool_options(\n        const RecyclingPoolOptions& recycling_pool_options) &&\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return std::move(set_recycling_pool_options(recycling_pool_options));\n    }\n    const RecyclingPoolOptions& recycling_pool_options() const\n        ABSL_ATTRIBUTE_LIFETIME_BOUND {\n      return recycling_pool_options_;\n    }\n\n   private:\n    int compression_level_ = kDefaultCompressionLevel;\n    int window_log_or_0_ = 0;\n    int target_cblock_size_or_0_ = 0;\n    ZstdDictionary dictionary_;\n    bool store_checksum_ = false;\n    std::optional<Position> pledged_size_;\n    bool reserve_max_size_ = false;\n    RecyclingPoolOptions recycling_pool_options_;\n  };\n\n  // Returns the compressed `Writer`. Unchanged by `Close()`.\n  virtual Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND = 0;\n\n  bool SupportsReadMode() override;\n\n protected:\n  explicit ZstdWriterBase(Closed) noexcept : BufferedWriter(kClosed) {}\n\n  explicit ZstdWriterBase(BufferOptions buffer_options,\n                          ZstdDictionary&& dictionary,\n                          std::optional<Position> pledged_size,\n                          bool reserve_max_size,\n                          const RecyclingPoolOptions& recycling_pool_options);\n\n  ZstdWriterBase(ZstdWriterBase&& that) noexcept;\n  ZstdWriterBase& operator=(ZstdWriterBase&& that) noexcept;\n\n  void Reset(Closed);\n  void Reset(BufferOptions buffer_options, ZstdDictionary&& dictionary,\n             std::optional<Position> pledged_size, bool reserve_max_size,\n             const RecyclingPoolOptions& recycling_pool_options);\n  void Initialize(Writer* dest, int compression_level, int window_log_or_0,\n                  int target_cblock_size_or_0, bool store_checksum);\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateOverDest(absl::Status status);\n\n  void DoneBehindBuffer(absl::string_view src) override;\n  void Done() override;\n  ABSL_ATTRIBUTE_COLD absl::Status AnnotateStatusImpl(\n      absl::Status status) override;\n  void SetWriteSizeHintImpl(std::optional<Position> write_size_hint) override;\n  bool WriteInternal(absl::string_view src) override;\n  bool FlushBehindBuffer(absl::string_view src, FlushType flush_type) override;\n  Reader* ReadModeBehindBuffer(Position initial_pos) override;\n\n private:\n  struct ZSTD_CCtxDeleter {\n    void operator()(ZSTD_CCtx* ptr) const { ZSTD_freeCCtx(ptr); }\n  };\n\n  bool WriteInternal(absl::string_view src, Writer& dest,\n                     ZSTD_EndDirective end_op);\n\n  ZstdDictionary dictionary_;\n  ZstdDictionary::ZSTD_CDictHandle compression_dictionary_;\n  std::optional<Position> pledged_size_;\n  bool reserve_max_size_ = false;\n  Position initial_compressed_pos_ = 0;\n  RecyclingPoolOptions recycling_pool_options_;\n  // If `ok()` but `compressor_ == nullptr` then `*pledged_size_` has been\n  // reached. In this case `ZSTD_compressStream()` must not be called again.\n  RecyclingPool<ZSTD_CCtx, ZSTD_CCtxDeleter>::Handle compressor_;\n\n  AssociatedReader<ZstdReader<Reader*>> associated_reader_;\n};\n\n// A `Writer` which compresses data with Zstd before passing it to another\n// `Writer`.\n//\n// The `Dest` template parameter specifies the type of the object providing and\n// possibly owning the compressed `Writer`. `Dest` must support\n// `Dependency<Writer*, Dest>`, e.g. `Writer*` (not owned, default),\n// `ChainWriter<>` (owned), `std::unique_ptr<Writer>` (owned),\n// `Any<Writer*>` (maybe owned).\n//\n// By relying on CTAD the template argument can be deduced as `TargetT` of the\n// type of the first constructor argument.\n//\n// The compressed `Writer` must not be accessed until the `ZstdWriter` is closed\n// or no longer used, except that it is allowed to read the destination of the\n// compressed `Writer` immediately after `Flush()`.\ntemplate <typename Dest = Writer*>\nclass ZstdWriter : public ZstdWriterBase {\n public:\n  // Creates a closed `ZstdWriter`.\n  explicit ZstdWriter(Closed) noexcept : ZstdWriterBase(kClosed) {}\n\n  // Will write to the compressed `Writer` provided by `dest`.\n  explicit ZstdWriter(Initializer<Dest> dest, Options options = Options());\n\n  ZstdWriter(ZstdWriter&& that) = default;\n  ZstdWriter& operator=(ZstdWriter&& that) = default;\n\n  // Makes `*this` equivalent to a newly constructed `ZstdWriter`. This avoids\n  // constructing a temporary `ZstdWriter` and moving from it.\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Closed);\n  ABSL_ATTRIBUTE_REINITIALIZES void Reset(Initializer<Dest> dest,\n                                          Options options = Options());\n\n  // Returns the object providing and possibly owning the compressed `Writer`.\n  // Unchanged by `Close()`.\n  Dest& dest() ABSL_ATTRIBUTE_LIFETIME_BOUND { return dest_.manager(); }\n  const Dest& dest() const ABSL_ATTRIBUTE_LIFETIME_BOUND {\n    return dest_.manager();\n  }\n  Writer* DestWriter() const ABSL_ATTRIBUTE_LIFETIME_BOUND override {\n    return dest_.get();\n  }\n\n protected:\n  void Done() override;\n  bool FlushImpl(FlushType flush_type) override;\n\n private:\n  // The object providing and possibly owning the compressed `Writer`.\n  Dependency<Writer*, Dest> dest_;\n};\n\nexplicit ZstdWriter(Closed) -> ZstdWriter<DeleteCtad<Closed>>;\ntemplate <typename Dest>\nexplicit ZstdWriter(Dest&& dest,\n                    ZstdWriterBase::Options options = ZstdWriterBase::Options())\n    -> ZstdWriter<TargetT<Dest>>;\n\n// Implementation details follow.\n\ninline ZstdWriterBase::ZstdWriterBase(\n    BufferOptions buffer_options, ZstdDictionary&& dictionary,\n    std::optional<Position> pledged_size, bool reserve_max_size,\n    const RecyclingPoolOptions& recycling_pool_options)\n    : BufferedWriter(buffer_options),\n      dictionary_(std::move(dictionary)),\n      pledged_size_(pledged_size),\n      reserve_max_size_(reserve_max_size),\n      recycling_pool_options_(recycling_pool_options) {}\n\ninline ZstdWriterBase::ZstdWriterBase(ZstdWriterBase&& that) noexcept\n    : BufferedWriter(static_cast<BufferedWriter&&>(that)),\n      dictionary_(std::move(that.dictionary_)),\n      compression_dictionary_(std::move(that.compression_dictionary_)),\n      pledged_size_(that.pledged_size_),\n      reserve_max_size_(that.reserve_max_size_),\n      initial_compressed_pos_(that.initial_compressed_pos_),\n      recycling_pool_options_(that.recycling_pool_options_),\n      compressor_(std::move(that.compressor_)),\n      associated_reader_(std::move(that.associated_reader_)) {}\n\ninline ZstdWriterBase& ZstdWriterBase::operator=(\n    ZstdWriterBase&& that) noexcept {\n  BufferedWriter::operator=(static_cast<BufferedWriter&&>(that));\n  dictionary_ = std::move(that.dictionary_);\n  compression_dictionary_ = std::move(that.compression_dictionary_);\n  pledged_size_ = that.pledged_size_;\n  reserve_max_size_ = that.reserve_max_size_;\n  initial_compressed_pos_ = that.initial_compressed_pos_;\n  recycling_pool_options_ = that.recycling_pool_options_;\n  compressor_ = std::move(that.compressor_);\n  associated_reader_ = std::move(that.associated_reader_);\n  return *this;\n}\n\ninline void ZstdWriterBase::Reset(Closed) {\n  BufferedWriter::Reset(kClosed);\n  pledged_size_ = std::nullopt;\n  reserve_max_size_ = false;\n  initial_compressed_pos_ = 0;\n  recycling_pool_options_ = RecyclingPoolOptions();\n  compressor_.reset();\n  // Must be destroyed after `compressor_`.\n  dictionary_ = ZstdDictionary();\n  compression_dictionary_.reset();\n  associated_reader_.Reset();\n}\n\ninline void ZstdWriterBase::Reset(\n    BufferOptions buffer_options, ZstdDictionary&& dictionary,\n    std::optional<Position> pledged_size, bool reserve_max_size,\n    const RecyclingPoolOptions& recycling_pool_options) {\n  BufferedWriter::Reset(buffer_options);\n  pledged_size_ = pledged_size;\n  reserve_max_size_ = reserve_max_size;\n  initial_compressed_pos_ = 0;\n  recycling_pool_options_ = recycling_pool_options;\n  compressor_.reset();\n  dictionary_ = std::move(dictionary);\n  compression_dictionary_.reset();\n  associated_reader_.Reset();\n}\n\ntemplate <typename Dest>\ninline ZstdWriter<Dest>::ZstdWriter(Initializer<Dest> dest, Options options)\n    : ZstdWriterBase(options.effective_buffer_options(),\n                     std::move(options.dictionary()), options.pledged_size(),\n                     options.reserve_max_size(),\n                     options.recycling_pool_options()),\n      dest_(std::move(dest)) {\n  Initialize(dest_.get(), options.compression_level(),\n             options.window_log().value_or(0),\n             options.target_cblock_size().value_or(0),\n             options.store_checksum());\n}\n\ntemplate <typename Dest>\ninline void ZstdWriter<Dest>::Reset(Closed) {\n  ZstdWriterBase::Reset(kClosed);\n  dest_.Reset();\n}\n\ntemplate <typename Dest>\ninline void ZstdWriter<Dest>::Reset(Initializer<Dest> dest, Options options) {\n  ZstdWriterBase::Reset(options.effective_buffer_options(),\n                        std::move(options.dictionary()), options.pledged_size(),\n                        options.reserve_max_size(),\n                        options.recycling_pool_options());\n  dest_.Reset(std::move(dest));\n  Initialize(dest_.get(), options.compression_level(),\n             options.window_log().value_or(0),\n             options.target_cblock_size().value_or(0),\n             options.store_checksum());\n}\n\ntemplate <typename Dest>\nvoid ZstdWriter<Dest>::Done() {\n  ZstdWriterBase::Done();\n  if (dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Close())) {\n      FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n}\n\ntemplate <typename Dest>\nbool ZstdWriter<Dest>::FlushImpl(FlushType flush_type) {\n  if (ABSL_PREDICT_FALSE(!ZstdWriterBase::FlushImpl(flush_type))) return false;\n  if (flush_type != FlushType::kFromObject || dest_.IsOwning()) {\n    if (ABSL_PREDICT_FALSE(!dest_->Flush(flush_type))) {\n      return FailWithoutAnnotation(AnnotateOverDest(dest_->status()));\n    }\n  }\n  return true;\n}\n\n}  // namespace riegeli\n\n#endif  // RIEGELI_ZSTD_ZSTD_WRITER_H_\n"
  },
  {
    "path": "tf_dependency/BUILD",
    "content": ""
  },
  {
    "path": "tf_dependency/BUILD.tpl",
    "content": "package(\n    default_visibility = [\"//visibility:public\"],\n    features = [\"header_modules\"],\n)\n\ncc_library(\n    name = \"tf_header_lib\",\n    hdrs = [\":tf_header_include\"],\n    includes = [\"include\"],\n    visibility = [\"//visibility:public\"],\n)\n\ncc_library(\n    name = \"libtensorflow_framework\",\n    srcs = [\":%{TF_SHARED_LIBRARY_NAME}\"],\n    visibility = [\"//visibility:public\"],\n)\n\n%{TF_HEADER_GENRULE}\n%{TF_SHARED_LIBRARY_GENRULE}\n"
  },
  {
    "path": "tf_dependency/tf_configure.bzl",
    "content": "\"\"\"Setup TensorFlow as external dependency\"\"\"\n\n_TF_HEADER_DIR = \"TF_HEADER_DIR\"\n_TF_SHARED_LIBRARY_DIR = \"TF_SHARED_LIBRARY_DIR\"\n_TF_SHARED_LIBRARY_NAME = \"TF_SHARED_LIBRARY_NAME\"\n\ndef _tpl(repository_ctx, tpl, substitutions = {}, out = None):\n    if not out:\n        out = tpl\n    repository_ctx.template(\n        out,\n        Label(\"//tf_dependency:{}.tpl\".format(tpl)),\n        substitutions,\n    )\n\ndef _fail(msg):\n    \"\"\"Outputs failure message when auto configuration fails.\"\"\"\n    red = \"\\033[0;31m\"\n    no_color = \"\\033[0m\"\n    fail(\"{}Python Configuration Error:{} {}\\n\".format(red, no_color, msg))\n\ndef _is_windows(repository_ctx):\n    \"\"\"Returns true if the host operating system is windows.\"\"\"\n    os_name = repository_ctx.os.name.lower()\n    return \"windows\" in os_name\n\ndef _execute(\n        repository_ctx,\n        cmdline,\n        error_msg = None,\n        error_details = None,\n        empty_stdout_fine = False):\n    \"\"\"Executes an arbitrary shell command.\n\n    Helper for executes an arbitrary shell command.\n\n    Args:\n      repository_ctx: the repository_ctx object.\n      cmdline: list of strings, the command to execute.\n      error_msg: string, a summary of the error if the command fails.\n      error_details: string, details about the error or steps to fix it.\n      empty_stdout_fine: bool, if True, an empty stdout result is fine,\n        otherwise it's an error.\n\n    Returns:\n      The result of repository_ctx.execute(cmdline).\n    \"\"\"\n    result = repository_ctx.execute(cmdline)\n    if result.stderr or not (empty_stdout_fine or result.stdout):\n        _fail(\"\\n\".join([\n            error_msg.strip() if error_msg else \"Repository command failed\",\n            result.stderr.strip(),\n            error_details if error_details else \"\",\n        ]))\n    return result\n\ndef _read_dir(repository_ctx, src_dir):\n    \"\"\"Returns a string with all files in a directory.\n\n    Finds all files inside a directory, traversing subfolders and following\n    symlinks. The returned string contains the full path of all files\n    separated by line breaks.\n\n    Args:\n        repository_ctx: the repository_ctx object.\n        src_dir: directory to find files from.\n\n    Returns:\n        A string of all files inside the given dir.\n    \"\"\"\n    if _is_windows(repository_ctx):\n        src_dir = src_dir.replace(\"/\", \"\\\\\")\n        find_result = _execute(\n            repository_ctx,\n            [\"cmd.exe\", \"/c\", \"dir\", src_dir, \"/b\", \"/s\", \"/a-d\"],\n            empty_stdout_fine = True,\n        )\n\n        # src_files will be used in genrule.outs where the paths must\n        # use forward slashes.\n        result = find_result.stdout.replace(\"\\\\\", \"/\")\n    else:\n        find_result = _execute(\n            repository_ctx,\n            [\"find\", src_dir, \"-follow\", \"-type\", \"f\"],\n            empty_stdout_fine = True,\n        )\n        result = find_result.stdout\n    return result\n\ndef _genrule(genrule_name, command, outs):\n    \"\"\"Returns a string with a genrule.\n\n    Genrule executes the given command and produces the given outputs.\n\n    Args:\n        genrule_name: A unique name for genrule target.\n        command: The command to run.\n        outs: A list of files generated by this rule.\n\n    Returns:\n        A genrule target.\n    \"\"\"\n    return (\n        \"genrule(\\n\" +\n        '    name = \"{}\",\\n' +\n        \"    outs = [\\n\" +\n        \"{}\\n\" +\n        \"    ],\\n\" +\n        '    cmd = \"\"\"\\n' +\n        \"{}\\n\" +\n        '   \"\"\",\\n' +\n        \")\\n\"\n    ).format(genrule_name, outs, command)\n\ndef _norm_path(path):\n    \"\"\"Returns a path with '/' and removes the trailing slash.\"\"\"\n    return path.replace(\"\\\\\", \"/\").rstrip(\"/\")\n\ndef _symlink_genrule_for_dir(\n        repository_ctx,\n        src_dir,\n        dest_dir,\n        genrule_name,\n        src_files = [],\n        dest_files = []):\n    \"\"\"Returns a genrule to symlink(or copy if on Windows) a set of files.\n\n    If src_dir is passed, files will be read from the given directory; otherwise\n    we assume files are in src_files and dest_files.\n\n    Args:\n        repository_ctx: the repository_ctx object.\n        src_dir: source directory.\n        dest_dir: directory to create symlink in.\n        genrule_name: genrule name.\n        src_files: list of source files instead of src_dir.\n        dest_files: list of corresonding destination files.\n\n    Returns:\n        genrule target that creates the symlinks.\n    \"\"\"\n    if src_dir != None:\n        src_dir = _norm_path(src_dir)\n        dest_dir = _norm_path(dest_dir)\n        files = \"\\n\".join(\n            sorted(_read_dir(repository_ctx, src_dir).splitlines()),\n        )\n\n        # Create a list with the src_dir stripped to use for outputs.\n        dest_files = files.replace(src_dir, \"\").splitlines()\n        src_files = files.splitlines()\n    command = []\n    outs = []\n    for i in range(len(dest_files)):\n        if dest_files[i] != \"\":\n            # If we have only one file to link we do not want to use the\n            # dest_dir, as $(@D) will include the full path to the file.\n            dest = \"$(@D)/{}{}\".format(\n                dest_dir if len(dest_files) != 1 else \"\",\n                dest_files[i],\n            )\n\n            # Copy the headers to create a sandboxable setup.\n            cmd = \"cp -f\"\n            command.append('{} \"{}\" \"{}\"'.format(cmd, src_files[i], dest))\n            outs.append('        \"{}{}\",'.format(dest_dir, dest_files[i]))\n    genrule = _genrule(\n        genrule_name,\n        \" && \".join(command),\n        \"\\n\".join(outs),\n    )\n    return genrule\n\ndef _tf_pip_impl(repository_ctx):\n    tf_header_dir = repository_ctx.os.environ[_TF_HEADER_DIR]\n    tf_header_rule = _symlink_genrule_for_dir(\n        repository_ctx,\n        tf_header_dir,\n        \"include\",\n        \"tf_header_include\",\n    )\n\n    tf_shared_library_dir = repository_ctx.os.environ[_TF_SHARED_LIBRARY_DIR]\n    tf_shared_library_name = repository_ctx.os.environ[_TF_SHARED_LIBRARY_NAME]\n    tf_shared_library_path = \"{}/{}\".format(\n        tf_shared_library_dir,\n        tf_shared_library_name,\n    )\n\n    tf_shared_library_rule = _symlink_genrule_for_dir(\n        repository_ctx,\n        None,\n        \"\",\n        tf_shared_library_name,\n        [tf_shared_library_path],\n        [tf_shared_library_name],\n    )\n\n    _tpl(repository_ctx, \"BUILD\", {\n        \"%{TF_HEADER_GENRULE}\": tf_header_rule,\n        \"%{TF_SHARED_LIBRARY_GENRULE}\": tf_shared_library_rule,\n        \"%{TF_SHARED_LIBRARY_NAME}\": tf_shared_library_name,\n    })\n\ntf_configure = repository_rule(\n    implementation = _tf_pip_impl,\n    environ = [\n        _TF_HEADER_DIR,\n        _TF_SHARED_LIBRARY_DIR,\n        _TF_SHARED_LIBRARY_NAME,\n    ],\n)\n"
  }
]