[ { "path": ".clang-format", "content": "BasedOnStyle: google\nSpaceBeforeParens: Always\nIndentCaseLabels: false\nAllowShortIfStatementsOnASingleLine: true\nAllowShortLoopsOnASingleLine: true\nSpaceAfterCStyleCast: true\nPointerAlignment: Right\nBreakBeforeBinaryOperators: All\nConstructorInitializerIndentWidth: 2\nContinuationIndentWidth: 2\nPenaltyBreakBeforeFirstCallParameter: 10000\nSortIncludes: false\nBreakStringLiterals: true\nBreakBeforeTernaryOperators: true\nAllowShortCaseLabelsOnASingleLine: true\n#AllowShortEnumsOnASingleLine: true\nColumnLimit: 100\nMaxEmptyLinesToKeep: 1\n#StatementMacros: [ 'REP2', 'REP3', 'REP4', 'REP5', 'REP6', 'REP7', 'REP8' ]\n#TypenameMacros: [ 'VARR', 'DLIST', 'HTAB' ]\n" }, { "path": "ChangeLog", "content": "2018-11-02 Vladimir Makarov \n\n\t* README.md: Add update about mum-prng. Correct typo for\n\txoshiro512** result.\n\t* mum-prng.h (_mum_prng_state): Change avx2_support onto\n\tupdate_func.\n\t(_mum_prng_setup_avx2): Setup update_func. Move below.\n\t(_start_mum_prng): Setup update_func for non x86-64. Move below.\n\t(init_mum_prng, set_mum_prng_seed): Move below.\n\t(_mum_prng_update_avx2, _mum_prng_update): Update a state word\n\tfrom the next word.\n\t(get_mum_prn): Simplify.\n\t* src/bench-prng: Use env. variable MUM_ONLY.\n\n2018-10-31 Vladimir Makarov \n\n\t* README.md: Minor editions. Add RAND failure on practrand.\n\n2018-10-31 Vladimir Makarov \n\n\t* README.md: Update for PRNGs and MUM-512.\n\t* mum-512.h (_MC_FRESH_GCC, _mc_hash_avx2): Removed.\n\t(_mc_hash_default): Don't check _MC_UNALIGNED_ACCESS.\n\t(mum512_keyed_hash): Remove avx2 probe. Use _mc_hash_aligned.\n\t* mum-prng.h (_mum_avx2): New.\n\t(_mum_prng_update_avx2): Use it.\n\t* src/bench-prng: Add new xo[ro]shiro tests.\n\t* src/bench-prng.c: Ditto. Add code for PRNs output.\n\t* src/xoroshiro128plus.c, src/xoroshiro128starstar.c: New files.\n\t* src/xoshiro256plus.c, src/xoshiro256starstar.c: New files.\n\t* src/xoshiro512plus.c, src/xoshiro512starstar.c: New files.\n\t* src/splitmix64.c, src/xoseed.c: New files.\n\n2018-10-30 Vladimir Makarov \n\n\t* README.md: Descrition of new version of mum-hash. New results\n\tfor modern CPUs.\n\t* mum-hash (_MUM_FRESH_GCC): Remove.\n\t(_mum_rotl): New.\n\t(_mum_hash_aligned, _mum_final): Add new version. Use MUM_V1 for\n\tthe old code.\n\t(_mum_hash_avx2): Remove.\n\t(_mum_hash_default, mum_hash): Modify and simplify.\n\t* src/bench: Use environment variable MUM_ONLY for runing mum-hash\n\tonly. Add Meow Hash runs. Use 16MB keys instead of 1KB ones.\n\t* src/bench.c (meowhash_test): New.\n\t(main): Use 16MB keys instead of 1KB ones.\n\t* meow_hash.h: New.\n\n2016-08-10 Aras Pranckevičius \n\t Vladimir Makarov \n\n\t* mum.h: Permit unaligned access for _M_AMD64 and _M_IX86\n\t(Windows).\n\t* mum512.h: Ditto.\n\t* bench: Use CC instead of CXX for siphash.\n\n2016-07-13 Vladimir Makarov \n\n\t* mum.h (_mum_hash_aligned, mum_hash_randomize): Make i type of\n\tsize_t.\n\t* mum512.h (_mc_hash_aligned): Ditto.\n\t(_mc_init_state, _mc_hash_avx2, _mc_hash_default): Use cont for\n\tseed.\n\n2016-06-14 Vladimir Makarov \n\n\t* README.md: Add results for Blake2.\n\t* src/{blake2b.c, blake2b-load-sse2.h, blake2b-load-sse41.h}: New.\n\t* src/{blake2b-round.h, blake2-config.h, blake2.h, blake2-impl.h}:\n\tNew.\n\t* src/bench-crypto.c: Add code for testing Blake2.\n\t* src/bench-crypto: Ditto.\n\n2016-06-07 Vladimir Makarov \n\n\t* README.md: Update speed numbers for all functions for aarch64.\n\ny2016-06-06 Vladimir Makarov \n\n\t* src/bench.c: Use faster interface for xxHash.\n\t* README.md: Update xxHash speed numbers for x86-64 and ppc64.\n\n2016-05-18 Vladimir Makarov \n\n\t* README.md: Update speed data for MUM and MUM512 and add info\n\tabout testing MUM PRNG on NIST bigger data.\n\n2016-05-18 Vladimir Makarov \n\t Vsevolod Stakhov \n\n\t* mum512.h (_MC_FRESH_GCC): New. Use it as a guard for avx2 version.\n\t* mum-prng.h (_MUM_PRNG_FRESH_GCC): New. Use it as a guard for\n\tavx2 version.\n\t* mum.h (_MUM_FRESH_GCC): New. Use it as a guard for avx2 version.\n\tRemove clang guard for _MUM_OPTIMIZE etc.\n\n2016-05-18 Vladimir Makarov \n\n\t* src/mum-prng.h: Move it to parent directory\n\n2016-05-13 Vladimir Makarov \n\n\t* README.md: Some minor changes.\n\n2016-05-13 Vladimir Makarov \n\t Vsevolod Stakhov \n\n\t* src/bench.c: Include test for metro hash.\n\t* src/bench (COPTFLAGS, CC, CXX, LTO): New. Use them. Add runs for\n\tmetro hash.\n\t* README.md: Update benchmark results, add results for MetroHash.\n\t* metrohash64.cpp: New.\n\t* metrohash64.h: New.\n\t* platform.h: New.\n\n2016-05-13 Vsevolod Stakhov \n\n\t* mum.h: Add support for LLVM.\n\n2016-05-13 Vladimir Makarov \n\n\t* mum.h (_mum_le32): New.\n\t(_mum_hash_aligned): Change the code to deal with uint64_t shifts\n\tand endianess.\n\n2016-05-12 Vladimir Makarov \n\n\t* README.md: Add results for 5-byte string tests.\n\t* mum.h (uint16_t): New.\n\t(_mum_hash_aligned): Modify code to process a tail < 8 bytes.\n\t(_mum_hash_default): Use memmove instead of memcpy.\n\t* src/bench.c: Add test for 5-byte strings.\n\t* src/bench: Ditto.\n\n2016-05-10 Vladimir Makarov \n\n\t* README.md: Highlight the new MUM PRNG speed.\n\n2016-05-10 Vladimir Makarov \n\n\t* README.md: Describe a new version of MUM PRNG and update its\n\tspeed.\n\t* src/mum-prng.h (MUM_PRNG_UNROLL): New.\n\t(EXPECT): New.\n\t(mum_prng_state): Rename to _mum_prng_state. Add fields count and\n\tavx2_support. Make state an array.\n\t(_mum_prng_setup_avx2, _mum_prn_update, _mum_prn_avx2_update):\n\tNew.\n\t(_start_mum_prng): New.\n\t(init_mum_prng): Use _start_mum_prng.\n\t(set_mum_seed): Rename to set_mum_prng_seed. Use _start_mum_prng.\n\t(get_mum_prn): Rewrite.\n\t* src/bench-prng.c (init_prng): Randomize multiplication\n\tconstants.\n\n2016-05-09 Vladimir Makarov \n\n\t* src/mum-prng.h (init_mum_prng): Use seed == 1.\n\t(get_mum_prn): Fix prns generation.\n\t* README.md: Update result for MUM PRNG.\n\n2016-05-09 Vladimir Makarov \n\n\t* README.md: Add results for xxHash64 on AARCH64 and PPC64 and for\n\txoroshiro128+.\n\t* src/xoroshiro128plus.c: New.\n\t* src/bench-prng.c: Add a code to test xoroshiro128+.\n\t* src/bench-prng: Add a run to test speed of xoroshiro128+.\n\n2016-05-09 Vladimir Makarov \n\n\t* README.md: Add results for xxHash64.\n\t* src/xxhash.[ch]: New.\n\t* src/bench.c: Add code for xxHash64.\n\t(state, xxHash64_test): New.\n\t* src/bench: Add runs for xxHash64.\n\n2016-05-08 Vladimir Makarov \n\n\t* README.md: Some editing.\n\n2016-05-08 Vladimir Makarov \n\n\t* mum512.h (_mc_rotr): Decrease sh for sh >= 64.\n\n2016-05-08 Vladimir Makarov \n\n\t* mum512.h (_mc_ti): Define depending on endianess.\n\n2016-05-08 Vladimir Makarov \n\n\t* mum512.h (_mc_mul64): Change multiplication result names.\n\t(_mc_permute): Use _mc_xor.\n\n2016-05-08 Vladimir Makarov \n\n\tsrc/bench.c (main): Change input every iteration.\n\n2016-05-08 Vladimir Makarov \n\n\t* mum.h: New file.\n\t* mum512.h: New file.\n\t* README.md: New file.\n\t* src/bbs-prng.h: Ditto.\n\t* src/bench: Ditto.\n\t* src/bench.c: Ditto.\n\t* src/bench-crypto: Ditto.\n\t* src/bench-crypto.c: Ditto.\n\t* src/bench-prng: Ditto.\n\t* src/bench-prng.c: Ditto.\n\t* src/byte_order.[ch]: Ditto.\n\t* src/chacha-prng.h: Ditto.\n\t* src/City.cpp: Ditto.\n\t* src/City.h: Ditto.\n\t* src/mum512-prng.h: Ditto.\n\t* src/mum-prng.h: Ditto.\n\t* src/sha3.[ch]: Ditto.\n\t* src/sha512.[ch]: Ditto.\n\t* src/sip24-prng.h: Ditto.\n\t* src/siphash24.c: Ditto.\n\t* src/Spooky.cpp: Ditto.\n\t* src/Spooky.h: Ditto.\n\t* src/ustd.h: Ditto.\n\n" }, { "path": "README.md", "content": "# **Update (Nov. 28, 2025): Implemented collision attack prevention in VMUM and MUM-V3**\n* The attack is described in Issue#18\n* The code in question looks like\n```\n state ^= _vmum (data[i] ^ _vmum_factors[i], data[i + 1] ^ _vmum_factors[i + 1]));\n```\n It is easy to generate data which makes the 1st operand of `_vmum`\n to be zero. In this case whatever the second operand is, the hash\n will be generated the same. So an adversary can generate a lot of\n data with the same hash\n* This is pretty common code mistake for a few fast hash-functions. At\n least I found the same vulnerability in [wyhash](https://github.com/wangyi-fudan/wyhash/blob/46cebe9dc4e51f94d0dca287733bc5a94f76a10d/wyhash.h#L130) and [rapidhash](https://github.com/Nicoshev/rapidhash/blob/d60698faa10916879f85b2799bfdc6996b94c2b7/rapidhash.h#L383)\n* After the code change, the safe variants of VMUM and MUM hashes are\n switched on by default. If you want previous variants, please use\n macros VMUM_V1 and MUM_V3 correspondingly. I believe there are still\n cases when they can be used, e.g. for hash tables in compilers.\n* The fix consist of checking _vmum operands on zero and use nonzero value instead\n * all checks are implemented to avoid branch instruction generations to keep hash calculation pipeline going\n * still the checks increase length of critical paths of calculation\n * in most cases, new versions of VMUM and MUM generates the same hashes as the previous versions\n* The fix results in slowing down hash speeds by about **10%** according to my benchmarks\n * I updated all benchmark data related to the new versions of VMUM and MUM below\n\t\n# MUM Hash\n* MUM hash is a **fast non-cryptographic hash function**\n suitable for different hash table implementations\n* MUM means **MU**ltiply and **M**ix\n * It is a name of the base transformation on which hashing is implemented\n * Modern processors have a fast logic to do long number multiplications\n * It is very attractive to use it for fast hashing\n * For example, 64x64-bit multiplication can do the same work as 32\n shifts and additions\n * I'd like to call it Multiply and Reduce. Unfortunately, MUR\n (MUltiply and Rotate) is already taken for famous hashing\n technique designed by Austin Appleby\n * I've chosen the name also as the first release happened on Mother's day\n* To use mum you just need one header file (mum.h)\n* MUM hash passes **all** [SMHasher](https://github.com/aappleby/smhasher) tests\n * For comparison, only 4 out of 15 non-cryptographic hash functions\n in SMHasher passes the tests, e.g. well known FNV, Murmur2,\n Lookup, and Superfast hashes fail the tests\n* MUM V3 hash does not pass the following tests of a more rigourous\n version of [SMHasher](https://github.com/rurban/smhasher):\n * It fails on Perlin noise and bad seeds tests. It means it still\n qualitative enough for the most applications\n * To make MUM V3 to pass the Rurban SMHasher, macro `MUM_QUALITY` has been\n added. Compilation with this defined macro makes MUM V3 to pass\n all tests of Rurban SMHasher. The slowdown is about 5% in average\n or 10% at most on keys of length 8. It also results in generating\n a target independent hash\n* For historic reasons mum.h contains code for older version V1 and\n V2. You can switch them on by defining macros **MUM_V1** and **MUM_V2**\n* MUM algorithm is **simpler** than the VMUM one\n* MUM is specifically **designed to be fast on 64-bit CPUs**\n * Still MUM will work for 32-bit CPUs and it will be sometimes\n faster than Spooky and City\n* MUM has a **fast startup**. It is particular good to hash small keys\n which are prevalent in hash table applications\n\n# MUM implementation details\n\n* Input 64-bit data is randomized by 64x64->128 bit multiplication and mixing\n high- and low-parts of the multiplication result by using addition.\n The result is mixed with the current internal state by using XOR\n * Instead of addition for mixing high- and low- parts, XOR could be\n used\n * Using addition instead of XOR improves performance by about\n 10% on Haswell and Power7\n* Factor numbers, randomly generated with an equal probability of their\n bit values, are used for the multiplication\n* When all factors are used once, the internal state is randomized, and the same\n factors are used again for subsequent data randomization\n* The main loop is formed to be **unrolled** by the compiler to benefit from the\n the compiler instruction scheduling optimization and OOO\n (out-of-order) instruction execution in modern CPUs\n* MUM code does not contain assembly (asm) code anymore. This makes MUM less\n machine-dependent. To have efficient mum implementation, the\n compiler should support 128-bit integer\n extension (true for GCC and Clang on many targets)\n\n# VMUM Hash\n* VMUM is a vector variant of mum hashing (see below)\n * It uses target SIMD instructions (insns)\n * In comparison with mum v3, vmum considerably (up to 3 times) improves the speed\n of hashing mid-range (32 to 256 bytes) to long-range (more 256 bytes) length keys\n * As with previous mum hashing, to use vmum you just need one header\n file (vmum.h)\n * vmum source code is considerably smaller than that of extremely\n fast xxHash3 and th1ha2 and competes with them on hashing speed\n * vmum passes a more rigorous version of\n [SMHasher](https://github.com/rurban/smhasher)\n \n# VMUM implementation details\n* For long keys vmum uses vector insns:\n * AVX2 256-bit vector insns on x86-64\n * Neon 128-bit vector insns on aarch64\n * Altivec 128-bit vector insns on ppc64\n * There is a scalar emulation of the vector insns, too, for other targets\n\t* This could be useful for understanding used the vector\n operations used\n* You can add usage of vector insns for other targets. For this you\n just need to add small functions `_vmum_update_block`,\n `_vmum_zero_block`, and `_vmum_fold_block`\n * For the beneficial usage of vector insns the target should have unsigned `32 x 32-bit ->\n 64-bit` vector multiplication\n* To run vector insns in parallel on OOO CPUs, two vmum code loops are formed\n to be **unrolled** by the compiler into one basic block\n* I experimented a lot with other vector insns and found that the usage of\n carry-less (sometimes called polynomial) vector multiplication insns does not work\n well enough for hashing\n\n# VMUM and MUM benchmarking vs other famous hash functions\n\n* Here are the results of benchmarking VMUM and MUM with the fastest\n non-cryptographic hash functions I know:\n * Google City64 (sources are taken from SMHasher)\n * Bob Jenkins Spooky (sources are taken from SMHasher)\n * Yann Collet's xxHash3 (sources are taken from the\n [original repository](https://github.com/Cyan4973/xxHash))\n* I also added J. Aumasson and D. Bernstein's\n [SipHash24](https://github.com/veorq/SipHash) for the comparison as it\n is a popular choice for hash table implementation these days\n* A [metro hash](https://github.com/jandrewrogers/MetroHash)\n was added as people asked and as metro hash is\n claimed to be the fastest hash function\n * metro hash is not portable as others functions as it does not deal\n with the unaligned accesses problem on some targets\n * metro hash will produce different hash for LE/BE targets\n* Measurements were done on 4 different architecture machines:\n * AMD Ryzen 9900X\n * Intel i5-1300K\n * IBM Power10\n * Apple M4 10 cores (mac mini)\n* Hashing 10,000 of 16MB keys (bulk)\n* Hashing 1,280M keys for all other length keys\n* Each test was run 3 times and the minimal time was taken\n * GCC-14.2.1 was used on AMD and M4 machine, GCC-12.3.1 on Intel\n machine, GCC-11.5.0 was used on Power10\n * `-O3` was used for all compilations\n * The keys were generated by `rand` calls\n * The keys were aligned to see a hashing speed better and to permit runs for Metro\n * Some people complaint that my comparison is unfair as most hash functions are not inlined\n * I believe that the interface is the part of the implementation. So when\n the interface does not provide an easy way for inlining, it is an\n implementation pitfall\n * Still to address the complaints I added `-flto` for benchmarking all hash\n functions excluding MUM and VMUM. This option makes cross-file inlining\n* Here are graphs summarizing the measurements:\n\n![AMD](./benchmarks/amd.png)\n\n![INTEL](./benchmarks/intel.png)\n\n![M4](./benchmarks/m4.png)\n\n![Power10](./benchmarks/power10.png)\n\n* Exact numbers are given in the last section\n\n# SMhasher Speed Measurements\n\n* SMhasher also measures hash speeds. It uses the CPU cycle counter (__rtdc)\n * __rtdc-based measurements might be inaccurate for a small number of\n executed insns as the process can migrate, not all insns can\n retire, and CPU freq can be different. That is why I prefer long\n running benchmarks\n* Here are the results on AMD Ryzen 9900X for the fastest quality hashes\n (chosen according to SMhasher bulk speed results from https://github.com/rurban/smhasher)\n* More GB/sec is better. Less cycles/hash is better\n* Some hashes are based on the use of x86\\_64 AES insns and are less portable.\n They are marked by \"Yes\" in the AES column \n* The SLOC column gives the source code lines to implement the hash\n \n| Hash | AES | Bulk Speed (256KB): GB/s |Av. Speed on keys (1-32 bytes): cycles/hash| SLOC|\n|:----------------|:----:|-------------------------:|------------------------------------------:|----:|\n|VMUM-V2 | - | 103.7 | 16.4 |459 |\n|VMUM-V1 | - | 143.5 | 16.8 |459 |\n|MUM-V4 | - | 28.6 | 15.8 |291 |\n|MUM-V3 | - | 40.4 | 16.3 |291 |\n|xxh3 | - | 66.6 | 17.6 |965 |\n|umash64 | - | 63.1 | 25.4 |1097 |\n|FarmHash32 | - | 39.8 | 32.6 |1423 |\n|wyhash | - | 39.3 | 18.3 | 194 |\n|clhash | - | 38.4 | 51.7 | 366 |\n|t1ha2\\_atonce | - | 34.7 | 25.5 |2262 |\n|t1ha0\\_aes\\_avx2 | Yes | 128.9 | 25.0 |2262 |\n|gxhash64 | Yes | 197.1 | 27.9 | 274 |\n|aesni | Yes | 38.7 | 28.5 | 132 |\n\n\n# Using cryptographic vs. non-cryptographic hash function\n * People worrying about denial attacks based on generating hash\n collisions started to use cryptographic hash functions in hash tables\n * Cryptographic functions are very slow\n * *sha1* is about 20-30 times slower than MUM and City on the bulk speed tests\n * The new fastest cryptographic hash function *SipHash* is up to 10\n times slower\n * MUM and VMUM are also *resistant* to preimage attack (finding a\n key with a given hash) \n * To make hard moving to previous state values we use mostly 1-to-1 one way\n function `lo(x*C) + hi(x*C)` where C is a constant. Brute force\n solution of equation `f(x) = a` probably requires `2^63` tries.\n Another used function equation `x ^ y = a` has a `2^64`\n solutions. It complicates finding the overal solution further\n * If somebody is not convinced, you can use **randomly chosen\n multiplication constants** (see functions `mum_hash_randomize` and\n `vmum_hash_randomize`).\n Finding a key with a given hash even if you know a key with such\n a hash probably will be close to finding two or more solutions of\n *Diophantine* equations\n * If somebody is still not convinced, you can implement hash tables\n to **recognize the attack and rebuild** the table using the MUM function\n with the new multiplication constants\n * Analogous approach can be used if you use weak hash function as\n MurMur or City. Instead of using cryptographic hash functions\n **all the time**, hash tables can be implemented to recognize the\n attack and rebuild the table and start using a cryptographic hash\n function\n * This approach solves the speed problem and permits us to switch easily to a new\n cryptographic hash function if a flaw is found in the old one, e.g., switching from\n SipHash to SHA2\n \n# How to use [V]MUM\n* Please just include file `[v]mum.h` into your C/C++ program and use the following functions:\n * optional `[v]mum_hash_randomize` for choosing multiplication constants randomly\n * `[v]mum_hash_init`, `[v]mum_hash_step`, and `[v]mum_hash_finish` for hashing complex data structures\n * `[v]mum_hash64` for hashing a 64-bit data\n * `[v]mum_hash` for hashing any continuous block of data\n * Compile `vmum.h` with other code using options switching on vector\n insns if necessary (e.g. -mavx2 for x86\\_64)\n* To compare MUM and VMUM speed with other hash functions on your machine go to\n the directory `benchmarks` and run a script `./bench.sh`\n* The script will compile source files and run the tests printing the\n results as a markdown table\n\n# Crypto-hash function MUM512\n * [V]MUM is not designed to be a crypto-hash\n * The key (seed) and state are only 64-bit which are not crypto-level ones\n * The result can be different for different targets (BE/LE\n machines, 32- and 64-bit machines) as for other hash functions, e.g. City (hash can be\n different on SSE4.2 nad non SSE4.2 targets) or Spooky (BE/LE machines)\n * If you need the same MUM hash independent on the target, please\n define macro `[V]MUM_TARGET_INDEPENDENT_HASH`. Defining the\n macro affects the performace only on big-endian targets or\n targets without int128 support\n * There is a variant of MUM called MUM512 which can be a **candidate**\n for a crypto-hash function and keyed crypto-hash function and\n might be interesting for researchers\n * The **key** is **256**-bit\n * The **state** and the **output** are **512**-bit\n * The **block** size is **512**-bit\n * It uses 128x128->256-bit multiplication which is analogous to about\n 64 shifts and additions for 128-bit block word instead of 80\n rounds of shifts, additions, logical operations for 512-bit block\n in sha2-512.\n * It is **only a candidate** for a crypto hash function\n * I did not make any differential crypto-analysis or investigated\n probabilities of different attacks on the hash function (sorry, it\n is too big job)\n * I might be do this in the future as I am interested in\n differential characteristics of the MUM512 base transformation\n step (128x128-bit multiplications with addition of high and\n low 128-bit parts)\n * I am also interested in the right choice of the multiplication constants\n * May be somebody will do the analysis. I will be glad to hear anything.\n Who knows, may be it can be easily broken as Nimbus cipher.\n * The current code might be also vulnerable to timing attack on\n systems with varying multiplication instruction latency time.\n There is no code for now to prevent it\n * To compare the MUM512 speed with the speed of SHA-2 (SHA512) and\n SHA-3 (SHA3-512) go to the directory `benchmarks` and run a script `./bench-crypto.sh`\n * SHA-2 and SHA-3 code is taken from [RHash](https://github.com/rhash/RHash.git)\n * Blake2 crypto-hash from [github.com/BLAKE2/BLAKE2](https://github.com/BLAKE2/BLAKE2)\n was added for comparison. I use sse version of 64-bit Blake2 (blake2b).\n * Here is the speed of the crypto hash functions on AMD 9900X:\n\n| | MUM512 | SHA2 | SHA3 | Blake2B|\n:------------------------|-------:|------:|-------:|-------:|\n10 bytes (20 M texts) | 0.27s | 0.27s | 0.44s | 0.81s |\n100 bytes (20 M texts) | 0.36s | 0.25s | 0.84s | 0.84s |\n1000 bytes (20 M texts) | 1.21s | 2.08s | 5.63s | 3.70s |\n10000 bytes (5 M texts) | 5.60s | 5.05s | 14.07s | 7.99s |\n\n# Pseudo-random generators\n * Files `mum-prng.h` and `mum512-prng.h` provide pseudo-random\n functions based on MUM and MUM512 hash functions\n * All PRNGs passed *NIST Statistical Test Suite for Random and\n Pseudorandom Number Generators for Cryptographic Applications*\n (version 2.2.1) with 1000 bitstreams each containing 1M bits\n * Although MUM PRNG passed the test, it is not a cryptographically\n secure PRNG as is the hash function used for it\n * To compare the PRNG speeds go to\n the directory `benchmarks` and run a script `./bench-prng.sh`\n * For the comparison I wrote crypto-secured Blum Blum Shub PRNG\n (file `bbs-prng.h`) and PRNGs based on fast cryto-level hash\n functions in ChaCha stream cipher (file `chacha-prng.h`) and\n SipHash24 (file `sip24-prng.h`).\n * The additional PRNGs also pass the Statistical Test Suite\n * For the comparison I also added the fastest PRNGs\n * [xoroshiro128+](http://xoroshiro.di.unimi.it/xoroshiro128plus.c)\n * [xoroshiro128**](http://xoroshiro.di.unimi.it/xoroshiro128starstar.c)\n * [xoshiro256+](http://xoroshiro.di.unimi.it/xoshiro256plus.c)\n * [xoshiro256**](http://xoroshiro.di.unimi.it/xoshiro256starstar.c)\n * [xoshiro512**](http://xoroshiro.di.unimi.it/xoshiro512starstar.c)\n * As recommended the first numbers generated by splitmix64 were used as a seed\n * I had no intention to tune MUM based PRNG first but\n after adding xoroshiro128+ and finding how fast it is, I've decided\n to speedup MUM PRNG\n * I added code to calculate a few PRNs at once to calculate them in parallel\n * I added AVX2 version functions to use the faster `MULX` instruction\n * The new version also passed NIST Statistical Test Suite. It was\n tested even on bigger data (10K bitstreams each containing 10M\n bits). The test took several days on i7-4790K\n * The new version is **almost 2 times** faster than the old one and MUM PRN\n speed became almost the same as xoroshiro/xoshiro ones\n * All xoroshiro/xoshiro and MUM PRNG functions are inlined in the benchmark program\n * Both code without inlining will be visibly slower and the speed\n difference will be negligible as one PRN calculation takes\n only about **3-4 machine cycle** for xoroshiro/xoshiro and MUM PRN.\n * **Update Nov.2 2019**: I found that MUM PRNG fails practrand on 512GB. So I modified it.\n Instead of basically 16 independent PRNGs with 64-bit state, I made it one PRNG with 1024-bit state.\n I also managed to speed up MUM PRNG by 15%.\n * All PRNG were tested by [practrand](http://pracrand.sourceforge.net/) with\n 4TB PRNG generated stream (it took a few days)\n * **GLIBC RAND, xoroshiro128+, xoshiro256+, and xoshiro512+ failed** on the first stages of practrand\n * The rest of the PRNGs passed\n * BBS PRNG was tested by only 64GB stream because it is too slow\n * Here is the speed of the PRNGs in millions generated PRNs\n per second:\n\n| M prns/sec | AMD 9900X |Intel i5-1360K| Apple M4 | Power10 |\n:--------------|------------:|-------------:|------------:|---------:|\nBBS | 0.0886 | 0.0827 | 0.122 | 0.021 |\nChaCha | 357.68 | 184.80 | 262.81 | 83.20 |\nSipHash24 | 702.10 | 567.43 | 760.13 | 231.48 |\nMUM512 | 91.54 | 179.62 | 268.04 | 44.28 |\nMUM |1947.27 |1620.65 |2263.68 | 694.42 |\nXOSHIRO128** |1797.02 |1386.87 |1095.37 | 477.67 |\nXOSHIRO256** |1866.35 |1364.85 |1466.15 | 607.65 |\nXOSHIRO512** |1663.86 |1235.15 |1423.90 | 631.90 |\nGLIBC RAND | 115.57 | 101.48 | 228.99 | 33.66 |\nXOROSHIRO128+ |1786.62 |1299.59 |1296.48 | 549.85 |\nXOSHIRO256+ |2321.99 |1720.67 |1690.96 | 711.41 |\nXOSHIRO512+ |1808.81 |1525.18 |1659.76 | 717.12 |\n\n# Table results for hash speed measurements\n* Here are table variants of my measurements for people wanting the\n exact numbers. The tables also contain time spent for hashing.\n \n* AMD Ryzen 9900X:\n\n| Length | VMUM-V2 | VMUM-V1 | MUM-V4 | MUM-V3 | Spooky | City | xxHash3 | t1ha2 | SipHash24 | Metro |\n|:----------|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|\n| 3 bytes |1.00 4.57s|0.98 4.67s|0.98 4.64s|0.97 4.73s|0.76 6.01s|0.60 7.61s|0.94 4.84s|0.61 7.47s|0.61 7.51s|0.69 6.67s|\n| 4 bytes |1.00 2.77s|1.00 2.78s|1.09 2.55s|1.09 2.55s|0.55 5.08s|0.39 7.15s|0.71 3.92s|0.69 4.03s|0.44 6.24s|0.75 3.69s|\n| 5 bytes |1.00 4.63s|1.00 4.64s|1.00 4.62s|1.00 4.63s|0.80 5.78s|0.65 7.07s|0.88 5.28s|0.62 7.41s|0.61 7.59s|0.88 5.24s|\n| 6 bytes |1.00 4.57s|1.00 4.56s|1.00 4.57s|1.00 4.56s|0.77 5.93s|0.65 7.06s|0.87 5.24s|0.62 7.38s|0.58 7.87s|0.87 5.24s|\n| 7 bytes |1.00 4.84s|1.01 4.80s|1.01 4.80s|1.01 4.79s|0.79 6.15s|0.69 7.06s|0.92 5.24s|0.66 7.38s|0.60 8.10s|0.76 6.38s|\n| 8 bytes |1.00 2.74s|1.00 2.74s|1.09 2.51s|1.09 2.51s|0.54 5.03s|0.39 7.06s|0.52 5.24s|0.69 3.97s|0.33 8.29s|0.75 3.67s|\n| 9 bytes |1.00 3.01s|1.08 2.78s|1.07 2.82s|1.06 2.83s|0.59 5.06s|0.27 11.03s|0.45 6.66s|0.41 7.37s|0.36 8.29s|0.60 5.04s|\n| 10 bytes |1.00 3.01s|1.08 2.78s|1.08 2.79s|1.04 2.89s|0.59 5.08s|0.27 11.02s|0.45 6.66s|0.41 7.36s|0.36 8.34s|0.60 5.05s|\n| 11 bytes |1.00 3.01s|1.08 2.79s|1.08 2.79s|1.08 2.78s|0.59 5.08s|0.27 11.04s|0.45 6.67s|0.41 7.37s|0.36 8.32s|0.49 6.20s|\n| 12 bytes |1.00 3.01s|1.09 2.77s|1.07 2.81s|1.07 2.82s|0.59 5.06s|0.27 11.03s|0.45 6.66s|0.41 7.35s|0.36 8.30s|0.60 5.02s|\n| 13 bytes |1.00 2.98s|1.08 2.77s|1.05 2.83s|1.08 2.77s|0.59 5.02s|0.27 10.94s|0.45 6.61s|0.41 7.28s|0.36 8.21s|0.48 6.15s|\n| 14 bytes |1.00 2.96s|1.08 2.74s|1.08 2.75s|1.08 2.74s|0.59 5.01s|0.27 10.95s|0.45 6.60s|0.41 7.29s|0.36 8.21s|0.48 6.16s|\n| 15 bytes |1.00 2.98s|1.09 2.74s|1.08 2.77s|1.06 2.80s|0.59 5.01s|0.27 10.93s|0.45 6.61s|0.41 7.29s|0.36 8.21s|0.41 7.28s|\n| 16 bytes |1.00 2.98s|1.09 2.74s|1.09 2.73s|1.09 2.73s|0.27 10.94s|0.41 7.28s|0.93 3.19s|0.59 5.08s|0.29 10.31s|0.62 4.78s|\n| 32 bytes |1.00 3.28s|1.08 3.05s|1.00 3.27s|0.98 3.34s|0.30 10.95s|0.40 8.19s|1.03 3.19s|0.44 7.50s|0.23 14.39s|0.33 9.82s|\n| 64 bytes |1.00 3.89s|1.09 3.58s|0.87 4.47s|0.85 4.58s|0.23 16.63s|0.47 8.36s|1.22 3.19s|0.68 5.69s|0.17 22.59s|0.37 10.49s|\n| 96 bytes |1.00 4.55s|1.08 4.20s|0.79 5.79s|0.78 5.83s|0.20 22.31s|0.36 12.54s|1.43 3.19s|0.66 6.88s|0.15 31.11s|0.40 11.27s|\n| 128 bytes |1.00 6.32s|1.40 4.52s|0.91 6.92s|0.90 7.06s|0.23 27.99s|0.50 12.54s|1.98 3.19s|0.83 7.57s|0.16 39.35s|0.53 11.85s|\n| 192 bytes |1.00 8.55s|1.29 6.63s|0.90 9.46s|0.99 8.65s|0.36 23.81s|0.59 14.48s|1.25 6.83s|0.88 9.74s|0.15 55.96s|0.65 13.21s|\n| 256 bytes |1.00 10.98s|1.38 7.95s|0.92 11.98s|1.06 10.32s|0.45 24.22s|0.66 16.71s|0.79 13.86s|0.92 11.89s|0.15 74.12s|0.75 14.57s|\n| 512 bytes |1.00 14.42s|1.12 12.91s|0.65 22.33s|0.79 18.16s|0.41 35.30s|0.53 26.98s|0.91 15.92s|0.69 21.04s|0.10 140.39s|0.63 22.76s|\n|1024 bytes |1.00 17.13s|1.09 15.75s|0.37 46.54s|0.50 34.26s|0.32 53.76s|0.38 45.34s|0.88 19.50s|0.44 38.78s|0.06 272.94s|0.44 39.07s|\n| Bulk |1.00 1.70s|1.36 1.25s|0.31 5.57s|0.44 3.85s|0.33 5.13s|0.34 4.94s|1.18 1.44s|0.37 4.62s|0.05 33.88s|0.40 4.26s|\n| Average |1.00 |1.11 |0.93 |0.96 |0.50 |0.43 |0.85 |0.58 |0.31 |0.59 |\n| Geomean |1.00 |1.10 |0.90 |0.93 |0.46 |0.41 |0.77 |0.55 |0.26 |0.56 |\n\n* Intel i5-13600K:\n\n| Length | VMUM-V2 | VMUM-V1 | MUM-V4 | MUM-V3 | Spooky | City | xxHash3 | t1ha2 | SipHash24 | Metro |\n|:----------|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|\n| 3 bytes |1.00 4.67s|1.02 4.59s|1.03 4.53s|1.03 4.53s|0.67 6.94s|0.58 8.05s|0.93 5.03s|0.50 9.31s|0.47 9.93s|0.69 6.79s|\n| 4 bytes |1.00 4.27s|1.00 4.27s|1.06 4.02s|1.06 4.02s|0.73 5.86s|0.51 8.37s|0.75 5.70s|0.77 5.58s|0.52 8.17s|0.81 5.28s|\n| 5 bytes |1.00 4.66s|1.02 4.59s|1.03 4.53s|1.03 4.53s|0.73 6.37s|0.57 8.17s|0.82 5.70s|0.50 9.31s|0.44 10.49s|0.69 6.79s|\n| 6 bytes |1.00 4.67s|1.02 4.56s|1.03 4.53s|1.03 4.53s|0.69 6.76s|0.57 8.17s|0.82 5.69s|0.50 9.31s|0.43 10.95s|0.69 6.79s|\n| 7 bytes |1.00 4.87s|1.00 4.89s|1.01 4.83s|1.01 4.83s|0.69 7.02s|0.60 8.17s|0.85 5.70s|0.52 9.31s|0.45 10.89s|0.69 7.04s|\n| 8 bytes |1.00 4.27s|1.00 4.28s|1.55 2.76s|1.55 2.76s|0.76 5.60s|0.53 8.08s|0.75 5.69s|0.99 4.30s|0.39 10.88s|1.06 4.02s|\n| 9 bytes |1.00 4.53s|1.06 4.29s|1.50 3.02s|1.50 3.01s|0.81 5.60s|0.33 13.59s|0.53 8.58s|0.48 9.51s|0.42 10.88s|0.82 5.53s|\n| 10 bytes |1.00 4.54s|1.06 4.29s|1.50 3.02s|1.51 3.01s|0.81 5.60s|0.33 13.59s|0.53 8.58s|0.48 9.51s|0.42 10.88s|0.82 5.53s|\n| 11 bytes |1.00 4.52s|1.06 4.27s|1.50 3.02s|1.50 3.02s|0.81 5.60s|0.33 13.59s|0.53 8.58s|0.48 9.51s|0.42 10.88s|0.67 6.79s|\n| 12 bytes |1.00 4.54s|1.06 4.29s|1.50 3.02s|1.51 3.01s|0.81 5.60s|0.33 13.59s|0.53 8.58s|0.48 9.51s|0.42 10.88s|0.82 5.53s|\n| 13 bytes |1.00 4.52s|1.06 4.28s|1.49 3.03s|1.50 3.02s|0.81 5.60s|0.33 13.59s|0.53 8.59s|0.48 9.51s|0.42 10.88s|0.67 6.79s|\n| 14 bytes |1.00 4.52s|1.06 4.27s|1.49 3.03s|1.50 3.02s|0.81 5.60s|0.33 13.59s|0.53 8.59s|0.48 9.51s|0.42 10.88s|0.67 6.79s|\n| 15 bytes |1.00 4.53s|1.06 4.29s|1.50 3.02s|1.50 3.02s|0.81 5.60s|0.33 13.59s|0.53 8.58s|0.48 9.51s|0.42 10.88s|0.56 8.05s|\n| 16 bytes |1.00 4.52s|1.06 4.28s|1.50 3.02s|1.50 3.01s|0.37 12.13s|0.56 8.05s|0.89 5.07s|0.85 5.29s|0.34 13.43s|0.83 5.46s|\n| 32 bytes |1.00 4.79s|1.05 4.58s|1.30 3.69s|1.33 3.59s|0.39 12.38s|0.51 9.39s|0.94 5.10s|0.67 7.15s|0.25 18.92s|0.43 11.07s|\n| 64 bytes |1.00 5.46s|1.06 5.15s|1.14 4.78s|1.11 4.91s|0.29 18.66s|0.58 9.36s|1.06 5.13s|0.88 6.22s|0.17 31.57s|0.46 11.83s|\n| 96 bytes |1.00 6.43s|1.10 5.83s|0.84 7.68s|0.84 7.67s|0.26 25.17s|0.46 13.88s|1.23 5.23s|0.85 7.60s|0.15 42.71s|0.51 12.70s|\n| 128 bytes |1.00 7.92s|1.25 6.36s|0.84 9.42s|0.86 9.19s|0.25 31.62s|0.57 13.87s|1.51 5.24s|0.91 8.67s|0.15 53.88s|0.59 13.51s|\n| 192 bytes |1.00 11.52s|1.43 8.06s|1.05 11.02s|1.08 10.68s|0.39 29.49s|0.71 16.25s|1.23 9.34s|1.03 11.18s|0.15 76.23s|0.76 15.07s|\n| 256 bytes |1.00 14.26s|1.60 8.89s|1.04 13.65s|1.18 12.11s|0.48 29.86s|0.75 19.06s|0.91 15.64s|1.03 13.82s|0.15 97.67s|0.85 16.68s|\n| 512 bytes |1.00 15.93s|1.04 15.31s|0.62 25.67s|0.81 19.65s|0.35 45.39s|0.46 34.70s|0.96 16.68s|0.58 27.38s|0.09 186.04s|0.53 29.86s|\n|1024 bytes |1.00 20.96s|1.08 19.32s|0.42 49.61s|0.56 37.74s|0.29 71.66s|0.34 61.75s|0.84 24.92s|0.42 49.86s|0.06 362.59s|0.36 58.67s|\n| Bulk |1.00 3.13s|1.09 2.86s|0.40 7.77s|0.61 5.17s|0.40 7.78s|0.42 7.37s|0.87 3.60s|0.49 6.38s|0.07 45.99s|0.45 6.88s|\n| Average |1.00 |1.10 |1.15 |1.18 |0.58 |0.48 |0.83 |0.65 |0.31 |0.67 |\n| Geomean |1.00 |1.09 |1.08 |1.13 |0.54 |0.46 |0.79 |0.62 |0.26 |0.65 |\n\n* Apple M4:\n\n| Length | VMUM-V2 | VMUM-V1 | MUM-V4 | MUM-V3 | Spooky | City | xxHash3 | t1ha2 | SipHash24 | Metro |\n|:----------|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|\n| 3 bytes |1.00 5.15s|1.02 5.03s|1.03 5.02s|1.02 5.03s|0.69 7.50s|0.52 9.91s|0.87 5.92s|1.08 4.77s|0.51 10.01s|0.63 8.17s|\n| 4 bytes |1.00 4.70s|1.01 4.66s|1.08 4.36s|1.08 4.36s|0.69 6.78s|0.49 9.64s|0.70 6.71s|0.99 4.77s|0.52 9.03s|0.73 6.41s|\n| 5 bytes |1.00 5.05s|1.00 5.03s|1.01 5.01s|1.01 5.01s|0.71 7.08s|0.52 9.64s|0.74 6.78s|1.06 4.77s|0.49 10.38s|0.62 8.17s|\n| 6 bytes |1.00 5.15s|1.02 5.03s|1.04 4.95s|1.04 4.95s|0.69 7.49s|0.53 9.64s|0.76 6.78s|1.08 4.76s|0.49 10.51s|0.63 8.17s|\n| 7 bytes |1.00 5.52s|1.03 5.34s|1.04 5.32s|1.04 5.32s|0.71 7.82s|0.57 9.64s|0.81 6.79s|1.16 4.76s|0.51 10.77s|0.65 8.46s|\n| 8 bytes |1.00 3.07s|1.02 3.02s|1.16 2.65s|1.16 2.65s|0.47 6.49s|0.32 9.64s|0.46 6.71s|0.68 4.53s|0.26 11.69s|0.66 4.66s|\n| 9 bytes |1.00 3.26s|1.07 3.04s|1.03 3.17s|1.03 3.18s|0.50 6.48s|0.27 11.96s|0.56 5.85s|0.55 5.98s|0.28 11.69s|0.51 6.41s|\n| 10 bytes |1.00 3.27s|1.08 3.03s|1.08 3.02s|1.08 3.02s|0.50 6.48s|0.27 11.96s|0.56 5.85s|0.55 5.98s|0.28 11.69s|0.51 6.41s|\n| 11 bytes |1.00 3.38s|1.10 3.07s|1.11 3.05s|1.07 3.15s|0.52 6.48s|0.28 11.96s|0.58 5.85s|0.57 5.98s|0.29 11.69s|0.43 7.87s|\n| 12 bytes |1.00 3.39s|1.08 3.13s|1.13 3.00s|1.13 3.00s|0.52 6.48s|0.28 11.96s|0.58 5.85s|0.57 5.98s|0.29 11.69s|0.53 6.41s|\n| 13 bytes |1.00 3.33s|1.08 3.08s|1.10 3.04s|1.10 3.04s|0.51 6.48s|0.28 11.95s|0.57 5.85s|0.56 5.98s|0.28 11.69s|0.42 7.87s|\n| 14 bytes |1.00 3.31s|1.09 3.05s|1.10 3.02s|1.09 3.03s|0.51 6.48s|0.28 11.96s|0.56 5.86s|0.55 5.98s|0.28 11.69s|0.42 7.87s|\n| 15 bytes |1.00 3.32s|1.06 3.12s|1.08 3.07s|1.08 3.07s|0.51 6.48s|0.28 11.96s|0.57 5.85s|0.56 5.98s|0.28 11.69s|0.36 9.33s|\n| 16 bytes |1.00 3.30s|1.10 3.00s|1.07 3.07s|1.07 3.08s|0.23 14.08s|0.35 9.33s|0.85 3.87s|0.55 5.98s|0.23 14.58s|0.54 6.12s|\n| 32 bytes |1.00 3.66s|1.07 3.42s|1.01 3.64s|1.00 3.65s|0.26 14.07s|0.35 10.51s|0.96 3.80s|0.41 9.03s|0.18 20.66s|0.29 12.57s|\n| 64 bytes |1.00 4.42s|1.09 4.07s|0.89 4.99s|0.89 4.99s|0.21 21.37s|0.41 10.84s|1.17 3.79s|0.62 7.11s|0.13 33.90s|0.33 13.44s|\n| 96 bytes |1.00 5.16s|1.07 4.82s|0.82 6.27s|0.84 6.17s|0.18 28.70s|0.32 16.19s|1.36 3.80s|0.60 8.57s|0.11 45.54s|0.36 14.34s|\n| 128 bytes |1.00 6.87s|1.13 6.08s|0.91 7.55s|0.93 7.40s|0.19 35.99s|0.42 16.19s|1.81 3.80s|0.72 9.53s|0.12 59.49s|0.45 15.22s|\n| 192 bytes |1.00 8.65s|1.12 7.69s|0.82 10.60s|0.88 9.86s|0.27 31.55s|0.46 18.64s|0.88 9.86s|0.71 12.16s|0.10 84.13s|0.51 16.99s|\n| 256 bytes |1.00 9.39s|1.30 7.20s|0.71 13.24s|0.76 12.29s|0.29 32.11s|0.44 21.28s|0.71 13.19s|0.63 14.87s|0.09 106.82s|0.50 18.77s|\n| 512 bytes |1.00 14.79s|1.07 13.76s|0.69 21.33s|0.92 15.99s|0.29 50.41s|0.40 37.15s|0.91 16.28s|0.49 30.25s|0.07 204.80s|0.46 31.88s|\n|1024 bytes |1.00 27.83s|1.56 17.79s|0.65 43.07s|1.04 26.70s|0.35 78.46s|0.44 62.77s|1.14 24.39s|0.51 54.28s|0.07 399.61s|0.55 50.90s|\n| Bulk |1.00 3.45s|1.39 2.49s|0.58 6.00s|1.13 3.06s|0.44 7.83s|0.51 6.70s|1.19 2.89s|0.54 6.36s|0.07 50.68s|0.67 5.13s|\n| Average |1.00 |1.11 |0.96 |1.02 |0.45 |0.39 |0.84 |0.68 |0.26 |0.51 |\n| Geomean |1.00 |1.10 |0.95 |1.01 |0.41 |0.38 |0.79 |0.66 |0.21 |0.50 |\n\n* IBM Power10:\n\n| Length | VMUM-V2 | VMUM-V1 | MUM-V4 | MUM-V3 | Spooky | City | xxHash3 | t1ha2 | SipHash24 | Metro |\n|:----------|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|\n| 3 bytes |1.00 11.52s|1.00 11.53s|1.03 11.18s|1.03 11.21s|0.68 16.87s|0.61 18.95s|0.95 12.16s|1.09 10.57s|0.52 22.13s|0.66 17.35s|\n| 4 bytes |1.00 10.86s|1.00 10.87s|1.07 10.19s|1.07 10.19s|0.72 15.18s|0.54 20.22s|0.89 12.27s|1.03 10.58s|0.51 21.13s|0.88 12.40s|\n| 5 bytes |1.00 11.53s|0.98 11.73s|1.03 11.17s|1.03 11.17s|0.71 16.24s|0.55 20.91s|0.90 12.76s|1.09 10.58s|0.49 23.74s|0.66 17.35s|\n| 6 bytes |1.00 11.52s|0.98 11.73s|1.03 11.17s|1.03 11.18s|0.68 16.87s|0.55 20.92s|0.90 12.76s|1.09 10.58s|0.48 23.91s|0.66 17.36s|\n| 7 bytes |1.00 12.23s|1.02 11.96s|0.98 12.51s|1.03 11.84s|0.69 17.69s|0.58 20.92s|0.96 12.76s|1.16 10.56s|0.50 24.38s|0.56 22.01s|\n| 8 bytes |1.00 10.85s|1.00 10.86s|1.06 10.19s|1.07 10.18s|0.76 14.27s|0.52 20.92s|0.85 12.75s|1.05 10.32s|0.37 29.14s|1.10 9.86s|\n| 9 bytes |1.00 11.54s|1.06 10.87s|1.06 10.85s|1.06 10.85s|0.79 14.55s|0.40 28.92s|0.72 16.11s|0.84 13.73s|0.40 29.08s|0.84 13.80s|\n| 10 bytes |1.00 11.53s|1.06 10.87s|1.06 10.85s|1.06 10.85s|0.79 14.54s|0.40 28.92s|0.72 16.10s|0.84 13.72s|0.40 29.17s|0.84 13.79s|\n| 11 bytes |1.00 11.22s|1.03 10.87s|1.03 10.85s|1.03 10.85s|0.77 14.55s|0.39 28.90s|0.70 16.11s|0.82 13.72s|0.38 29.21s|0.66 17.08s|\n| 12 bytes |1.00 11.53s|1.06 10.86s|1.06 10.86s|1.06 10.85s|0.79 14.54s|0.40 28.92s|0.72 16.11s|0.84 13.72s|0.40 29.13s|0.84 13.80s|\n| 13 bytes |1.00 11.23s|1.03 10.87s|1.04 10.85s|1.04 10.85s|0.77 14.56s|0.39 28.91s|0.70 16.11s|0.82 13.72s|0.39 29.14s|0.66 17.09s|\n| 14 bytes |1.00 11.23s|1.03 10.88s|1.03 10.87s|1.04 10.84s|0.77 14.55s|0.39 28.92s|0.70 16.11s|0.82 13.71s|0.38 29.20s|0.66 17.09s|\n| 15 bytes |1.00 11.53s|1.06 10.88s|1.06 10.89s|1.06 10.89s|0.79 14.56s|0.40 28.91s|0.72 16.11s|0.84 13.72s|0.40 29.17s|0.57 20.38s|\n| 16 bytes |1.00 12.20s|1.12 10.91s|1.12 10.85s|1.12 10.89s|0.44 27.70s|0.61 20.05s|1.36 8.96s|0.89 13.72s|0.31 39.95s|1.00 12.16s|\n| 32 bytes |1.00 12.92s|1.11 11.59s|1.06 12.22s|1.06 12.22s|0.45 28.63s|0.58 22.34s|1.57 8.23s|0.64 20.32s|0.24 54.90s|0.52 24.97s|\n| 64 bytes |1.00 14.54s|1.11 13.09s|0.97 15.02s|0.96 15.07s|0.35 41.29s|0.62 23.31s|1.70 8.54s|0.90 16.20s|0.16 88.17s|0.54 26.78s|\n| 96 bytes |1.00 15.93s|1.13 14.07s|0.82 19.44s|0.96 16.64s|0.29 54.32s|0.45 35.33s|1.93 8.24s|0.81 19.57s|0.13 119.66s|0.55 28.81s|\n| 128 bytes |1.00 16.71s|1.08 15.47s|0.75 22.21s|0.86 19.52s|0.24 68.72s|0.47 35.31s|2.03 8.22s|0.77 21.78s|0.11 156.55s|0.54 30.78s|\n| 192 bytes |1.00 21.98s|1.16 18.99s|0.83 26.51s|0.83 26.60s|0.36 60.97s|0.53 41.32s|0.76 28.78s|0.76 29.01s|0.11 195.56s|0.62 35.17s|\n| 256 bytes |1.00 22.31s|1.25 17.89s|0.73 30.47s|0.76 29.34s|0.36 62.61s|0.47 47.41s|0.67 33.50s|0.64 35.02s|0.09 252.22s|0.57 39.10s|\n| 512 bytes |1.00 33.65s|1.19 28.16s|0.71 47.60s|0.74 45.60s|0.33 100.76s|0.46 73.34s|0.73 46.38s|0.56 60.20s|0.07 483.42s|0.60 55.93s|\n|1024 bytes |1.00 56.70s|1.41 40.15s|0.69 81.61s|0.73 78.10s|0.34 167.11s|0.45 126.45s|0.80 70.61s|0.49 116.16s|0.06 944.76s|0.45 127.01s|\n| Bulk |1.00 6.75s|1.42 4.75s|0.70 9.62s|0.67 10.01s|0.38 17.80s|0.49 13.74s|0.93 7.27s|0.49 13.91s|0.06 118.69s|0.46 14.52s|\n| Average |1.00 |1.10 |0.95 |0.97 |0.58 |0.49 |1.00 |0.84 |0.30 |0.67 |\n| Geomean |1.00 |1.09 |0.94 |0.96 |0.54 |0.48 |0.93 |0.82 |0.24 |0.65 |\n\n" }, { "path": "benchmarks/City.cpp", "content": "// Copyright (c) 2011 Google, Inc.\n//\n// Permission is hereby granted, free of charge, to any person obtaining a copy\n// of this software and associated documentation files (the \"Software\"), to deal\n// in the Software without restriction, including without limitation the rights\n// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n// copies of the Software, and to permit persons to whom the Software is\n// furnished to do so, subject to the following conditions:\n//\n// The above copyright notice and this permission notice shall be included in\n// all copies or substantial portions of the Software.\n//\n// THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\n// THE SOFTWARE.\n//\n// CityHash, by Geoff Pike and Jyrki Alakuijala\n//\n// This file provides CityHash64() and related functions.\n//\n// It's probably possible to create even faster hash functions by\n// writing a program that systematically explores some of the space of\n// possible hash functions, by using SIMD instructions, or by\n// compromising on hash quality.\n\n#include \"City.h\"\n\n#include \n#include // for memcpy and memset\n\nusing namespace std;\n\nstatic uint64 UNALIGNED_LOAD64(const char *p) {\n uint64 result;\n memcpy(&result, p, sizeof(result));\n return result;\n}\n\nstatic uint32 UNALIGNED_LOAD32(const char *p) {\n uint32 result;\n memcpy(&result, p, sizeof(result));\n return result;\n}\n\n#ifndef __BIG_ENDIAN__\n\n#define uint32_in_expected_order(x) (x)\n#define uint64_in_expected_order(x) (x)\n\n#else\n\n#ifdef _MSC_VER\n#include \n#define bswap_32(x) _byteswap_ulong(x)\n#define bswap_64(x) _byteswap_uint64(x)\n\n#elif defined(__APPLE__)\n// Mac OS X / Darwin features\n#include \n#define bswap_32(x) OSSwapInt32(x)\n#define bswap_64(x) OSSwapInt64(x)\n\n#else\n#include \n#endif\n\n#define uint32_in_expected_order(x) (bswap_32(x))\n#define uint64_in_expected_order(x) (bswap_64(x))\n\n#endif // __BIG_ENDIAN__\n\n#if !defined(LIKELY)\n#if defined(__GNUC__) || defined(__INTEL_COMPILER)\n#define LIKELY(x) (__builtin_expect(!!(x), 1))\n#else\n#define LIKELY(x) (x)\n#endif\n#endif\n\nstatic uint64 Fetch64(const char *p) {\n return uint64_in_expected_order(UNALIGNED_LOAD64(p));\n}\n\nstatic uint32 Fetch32(const char *p) {\n return uint32_in_expected_order(UNALIGNED_LOAD32(p));\n}\n\n// Some primes between 2^63 and 2^64 for various uses.\nstatic const uint64 k0 = 0xc3a5c85c97cb3127ULL;\nstatic const uint64 k1 = 0xb492b66fbe98f273ULL;\nstatic const uint64 k2 = 0x9ae16a3b2f90404fULL;\nstatic const uint64 k3 = 0xc949d7c7509e6557ULL;\n\n// Bitwise right rotate. Normally this will compile to a single\n// instruction, especially if the shift is a manifest constant.\nstatic uint64 Rotate(uint64 val, int shift) {\n // Avoid shifting by 64: doing so yields an undefined result.\n return shift == 0 ? val : ((val >> shift) | (val << (64 - shift)));\n}\n\n// Equivalent to Rotate(), but requires the second arg to be non-zero.\n// On x86-64, and probably others, it's possible for this to compile\n// to a single instruction if both args are already in registers.\nstatic uint64 RotateByAtLeast1(uint64 val, int shift) {\n return (val >> shift) | (val << (64 - shift));\n}\n\nstatic uint64 ShiftMix(uint64 val) {\n return val ^ (val >> 47);\n}\n\nstatic uint64 HashLen16(uint64 u, uint64 v) {\n return Hash128to64(uint128(u, v));\n}\n\nstatic uint64 HashLen0to16(const char *s, size_t len) {\n if (len > 8) {\n uint64 a = Fetch64(s);\n uint64 b = Fetch64(s + len - 8);\n return HashLen16(a, RotateByAtLeast1(b + len, len)) ^ b;\n }\n if (len >= 4) {\n uint64 a = Fetch32(s);\n return HashLen16(len + (a << 3), Fetch32(s + len - 4));\n }\n if (len > 0) {\n uint8 a = s[0];\n uint8 b = s[len >> 1];\n uint8 c = s[len - 1];\n uint32 y = static_cast(a) + (static_cast(b) << 8);\n uint32 z = len + (static_cast(c) << 2);\n return ShiftMix(y * k2 ^ z * k3) * k2;\n }\n return k2;\n}\n\n// This probably works well for 16-byte strings as well, but it may be overkill\n// in that case.\nstatic uint64 HashLen17to32(const char *s, size_t len) {\n uint64 a = Fetch64(s) * k1;\n uint64 b = Fetch64(s + 8);\n uint64 c = Fetch64(s + len - 8) * k2;\n uint64 d = Fetch64(s + len - 16) * k0;\n return HashLen16(Rotate(a - b, 43) + Rotate(c, 30) + d,\n a + Rotate(b ^ k3, 20) - c + len);\n}\n\n// Return a 16-byte hash for 48 bytes. Quick and dirty.\n// Callers do best to use \"random-looking\" values for a and b.\nstatic pair WeakHashLen32WithSeeds(\n uint64 w, uint64 x, uint64 y, uint64 z, uint64 a, uint64 b) {\n a += w;\n b = Rotate(b + a + z, 21);\n uint64 c = a;\n a += x;\n a += y;\n b += Rotate(a, 44);\n return make_pair(a + z, b + c);\n}\n\n// Return a 16-byte hash for s[0] ... s[31], a, and b. Quick and dirty.\nstatic pair WeakHashLen32WithSeeds(\n const char* s, uint64 a, uint64 b) {\n return WeakHashLen32WithSeeds(Fetch64(s),\n Fetch64(s + 8),\n Fetch64(s + 16),\n Fetch64(s + 24),\n a,\n b);\n}\n\n// Return an 8-byte hash for 33 to 64 bytes.\nstatic uint64 HashLen33to64(const char *s, size_t len) {\n uint64 z = Fetch64(s + 24);\n uint64 a = Fetch64(s) + (len + Fetch64(s + len - 16)) * k0;\n uint64 b = Rotate(a + z, 52);\n uint64 c = Rotate(a, 37);\n a += Fetch64(s + 8);\n c += Rotate(a, 7);\n a += Fetch64(s + 16);\n uint64 vf = a + z;\n uint64 vs = b + Rotate(a, 31) + c;\n a = Fetch64(s + 16) + Fetch64(s + len - 32);\n z = Fetch64(s + len - 8);\n b = Rotate(a + z, 52);\n c = Rotate(a, 37);\n a += Fetch64(s + len - 24);\n c += Rotate(a, 7);\n a += Fetch64(s + len - 16);\n uint64 wf = a + z;\n uint64 ws = b + Rotate(a, 31) + c;\n uint64 r = ShiftMix((vf + ws) * k2 + (wf + vs) * k0);\n return ShiftMix(r * k0 + vs) * k2;\n}\n\nuint64 CityHash64(const char *s, size_t len) {\n if (len <= 32) {\n if (len <= 16) {\n return HashLen0to16(s, len);\n } else {\n return HashLen17to32(s, len);\n }\n } else if (len <= 64) {\n return HashLen33to64(s, len);\n }\n\n // For strings over 64 bytes we hash the end first, and then as we\n // loop we keep 56 bytes of state: v, w, x, y, and z.\n uint64 x = Fetch64(s + len - 40);\n uint64 y = Fetch64(s + len - 16) + Fetch64(s + len - 56);\n uint64 z = HashLen16(Fetch64(s + len - 48) + len, Fetch64(s + len - 24));\n pair v = WeakHashLen32WithSeeds(s + len - 64, len, z);\n pair w = WeakHashLen32WithSeeds(s + len - 32, y + k1, x);\n x = x * k1 + Fetch64(s);\n\n // Decrease len to the nearest multiple of 64, and operate on 64-byte chunks.\n len = (len - 1) & ~static_cast(63);\n do {\n x = Rotate(x + y + v.first + Fetch64(s + 8), 37) * k1;\n y = Rotate(y + v.second + Fetch64(s + 48), 42) * k1;\n x ^= w.second;\n y += v.first + Fetch64(s + 40);\n z = Rotate(z + w.first, 33) * k1;\n v = WeakHashLen32WithSeeds(s, v.second * k1, x + w.first);\n w = WeakHashLen32WithSeeds(s + 32, z + w.second, y + Fetch64(s + 16));\n std::swap(z, x);\n s += 64;\n len -= 64;\n } while (len != 0);\n return HashLen16(HashLen16(v.first, w.first) + ShiftMix(y) * k1 + z,\n HashLen16(v.second, w.second) + x);\n}\n\nuint64 CityHash64WithSeed(const char *s, size_t len, uint64 seed) {\n return CityHash64WithSeeds(s, len, k2, seed);\n}\n\nuint64 CityHash64WithSeeds(const char *s, size_t len,\n uint64 seed0, uint64 seed1) {\n return HashLen16(CityHash64(s, len) - seed0, seed1);\n}\n\n// A subroutine for CityHash128(). Returns a decent 128-bit hash for strings\n// of any length representable in signed long. Based on City and Murmur.\nstatic uint128 CityMurmur(const char *s, size_t len, uint128 seed) {\n uint64 a = Uint128Low64(seed);\n uint64 b = Uint128High64(seed);\n uint64 c = 0;\n uint64 d = 0;\n signed long l = len - 16;\n if (l <= 0) { // len <= 16\n a = ShiftMix(a * k1) * k1;\n c = b * k1 + HashLen0to16(s, len);\n d = ShiftMix(a + (len >= 8 ? Fetch64(s) : c));\n } else { // len > 16\n c = HashLen16(Fetch64(s + len - 8) + k1, a);\n d = HashLen16(b + len, c + Fetch64(s + len - 16));\n a += d;\n do {\n a ^= ShiftMix(Fetch64(s) * k1) * k1;\n a *= k1;\n b ^= a;\n c ^= ShiftMix(Fetch64(s + 8) * k1) * k1;\n c *= k1;\n d ^= c;\n s += 16;\n l -= 16;\n } while (l > 0);\n }\n a = HashLen16(a, c);\n b = HashLen16(d, b);\n return uint128(a ^ b, HashLen16(b, a));\n}\n\nuint128 CityHash128WithSeed(const char *s, size_t len, uint128 seed) {\n if (len < 128) {\n return CityMurmur(s, len, seed);\n }\n\n // We expect len >= 128 to be the common case. Keep 56 bytes of state:\n // v, w, x, y, and z.\n pair v, w;\n uint64 x = Uint128Low64(seed);\n uint64 y = Uint128High64(seed);\n uint64 z = len * k1;\n v.first = Rotate(y ^ k1, 49) * k1 + Fetch64(s);\n v.second = Rotate(v.first, 42) * k1 + Fetch64(s + 8);\n w.first = Rotate(y + z, 35) * k1 + x;\n w.second = Rotate(x + Fetch64(s + 88), 53) * k1;\n\n // This is the same inner loop as CityHash64(), manually unrolled.\n do {\n x = Rotate(x + y + v.first + Fetch64(s + 8), 37) * k1;\n y = Rotate(y + v.second + Fetch64(s + 48), 42) * k1;\n x ^= w.second;\n y += v.first + Fetch64(s + 40);\n z = Rotate(z + w.first, 33) * k1;\n v = WeakHashLen32WithSeeds(s, v.second * k1, x + w.first);\n w = WeakHashLen32WithSeeds(s + 32, z + w.second, y + Fetch64(s + 16));\n std::swap(z, x);\n s += 64;\n x = Rotate(x + y + v.first + Fetch64(s + 8), 37) * k1;\n y = Rotate(y + v.second + Fetch64(s + 48), 42) * k1;\n x ^= w.second;\n y += v.first + Fetch64(s + 40);\n z = Rotate(z + w.first, 33) * k1;\n v = WeakHashLen32WithSeeds(s, v.second * k1, x + w.first);\n w = WeakHashLen32WithSeeds(s + 32, z + w.second, y + Fetch64(s + 16));\n std::swap(z, x);\n s += 64;\n len -= 128;\n } while (LIKELY(len >= 128));\n x += Rotate(v.first + z, 49) * k0;\n z += Rotate(w.first, 37) * k0;\n // If 0 < len < 128, hash up to 4 chunks of 32 bytes each from the end of s.\n for (size_t tail_done = 0; tail_done < len; ) {\n tail_done += 32;\n y = Rotate(x + y, 42) * k0 + v.second;\n w.first += Fetch64(s + len - tail_done + 16);\n x = x * k0 + w.first;\n z += w.second + Fetch64(s + len - tail_done);\n w.second += v.first;\n v = WeakHashLen32WithSeeds(s + len - tail_done, v.first + z, v.second);\n }\n // At this point our 56 bytes of state should contain more than\n // enough information for a strong 128-bit hash. We use two\n // different 56-byte-to-8-byte hashes to get a 16-byte final result.\n x = HashLen16(x, v.first);\n y = HashLen16(y + z, w.first);\n return uint128(HashLen16(x + v.second, w.second) + y,\n HashLen16(x + w.second, y + v.second));\n}\n\nuint128 CityHash128(const char *s, size_t len) {\n if (len >= 16) {\n return CityHash128WithSeed(s + 16,\n len - 16,\n uint128(Fetch64(s) ^ k3,\n Fetch64(s + 8)));\n } else if (len >= 8) {\n return CityHash128WithSeed(NULL,\n 0,\n uint128(Fetch64(s) ^ (len * k0),\n Fetch64(s + len - 8) ^ k1));\n } else {\n return CityHash128WithSeed(s, len, uint128(k0, k1));\n }\n}\n\n#if defined(__SSE4_2__) && defined(__x86_64__)\n#include \n\n// Requires len >= 240.\nstatic void CityHashCrc256Long(const char *s, size_t len,\n uint32 seed, uint64 *result) {\n uint64 a = Fetch64(s + 56) + k0;\n uint64 b = Fetch64(s + 96) + k0;\n uint64 c = result[0] = HashLen16(b, len);\n uint64 d = result[1] = Fetch64(s + 120) * k0 + len;\n uint64 e = Fetch64(s + 184) + seed;\n uint64 f = seed;\n uint64 g = 0;\n uint64 h = 0;\n uint64 i = 0;\n uint64 j = 0;\n uint64 t = c + d;\n\n // 240 bytes of input per iter.\n size_t iters = len / 240;\n len -= iters * 240;\n do {\n#define CHUNK(multiplier, z) \\\n { \\\n uint64 old_a = a; \\\n a = Rotate(b, 41 ^ z) * multiplier + Fetch64(s); \\\n b = Rotate(c, 27 ^ z) * multiplier + Fetch64(s + 8); \\\n c = Rotate(d, 41 ^ z) * multiplier + Fetch64(s + 16); \\\n d = Rotate(e, 33 ^ z) * multiplier + Fetch64(s + 24); \\\n e = Rotate(t, 25 ^ z) * multiplier + Fetch64(s + 32); \\\n t = old_a; \\\n } \\\n f = _mm_crc32_u64(f, a); \\\n g = _mm_crc32_u64(g, b); \\\n h = _mm_crc32_u64(h, c); \\\n i = _mm_crc32_u64(i, d); \\\n j = _mm_crc32_u64(j, e); \\\n s += 40\n\n CHUNK(1, 1); CHUNK(k0, 0);\n CHUNK(1, 1); CHUNK(k0, 0);\n CHUNK(1, 1); CHUNK(k0, 0);\n } while (--iters > 0);\n\n while (len >= 40) {\n CHUNK(k0, 0);\n len -= 40;\n }\n if (len > 0) {\n s = s + len - 40;\n CHUNK(k0, 0);\n }\n j += i << 32;\n a = HashLen16(a, j);\n h += g << 32;\n b += h;\n c = HashLen16(c, f) + i;\n d = HashLen16(d, e + result[0]);\n j += e;\n i += HashLen16(h, t);\n e = HashLen16(a, d) + j;\n f = HashLen16(b, c) + a;\n g = HashLen16(j, i) + c;\n result[0] = e + f + g + h;\n a = ShiftMix((a + g) * k0) * k0 + b;\n result[1] += a + result[0];\n a = ShiftMix(a * k0) * k0 + c;\n result[2] = a + result[1];\n a = ShiftMix((a + e) * k0) * k0;\n result[3] = a + result[2];\n}\n\n// Requires len < 240.\nstatic void CityHashCrc256Short(const char *s, size_t len, uint64 *result) {\n char buf[240];\n memcpy(buf, s, len);\n memset(buf + len, 0, 240 - len);\n CityHashCrc256Long(buf, 240, ~static_cast(len), result);\n}\n\nvoid CityHashCrc256(const char *s, size_t len, uint64 *result) {\n if (LIKELY(len >= 240)) {\n CityHashCrc256Long(s, len, 0, result);\n } else {\n CityHashCrc256Short(s, len, result);\n }\n}\n\nuint128 CityHashCrc128WithSeed(const char *s, size_t len, uint128 seed) {\n if (len <= 900) {\n return CityHash128WithSeed(s, len, seed);\n } else {\n uint64 result[4];\n CityHashCrc256(s, len, result);\n uint64 u = Uint128High64(seed) + result[0];\n uint64 v = Uint128Low64(seed) + result[1];\n return uint128(HashLen16(u, v + result[2]),\n HashLen16(Rotate(v, 32), u * k0 + result[3]));\n }\n}\n\nuint128 CityHashCrc128(const char *s, size_t len) {\n if (len <= 900) {\n return CityHash128(s, len);\n } else {\n uint64 result[4];\n CityHashCrc256(s, len, result);\n return uint128(result[2], result[3]);\n }\n}\n\n#endif\n" }, { "path": "benchmarks/City.h", "content": "// Copyright (c) 2011 Google, Inc.\n//\n// Permission is hereby granted, free of charge, to any person obtaining a copy\n// of this software and associated documentation files (the \"Software\"), to deal\n// in the Software without restriction, including without limitation the rights\n// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n// copies of the Software, and to permit persons to whom the Software is\n// furnished to do so, subject to the following conditions:\n//\n// The above copyright notice and this permission notice shall be included in\n// all copies or substantial portions of the Software.\n//\n// THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\n// THE SOFTWARE.\n//\n// CityHash, by Geoff Pike and Jyrki Alakuijala\n//\n// This file provides a few functions for hashing strings. On x86-64\n// hardware in 2011, CityHash64() is faster than other high-quality\n// hash functions, such as Murmur. This is largely due to higher\n// instruction-level parallelism. CityHash64() and CityHash128() also perform\n// well on hash-quality tests.\n//\n// CityHash128() is optimized for relatively long strings and returns\n// a 128-bit hash. For strings more than about 2000 bytes it can be\n// faster than CityHash64().\n//\n// Functions in the CityHash family are not suitable for cryptography.\n//\n// WARNING: This code has not been tested on big-endian platforms!\n// It is known to work well on little-endian platforms that have a small penalty\n// for unaligned reads, such as current Intel and AMD moderate-to-high-end CPUs.\n//\n// By the way, for some hash functions, given strings a and b, the hash\n// of a+b is easily derived from the hashes of a and b. This property\n// doesn't hold for any hash functions in this file.\n\n#ifndef CITY_HASH_H_\n#define CITY_HASH_H_\n\n#include // for size_t.\n#include \n\n// Microsoft Visual Studio may not have stdint.h.\n#if defined(_MSC_VER) && (_MSC_VER < 1600)\ntypedef unsigned char uint8_t;\ntypedef unsigned int uint32_t;\ntypedef unsigned __int64 uint64_t;\n#else // defined(_MSC_VER)\n#include \n#endif // !defined(_MSC_VER)\n\ntypedef uint8_t uint8;\ntypedef uint32_t uint32;\ntypedef uint64_t uint64;\ntypedef std::pair uint128;\n\ninline uint64 Uint128Low64(const uint128& x) { return x.first; }\ninline uint64 Uint128High64(const uint128& x) { return x.second; }\n\n// Hash function for a byte array.\nuint64 CityHash64(const char *buf, size_t len);\n\n// Hash function for a byte array. For convenience, a 64-bit seed is also\n// hashed into the result.\nuint64 CityHash64WithSeed(const char *buf, size_t len, uint64 seed);\n\n// Hash function for a byte array. For convenience, two seeds are also\n// hashed into the result.\nuint64 CityHash64WithSeeds(const char *buf, size_t len,\n uint64 seed0, uint64 seed1);\n\n// Hash function for a byte array.\nuint128 CityHash128(const char *s, size_t len);\n\n// Hash function for a byte array. For convenience, a 128-bit seed is also\n// hashed into the result.\nuint128 CityHash128WithSeed(const char *s, size_t len, uint128 seed);\n\n// Hash 128 input bits down to 64 bits of output.\n// This is intended to be a reasonably good hash function.\ninline uint64 Hash128to64(const uint128& x) {\n // Murmur-inspired hashing.\n const uint64 kMul = 0x9ddfea08eb382d69ULL;\n uint64 a = (Uint128Low64(x) ^ Uint128High64(x)) * kMul;\n a ^= (a >> 47);\n uint64 b = (Uint128High64(x) ^ a) * kMul;\n b ^= (b >> 47);\n b *= kMul;\n return b;\n}\n\n// Conditionally include declarations for versions of City that require SSE4.2\n// instructions to be available.\n#if defined(__SSE4_2__) && defined(__x86_64__)\n\n// Hash function for a byte array.\nuint128 CityHashCrc128(const char *s, size_t len);\n\n// Hash function for a byte array. For convenience, a 128-bit seed is also\n// hashed into the result.\nuint128 CityHashCrc128WithSeed(const char *s, size_t len, uint128 seed);\n\n// Hash function for a byte array. Sets result[0] ... result[3].\nvoid CityHashCrc256(const char *s, size_t len, uint64 *result);\n\n#endif // __SSE4_2__\n\n#endif // CITY_HASH_H_\n" }, { "path": "benchmarks/SpookyV2.cpp", "content": "// Spooky Hash\n// A 128-bit noncryptographic hash, for checksums and table lookup\n// By Bob Jenkins. Public domain.\n// Oct 31 2010: published framework, disclaimer ShortHash isn't right\n// Nov 7 2010: disabled ShortHash\n// Oct 31 2011: replace End, ShortMix, ShortEnd, enable ShortHash again\n// April 10 2012: buffer overflow on platforms without unaligned reads\n// July 12 2012: was passing out variables in final to in/out in short\n// July 30 2012: I reintroduced the buffer overflow\n// August 5 2012: SpookyV2: d = should be d += in short hash, and remove extra mix from long hash\n\n#include \n#include \"SpookyV2.h\"\n\n#define ALLOW_UNALIGNED_READS 1\n\n//\n// short hash ... it could be used on any message, \n// but it's used by Spooky just for short messages.\n//\nvoid SpookyHash::Short(\n const void *message,\n size_t length,\n uint64 *hash1,\n uint64 *hash2)\n{\n uint64 buf[2*sc_numVars];\n union \n { \n const uint8 *p8; \n uint32 *p32;\n uint64 *p64; \n size_t i; \n } u;\n\n u.p8 = (const uint8 *)message;\n \n if (!ALLOW_UNALIGNED_READS && (u.i & 0x7))\n {\n memcpy(buf, message, length);\n u.p64 = buf;\n }\n\n size_t remainder = length%32;\n uint64 a=*hash1;\n uint64 b=*hash2;\n uint64 c=sc_const;\n uint64 d=sc_const;\n\n if (length > 15)\n {\n const uint64 *end = u.p64 + (length/32)*4;\n \n // handle all complete sets of 32 bytes\n for (; u.p64 < end; u.p64 += 4)\n {\n c += u.p64[0];\n d += u.p64[1];\n ShortMix(a,b,c,d);\n a += u.p64[2];\n b += u.p64[3];\n }\n \n //Handle the case of 16+ remaining bytes.\n if (remainder >= 16)\n {\n c += u.p64[0];\n d += u.p64[1];\n ShortMix(a,b,c,d);\n u.p64 += 2;\n remainder -= 16;\n }\n }\n \n // Handle the last 0..15 bytes, and its length\n d += ((uint64)length) << 56;\n switch (remainder)\n {\n case 15:\n d += ((uint64)u.p8[14]) << 48;\n case 14:\n d += ((uint64)u.p8[13]) << 40;\n case 13:\n d += ((uint64)u.p8[12]) << 32;\n case 12:\n d += u.p32[2];\n c += u.p64[0];\n break;\n case 11:\n d += ((uint64)u.p8[10]) << 16;\n case 10:\n d += ((uint64)u.p8[9]) << 8;\n case 9:\n d += (uint64)u.p8[8];\n case 8:\n c += u.p64[0];\n break;\n case 7:\n c += ((uint64)u.p8[6]) << 48;\n case 6:\n c += ((uint64)u.p8[5]) << 40;\n case 5:\n c += ((uint64)u.p8[4]) << 32;\n case 4:\n c += u.p32[0];\n break;\n case 3:\n c += ((uint64)u.p8[2]) << 16;\n case 2:\n c += ((uint64)u.p8[1]) << 8;\n case 1:\n c += (uint64)u.p8[0];\n break;\n case 0:\n c += sc_const;\n d += sc_const;\n }\n ShortEnd(a,b,c,d);\n *hash1 = a;\n *hash2 = b;\n}\n\n\n\n\n// do the whole hash in one call\nvoid SpookyHash::Hash128(\n const void *message, \n size_t length, \n uint64 *hash1, \n uint64 *hash2)\n{\n if (length < sc_bufSize)\n {\n Short(message, length, hash1, hash2);\n return;\n }\n\n uint64 h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11;\n uint64 buf[sc_numVars];\n uint64 *end;\n union \n { \n const uint8 *p8; \n uint64 *p64; \n size_t i; \n } u;\n size_t remainder;\n \n h0=h3=h6=h9 = *hash1;\n h1=h4=h7=h10 = *hash2;\n h2=h5=h8=h11 = sc_const;\n \n u.p8 = (const uint8 *)message;\n end = u.p64 + (length/sc_blockSize)*sc_numVars;\n\n // handle all whole sc_blockSize blocks of bytes\n if (ALLOW_UNALIGNED_READS || ((u.i & 0x7) == 0))\n {\n while (u.p64 < end)\n { \n Mix(u.p64, h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11);\n\t u.p64 += sc_numVars;\n }\n }\n else\n {\n while (u.p64 < end)\n {\n memcpy(buf, u.p64, sc_blockSize);\n Mix(buf, h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11);\n\t u.p64 += sc_numVars;\n }\n }\n\n // handle the last partial block of sc_blockSize bytes\n remainder = (length - ((const uint8 *)end-(const uint8 *)message));\n memcpy(buf, end, remainder);\n memset(((uint8 *)buf)+remainder, 0, sc_blockSize-remainder);\n ((uint8 *)buf)[sc_blockSize-1] = remainder;\n \n // do some final mixing \n End(buf, h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11);\n *hash1 = h0;\n *hash2 = h1;\n}\n\n\n\n// init spooky state\nvoid SpookyHash::Init(uint64 seed1, uint64 seed2)\n{\n m_length = 0;\n m_remainder = 0;\n m_state[0] = seed1;\n m_state[1] = seed2;\n}\n\n\n// add a message fragment to the state\nvoid SpookyHash::Update(const void *message, size_t length)\n{\n uint64 h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11;\n size_t newLength = length + m_remainder;\n uint8 remainder;\n union \n { \n const uint8 *p8; \n uint64 *p64; \n size_t i; \n } u;\n const uint64 *end;\n \n // Is this message fragment too short? If it is, stuff it away.\n if (newLength < sc_bufSize)\n {\n memcpy(&((uint8 *)m_data)[m_remainder], message, length);\n m_length = length + m_length;\n m_remainder = (uint8)newLength;\n return;\n }\n \n // init the variables\n if (m_length < sc_bufSize)\n {\n h0=h3=h6=h9 = m_state[0];\n h1=h4=h7=h10 = m_state[1];\n h2=h5=h8=h11 = sc_const;\n }\n else\n {\n h0 = m_state[0];\n h1 = m_state[1];\n h2 = m_state[2];\n h3 = m_state[3];\n h4 = m_state[4];\n h5 = m_state[5];\n h6 = m_state[6];\n h7 = m_state[7];\n h8 = m_state[8];\n h9 = m_state[9];\n h10 = m_state[10];\n h11 = m_state[11];\n }\n m_length = length + m_length;\n \n // if we've got anything stuffed away, use it now\n if (m_remainder)\n {\n uint8 prefix = sc_bufSize-m_remainder;\n memcpy(&(((uint8 *)m_data)[m_remainder]), message, prefix);\n u.p64 = m_data;\n Mix(u.p64, h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11);\n Mix(&u.p64[sc_numVars], h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11);\n u.p8 = ((const uint8 *)message) + prefix;\n length -= prefix;\n }\n else\n {\n u.p8 = (const uint8 *)message;\n }\n \n // handle all whole blocks of sc_blockSize bytes\n end = u.p64 + (length/sc_blockSize)*sc_numVars;\n remainder = (uint8)(length-((const uint8 *)end-u.p8));\n if (ALLOW_UNALIGNED_READS || (u.i & 0x7) == 0)\n {\n while (u.p64 < end)\n { \n Mix(u.p64, h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11);\n\t u.p64 += sc_numVars;\n }\n }\n else\n {\n while (u.p64 < end)\n { \n memcpy(m_data, u.p8, sc_blockSize);\n Mix(m_data, h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11);\n\t u.p64 += sc_numVars;\n }\n }\n\n // stuff away the last few bytes\n m_remainder = remainder;\n memcpy(m_data, end, remainder);\n \n // stuff away the variables\n m_state[0] = h0;\n m_state[1] = h1;\n m_state[2] = h2;\n m_state[3] = h3;\n m_state[4] = h4;\n m_state[5] = h5;\n m_state[6] = h6;\n m_state[7] = h7;\n m_state[8] = h8;\n m_state[9] = h9;\n m_state[10] = h10;\n m_state[11] = h11;\n}\n\n\n// report the hash for the concatenation of all message fragments so far\nvoid SpookyHash::Final(uint64 *hash1, uint64 *hash2)\n{\n // init the variables\n if (m_length < sc_bufSize)\n {\n *hash1 = m_state[0];\n *hash2 = m_state[1];\n Short( m_data, m_length, hash1, hash2);\n return;\n }\n \n const uint64 *data = (const uint64 *)m_data;\n uint8 remainder = m_remainder;\n \n uint64 h0 = m_state[0];\n uint64 h1 = m_state[1];\n uint64 h2 = m_state[2];\n uint64 h3 = m_state[3];\n uint64 h4 = m_state[4];\n uint64 h5 = m_state[5];\n uint64 h6 = m_state[6];\n uint64 h7 = m_state[7];\n uint64 h8 = m_state[8];\n uint64 h9 = m_state[9];\n uint64 h10 = m_state[10];\n uint64 h11 = m_state[11];\n\n if (remainder >= sc_blockSize)\n {\n // m_data can contain two blocks; handle any whole first block\n Mix(data, h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11);\n data += sc_numVars;\n remainder -= sc_blockSize;\n }\n\n // mix in the last partial block, and the length mod sc_blockSize\n memset(&((uint8 *)data)[remainder], 0, (sc_blockSize-remainder));\n\n ((uint8 *)data)[sc_blockSize-1] = remainder;\n \n // do some final mixing\n End(data, h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11);\n\n *hash1 = h0;\n *hash2 = h1;\n}\n\n" }, { "path": "benchmarks/SpookyV2.h", "content": "//\n// SpookyHash: a 128-bit noncryptographic hash function\n// By Bob Jenkins, public domain\n// Oct 31 2010: alpha, framework + SpookyHash::Mix appears right\n// Oct 31 2011: alpha again, Mix only good to 2^^69 but rest appears right\n// Dec 31 2011: beta, improved Mix, tested it for 2-bit deltas\n// Feb 2 2012: production, same bits as beta\n// Feb 5 2012: adjusted definitions of uint* to be more portable\n// Mar 30 2012: 3 bytes/cycle, not 4. Alpha was 4 but wasn't thorough enough.\n// August 5 2012: SpookyV2 (different results)\n// \n// Up to 3 bytes/cycle for long messages. Reasonably fast for short messages.\n// All 1 or 2 bit deltas achieve avalanche within 1% bias per output bit.\n//\n// This was developed for and tested on 64-bit x86-compatible processors.\n// It assumes the processor is little-endian. There is a macro\n// controlling whether unaligned reads are allowed (by default they are).\n// This should be an equally good hash on big-endian machines, but it will\n// compute different results on them than on little-endian machines.\n//\n// Google's CityHash has similar specs to SpookyHash, and CityHash is faster\n// on new Intel boxes. MD4 and MD5 also have similar specs, but they are orders\n// of magnitude slower. CRCs are two or more times slower, but unlike \n// SpookyHash, they have nice math for combining the CRCs of pieces to form \n// the CRCs of wholes. There are also cryptographic hashes, but those are even \n// slower than MD5.\n//\n\n#include \n\n#ifdef _MSC_VER\n# define INLINE __forceinline\n typedef unsigned __int64 uint64;\n typedef unsigned __int32 uint32;\n typedef unsigned __int16 uint16;\n typedef unsigned __int8 uint8;\n#else\n# include \n# define INLINE inline\n typedef uint64_t uint64;\n typedef uint32_t uint32;\n typedef uint16_t uint16;\n typedef uint8_t uint8;\n#endif\n\n\nclass SpookyHash\n{\npublic:\n //\n // SpookyHash: hash a single message in one call, produce 128-bit output\n //\n static void Hash128(\n const void *message, // message to hash\n size_t length, // length of message in bytes\n uint64 *hash1, // in/out: in seed 1, out hash value 1\n uint64 *hash2); // in/out: in seed 2, out hash value 2\n\n //\n // Hash64: hash a single message in one call, return 64-bit output\n //\n static uint64 Hash64(\n const void *message, // message to hash\n size_t length, // length of message in bytes\n uint64 seed) // seed\n {\n uint64 hash1 = seed;\n Hash128(message, length, &hash1, &seed);\n return hash1;\n }\n\n //\n // Hash32: hash a single message in one call, produce 32-bit output\n //\n static uint32 Hash32(\n const void *message, // message to hash\n size_t length, // length of message in bytes\n uint32 seed) // seed\n {\n uint64 hash1 = seed, hash2 = seed;\n Hash128(message, length, &hash1, &hash2);\n return (uint32)hash1;\n }\n\n //\n // Init: initialize the context of a SpookyHash\n //\n void Init(\n uint64 seed1, // any 64-bit value will do, including 0\n uint64 seed2); // different seeds produce independent hashes\n \n //\n // Update: add a piece of a message to a SpookyHash state\n //\n void Update(\n const void *message, // message fragment\n size_t length); // length of message fragment in bytes\n\n\n //\n // Final: compute the hash for the current SpookyHash state\n //\n // This does not modify the state; you can keep updating it afterward\n //\n // The result is the same as if SpookyHash() had been called with\n // all the pieces concatenated into one message.\n //\n void Final(\n uint64 *hash1, // out only: first 64 bits of hash value.\n uint64 *hash2); // out only: second 64 bits of hash value.\n\n //\n // left rotate a 64-bit value by k bytes\n //\n static INLINE uint64 Rot64(uint64 x, int k)\n {\n return (x << k) | (x >> (64 - k));\n }\n\n //\n // This is used if the input is 96 bytes long or longer.\n //\n // The internal state is fully overwritten every 96 bytes.\n // Every input bit appears to cause at least 128 bits of entropy\n // before 96 other bytes are combined, when run forward or backward\n // For every input bit,\n // Two inputs differing in just that input bit\n // Where \"differ\" means xor or subtraction\n // And the base value is random\n // When run forward or backwards one Mix\n // I tried 3 pairs of each; they all differed by at least 212 bits.\n //\n static INLINE void Mix(\n const uint64 *data, \n uint64 &s0, uint64 &s1, uint64 &s2, uint64 &s3,\n uint64 &s4, uint64 &s5, uint64 &s6, uint64 &s7,\n uint64 &s8, uint64 &s9, uint64 &s10,uint64 &s11)\n {\n s0 += data[0]; s2 ^= s10; s11 ^= s0; s0 = Rot64(s0,11); s11 += s1;\n s1 += data[1]; s3 ^= s11; s0 ^= s1; s1 = Rot64(s1,32); s0 += s2;\n s2 += data[2]; s4 ^= s0; s1 ^= s2; s2 = Rot64(s2,43); s1 += s3;\n s3 += data[3]; s5 ^= s1; s2 ^= s3; s3 = Rot64(s3,31); s2 += s4;\n s4 += data[4]; s6 ^= s2; s3 ^= s4; s4 = Rot64(s4,17); s3 += s5;\n s5 += data[5]; s7 ^= s3; s4 ^= s5; s5 = Rot64(s5,28); s4 += s6;\n s6 += data[6]; s8 ^= s4; s5 ^= s6; s6 = Rot64(s6,39); s5 += s7;\n s7 += data[7]; s9 ^= s5; s6 ^= s7; s7 = Rot64(s7,57); s6 += s8;\n s8 += data[8]; s10 ^= s6; s7 ^= s8; s8 = Rot64(s8,55); s7 += s9;\n s9 += data[9]; s11 ^= s7; s8 ^= s9; s9 = Rot64(s9,54); s8 += s10;\n s10 += data[10]; s0 ^= s8; s9 ^= s10; s10 = Rot64(s10,22); s9 += s11;\n s11 += data[11]; s1 ^= s9; s10 ^= s11; s11 = Rot64(s11,46); s10 += s0;\n }\n\n //\n // Mix all 12 inputs together so that h0, h1 are a hash of them all.\n //\n // For two inputs differing in just the input bits\n // Where \"differ\" means xor or subtraction\n // And the base value is random, or a counting value starting at that bit\n // The final result will have each bit of h0, h1 flip\n // For every input bit,\n // with probability 50 +- .3%\n // For every pair of input bits,\n // with probability 50 +- 3%\n //\n // This does not rely on the last Mix() call having already mixed some.\n // Two iterations was almost good enough for a 64-bit result, but a\n // 128-bit result is reported, so End() does three iterations.\n //\n static INLINE void EndPartial(\n uint64 &h0, uint64 &h1, uint64 &h2, uint64 &h3,\n uint64 &h4, uint64 &h5, uint64 &h6, uint64 &h7, \n uint64 &h8, uint64 &h9, uint64 &h10,uint64 &h11)\n {\n h11+= h1; h2 ^= h11; h1 = Rot64(h1,44);\n h0 += h2; h3 ^= h0; h2 = Rot64(h2,15);\n h1 += h3; h4 ^= h1; h3 = Rot64(h3,34);\n h2 += h4; h5 ^= h2; h4 = Rot64(h4,21);\n h3 += h5; h6 ^= h3; h5 = Rot64(h5,38);\n h4 += h6; h7 ^= h4; h6 = Rot64(h6,33);\n h5 += h7; h8 ^= h5; h7 = Rot64(h7,10);\n h6 += h8; h9 ^= h6; h8 = Rot64(h8,13);\n h7 += h9; h10^= h7; h9 = Rot64(h9,38);\n h8 += h10; h11^= h8; h10= Rot64(h10,53);\n h9 += h11; h0 ^= h9; h11= Rot64(h11,42);\n h10+= h0; h1 ^= h10; h0 = Rot64(h0,54);\n }\n\n static INLINE void End(\n const uint64 *data, \n uint64 &h0, uint64 &h1, uint64 &h2, uint64 &h3,\n uint64 &h4, uint64 &h5, uint64 &h6, uint64 &h7, \n uint64 &h8, uint64 &h9, uint64 &h10,uint64 &h11)\n {\n h0 += data[0]; h1 += data[1]; h2 += data[2]; h3 += data[3];\n h4 += data[4]; h5 += data[5]; h6 += data[6]; h7 += data[7];\n h8 += data[8]; h9 += data[9]; h10 += data[10]; h11 += data[11];\n EndPartial(h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11);\n EndPartial(h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11);\n EndPartial(h0,h1,h2,h3,h4,h5,h6,h7,h8,h9,h10,h11);\n }\n\n //\n // The goal is for each bit of the input to expand into 128 bits of \n // apparent entropy before it is fully overwritten.\n // n trials both set and cleared at least m bits of h0 h1 h2 h3\n // n: 2 m: 29\n // n: 3 m: 46\n // n: 4 m: 57\n // n: 5 m: 107\n // n: 6 m: 146\n // n: 7 m: 152\n // when run forwards or backwards\n // for all 1-bit and 2-bit diffs\n // with diffs defined by either xor or subtraction\n // with a base of all zeros plus a counter, or plus another bit, or random\n //\n static INLINE void ShortMix(uint64 &h0, uint64 &h1, uint64 &h2, uint64 &h3)\n {\n h2 = Rot64(h2,50); h2 += h3; h0 ^= h2;\n h3 = Rot64(h3,52); h3 += h0; h1 ^= h3;\n h0 = Rot64(h0,30); h0 += h1; h2 ^= h0;\n h1 = Rot64(h1,41); h1 += h2; h3 ^= h1;\n h2 = Rot64(h2,54); h2 += h3; h0 ^= h2;\n h3 = Rot64(h3,48); h3 += h0; h1 ^= h3;\n h0 = Rot64(h0,38); h0 += h1; h2 ^= h0;\n h1 = Rot64(h1,37); h1 += h2; h3 ^= h1;\n h2 = Rot64(h2,62); h2 += h3; h0 ^= h2;\n h3 = Rot64(h3,34); h3 += h0; h1 ^= h3;\n h0 = Rot64(h0,5); h0 += h1; h2 ^= h0;\n h1 = Rot64(h1,36); h1 += h2; h3 ^= h1;\n }\n\n //\n // Mix all 4 inputs together so that h0, h1 are a hash of them all.\n //\n // For two inputs differing in just the input bits\n // Where \"differ\" means xor or subtraction\n // And the base value is random, or a counting value starting at that bit\n // The final result will have each bit of h0, h1 flip\n // For every input bit,\n // with probability 50 +- .3% (it is probably better than that)\n // For every pair of input bits,\n // with probability 50 +- .75% (the worst case is approximately that)\n //\n static INLINE void ShortEnd(uint64 &h0, uint64 &h1, uint64 &h2, uint64 &h3)\n {\n h3 ^= h2; h2 = Rot64(h2,15); h3 += h2;\n h0 ^= h3; h3 = Rot64(h3,52); h0 += h3;\n h1 ^= h0; h0 = Rot64(h0,26); h1 += h0;\n h2 ^= h1; h1 = Rot64(h1,51); h2 += h1;\n h3 ^= h2; h2 = Rot64(h2,28); h3 += h2;\n h0 ^= h3; h3 = Rot64(h3,9); h0 += h3;\n h1 ^= h0; h0 = Rot64(h0,47); h1 += h0;\n h2 ^= h1; h1 = Rot64(h1,54); h2 += h1;\n h3 ^= h2; h2 = Rot64(h2,32); h3 += h2;\n h0 ^= h3; h3 = Rot64(h3,25); h0 += h3;\n h1 ^= h0; h0 = Rot64(h0,63); h1 += h0;\n }\n \nprivate:\n\n //\n // Short is used for messages under 192 bytes in length\n // Short has a low startup cost, the normal mode is good for long\n // keys, the cost crossover is at about 192 bytes. The two modes were\n // held to the same quality bar.\n // \n static void Short(\n const void *message, // message (array of bytes, not necessarily aligned)\n size_t length, // length of message (in bytes)\n uint64 *hash1, // in/out: in the seed, out the hash value\n uint64 *hash2); // in/out: in the seed, out the hash value\n\n // number of uint64's in internal state\n static const size_t sc_numVars = 12;\n\n // size of the internal state\n static const size_t sc_blockSize = sc_numVars*8;\n\n // size of buffer of unhashed data, in bytes\n static const size_t sc_bufSize = 2*sc_blockSize;\n\n //\n // sc_const: a constant which:\n // * is not zero\n // * is odd\n // * is a not-very-regular mix of 1's and 0's\n // * does not need any other special mathematical properties\n //\n static const uint64 sc_const = 0xdeadbeefdeadbeefLL;\n\n uint64 m_data[2*sc_numVars]; // unhashed data, for partial messages\n uint64 m_state[sc_numVars]; // internal state of the hash\n size_t m_length; // total length of the input so far\n uint8 m_remainder; // length of unhashed data stashed in m_data\n};\n\n\n\n" }, { "path": "benchmarks/bbs-prng.h", "content": "/* Copyright (c) 2016 Vladimir Makarov \n\n Permission is hereby granted, free of charge, to any person\n obtaining a copy of this software and associated documentation\n files (the \"Software\"), to deal in the Software without\n restriction, including without limitation the rights to use, copy,\n modify, merge, publish, distribute, sublicense, and/or sell copies\n of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n\n The above copyright notice and this permission notice shall be\n included in all copies or substantial portions of the Software.\n\n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\n MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\n BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\n ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\n CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.\n*/\n\n/* Blum-Blum-Shub Pseudo Random Number Generator (PRNG). It is a\n crypto level PRNG: asymptotically the prediction of the next\n generated number is NP-complete task as finding the solution is\n equivalent to solving quadratic residue modulo N problem.\n\n The PRNG equation is simple x[n+1] = (x[n] * x[n]) mod N, where N\n is a product of two large prime numbers.\n\n The PRNG finds the two prime numbers during the PRNG\n initialization. The prime number search is a probabilistic one.\n The numbers might be composite but the probability of this is very\n small 4^(-100).\n\n Working with big numbers are implemented by GMP. So you need link\n a GMP Library (-lgmp) if you are going to use the PRNG.\n\n To use a generator call `init_bbs_prng` first, then call\n `get_bbs_prn` as much as you want to get a new PRN. At the end of\n the PRNG use, call `finish_bbs_prng`. You can change the default\n seed by calling set_bbs_seed.\n\n The PRNG passes NIST Statistical Test Suite for Random and\n Pseudorandom Number Generators for Cryptographic Applications\n (version 2.2.1) with 1000 bitstreams each containing 1M bits.\n\n The generation of a new number takes about 73K CPU cycles on x86_64\n (Intel 4.2GHz i7-4790K), or speed of the generation is about 58K\n numbers per sec. */\n\n#ifndef __BBS_PRNG__\n#define __BBS_PRNG__\n\n#ifdef _MSC_VER\ntypedef unsigned __int32 uint32_t;\ntypedef unsigned __int64 uint64_t;\n#else\n#include \n#endif\n\n#include \n#include \n\n/* What size numbers should we use to make the next number prediction\n hard and how hard the prediction can be for a given number is a\n tricky question. Please, read the Blum Blum Shub article and\n numerous discussion on the Internet. I believe the default value\n is good for my purposes. */\n#ifndef BBS_PRIME_BITS\n#define BBS_PRIME_BITS 512\n#endif\n\n/* BBS N which is a factor of two primes and the last generated pseudo\n random number which also can be considered as the current state of\n the PRNG. */\nstatic MP_INT _BBS_N, _BBS_xn;\n\n/* Set up _BBS_N and _BBS_xn. */\nstatic inline void\ninit_bbs_prng (void) {\n int i, n;\n MP_INT start;\n \n mpz_init (&start);\n mpz_init (&_BBS_N);\n mpz_init (&_BBS_xn);\n for (n = 0; n != 3;) {\n mpz_set_ui (&start, 0);\n for (i = 0; i < BBS_PRIME_BITS; i++) {\n mpz_mul_ui (&start, &start, 2);\n mpz_add_ui (&start, &start, rand () % 2);\n }\n if (mpz_probab_prime_p (&start, 100) == 0\n\t/* The following is BBS requirement to have only one solution\n\t for the quadratic residue equation. */\n\t|| mpz_tdiv_ui (&start, 4) != 3)\n continue;\n if (n == 0) mpz_set (&_BBS_N, &start);\n else if (n == 1) mpz_mul (&_BBS_N, &_BBS_N, &start);\n else mpz_set (&_BBS_xn, &start);\n n++;\n }\n mpz_clear (&start);\n}\n\n/* Make _BBS_xn equal to SEED. */\nstatic inline void\nset_bbs_seed (uint32_t seed) {\n mpz_set_ui (&_BBS_xn, seed);\n}\n\n\n/* Update _BBS_xn. */\nstatic inline void\n_update_bbs_prng (void) {\n mpz_mul (&_BBS_xn, &_BBS_xn, &_BBS_xn);\n mpz_tdiv_r (&_BBS_xn, &_BBS_xn, &_BBS_N);\n}\n\n/* Return the next pseudo random number. */\nstatic inline uint64_t\nget_bbs_prn (void) {\n int i;\n uint64_t res = 0;\n \n for (i = 0; i < 32; i++) {\n _update_bbs_prng ();\n /* We conservatively use only 2 LS bits. Depending of\n BBS_PRIME_BITS value it could be more. */\n res = res << 2 | mpz_tdiv_ui (&_BBS_xn, 4);\n }\n return res;\n}\n\n/* Finish work with the PRNG. */\nstatic inline void\nfinish_bbs_prng (void) {\n mpz_clear (&_BBS_N);\n mpz_clear (&_BBS_xn);\n}\n\n#endif\n" }, { "path": "benchmarks/bench-crypto.c", "content": "#if defined(SHA2)\n\n#include \"sha512.h\"\nvoid sha512_test (const void *msg, int len, void *out) {\n sha512_ctx ctx;\n \n rhash_sha512_init (&ctx);\n rhash_sha512_update (&ctx, msg, len);\n rhash_sha512_final (&ctx, out);\n}\n\n#define test sha512_test\n\n#elif defined(SHA3)\n\n#include \"sha3.h\"\nvoid sha3_512_test (const void *msg, int len, void *out) {\n sha3_ctx ctx;\n \n rhash_sha3_512_init (&ctx);\n rhash_sha3_update (&ctx, msg, len);\n rhash_sha3_final (&ctx, out);\n}\n\n#define test sha3_512_test\n\n#elif defined(BLAKE2B)\n\n#include \"blake2.h\"\nvoid blake2b_test (const void *msg, int len, void *out) {\n static const uint64_t key[4] = {0, 0, 0, 0};\n blake2b ((uint8_t *) out, msg, key, 64, len, 32);\n}\n\n#define test blake2b_test\n\n#elif defined(MUM512)\n\n#include \"mum512.h\"\n\nvoid mum512_test (const void *msg, int len, void *out) {\n mum512_hash (msg, len, out);\n}\n\n#define test mum512_test\n\n#else\n#error \"I don't know what to test\"\n#endif\n\nstatic const char msg[] =\n\"SATURDAY morning was come, and all the summer world was bright and\\n\\\nfresh, and brimming with life. There was a song in every heart; and if\\n\\\nthe heart was young the music issued at the lips. There was cheer in\\n\\\nevery face and a spring in every step. The locust-trees were in bloom\\n\\\nand the fragrance of the blossoms filled the air. Cardiff Hill, beyond\\n\\\nthe village and above it, was green with vegetation and it lay just far\\n\\\nenough away to seem a Delectable Land, dreamy, reposeful, and inviting.\\n\\\n\\n\\\nTom appeared on the sidewalk with a bucket of whitewash and a\\n\\\nlong-handled brush. He surveyed the fence, and all gladness left him and\\n\\\na deep melancholy settled down upon his spirit. Thirty yards of board\\n\\\nfence nine feet high. Life to him seemed hollow, and existence but a\\n\\\nburden. Sighing, he dipped his brush and passed it along the topmost\\n\\\nplank; repeated the operation; did it again; compared the insignificant\\n\\\nwhitewashed streak with the far-reaching continent of unwhitewashed\\n\\\nfence, and sat down on a tree-box discouraged. Jim came skipping out at\\n\\\nthe gate with a tin pail, and singing Buffalo Gals. Bringing water from\\n\\\nthe town pump had always been hateful work in Tom's eyes, before, but\\n\\\nnow it did not strike him so. He remembered that there was company at\\n\\\nthe pump. White, mulatto, and negro boys and girls were always there\\n\\\nwaiting their turns, resting, trading playthings, quarrelling, fighting,\\n\\\nskylarking. And he remembered that although the pump was only a hundred\\n\\\nand fifty yards off, Jim never got back with a bucket of water under an\\n\\\nhour--and even then somebody generally had to go after him. Tom said:\\n\\\n\\n\\\n\\\"Say, Jim, I'll fetch the water if you'll whitewash some.\\\"\\n\\\n\\n\\\nJim shook his head and said:\\n\\\n\\n\\\n\\\"Can't, Mars Tom. Ole missis, she tole me I got to go an' git dis water\\n\\\nan' not stop foolin' roun' wid anybody. She say she spec' Mars Tom gwine\\n\\\nto ax me to whitewash, an' so she tole me go 'long an' 'tend to my own\\n\\\nbusiness--she 'lowed _she'd_ 'tend to de whitewashin'.\\\"\\n\\\n\\n\\\n\\\"Oh, never you mind what she said, Jim. That's the way she always talks.\\n\\\nGimme the bucket--I won't be gone only a a minute. _She_ won't ever\\n\\\nknow.\\\"\\n\\\n\\n\\\n\\\"Oh, I dasn't, Mars Tom. Ole missis she'd take an' tar de head off'n me.\\n\\\n'Deed she would.\\\"\\n\\\n\\n\\\n\\\"_She_! She never licks anybody--whacks 'em over the head with her\\n\\\nthimble--and who cares for that, I'd like to know. She talks awful, but\\n\\\ntalk don't hurt--anyways it don't if she don't cry. Jim, I'll give you a\\n\\\nmarvel. I'll give you a white alley!\\\"\\n\\\n\\n\\\nJim began to waver.\\n\\\n\\n\\\n\\\"White alley, Jim! And it's a bully taw.\\\"\\n\\\n\\n\\\n\\\"My! Dat's a mighty gay marvel, I tell you! But Mars Tom I's powerful\\n\\\n'fraid ole missis--\\\"\\n\\\n\\n\\\n\\\"And besides, if you will I'll show you my sore toe.\\\"\\n\\\n\\n\\\nJim was only human--this attraction was too much for him. He put down\\n\\\nhis pail, took the white alley, and bent over the toe with absorbing\\n\\\ninterest while the bandage was being unwound. In another moment he\\n\\\nwas flying down the street with his pail and a tingling rear, Tom was\\n\\\nwhitewashing with vigor, and Aunt Polly was retiring from the field with\\n\\\na slipper in her hand and triumph in her eye.\\n\\\n\\n\\\nBut Tom's energy did not last. He began to think of the fun he had\\n\\\nplanned for this day, and his sorrows multiplied. Soon the free boys\\n\\\nwould come tripping along on all sorts of delicious expeditions, and\\n\\\nthey would make a world of fun of him for having to work--the very\\n\\\nthought of it burnt him like fire. He got out his worldly wealth and\\n\\\nexamined it--bits of toys, marbles, and trash; enough to buy an exchange\\n\\\nof _work_, maybe, but not half enough to buy so much as half an hour\\n\\\nof pure freedom. So he returned his straitened means to his pocket, and\\n\\\ngave up the idea of trying to buy the boys. At this dark and hopeless\\n\\\nmoment an inspiration burst upon him! Nothing less than a great,\\n\\\nmagnificent inspiration.\\n\\\n\\n\\\nHe took up his brush and went tranquilly to work. Ben Rogers hove in\\n\\\nsight presently--the very boy, of all boys, whose ridicule he had been\\n\\\ndreading. Ben's gait was the hop-skip-and-jump--proof enough that his\\n\\\nheart was light and his anticipations high. He was eating an apple, and\\n\\\ngiving a long, melodious whoop, at intervals, followed by a deep-toned\\n\\\nding-dong-dong, ding-dong-dong, for he was personating a steamboat. As\\n\\\nhe drew near, he slackened speed, took the middle of the street, leaned\\n\\\nfar over to starboard and rounded to ponderously and with laborious pomp\\n\\\nand circumstance--for he was personating the Big Missouri, and considered\\n\\\nhimself to be drawing nine feet of water. He was boat and captain and\\n\\\nengine-bells combined, so he had to imagine himself standing on his own\\n\\\nhurricane-deck giving the orders and executing them:\\n\\\n\\n\\\n\\\"Stop her, sir! Ting-a-ling-ling!\\\" The headway ran almost out, and he\\n\\\ndrew up slowly toward the sidewalk.\\n\\\n\\n\\\n\\\"Ship up to back! Ting-a-ling-ling!\\\" His arms straightened and stiffened\\n\\\ndown his sides.\\n\\\n\\n\\\n\\\"Set her back on the stabboard! Ting-a-ling-ling! Chow! ch-chow-wow!\\n\\\nChow!\\\" His right hand, mean-time, describing stately circles--for it was\\n\\\nrepresenting a forty-foot wheel.\\n\\\n\\n\\\n\\\"Let her go back on the labboard! Ting-a-ling-ling! Chow-ch-chow-chow!\\\"\\n\\\nThe left hand began to describe circles.\\n\\\n\\n\\\n\\\"Stop the stabboard! Ting-a-ling-ling! Stop the labboard! Come ahead on\\n\\\nthe stabboard! Stop her! Let your outside turn over slow! Ting-a-ling-ling!\\n\\\nChow-ow-ow! Get out that head-line! _lively_ now! Come--out with\\n\\\nyour spring-line--what're you about there! Take a turn round that stump\\n\\\nwith the bight of it! Stand by that stage, now--let her go! Done with\\n\\\nthe engines, sir! Ting-a-ling-ling! SH'T! S'H'T! SH'T!\\\" (trying the\\n\\\ngauge-cocks).\\n\\\n\\n\\\nTom went on whitewashing--paid no attention to the steamboat. Ben stared\\n\\\na moment and then said: \\\"_Hi-Yi! You're_ up a stump, ain't you!\\\"\\n\\\n\\n\\\nNo answer. Tom surveyed his last touch with the eye of an artist, then\\n\\\nhe gave his brush another gentle sweep and surveyed the result, as\\n\\\nbefore. Ben ranged up alongside of him. Tom's mouth watered for the\\n\\\napple, but he stuck to his work. Ben said:\\n\\\n\\n\\\n\\\"Hello, old chap, you got to work, hey?\\\"\\n\\\n\\n\\\nTom wheeled suddenly and said:\\n\\\n\\n\\\n\\\"Why, it's you, Ben! I warn't noticing.\\\"\\n\\\n\\n\\\n\\\"Say--I'm going in a-swimming, I am. Don't you wish you could? But of\\n\\\ncourse you'd druther _work_--wouldn't you? Course you would!\\\"\\n\\\n\\n\\\nTom contemplated the boy a bit, and said:\\n\\\n\\n\\\n\\\"What do you call work?\\\"\\n\\\n\\n\\\n\\\"Why, ain't _that_ work?\\\"\\n\\\n\\n\\\nTom resumed his whitewashing, and answered carelessly:\\n\\\n\\n\\\n\\\"Well, maybe it is, and maybe it ain't. All I know, is, it suits Tom\\n\\\nSawyer.\\\"\\n\\\n\\n\\\n\\\"Oh come, now, you don't mean to let on that you _like_ it?\\\"\\n\\\n\\n\\\nThe brush continued to move.\\n\\\n\\n\\\n\\\"Like it? Well, I don't see why I oughtn't to like it. Does a boy get a\\n\\\nchance to whitewash a fence every day?\\\"\\n\\\n\\n\\\nThat put the thing in a new light. Ben stopped nibbling his apple.\\n\\\nTom swept his brush daintily back and forth--stepped back to note the\\n\\\neffect--added a touch here and there--criticised the effect again--Ben\\n\\\nwatching every move and getting more and more interested, more and more\\n\\\nabsorbed. Presently he said:\\n\\\n\\n\\\n\\\"Say, Tom, let _me_ whitewash a little.\\\"\\n\\\n\\n\\\nTom considered, was about to consent; but he altered his mind:\\n\\\n\\n\\\n\\\"No--no--I reckon it wouldn't hardly do, Ben. You see, Aunt Polly's awful\\n\\\nparticular about this fence--right here on the street, you know--but if it\\n\\\nwas the back fence I wouldn't mind and _she_ wouldn't. Yes, she's awful\\n\\\nparticular about this fence; it's got to be done very careful; I reckon\\n\\\nthere ain't one boy in a thousand, maybe two thousand, that can do it\\n\\\nthe way it's got to be done.\\\"\\n\\\n\\n\\\n\\\"No--is that so? Oh come, now--lemme just try. Only just a little--I'd let\\n\\\n_you_, if you was me, Tom.\\\"\\n\\\n\\n\\\n\\\"Ben, I'd like to, honest injun; but Aunt Polly--well, Jim wanted to do\\n\\\nit, but she wouldn't let him; Sid wanted to do it, and she wouldn't let\\n\\\nSid. Now don't you see how I'm fixed? If you was to tackle this fence\\n\\\nand anything was to happen to it--\\\"\\n\\\n\\n\\\n\\\"Oh, shucks, I'll be just as careful. Now lemme try. Say--I'll give you\\n\\\nthe core of my apple.\\\"\\n\\\n\\n\\\n\\\"Well, here--No, Ben, now don't. I'm afeard--\\\"\\n\\\n\\n\\\n\\\"I'll give you _all_ of it!\\\"\\n\\\n\\n\\\nTom gave up the brush with reluctance in his face, but alacrity in his\\n\\\nheart. And while the late steamer Big Missouri worked and sweated in the\\n\\\nsun, the retired artist sat on a barrel in the shade close by,\\n\\\ndangled his legs, munched his apple, and planned the slaughter of more\\n\\\ninnocents. There was no lack of material; boys happened along every\\n\\\nlittle while; they came to jeer, but remained to whitewash. By the time\\n\\\nBen was fagged out, Tom had traded the next chance to Billy Fisher for\\n\\\na kite, in good repair; and when he played out, Johnny Miller bought in\\n\\\nfor a dead rat and a string to swing it with--and so on, and so on, hour\\n\\\nafter hour. And when the middle of the afternoon came, from being a\\n\\\npoor poverty-stricken boy in the morning, Tom was literally rolling in\\n\\\nwealth. He had besides the things before mentioned, twelve marbles, part\\n\\\nof a jews-harp, a piece of blue bottle-glass to look through, a spool\\n\\\ncannon, a key that wouldn't unlock anything, a fragment of chalk, a\\n\\\nglass stopper of a decanter, a tin soldier, a couple of tadpoles,\\n\\\nsix fire-crackers, a kitten with only one eye, a brass door-knob, a\\n\\\ndog-collar--but no dog--the handle of a knife, four pieces of orange-peel,\\n\\\nand a dilapidated old window sash.\\n\\\n\\n\\\nHe had had a nice, good, idle time all the while--plenty of company--and\\n\\\nthe fence had three coats of whitewash on it! If he hadn't run out of\\n\\\nwhitewash he would have bankrupted every boy in the village.\\n\\\n\\n\\\nTom said to himself that it was not such a hollow world, after all. He\\n\\\nhad discovered a great law of human action, without knowing it--namely,\\n\\\nthat in order to make a man or a boy covet a thing, it is only necessary\\n\\\nto make the thing difficult to attain. If he had been a great and\\n\\\nwise philosopher, like the writer of this book, he would now have\\n\\\ncomprehended that Work consists of whatever a body is _obliged_ to do,\\n\\\nand that Play consists of whatever a body is not obliged to do. And\\n\\\nthis would help him to understand why constructing artificial flowers or\\n\\\nperforming on a tread-mill is work, while rolling ten-pins or climbing\\n\\\nMont Blanc is only amusement. There are wealthy gentlemen in England\\n\\\nwho drive four-horse passenger-coaches twenty or thirty miles on a\\n\\\ndaily line, in the summer, because the privilege costs them considerable\\n\\\nmoney; but if they were offered wages for the service, that would turn\\n\\\nit into work and then they would resign.\\n\\\n\\n\\\nThe boy mused awhile over the substantial change which had taken place\\n\\\nin his worldly circumstances, and then wended toward headquarters to\\n\\\nreport.\\n\\\n\";\n\n#include \n#include \nstatic void\nprint512 (unsigned char digest[64]) {\n int i;\n \n for (i = 0; i < 64; i++)\n printf (\"%x\", digest[i]);\n printf (\"\\n\");\n}\n\n#ifdef SPEED1\nint main () {\n int i; uint64_t k = rand (); unsigned char out[64];\n \n for (i = 0; i < 2000000; i++)\n test (msg, 10, &out);\n printf (\"10 byte: %s:\", (size_t)&k & 0x7 ? \"unaligned\" : \"aligned\");\n print512 (out);\n return 0;\n}\n#endif\n\n#ifdef SPEED2\nint main () {\n int i; uint64_t k = rand (); unsigned char out[64];\n \n for (i = 0; i < 2000000; i++)\n test (msg, 100, &out);\n printf (\"100 byte: %s:\", (size_t)&k & 0x7 ? \"unaligned\" : \"aligned\");\n print512 (out);\n return 0;\n}\n#endif\n\n#ifdef SPEED3\nint main () {\n int i; uint64_t k = rand (); unsigned char out[64];\n \n for (i = 0; i < 2000000; i++)\n test (msg, 1000, &out);\n printf (\"1000 byte: %s:\", (size_t)&k & 0x7 ? \"unaligned\" : \"aligned\");\n print512 (out);\n return 0;\n}\n#endif\n\n#ifdef SPEED4\nint main () {\nint i; uint64_t k = rand (); unsigned char out[64];\n\n for (i = 0; i < 500000; i++)\n test (msg, 10000, &out);\n printf (\"10000 byte: %s:\", (size_t)&k & 0x7 ? \"unaligned\" : \"aligned\");\n print512 (out);\n return 0;\n}\n#endif\n\n" }, { "path": "benchmarks/bench-crypto.sh", "content": "#!/bin/bash\n\n# Benchmarking different crypto hash functions.\n\nIFS='%'\ntemp=__temp\n\n\nprint() {\n s=`grep -E 'user[ \t]*[0-9]' $2 | sed s/.*user// | sed s/\\\\t//`\n echo $1 \"$s\"s\n}\n\necho compiling sha512\ngcc -O3 -w -c sha512.c byte_order.c || exit 1\necho compiling sha3\ngcc -O3 -w -c sha3.c || exit 1\n\nstr86_64=\"`uname -a|grep x86_64`\"\nif test -n str86_64; then\n echo compiling blake2b\n gcc -O3 -w -c -I. -std=gnu99 blake2b.c || exit 1\nfi\n\necho +++10-byte speed '(20M texts)':\ngcc -DSPEED1 -O3 -w -DSHA2 sha512.o byte_order.o bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"sha512:\" $temp\ngcc -DSPEED1 -O3 -w -DSHA3 sha3.o byte_order.o bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"sha3 :\" $temp\nif test -n str86_64; then\n gcc -DSPEED1 -O3 -w -DBLAKE2B blake2b.o bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"blake2:\" $temp\nfi\ngcc -DSPEED1 -O3 -w -DMUM512 -I../ bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"mum512:\" $temp\n\necho +++100-byte speed '(20M texts)':\ngcc -DSPEED2 -O3 -w -DSHA2 sha512.o byte_order.o bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"sha512:\" $temp\ngcc -DSPEED2 -O3 -w -DSHA3 sha3.o byte_order.o bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"sha3 :\" $temp\nif test -n str86_64; then\n gcc -DSPEED2 -O3 -w -DBLAKE2B blake2b.o bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"blake2:\" $temp\nfi\ngcc -DSPEED2 -O3 -w -DMUM512 -I../ bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"mum512:\" $temp\n\necho +++1000-byte speed '(20M texts)':\ngcc -DSPEED3 -O3 -w -DSHA2 sha512.o byte_order.o bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"sha512:\" $temp\ngcc -DSPEED3 -O3 -w -DSHA3 sha3.o byte_order.o bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"sha3 :\" $temp\nif test -n str86_64; then\n gcc -DSPEED3 -O3 -w -DBLAKE2B blake2b.o bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"blake2:\" $temp\nfi\ngcc -DSPEED3 -O3 -w -DMUM512 -I../ bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"mum512:\" $temp\n\necho +++10000-byte speed '(5M texts)':\ngcc -DSPEED4 -O3 -w -DSHA2 sha512.o byte_order.o bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"sha512:\" $temp\ngcc -DSPEED4 -O3 -w -DSHA3 sha3.o byte_order.o bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"sha3 :\" $temp\nif test -n str86_64; then\n gcc -DSPEED4 -O3 -w -DBLAKE2B blake2b.o bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"blake2:\" $temp\nfi\ngcc -DSPEED4 -O3 -w -DMUM512 -I../ bench-crypto.c && (time -p ./a.out) >$temp 2>&1 && print \"mum512:\" $temp\n\nrm -rf ./a.out $temp sha512.o sha3.o byte_order.o blake2b.o\n" }, { "path": "benchmarks/bench-prng.c", "content": "#define N1 100000\n#if defined(BBS)\n#include \"bbs-prng.h\"\n#define N2 2\nstatic void init_prng (void) { init_bbs_prng (); }\nstatic uint64_t get_prn (void) { return get_bbs_prn (); }\nstatic void finish_prng (void) { finish_bbs_prng (); }\n#elif defined(CHACHA)\n#include \"chacha-prng.h\"\n#define N2 5000\nstatic void init_prng (void) { init_chacha_prng (); }\nstatic uint64_t get_prn (void) { return get_chacha_prn (); }\nstatic void finish_prng (void) { finish_chacha_prng (); }\n#elif defined(SIP24)\n#include \"sip24-prng.h\"\n#define N2 5000\nstatic void init_prng (void) { init_sip24_prng (); }\nstatic uint64_t get_prn (void) { return get_sip24_prn (); }\nstatic void finish_prng (void) { finish_sip24_prng (); }\n#elif defined(MUM)\n#include \"mum-prng.h\"\n#define N2 30000\nstatic void init_prng (void) { mum_hash_randomize (0); init_mum_prng (); }\nstatic uint64_t get_prn (void) { return get_mum_prn (); }\nstatic void finish_prng (void) { finish_mum_prng (); }\n#elif defined(MUM512)\n#include \"mum512-prng.h\"\n#define N2 2000\nstatic void init_prng (void) { init_mum512_prng (); }\nstatic uint64_t get_prn (void) { return get_mum512_prn (); }\nstatic void finish_prng (void) { finish_mum512_prng (); }\n#elif defined(XOROSHIRO128P)\n#include \"xoroshiro128plus.c\"\n#define N2 30000\nstatic void init_prng (void) { s[0] = 0xe220a8397b1dcdaf; s[1] = 0x6e789e6aa1b965f4; }\nstatic uint64_t get_prn (void) { return next (); }\nstatic void finish_prng (void) { }\n#elif defined(XOROSHIRO128STARSTAR)\n#include \"xoroshiro128starstar.c\"\n#define N2 30000\nstatic void init_prng (void) { s[0] = 0xe220a8397b1dcdaf; s[1] = 0x6e789e6aa1b965f4; }\nstatic uint64_t get_prn (void) { return next (); }\nstatic void finish_prng (void) { }\n#elif defined(XOSHIRO256P)\n#include \"xoshiro256plus.c\"\n#define N2 30000\nstatic void init_prng (void) { s[0] = 0xe220a8397b1dcdaf; s[1] = 0x6e789e6aa1b965f4; s[2] = 0x6c45d188009454f; s[3] = 0xf88bb8a8724c81ec; }\nstatic uint64_t get_prn (void) { return next (); }\nstatic void finish_prng (void) { }\n#elif defined(XOSHIRO256STARSTAR)\n#include \"xoshiro256starstar.c\"\n#define N2 30000\nstatic void init_prng (void) { s[0] = 0xe220a8397b1dcdaf; s[1] = 0x6e789e6aa1b965f4; s[2] = 0x6c45d188009454f; s[3] = 0xf88bb8a8724c81ec; }\nstatic uint64_t get_prn (void) { return next (); }\nstatic void finish_prng (void) { }\n#elif defined(XOSHIRO512P)\n#include \"xoshiro512plus.c\"\n#define N2 30000\nstatic void init_prng (void) { s[0] = 0xe220a8397b1dcdaf; s[1] = 0x6e789e6aa1b965f4; s[2] = 0x6c45d188009454f; s[3] = 0xf88bb8a8724c81ec; s[4] = 0x1b39896a51a8749b; s[5] = 0x53cb9f0c747ea2ea; s[6] = 0x2c829abe1f4532e1; s[7] = 0xc584133ac916ab3c; }\nstatic uint64_t get_prn (void) { return next (); }\nstatic void finish_prng (void) { }\n#elif defined(XOSHIRO512STARSTAR)\n#include \"xoshiro512starstar.c\"\n#define N2 30000\nstatic void init_prng (void) { s[0] = 0xe220a8397b1dcdaf; s[1] = 0x6e789e6aa1b965f4; s[2] = 0x6c45d188009454f; s[3] = 0xf88bb8a8724c81ec; s[4] = 0x1b39896a51a8749b; s[5] = 0x53cb9f0c747ea2ea; s[6] = 0x2c829abe1f4532e1; s[7] = 0xc584133ac916ab3c; }\nstatic uint64_t get_prn (void) { return next (); }\nstatic void finish_prng (void) { }\n#elif defined(RAND)\n#include \n#include \n#define N2 7000\nstatic void init_prng (void) { }\nstatic uint64_t get_prn (void) { return rand (); }\nstatic void finish_prng (void) { }\n#endif\n\nuint64_t dummy;\n#include \n#include \n\n#ifdef OUTPUT\n#include \n\nint main(void)\n{\n init_prng ();\n while (1) {\n uint64_t value = get_prn();\n fwrite((void*) &value, sizeof(value), 1, stdout);\n }\n}\n\n#else\nint main (void) {\n int i, j; double d; uint64_t res = 0;\n clock_t t = clock ();\n \n init_prng ();\n for (i = 0; i < N2; i++)\n for (j = 0; j < N1; j++)\n res ^= get_prn ();\n finish_prng ();\n t = clock () - t;\n d = (N1 + 0.0) * N2 * CLOCKS_PER_SEC / t / 1000;\n if (d > 1000)\n printf (\"%7.2fM prns/sec\\n\", d / 1000);\n else\n printf (\"%7.2fK prns/sec\\n\", d);\n dummy = res;\n return 0;\n}\n#endif\n" }, { "path": "benchmarks/bench-prng.sh", "content": "#!/bin/bash\n\n# Benchmarking different Pseudo Random Generators\n\necho +++pseudo random number generation speed '(PRNs/sec)':\nif test x${MUM_ONLY} == x; then\n gcc -DBBS -O3 -w bench-prng.c -lgmp && echo -n 'BBS : ' && ./a.out 2>&1\n gcc -DCHACHA -O3 -w bench-prng.c && echo -n 'ChaCha : ' && ./a.out 2>&1\n gcc -DSIP24 -O3 -w bench-prng.c && echo -n 'Sip24 : ' && ./a.out 2>&1\nfi\ngcc -DMUM512 -DMUM512_ROUNDS=2 -I../ -O3 -w bench-prng.c && echo -n 'MUM512 : ' && ./a.out 2>&1\ngcc -DMUM -I../ -O3 -w bench-prng.c && echo -n 'MUM : ' && ./a.out 2>&1\nif test x${MUM_ONLY} == x; then\n gcc -DXOROSHIRO128STARSTAR -I../ -std=c99 -O3 -w bench-prng.c && echo -n 'XOROSHIRO128**: ' && ./a.out 2>&1\n gcc -DXOSHIRO256STARSTAR -I../ -std=c99 -O3 -w bench-prng.c && echo -n 'XOSHIRO256** : ' && ./a.out 2>&1\n gcc -DXOSHIRO512STARSTAR -I../ -std=c99 -O3 -w bench-prng.c && echo -n 'XOSHIRO512** : ' && ./a.out 2>&1\n gcc -DRAND -I../ -O3 -w bench-prng.c && echo -n 'RAND : ' && ./a.out 2>&1\n gcc -DXOROSHIRO128P -I../ -std=c99 -O3 -w bench-prng.c && echo -n 'XOROSHIRO128+ : ' && ./a.out 2>&1\n gcc -DXOSHIRO256P -I../ -std=c99 -O3 -w bench-prng.c && echo -n 'XOSHIRO256+ : ' && ./a.out 2>&1\n gcc -DXOSHIRO512P -I../ -std=c99 -O3 -w bench-prng.c && echo -n 'XOSHIRO512+ : ' && ./a.out 2>&1\nfi\n\nrm -rf ./a.out\n" }, { "path": "benchmarks/bench.c", "content": "#if defined(Spooky)\n\n#include \"SpookyV2.h\"\nstatic void SpookyHash64_test (const void *key, int len, uint32_t seed, void *out) {\n *(uint64_t *) out = SpookyHash::Hash64 (key, len, seed);\n}\n\n#define test SpookyHash64_test\n#define test64 test\n\n#elif defined(City)\n\n#include \"City.h\"\nstatic void CityHash64_test (const void *key, int len, uint32_t seed, void *out) {\n *(uint64 *) out = CityHash64WithSeed ((const char *) key, len, seed);\n}\n\n#define test CityHash64_test\n#define test64 test\n\n#elif defined(SipHash)\n\n#include \n\nextern int siphash (uint8_t *out, const uint8_t *in, uint64_t inlen, const uint8_t *k);\n\nstatic void siphash_test (const void *key, int len, uint32_t seed, void *out) {\n uint64_t s[2];\n\n s[0] = seed;\n s[1] = 0;\n siphash (out, (const uint8_t *) key, len, (const uint8_t *) s);\n}\n\n#define test siphash_test\n#define test64 test\n\n#elif defined(xxHash)\n\n#ifdef _MSC_VER\ntypedef unsigned __int32 uint32_t;\ntypedef unsigned __int64 uint64_t;\n#else\n#include \n#endif\n\n#include \"xxhash.c\"\n\nstatic void xxHash64_test (const void *key, int len, uint32_t seed, void *out) {\n *(uint64_t *) out = XXH64 (key, len, seed);\n}\n\n#define test xxHash64_test\n#define test64 test\n\n#elif defined(xxh3)\n\n#include \"xxh3.h\"\n\nstatic void xxh3_test (const void *key, int len, uint32_t seed, void *out) {\n *(uint64_t *) out = XXH3_64bits (key, len);\n}\n\n#define test xxh3_test\n#define test64 test\n\n#elif defined(T1HA2)\n\n#include \"t1ha.h\"\nstatic void t1ha_test (const void *key, int len, uint32_t seed, void *out) {\n *(uint64_t *) out = t1ha2_atonce (key, len, seed);\n}\n\n#define test t1ha_test\n#define test64 test\n\n#elif defined(City)\n\n#include \"City.h\"\nstatic void CityHash64_test (const void *key, int len, uint32_t seed, void *out) {\n *(uint64 *) out = CityHash64WithSeed ((const char *) key, len, seed);\n}\n\n#define test CityHash64_test\n#define test64 test\n\n#elif defined(METRO)\n\n#include \"metrohash64.h\"\nstatic void metro_test (const void *key, int len, uint32_t seed, void *out) {\n MetroHash64::Hash ((const uint8_t *) key, len, (uint8_t *) out, seed);\n}\n\n#define test metro_test\n#define test64 test\n\n#elif defined(MeowHash)\n\n#ifdef _MSC_VER\ntypedef unsigned __int32 uint32_t;\ntypedef unsigned __int64 uint64_t;\n#else\n#include \n#endif\n\n#include \"meow_intrinsics.h\"\n#include \"meow_hash.h\"\n\nstatic void meowhash_test (const void *key, int len, uint32_t seed, void *out) {\n *(uint64_t *) out = MeowU64From (MeowHash_Accelerated (seed, len, key), 0);\n}\n\n#define test meowhash_test\n#define test64 test\n\n#elif defined(MUM)\n\n#include \"mum.h\"\nstatic void mum_test (const void *key, int len, uint32_t seed, void *out) {\n *(uint64_t *) out = mum_hash (key, len, seed);\n}\n\nstatic void mum_test64 (const void *key, int len, uint32_t seed, void *out) {\n *(uint64_t *) out = mum_hash64 (*(uint64_t *) key, seed);\n}\n\n#define test mum_test\n#define test64 mum_test64\n\n#elif defined(VMUM)\n\n#include \"vmum.h\"\nstatic void mum_test (const void *key, int len, uint32_t seed, void *out) {\n *(uint64_t *) out = vmum_hash (key, len, seed);\n}\n\nstatic void mum_test64 (const void *key, int len, uint32_t seed, void *out) {\n *(uint64_t *) out = vmum_hash64 (*(uint64_t *) key, seed);\n}\n\n#define test mum_test\n#define test64 mum_test64\n\n#elif defined(RAPID)\n\n#include \"rapidhash.h\"\nstatic void rapid_test (const void *key, int len, uint32_t seed, void *out) {\n *(uint64_t *) out = rapidhash_withSeed (key, len, seed);\n}\n\n#define test rapid_test\n#define test64 rapid_test\n\n#else\n#error \"I don't know what to test\"\n#endif\n\n#if DATA_LEN == 0\n\n#include \n#include \nuint32_t arr[16 * 256 * 1024];\nint main () {\n int i;\n uint64_t out;\n\n for (i = 0; i < 16 * 256 * 1024; i++) {\n arr[i] = rand ();\n }\n for (i = 0; i < 10000; i++) test (arr, 16 * 256 * 1024 * 4, 2, &out), arr[0] = out;\n printf (\"%s:%llx\\n\", (size_t) arr & 0x7 ? \"unaligned\" : \"aligned\", out);\n return 0;\n}\n\n#else\n\nint len = DATA_LEN;\nuint64_t k[(DATA_LEN + 7) / 8];\n#include \n#include \n#include \n/* We should use external to prevent optimizations for MUM after\n inlining. Otherwise MUM results will be too good. */\nint main () {\n int i, j, n;\n uint64_t out;\n\n assert (len <= 1024);\n printf (\"%d-byte: %s:\\n\", len, (size_t) k & 0x7 ? \"unaligned\" : \"aligned\");\n for (i = 0; i < sizeof (k) / sizeof (uint64_t); i++) k[i] = i;\n for (j = 0; j < 128; j++)\n for (n = i = 0; i < 10000000; i++) test (k, len, 2, &out), k[0] = out;\n printf (\"%llx\\n\", out);\n return 0;\n}\n\n#endif\n" }, { "path": "benchmarks/bench.sh", "content": "#!/bin/bash\n\n# Benchmarking different hash functions.\n\ntemp=__hash-temp.out\ntemp2=__hash-temp2.out\ntemp3=__hash-temp3.out\n\nCOPTFLAGS=${COPTFLAGS:--O3}\nif test `uname -m` == x86_64; then\n COPTFLAGS=`echo $COPTFLAGS -march=native`\nelif test `uname -m` == ppc64; then\n COPTFLAGS=`echo $COPTFLAGS -mcpu=native`\nfi\nLTO=${LTO:--flto}\nCC=${CC:-cc}\nCXX=${CXX:-c++}\n\necho Using ${CC} and ${CXX}\n\nif test x${MUM_ONLY} == x; then\n echo compiling Spooky\n ${CXX} ${COPTFLAGS} ${LTO} -w -c SpookyV2.cpp || exit 1\n echo compiling City\n ${CXX} ${COPTFLAGS} ${LTO} -w -c City.cpp || exit 1\n echo compiling t1ha\n ${CC} ${COPTFLAGS} ${LTO} -w -c t1ha/src/t1ha*.c || exit 1\n echo compiling metrohash64\n ${CXX} ${COPTFLAGS} ${LTO} -w -c metrohash64.cpp || exit 1\n echo compiling SipHash24\n ${CC} ${COPTFLAGS} ${LTO} -w -c siphash24.c || exit 1\nfi\n\nrm -f $temp3\n\npercent () {\n val=`awk \"BEGIN {if ($2==0) print \\\"Inf\\\"; else printf \\\"%.2f\\n\\\", $1/$2;}\"`\n echo \"$val\"\n echo \"$3:$val\" >>$temp3\n}\n\nskip () {\n l=$1\n n=$2\n while test $l -le $n; do echo -n \" \"; l=`expr $l + 1`; done\n}\n\nprint_time() {\n title=\"$1\"\n secs=$2\n printf '%-.2f %5.2fs|' `percent $base_time $secs \"$title\"` $secs\n}\n\nTASKSET=\"\"\nif type taskset >/dev/null 2>&1;then TASKSET=\"taskset -c 0\";fi\necho $TASKSET\n\nrun () {\n title=$1\n program=$2\n flag=$3\n ok=\n if (time -p $TASKSET $program) >$temp 2>$temp2; then\n ok=y\n (time -p $TASKSET $program) >$temp 2>>$temp2\n (time -p $TASKSET $program) >$temp 2>>$temp2\n fi\n if test x$ok = x;then echo $program: FAILED; return 1; fi\n secs=`grep -E 'user[ \t]*[0-9]' $temp2 | grep -F -v : | sed s/.*user// | sed s/\\\\t// | sort -n | head -1`\n if test x$flag != x;then base_time=$secs;fi\n print_time \"$title\" $secs\n}\n\nmach=`uname -m`\ncheck_meow=`(test $mach == x86_64 || test $mach == aarch64) && echo yes`\n\ncheck_meow=\ncheck_xxHash=\n\necho -n '| Length | VMUM-V2 | VMUM-V1 | MUM-V4 | MUM-V3 | Spooky | City |'\nif test \"$check_xxHash\" == yes; then echo -n ' xxHash |';fi\nif test \"$check_rapid\" == yes; then echo -n ' Rapidh |';fi\necho -n ' xxHash3 | t1ha2 | SipHash24 | Metro |'\nif test \"$check_meow\" == yes; then echo ' MeowHash |'; else echo; fi\necho -n '|:----------|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|'\nif test \"$check_xxHash\" == yes; then echo -n ':---------:|';fi\nif test \"$check_rapid\" == yes; then echo -n ':---------:|';fi\necho -n ':---------:|:---------:|:---------:|:---------:|'\nif test \"$check_meow\" == yes; then echo ':---------:|'; else echo; fi\n\nfor i in 3 4 5 6 7 8 9 10 11 12 13 14 15 16 32 64 96 128 192 256 512 1024 0;do\n if test $i == 0; then echo -n '| Bulk |'; else printf '|%4d bytes |' $i;fi\n ${CXX} -DDATA_LEN=$i ${COPTFLAGS} -w -fpermissive -DVMUM -I../ bench.c && run \"00vMUM-V2\" \"./a.out\" first\n ${CXX} -DDATA_LEN=$i ${COPTFLAGS} -w -fpermissive -DVMUM -DVMUM_V1 -I../ bench.c && run \"01vMUM-V1\" \"./a.out\"\n ${CXX} -DDATA_LEN=$i ${COPTFLAGS} -w -fpermissive -DMUM -I../ bench.c && run \"02MUM-V4\" \"./a.out\"\n ${CXX} -DDATA_LEN=$i ${COPTFLAGS} -w -fpermissive -DMUM -DMUM_V3 -I../ bench.c && run \"03MUM-V3\" \"./a.out\"\n if test x${MUM_ONLY} == x; then\n\t${CXX} -DDATA_LEN=$i ${COPTFLAGS} ${LTO} -w -fpermissive -DSpooky SpookyV2.o bench.c && run \"04Spooky\" \"./a.out\"\n\t${CXX} -DDATA_LEN=$i ${COPTFLAGS} ${LTO} -w -fpermissive -DCity City.o bench.c && run \"05City\" \"./a.out\"\n\tif test \"$check_xxHash\" == yes;then\n\t ${CXX} -DDATA_LEN=$i ${COPTFLAGS} ${LTO} -w -fpermissive -DxxHash bench.c && run \"06xxHash\" \"./a.out\"\n\tfi\n\tif test \"$check_rapid\" == yes;then\n ${CXX} -DDATA_LEN=$i ${COPTFLAGS} -w -fpermissive -DRAPID -I../ bench.c && run \"07vRAPID\" \"./a.out\"\n\tfi\n\t${CXX} -DDATA_LEN=$i ${COPTFLAGS} ${LTO} -w -fpermissive -Dxxh3 bench.c && run \"08xxh3\" \"./a.out\"\n\t${CC} -DDATA_LEN=$i ${COPTFLAGS} ${LTO} -w -fpermissive -It1ha -DT1HA2 t1ha*.o bench.c && run \"09t1ha2\" \"./a.out\"\n\t${CC} -DDATA_LEN=$i ${COPTFLAGS} ${LTO} -w -fpermissive -DSipHash siphash24.o bench.c && run \"10Siphash24\" \"./a.out\"\n\t${CXX} -DDATA_LEN=$i ${COPTFLAGS} ${LTO} -w -fpermissive -DMETRO metrohash64.o -I../ bench.c && run \"11Metro\" \"./a.out\"\n\tif test \"$check_meow\" == yes && test $mach == x86_64; then\n\t ${CXX} -DDATA_LEN=$i ${COPTFLAGS} -w -mavx2 -maes -fpermissive -DMeowHash -I../ bench.c && run \"12Meowhash\" \"./a.out\"\n\telif test \"$check_meow\" == yes && test $mach == aarch64; then\n\t ${CXX} -DDATA_LEN=$i ${COPTFLAGS} -w -march=native -fpermissive -DMeowHash -I../ bench.c && run \"13Meowhash\" \"./a.out\"\n\tfi\n fi\n echo\ndone\n\necho -n '| Average |'\nfor i in `awk -F: '{print $1}' $temp3|sort|uniq`; do\n printf '%-10.2f |' `awk -F: -v name=\"$i\" 'name==$1 {f = f + $2; n++} END {printf \"%0.2f\\n\", f / n}' $temp3`\ndone\necho\n\necho -n '| Geomean |'\nfor i in `awk -F: '{print $1}' $temp3|sort|uniq`; do\n printf '%-10.2f |' `awk -F: -v name=\"$i\" 'BEGIN{f=1.0} name==$1 {f = f * $2; n++} END {printf \"%0.2f\\n\", exp (log(f)/n)}' $temp3`\ndone\necho\n\nrm -rf ./a.out $temp $temp2 $temp3 SpookyV2.o City.o siphash24.o t1ha*.o\n" }, { "path": "benchmarks/blake2-config.h", "content": "/*\n BLAKE2 reference source code package - optimized C implementations\n\n Copyright 2012, Samuel Neves . You may use this under the\n terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at\n your option. The terms of these licenses can be found at:\n\n - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0\n - OpenSSL license : https://www.openssl.org/source/license.html\n - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0\n\n More information about the BLAKE2 hash function can be found at\n https://blake2.net.\n*/\n#pragma once\n#ifndef __BLAKE2_CONFIG_H__\n#define __BLAKE2_CONFIG_H__\n\n/* These don't work everywhere */\n#if defined(__SSE2__) || defined(__x86_64__) || defined(__amd64__)\n#define HAVE_SSE2\n#endif\n\n#if defined(__SSSE3__)\n#define HAVE_SSSE3\n#endif\n\n#if defined(__SSE4_1__)\n#define HAVE_SSE41\n#endif\n\n#if defined(__AVX__)\n#define HAVE_AVX\n#endif\n\n#if defined(__XOP__)\n#define HAVE_XOP\n#endif\n\n\n#ifdef HAVE_AVX2\n#ifndef HAVE_AVX\n#define HAVE_AVX\n#endif\n#endif\n\n#ifdef HAVE_XOP\n#ifndef HAVE_AVX\n#define HAVE_AVX\n#endif\n#endif\n\n#ifdef HAVE_AVX\n#ifndef HAVE_SSE41\n#define HAVE_SSE41\n#endif\n#endif\n\n#ifdef HAVE_SSE41\n#ifndef HAVE_SSSE3\n#define HAVE_SSSE3\n#endif\n#endif\n\n#ifdef HAVE_SSSE3\n#define HAVE_SSE2\n#endif\n\n#if !defined(HAVE_SSE2)\n#error \"This code requires at least SSE2.\"\n#endif\n\n#endif\n\n" }, { "path": "benchmarks/blake2-impl.h", "content": "/*\n BLAKE2 reference source code package - optimized C implementations\n \n Copyright 2012, Samuel Neves . You may use this under the\n terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at\n your option. The terms of these licenses can be found at:\n \n - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0\n - OpenSSL license : https://www.openssl.org/source/license.html\n - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0\n \n More information about the BLAKE2 hash function can be found at\n https://blake2.net.\n*/\n#pragma once\n#ifndef __BLAKE2_IMPL_H__\n#define __BLAKE2_IMPL_H__\n\n#include \n#include \n\nBLAKE2_LOCAL_INLINE(uint32_t) load32( const void *src )\n{\n#if defined(NATIVE_LITTLE_ENDIAN)\n uint32_t w;\n memcpy(&w, src, sizeof w);\n return w;\n#else\n const uint8_t *p = ( const uint8_t * )src;\n uint32_t w = *p++;\n w |= ( uint32_t )( *p++ ) << 8;\n w |= ( uint32_t )( *p++ ) << 16;\n w |= ( uint32_t )( *p++ ) << 24;\n return w;\n#endif\n}\n\nBLAKE2_LOCAL_INLINE(uint64_t) load64( const void *src )\n{\n#if defined(NATIVE_LITTLE_ENDIAN)\n uint64_t w;\n memcpy(&w, src, sizeof w);\n return w;\n#else\n const uint8_t *p = ( const uint8_t * )src;\n uint64_t w = *p++;\n w |= ( uint64_t )( *p++ ) << 8;\n w |= ( uint64_t )( *p++ ) << 16;\n w |= ( uint64_t )( *p++ ) << 24;\n w |= ( uint64_t )( *p++ ) << 32;\n w |= ( uint64_t )( *p++ ) << 40;\n w |= ( uint64_t )( *p++ ) << 48;\n w |= ( uint64_t )( *p++ ) << 56;\n return w;\n#endif\n}\n\nBLAKE2_LOCAL_INLINE(void) store32( void *dst, uint32_t w )\n{\n#if defined(NATIVE_LITTLE_ENDIAN)\n memcpy(dst, &w, sizeof w);\n#else\n uint8_t *p = ( uint8_t * )dst;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w;\n#endif\n}\n\nBLAKE2_LOCAL_INLINE(void) store64( void *dst, uint64_t w )\n{\n#if defined(NATIVE_LITTLE_ENDIAN)\n memcpy(dst, &w, sizeof w);\n#else\n uint8_t *p = ( uint8_t * )dst;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w;\n#endif\n}\n\nBLAKE2_LOCAL_INLINE(uint64_t) load48( const void *src )\n{\n const uint8_t *p = ( const uint8_t * )src;\n uint64_t w = *p++;\n w |= ( uint64_t )( *p++ ) << 8;\n w |= ( uint64_t )( *p++ ) << 16;\n w |= ( uint64_t )( *p++ ) << 24;\n w |= ( uint64_t )( *p++ ) << 32;\n w |= ( uint64_t )( *p++ ) << 40;\n return w;\n}\n\nBLAKE2_LOCAL_INLINE(void) store48( void *dst, uint64_t w )\n{\n uint8_t *p = ( uint8_t * )dst;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w; w >>= 8;\n *p++ = ( uint8_t )w;\n}\n\nBLAKE2_LOCAL_INLINE(uint32_t) rotl32( const uint32_t w, const unsigned c )\n{\n return ( w << c ) | ( w >> ( 32 - c ) );\n}\n\nBLAKE2_LOCAL_INLINE(uint64_t) rotl64( const uint64_t w, const unsigned c )\n{\n return ( w << c ) | ( w >> ( 64 - c ) );\n}\n\nBLAKE2_LOCAL_INLINE(uint32_t) rotr32( const uint32_t w, const unsigned c )\n{\n return ( w >> c ) | ( w << ( 32 - c ) );\n}\n\nBLAKE2_LOCAL_INLINE(uint64_t) rotr64( const uint64_t w, const unsigned c )\n{\n return ( w >> c ) | ( w << ( 64 - c ) );\n}\n\n/* prevents compiler optimizing out memset() */\nBLAKE2_LOCAL_INLINE(void) secure_zero_memory(void *v, size_t n)\n{\n static void *(*const volatile memset_v)(void *, int, size_t) = &memset;\n memset_v(v, 0, n);\n}\n\n#endif\n\n" }, { "path": "benchmarks/blake2.h", "content": "/*\n BLAKE2 reference source code package - reference C implementations\n \n Copyright 2012, Samuel Neves . You may use this under the\n terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at\n your option. The terms of these licenses can be found at:\n \n - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0\n - OpenSSL license : https://www.openssl.org/source/license.html\n - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0\n \n More information about the BLAKE2 hash function can be found at\n https://blake2.net.\n*/\n#pragma once\n#ifndef __BLAKE2_H__\n#define __BLAKE2_H__\n\n#include \n#include \n\n#ifdef BLAKE2_NO_INLINE\n#define BLAKE2_LOCAL_INLINE(type) static type\n#endif\n\n#ifndef BLAKE2_LOCAL_INLINE\n#define BLAKE2_LOCAL_INLINE(type) static inline type\n#endif\n\n#if defined(__cplusplus)\nextern \"C\" {\n#endif\n\n enum blake2s_constant\n {\n BLAKE2S_BLOCKBYTES = 64,\n BLAKE2S_OUTBYTES = 32,\n BLAKE2S_KEYBYTES = 32,\n BLAKE2S_SALTBYTES = 8,\n BLAKE2S_PERSONALBYTES = 8\n };\n\n enum blake2b_constant\n {\n BLAKE2B_BLOCKBYTES = 128,\n BLAKE2B_OUTBYTES = 64,\n BLAKE2B_KEYBYTES = 64,\n BLAKE2B_SALTBYTES = 16,\n BLAKE2B_PERSONALBYTES = 16\n };\n\n typedef struct __blake2s_state\n {\n uint32_t h[8];\n uint32_t t[2];\n uint32_t f[2];\n uint8_t buf[2 * BLAKE2S_BLOCKBYTES];\n size_t buflen;\n uint8_t last_node;\n } blake2s_state;\n\n typedef struct __blake2b_state\n {\n uint64_t h[8];\n uint64_t t[2];\n uint64_t f[2];\n uint8_t buf[2 * BLAKE2B_BLOCKBYTES];\n size_t buflen;\n uint8_t last_node;\n } blake2b_state;\n\n typedef struct __blake2sp_state\n {\n blake2s_state S[8][1];\n blake2s_state R[1];\n uint8_t buf[8 * BLAKE2S_BLOCKBYTES];\n size_t buflen;\n } blake2sp_state;\n\n typedef struct __blake2bp_state\n {\n blake2b_state S[4][1];\n blake2b_state R[1];\n uint8_t buf[4 * BLAKE2B_BLOCKBYTES];\n size_t buflen;\n } blake2bp_state;\n\n\n#pragma pack(push, 1)\n typedef struct __blake2s_param\n {\n uint8_t digest_length; /* 1 */\n uint8_t key_length; /* 2 */\n uint8_t fanout; /* 3 */\n uint8_t depth; /* 4 */\n uint32_t leaf_length; /* 8 */\n uint8_t node_offset[6];// 14\n uint8_t node_depth; /* 15 */\n uint8_t inner_length; /* 16 */\n /* uint8_t reserved[0]; */\n uint8_t salt[BLAKE2S_SALTBYTES]; /* 24 */\n uint8_t personal[BLAKE2S_PERSONALBYTES]; /* 32 */\n } blake2s_param;\n\n typedef struct __blake2b_param\n {\n uint8_t digest_length; /* 1 */\n uint8_t key_length; /* 2 */\n uint8_t fanout; /* 3 */\n uint8_t depth; /* 4 */\n uint32_t leaf_length; /* 8 */\n uint64_t node_offset; /* 16 */\n uint8_t node_depth; /* 17 */\n uint8_t inner_length; /* 18 */\n uint8_t reserved[14]; /* 32 */\n uint8_t salt[BLAKE2B_SALTBYTES]; /* 48 */\n uint8_t personal[BLAKE2B_PERSONALBYTES]; /* 64 */\n } blake2b_param;\n#pragma pack(pop)\n\n /* Streaming API */\n int blake2s_init( blake2s_state *S, const uint8_t outlen );\n int blake2s_init_key( blake2s_state *S, const uint8_t outlen, const void *key, const uint8_t keylen );\n int blake2s_init_param( blake2s_state *S, const blake2s_param *P );\n int blake2s_update( blake2s_state *S, const uint8_t *in, uint64_t inlen );\n int blake2s_final( blake2s_state *S, uint8_t *out, uint8_t outlen );\n\n int blake2b_init( blake2b_state *S, const uint8_t outlen );\n int blake2b_init_key( blake2b_state *S, const uint8_t outlen, const void *key, const uint8_t keylen );\n int blake2b_init_param( blake2b_state *S, const blake2b_param *P );\n int blake2b_update( blake2b_state *S, const uint8_t *in, uint64_t inlen );\n int blake2b_final( blake2b_state *S, uint8_t *out, uint8_t outlen );\n\n int blake2sp_init( blake2sp_state *S, const uint8_t outlen );\n int blake2sp_init_key( blake2sp_state *S, const uint8_t outlen, const void *key, const uint8_t keylen );\n int blake2sp_update( blake2sp_state *S, const uint8_t *in, uint64_t inlen );\n int blake2sp_final( blake2sp_state *S, uint8_t *out, uint8_t outlen );\n\n int blake2bp_init( blake2bp_state *S, const uint8_t outlen );\n int blake2bp_init_key( blake2bp_state *S, const uint8_t outlen, const void *key, const uint8_t keylen );\n int blake2bp_update( blake2bp_state *S, const uint8_t *in, uint64_t inlen );\n int blake2bp_final( blake2bp_state *S, uint8_t *out, uint8_t outlen );\n\n /* Simple API */\n int blake2s( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen );\n int blake2b( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen );\n\n int blake2sp( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen );\n int blake2bp( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen );\n\n static inline int blake2( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen )\n {\n return blake2b( out, in, key, outlen, inlen, keylen );\n }\n\n#if defined(__cplusplus)\n}\n#endif\n\n#endif\n\n" }, { "path": "benchmarks/blake2b-load-sse2.h", "content": "/*\n BLAKE2 reference source code package - optimized C implementations\n \n Copyright 2012, Samuel Neves . You may use this under the\n terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at\n your option. The terms of these licenses can be found at:\n \n - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0\n - OpenSSL license : https://www.openssl.org/source/license.html\n - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0\n \n More information about the BLAKE2 hash function can be found at\n https://blake2.net.\n*/\n#pragma once\n#ifndef __BLAKE2B_LOAD_SSE2_H__\n#define __BLAKE2B_LOAD_SSE2_H__\n\n#define LOAD_MSG_0_1(b0, b1) b0 = _mm_set_epi64x(m2, m0); b1 = _mm_set_epi64x(m6, m4)\n#define LOAD_MSG_0_2(b0, b1) b0 = _mm_set_epi64x(m3, m1); b1 = _mm_set_epi64x(m7, m5)\n#define LOAD_MSG_0_3(b0, b1) b0 = _mm_set_epi64x(m10, m8); b1 = _mm_set_epi64x(m14, m12)\n#define LOAD_MSG_0_4(b0, b1) b0 = _mm_set_epi64x(m11, m9); b1 = _mm_set_epi64x(m15, m13)\n#define LOAD_MSG_1_1(b0, b1) b0 = _mm_set_epi64x(m4, m14); b1 = _mm_set_epi64x(m13, m9)\n#define LOAD_MSG_1_2(b0, b1) b0 = _mm_set_epi64x(m8, m10); b1 = _mm_set_epi64x(m6, m15)\n#define LOAD_MSG_1_3(b0, b1) b0 = _mm_set_epi64x(m0, m1); b1 = _mm_set_epi64x(m5, m11)\n#define LOAD_MSG_1_4(b0, b1) b0 = _mm_set_epi64x(m2, m12); b1 = _mm_set_epi64x(m3, m7)\n#define LOAD_MSG_2_1(b0, b1) b0 = _mm_set_epi64x(m12, m11); b1 = _mm_set_epi64x(m15, m5)\n#define LOAD_MSG_2_2(b0, b1) b0 = _mm_set_epi64x(m0, m8); b1 = _mm_set_epi64x(m13, m2)\n#define LOAD_MSG_2_3(b0, b1) b0 = _mm_set_epi64x(m3, m10); b1 = _mm_set_epi64x(m9, m7)\n#define LOAD_MSG_2_4(b0, b1) b0 = _mm_set_epi64x(m6, m14); b1 = _mm_set_epi64x(m4, m1)\n#define LOAD_MSG_3_1(b0, b1) b0 = _mm_set_epi64x(m3, m7); b1 = _mm_set_epi64x(m11, m13)\n#define LOAD_MSG_3_2(b0, b1) b0 = _mm_set_epi64x(m1, m9); b1 = _mm_set_epi64x(m14, m12)\n#define LOAD_MSG_3_3(b0, b1) b0 = _mm_set_epi64x(m5, m2); b1 = _mm_set_epi64x(m15, m4)\n#define LOAD_MSG_3_4(b0, b1) b0 = _mm_set_epi64x(m10, m6); b1 = _mm_set_epi64x(m8, m0)\n#define LOAD_MSG_4_1(b0, b1) b0 = _mm_set_epi64x(m5, m9); b1 = _mm_set_epi64x(m10, m2)\n#define LOAD_MSG_4_2(b0, b1) b0 = _mm_set_epi64x(m7, m0); b1 = _mm_set_epi64x(m15, m4)\n#define LOAD_MSG_4_3(b0, b1) b0 = _mm_set_epi64x(m11, m14); b1 = _mm_set_epi64x(m3, m6)\n#define LOAD_MSG_4_4(b0, b1) b0 = _mm_set_epi64x(m12, m1); b1 = _mm_set_epi64x(m13, m8)\n#define LOAD_MSG_5_1(b0, b1) b0 = _mm_set_epi64x(m6, m2); b1 = _mm_set_epi64x(m8, m0)\n#define LOAD_MSG_5_2(b0, b1) b0 = _mm_set_epi64x(m10, m12); b1 = _mm_set_epi64x(m3, m11)\n#define LOAD_MSG_5_3(b0, b1) b0 = _mm_set_epi64x(m7, m4); b1 = _mm_set_epi64x(m1, m15)\n#define LOAD_MSG_5_4(b0, b1) b0 = _mm_set_epi64x(m5, m13); b1 = _mm_set_epi64x(m9, m14)\n#define LOAD_MSG_6_1(b0, b1) b0 = _mm_set_epi64x(m1, m12); b1 = _mm_set_epi64x(m4, m14)\n#define LOAD_MSG_6_2(b0, b1) b0 = _mm_set_epi64x(m15, m5); b1 = _mm_set_epi64x(m10, m13)\n#define LOAD_MSG_6_3(b0, b1) b0 = _mm_set_epi64x(m6, m0); b1 = _mm_set_epi64x(m8, m9)\n#define LOAD_MSG_6_4(b0, b1) b0 = _mm_set_epi64x(m3, m7); b1 = _mm_set_epi64x(m11, m2)\n#define LOAD_MSG_7_1(b0, b1) b0 = _mm_set_epi64x(m7, m13); b1 = _mm_set_epi64x(m3, m12)\n#define LOAD_MSG_7_2(b0, b1) b0 = _mm_set_epi64x(m14, m11); b1 = _mm_set_epi64x(m9, m1)\n#define LOAD_MSG_7_3(b0, b1) b0 = _mm_set_epi64x(m15, m5); b1 = _mm_set_epi64x(m2, m8)\n#define LOAD_MSG_7_4(b0, b1) b0 = _mm_set_epi64x(m4, m0); b1 = _mm_set_epi64x(m10, m6)\n#define LOAD_MSG_8_1(b0, b1) b0 = _mm_set_epi64x(m14, m6); b1 = _mm_set_epi64x(m0, m11)\n#define LOAD_MSG_8_2(b0, b1) b0 = _mm_set_epi64x(m9, m15); b1 = _mm_set_epi64x(m8, m3)\n#define LOAD_MSG_8_3(b0, b1) b0 = _mm_set_epi64x(m13, m12); b1 = _mm_set_epi64x(m10, m1)\n#define LOAD_MSG_8_4(b0, b1) b0 = _mm_set_epi64x(m7, m2); b1 = _mm_set_epi64x(m5, m4)\n#define LOAD_MSG_9_1(b0, b1) b0 = _mm_set_epi64x(m8, m10); b1 = _mm_set_epi64x(m1, m7)\n#define LOAD_MSG_9_2(b0, b1) b0 = _mm_set_epi64x(m4, m2); b1 = _mm_set_epi64x(m5, m6)\n#define LOAD_MSG_9_3(b0, b1) b0 = _mm_set_epi64x(m9, m15); b1 = _mm_set_epi64x(m13, m3)\n#define LOAD_MSG_9_4(b0, b1) b0 = _mm_set_epi64x(m14, m11); b1 = _mm_set_epi64x(m0, m12)\n#define LOAD_MSG_10_1(b0, b1) b0 = _mm_set_epi64x(m2, m0); b1 = _mm_set_epi64x(m6, m4)\n#define LOAD_MSG_10_2(b0, b1) b0 = _mm_set_epi64x(m3, m1); b1 = _mm_set_epi64x(m7, m5)\n#define LOAD_MSG_10_3(b0, b1) b0 = _mm_set_epi64x(m10, m8); b1 = _mm_set_epi64x(m14, m12)\n#define LOAD_MSG_10_4(b0, b1) b0 = _mm_set_epi64x(m11, m9); b1 = _mm_set_epi64x(m15, m13)\n#define LOAD_MSG_11_1(b0, b1) b0 = _mm_set_epi64x(m4, m14); b1 = _mm_set_epi64x(m13, m9)\n#define LOAD_MSG_11_2(b0, b1) b0 = _mm_set_epi64x(m8, m10); b1 = _mm_set_epi64x(m6, m15)\n#define LOAD_MSG_11_3(b0, b1) b0 = _mm_set_epi64x(m0, m1); b1 = _mm_set_epi64x(m5, m11)\n#define LOAD_MSG_11_4(b0, b1) b0 = _mm_set_epi64x(m2, m12); b1 = _mm_set_epi64x(m3, m7)\n\n\n#endif\n\n" }, { "path": "benchmarks/blake2b-load-sse41.h", "content": "/*\n BLAKE2 reference source code package - optimized C implementations\n \n Copyright 2012, Samuel Neves . You may use this under the\n terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at\n your option. The terms of these licenses can be found at:\n \n - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0\n - OpenSSL license : https://www.openssl.org/source/license.html\n - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0\n \n More information about the BLAKE2 hash function can be found at\n https://blake2.net.\n*/\n#pragma once\n#ifndef __BLAKE2B_LOAD_SSE41_H__\n#define __BLAKE2B_LOAD_SSE41_H__\n\n#define LOAD_MSG_0_1(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m0, m1); \\\nb1 = _mm_unpacklo_epi64(m2, m3); \\\n} while(0)\n\n\n#define LOAD_MSG_0_2(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m0, m1); \\\nb1 = _mm_unpackhi_epi64(m2, m3); \\\n} while(0)\n\n\n#define LOAD_MSG_0_3(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m4, m5); \\\nb1 = _mm_unpacklo_epi64(m6, m7); \\\n} while(0)\n\n\n#define LOAD_MSG_0_4(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m4, m5); \\\nb1 = _mm_unpackhi_epi64(m6, m7); \\\n} while(0)\n\n\n#define LOAD_MSG_1_1(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m7, m2); \\\nb1 = _mm_unpackhi_epi64(m4, m6); \\\n} while(0)\n\n\n#define LOAD_MSG_1_2(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m5, m4); \\\nb1 = _mm_alignr_epi8(m3, m7, 8); \\\n} while(0)\n\n\n#define LOAD_MSG_1_3(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_shuffle_epi32(m0, _MM_SHUFFLE(1,0,3,2)); \\\nb1 = _mm_unpackhi_epi64(m5, m2); \\\n} while(0)\n\n\n#define LOAD_MSG_1_4(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m6, m1); \\\nb1 = _mm_unpackhi_epi64(m3, m1); \\\n} while(0)\n\n\n#define LOAD_MSG_2_1(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_alignr_epi8(m6, m5, 8); \\\nb1 = _mm_unpackhi_epi64(m2, m7); \\\n} while(0)\n\n\n#define LOAD_MSG_2_2(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m4, m0); \\\nb1 = _mm_blend_epi16(m1, m6, 0xF0); \\\n} while(0)\n\n\n#define LOAD_MSG_2_3(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_blend_epi16(m5, m1, 0xF0); \\\nb1 = _mm_unpackhi_epi64(m3, m4); \\\n} while(0)\n\n\n#define LOAD_MSG_2_4(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m7, m3); \\\nb1 = _mm_alignr_epi8(m2, m0, 8); \\\n} while(0)\n\n\n#define LOAD_MSG_3_1(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m3, m1); \\\nb1 = _mm_unpackhi_epi64(m6, m5); \\\n} while(0)\n\n\n#define LOAD_MSG_3_2(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m4, m0); \\\nb1 = _mm_unpacklo_epi64(m6, m7); \\\n} while(0)\n\n\n#define LOAD_MSG_3_3(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_blend_epi16(m1, m2, 0xF0); \\\nb1 = _mm_blend_epi16(m2, m7, 0xF0); \\\n} while(0)\n\n\n#define LOAD_MSG_3_4(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m3, m5); \\\nb1 = _mm_unpacklo_epi64(m0, m4); \\\n} while(0)\n\n\n#define LOAD_MSG_4_1(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m4, m2); \\\nb1 = _mm_unpacklo_epi64(m1, m5); \\\n} while(0)\n\n\n#define LOAD_MSG_4_2(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_blend_epi16(m0, m3, 0xF0); \\\nb1 = _mm_blend_epi16(m2, m7, 0xF0); \\\n} while(0)\n\n\n#define LOAD_MSG_4_3(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_blend_epi16(m7, m5, 0xF0); \\\nb1 = _mm_blend_epi16(m3, m1, 0xF0); \\\n} while(0)\n\n\n#define LOAD_MSG_4_4(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_alignr_epi8(m6, m0, 8); \\\nb1 = _mm_blend_epi16(m4, m6, 0xF0); \\\n} while(0)\n\n\n#define LOAD_MSG_5_1(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m1, m3); \\\nb1 = _mm_unpacklo_epi64(m0, m4); \\\n} while(0)\n\n\n#define LOAD_MSG_5_2(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m6, m5); \\\nb1 = _mm_unpackhi_epi64(m5, m1); \\\n} while(0)\n\n\n#define LOAD_MSG_5_3(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_blend_epi16(m2, m3, 0xF0); \\\nb1 = _mm_unpackhi_epi64(m7, m0); \\\n} while(0)\n\n\n#define LOAD_MSG_5_4(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m6, m2); \\\nb1 = _mm_blend_epi16(m7, m4, 0xF0); \\\n} while(0)\n\n\n#define LOAD_MSG_6_1(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_blend_epi16(m6, m0, 0xF0); \\\nb1 = _mm_unpacklo_epi64(m7, m2); \\\n} while(0)\n\n\n#define LOAD_MSG_6_2(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m2, m7); \\\nb1 = _mm_alignr_epi8(m5, m6, 8); \\\n} while(0)\n\n\n#define LOAD_MSG_6_3(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m0, m3); \\\nb1 = _mm_shuffle_epi32(m4, _MM_SHUFFLE(1,0,3,2)); \\\n} while(0)\n\n\n#define LOAD_MSG_6_4(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m3, m1); \\\nb1 = _mm_blend_epi16(m1, m5, 0xF0); \\\n} while(0)\n\n\n#define LOAD_MSG_7_1(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m6, m3); \\\nb1 = _mm_blend_epi16(m6, m1, 0xF0); \\\n} while(0)\n\n\n#define LOAD_MSG_7_2(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_alignr_epi8(m7, m5, 8); \\\nb1 = _mm_unpackhi_epi64(m0, m4); \\\n} while(0)\n\n\n#define LOAD_MSG_7_3(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m2, m7); \\\nb1 = _mm_unpacklo_epi64(m4, m1); \\\n} while(0)\n\n\n#define LOAD_MSG_7_4(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m0, m2); \\\nb1 = _mm_unpacklo_epi64(m3, m5); \\\n} while(0)\n\n\n#define LOAD_MSG_8_1(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m3, m7); \\\nb1 = _mm_alignr_epi8(m0, m5, 8); \\\n} while(0)\n\n\n#define LOAD_MSG_8_2(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m7, m4); \\\nb1 = _mm_alignr_epi8(m4, m1, 8); \\\n} while(0)\n\n\n#define LOAD_MSG_8_3(b0, b1) \\\ndo \\\n{ \\\nb0 = m6; \\\nb1 = _mm_alignr_epi8(m5, m0, 8); \\\n} while(0)\n\n\n#define LOAD_MSG_8_4(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_blend_epi16(m1, m3, 0xF0); \\\nb1 = m2; \\\n} while(0)\n\n\n#define LOAD_MSG_9_1(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m5, m4); \\\nb1 = _mm_unpackhi_epi64(m3, m0); \\\n} while(0)\n\n\n#define LOAD_MSG_9_2(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m1, m2); \\\nb1 = _mm_blend_epi16(m3, m2, 0xF0); \\\n} while(0)\n\n\n#define LOAD_MSG_9_3(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m7, m4); \\\nb1 = _mm_unpackhi_epi64(m1, m6); \\\n} while(0)\n\n\n#define LOAD_MSG_9_4(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_alignr_epi8(m7, m5, 8); \\\nb1 = _mm_unpacklo_epi64(m6, m0); \\\n} while(0)\n\n\n#define LOAD_MSG_10_1(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m0, m1); \\\nb1 = _mm_unpacklo_epi64(m2, m3); \\\n} while(0)\n\n\n#define LOAD_MSG_10_2(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m0, m1); \\\nb1 = _mm_unpackhi_epi64(m2, m3); \\\n} while(0)\n\n\n#define LOAD_MSG_10_3(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m4, m5); \\\nb1 = _mm_unpacklo_epi64(m6, m7); \\\n} while(0)\n\n\n#define LOAD_MSG_10_4(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpackhi_epi64(m4, m5); \\\nb1 = _mm_unpackhi_epi64(m6, m7); \\\n} while(0)\n\n\n#define LOAD_MSG_11_1(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m7, m2); \\\nb1 = _mm_unpackhi_epi64(m4, m6); \\\n} while(0)\n\n\n#define LOAD_MSG_11_2(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m5, m4); \\\nb1 = _mm_alignr_epi8(m3, m7, 8); \\\n} while(0)\n\n\n#define LOAD_MSG_11_3(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_shuffle_epi32(m0, _MM_SHUFFLE(1,0,3,2)); \\\nb1 = _mm_unpackhi_epi64(m5, m2); \\\n} while(0)\n\n\n#define LOAD_MSG_11_4(b0, b1) \\\ndo \\\n{ \\\nb0 = _mm_unpacklo_epi64(m6, m1); \\\nb1 = _mm_unpackhi_epi64(m3, m1); \\\n} while(0)\n\n\n#endif\n\n" }, { "path": "benchmarks/blake2b-round.h", "content": "/*\n BLAKE2 reference source code package - optimized C implementations\n \n Copyright 2012, Samuel Neves . You may use this under the\n terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at\n your option. The terms of these licenses can be found at:\n \n - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0\n - OpenSSL license : https://www.openssl.org/source/license.html\n - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0\n \n More information about the BLAKE2 hash function can be found at\n https://blake2.net.\n*/\n#pragma once\n#ifndef __BLAKE2B_ROUND_H__\n#define __BLAKE2B_ROUND_H__\n\n#define LOADU(p) _mm_loadu_si128( (const __m128i *)(p) )\n#define STOREU(p,r) _mm_storeu_si128((__m128i *)(p), r)\n\n#define TOF(reg) _mm_castsi128_ps((reg))\n#define TOI(reg) _mm_castps_si128((reg))\n\n#define LIKELY(x) __builtin_expect((x),1)\n\n\n/* Microarchitecture-specific macros */\n#ifndef HAVE_XOP\n#ifdef HAVE_SSSE3\n#define _mm_roti_epi64(x, c) \\\n (-(c) == 32) ? _mm_shuffle_epi32((x), _MM_SHUFFLE(2,3,0,1)) \\\n : (-(c) == 24) ? _mm_shuffle_epi8((x), r24) \\\n : (-(c) == 16) ? _mm_shuffle_epi8((x), r16) \\\n : (-(c) == 63) ? _mm_xor_si128(_mm_srli_epi64((x), -(c)), _mm_add_epi64((x), (x))) \\\n : _mm_xor_si128(_mm_srli_epi64((x), -(c)), _mm_slli_epi64((x), 64-(-(c))))\n#else\n#define _mm_roti_epi64(r, c) _mm_xor_si128(_mm_srli_epi64( (r), -(c) ),_mm_slli_epi64( (r), 64-(-(c)) ))\n#endif\n#else\n/* ... */\n#endif\n\n\n\n#define G1(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h,b0,b1) \\\n row1l = _mm_add_epi64(_mm_add_epi64(row1l, b0), row2l); \\\n row1h = _mm_add_epi64(_mm_add_epi64(row1h, b1), row2h); \\\n \\\n row4l = _mm_xor_si128(row4l, row1l); \\\n row4h = _mm_xor_si128(row4h, row1h); \\\n \\\n row4l = _mm_roti_epi64(row4l, -32); \\\n row4h = _mm_roti_epi64(row4h, -32); \\\n \\\n row3l = _mm_add_epi64(row3l, row4l); \\\n row3h = _mm_add_epi64(row3h, row4h); \\\n \\\n row2l = _mm_xor_si128(row2l, row3l); \\\n row2h = _mm_xor_si128(row2h, row3h); \\\n \\\n row2l = _mm_roti_epi64(row2l, -24); \\\n row2h = _mm_roti_epi64(row2h, -24); \\\n \n#define G2(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h,b0,b1) \\\n row1l = _mm_add_epi64(_mm_add_epi64(row1l, b0), row2l); \\\n row1h = _mm_add_epi64(_mm_add_epi64(row1h, b1), row2h); \\\n \\\n row4l = _mm_xor_si128(row4l, row1l); \\\n row4h = _mm_xor_si128(row4h, row1h); \\\n \\\n row4l = _mm_roti_epi64(row4l, -16); \\\n row4h = _mm_roti_epi64(row4h, -16); \\\n \\\n row3l = _mm_add_epi64(row3l, row4l); \\\n row3h = _mm_add_epi64(row3h, row4h); \\\n \\\n row2l = _mm_xor_si128(row2l, row3l); \\\n row2h = _mm_xor_si128(row2h, row3h); \\\n \\\n row2l = _mm_roti_epi64(row2l, -63); \\\n row2h = _mm_roti_epi64(row2h, -63); \\\n \n#if defined(HAVE_SSSE3)\n#define DIAGONALIZE(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h) \\\n t0 = _mm_alignr_epi8(row2h, row2l, 8); \\\n t1 = _mm_alignr_epi8(row2l, row2h, 8); \\\n row2l = t0; \\\n row2h = t1; \\\n \\\n t0 = row3l; \\\n row3l = row3h; \\\n row3h = t0; \\\n \\\n t0 = _mm_alignr_epi8(row4h, row4l, 8); \\\n t1 = _mm_alignr_epi8(row4l, row4h, 8); \\\n row4l = t1; \\\n row4h = t0;\n\n#define UNDIAGONALIZE(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h) \\\n t0 = _mm_alignr_epi8(row2l, row2h, 8); \\\n t1 = _mm_alignr_epi8(row2h, row2l, 8); \\\n row2l = t0; \\\n row2h = t1; \\\n \\\n t0 = row3l; \\\n row3l = row3h; \\\n row3h = t0; \\\n \\\n t0 = _mm_alignr_epi8(row4l, row4h, 8); \\\n t1 = _mm_alignr_epi8(row4h, row4l, 8); \\\n row4l = t1; \\\n row4h = t0;\n#else\n\n#define DIAGONALIZE(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h) \\\n t0 = row4l;\\\n t1 = row2l;\\\n row4l = row3l;\\\n row3l = row3h;\\\n row3h = row4l;\\\n row4l = _mm_unpackhi_epi64(row4h, _mm_unpacklo_epi64(t0, t0)); \\\n row4h = _mm_unpackhi_epi64(t0, _mm_unpacklo_epi64(row4h, row4h)); \\\n row2l = _mm_unpackhi_epi64(row2l, _mm_unpacklo_epi64(row2h, row2h)); \\\n row2h = _mm_unpackhi_epi64(row2h, _mm_unpacklo_epi64(t1, t1))\n\n#define UNDIAGONALIZE(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h) \\\n t0 = row3l;\\\n row3l = row3h;\\\n row3h = t0;\\\n t0 = row2l;\\\n t1 = row4l;\\\n row2l = _mm_unpackhi_epi64(row2h, _mm_unpacklo_epi64(row2l, row2l)); \\\n row2h = _mm_unpackhi_epi64(t0, _mm_unpacklo_epi64(row2h, row2h)); \\\n row4l = _mm_unpackhi_epi64(row4l, _mm_unpacklo_epi64(row4h, row4h)); \\\n row4h = _mm_unpackhi_epi64(row4h, _mm_unpacklo_epi64(t1, t1))\n\n#endif\n\n#if defined(HAVE_SSE41)\n#include \"blake2b-load-sse41.h\"\n#else\n#include \"blake2b-load-sse2.h\"\n#endif\n\n#define ROUND(r) \\\n LOAD_MSG_ ##r ##_1(b0, b1); \\\n G1(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h,b0,b1); \\\n LOAD_MSG_ ##r ##_2(b0, b1); \\\n G2(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h,b0,b1); \\\n DIAGONALIZE(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h); \\\n LOAD_MSG_ ##r ##_3(b0, b1); \\\n G1(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h,b0,b1); \\\n LOAD_MSG_ ##r ##_4(b0, b1); \\\n G2(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h,b0,b1); \\\n UNDIAGONALIZE(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h);\n\n#endif\n\n" }, { "path": "benchmarks/blake2b.c", "content": "/*\n BLAKE2 reference source code package - optimized C implementations\n \n Copyright 2012, Samuel Neves . You may use this under the\n terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at\n your option. The terms of these licenses can be found at:\n \n - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0\n - OpenSSL license : https://www.openssl.org/source/license.html\n - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0\n \n More information about the BLAKE2 hash function can be found at\n https://blake2.net.\n*/\n\n#include \n#include \n#include \n\n#include \"blake2.h\"\n#include \"blake2-impl.h\"\n\n#include \"blake2-config.h\"\n\n#ifdef _MSC_VER\n#include /* for _mm_set_epi64x */\n#endif\n#include \n#if defined(HAVE_SSSE3)\n#include \n#endif\n#if defined(HAVE_SSE41)\n#include \n#endif\n#if defined(HAVE_AVX)\n#include \n#endif\n#if defined(HAVE_XOP)\n#include \n#endif\n\n#include \"blake2b-round.h\"\n\nstatic const uint64_t blake2b_IV[8] =\n{\n 0x6a09e667f3bcc908ULL, 0xbb67ae8584caa73bULL,\n 0x3c6ef372fe94f82bULL, 0xa54ff53a5f1d36f1ULL,\n 0x510e527fade682d1ULL, 0x9b05688c2b3e6c1fULL,\n 0x1f83d9abfb41bd6bULL, 0x5be0cd19137e2179ULL\n};\n\nstatic const uint8_t blake2b_sigma[12][16] =\n{\n { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 } ,\n { 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 } ,\n { 11, 8, 12, 0, 5, 2, 15, 13, 10, 14, 3, 6, 7, 1, 9, 4 } ,\n { 7, 9, 3, 1, 13, 12, 11, 14, 2, 6, 5, 10, 4, 0, 15, 8 } ,\n { 9, 0, 5, 7, 2, 4, 10, 15, 14, 1, 11, 12, 6, 8, 3, 13 } ,\n { 2, 12, 6, 10, 0, 11, 8, 3, 4, 13, 7, 5, 15, 14, 1, 9 } ,\n { 12, 5, 1, 15, 14, 13, 4, 10, 0, 7, 6, 3, 9, 2, 8, 11 } ,\n { 13, 11, 7, 14, 12, 1, 3, 9, 5, 0, 15, 4, 8, 6, 2, 10 } ,\n { 6, 15, 14, 9, 11, 3, 0, 8, 12, 2, 13, 7, 1, 4, 10, 5 } ,\n { 10, 2, 8, 4, 7, 6, 1, 5, 15, 11, 9, 14, 3, 12, 13 , 0 } ,\n { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 } ,\n { 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 }\n};\n\n\n/* Some helper functions, not necessarily useful */\nBLAKE2_LOCAL_INLINE(int) blake2b_set_lastnode( blake2b_state *S )\n{\n S->f[1] = -1;\n return 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_clear_lastnode( blake2b_state *S )\n{\n S->f[1] = 0;\n return 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_is_lastblock( const blake2b_state *S )\n{\n return S->f[0] != 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_set_lastblock( blake2b_state *S )\n{\n if( S->last_node ) blake2b_set_lastnode( S );\n\n S->f[0] = -1;\n return 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_clear_lastblock( blake2b_state *S )\n{\n if( S->last_node ) blake2b_clear_lastnode( S );\n\n S->f[0] = 0;\n return 0;\n}\n\n\nBLAKE2_LOCAL_INLINE(int) blake2b_increment_counter( blake2b_state *S, const uint64_t inc )\n{\n#if __x86_64__\n /* ADD/ADC chain */\n __uint128_t t = ( ( __uint128_t )S->t[1] << 64 ) | S->t[0];\n t += inc;\n S->t[0] = ( uint64_t )( t >> 0 );\n S->t[1] = ( uint64_t )( t >> 64 );\n#else\n S->t[0] += inc;\n S->t[1] += ( S->t[0] < inc );\n#endif\n return 0;\n}\n\n\n/* Parameter-related functions */\nBLAKE2_LOCAL_INLINE(int) blake2b_param_set_digest_length( blake2b_param *P, const uint8_t digest_length )\n{\n P->digest_length = digest_length;\n return 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_param_set_fanout( blake2b_param *P, const uint8_t fanout )\n{\n P->fanout = fanout;\n return 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_param_set_max_depth( blake2b_param *P, const uint8_t depth )\n{\n P->depth = depth;\n return 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_param_set_leaf_length( blake2b_param *P, const uint32_t leaf_length )\n{\n P->leaf_length = leaf_length;\n return 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_param_set_node_offset( blake2b_param *P, const uint64_t node_offset )\n{\n P->node_offset = node_offset;\n return 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_param_set_node_depth( blake2b_param *P, const uint8_t node_depth )\n{\n P->node_depth = node_depth;\n return 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_param_set_inner_length( blake2b_param *P, const uint8_t inner_length )\n{\n P->inner_length = inner_length;\n return 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_param_set_salt( blake2b_param *P, const uint8_t salt[BLAKE2B_SALTBYTES] )\n{\n memcpy( P->salt, salt, BLAKE2B_SALTBYTES );\n return 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_param_set_personal( blake2b_param *P, const uint8_t personal[BLAKE2B_PERSONALBYTES] )\n{\n memcpy( P->personal, personal, BLAKE2B_PERSONALBYTES );\n return 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_init0( blake2b_state *S )\n{\n memset( S, 0, sizeof( blake2b_state ) );\n\n for( int i = 0; i < 8; ++i ) S->h[i] = blake2b_IV[i];\n\n return 0;\n}\n\n/* init xors IV with input parameter block */\nint blake2b_init_param( blake2b_state *S, const blake2b_param *P )\n{\n /*blake2b_init0( S ); */\n const uint8_t * v = ( const uint8_t * )( blake2b_IV );\n const uint8_t * p = ( const uint8_t * )( P );\n uint8_t * h = ( uint8_t * )( S->h );\n /* IV XOR ParamBlock */\n memset( S, 0, sizeof( blake2b_state ) );\n\n for( int i = 0; i < BLAKE2B_OUTBYTES; ++i ) h[i] = v[i] ^ p[i];\n\n return 0;\n}\n\n\n/* Some sort of default parameter block initialization, for sequential blake2b */\nint blake2b_init( blake2b_state *S, const uint8_t outlen )\n{\n const blake2b_param P =\n {\n outlen,\n 0,\n 1,\n 1,\n 0,\n 0,\n 0,\n 0,\n {0},\n {0},\n {0}\n };\n\n if ( ( !outlen ) || ( outlen > BLAKE2B_OUTBYTES ) ) return -1;\n\n return blake2b_init_param( S, &P );\n}\n\nint blake2b_init_key( blake2b_state *S, const uint8_t outlen, const void *key, const uint8_t keylen )\n{\n const blake2b_param P =\n {\n outlen,\n keylen,\n 1,\n 1,\n 0,\n 0,\n 0,\n 0,\n {0},\n {0},\n {0}\n };\n\n if ( ( !outlen ) || ( outlen > BLAKE2B_OUTBYTES ) ) return -1;\n\n if ( ( !keylen ) || keylen > BLAKE2B_KEYBYTES ) return -1;\n\n if( blake2b_init_param( S, &P ) < 0 )\n return 0;\n\n {\n uint8_t block[BLAKE2B_BLOCKBYTES];\n memset( block, 0, BLAKE2B_BLOCKBYTES );\n memcpy( block, key, keylen );\n blake2b_update( S, block, BLAKE2B_BLOCKBYTES );\n secure_zero_memory( block, BLAKE2B_BLOCKBYTES ); /* Burn the key from stack */\n }\n return 0;\n}\n\nBLAKE2_LOCAL_INLINE(int) blake2b_compress( blake2b_state *S, const uint8_t block[BLAKE2B_BLOCKBYTES] )\n{\n __m128i row1l, row1h;\n __m128i row2l, row2h;\n __m128i row3l, row3h;\n __m128i row4l, row4h;\n __m128i b0, b1;\n __m128i t0, t1;\n#if defined(HAVE_SSSE3) && !defined(HAVE_XOP)\n const __m128i r16 = _mm_setr_epi8( 2, 3, 4, 5, 6, 7, 0, 1, 10, 11, 12, 13, 14, 15, 8, 9 );\n const __m128i r24 = _mm_setr_epi8( 3, 4, 5, 6, 7, 0, 1, 2, 11, 12, 13, 14, 15, 8, 9, 10 );\n#endif\n#if defined(HAVE_SSE41)\n const __m128i m0 = LOADU( block + 00 );\n const __m128i m1 = LOADU( block + 16 );\n const __m128i m2 = LOADU( block + 32 );\n const __m128i m3 = LOADU( block + 48 );\n const __m128i m4 = LOADU( block + 64 );\n const __m128i m5 = LOADU( block + 80 );\n const __m128i m6 = LOADU( block + 96 );\n const __m128i m7 = LOADU( block + 112 );\n#else\n const uint64_t m0 = ( ( uint64_t * )block )[ 0];\n const uint64_t m1 = ( ( uint64_t * )block )[ 1];\n const uint64_t m2 = ( ( uint64_t * )block )[ 2];\n const uint64_t m3 = ( ( uint64_t * )block )[ 3];\n const uint64_t m4 = ( ( uint64_t * )block )[ 4];\n const uint64_t m5 = ( ( uint64_t * )block )[ 5];\n const uint64_t m6 = ( ( uint64_t * )block )[ 6];\n const uint64_t m7 = ( ( uint64_t * )block )[ 7];\n const uint64_t m8 = ( ( uint64_t * )block )[ 8];\n const uint64_t m9 = ( ( uint64_t * )block )[ 9];\n const uint64_t m10 = ( ( uint64_t * )block )[10];\n const uint64_t m11 = ( ( uint64_t * )block )[11];\n const uint64_t m12 = ( ( uint64_t * )block )[12];\n const uint64_t m13 = ( ( uint64_t * )block )[13];\n const uint64_t m14 = ( ( uint64_t * )block )[14];\n const uint64_t m15 = ( ( uint64_t * )block )[15];\n#endif\n row1l = LOADU( &S->h[0] );\n row1h = LOADU( &S->h[2] );\n row2l = LOADU( &S->h[4] );\n row2h = LOADU( &S->h[6] );\n row3l = LOADU( &blake2b_IV[0] );\n row3h = LOADU( &blake2b_IV[2] );\n row4l = _mm_xor_si128( LOADU( &blake2b_IV[4] ), LOADU( &S->t[0] ) );\n row4h = _mm_xor_si128( LOADU( &blake2b_IV[6] ), LOADU( &S->f[0] ) );\n ROUND( 0 );\n ROUND( 1 );\n ROUND( 2 );\n ROUND( 3 );\n ROUND( 4 );\n ROUND( 5 );\n ROUND( 6 );\n ROUND( 7 );\n ROUND( 8 );\n ROUND( 9 );\n ROUND( 10 );\n ROUND( 11 );\n row1l = _mm_xor_si128( row3l, row1l );\n row1h = _mm_xor_si128( row3h, row1h );\n STOREU( &S->h[0], _mm_xor_si128( LOADU( &S->h[0] ), row1l ) );\n STOREU( &S->h[2], _mm_xor_si128( LOADU( &S->h[2] ), row1h ) );\n row2l = _mm_xor_si128( row4l, row2l );\n row2h = _mm_xor_si128( row4h, row2h );\n STOREU( &S->h[4], _mm_xor_si128( LOADU( &S->h[4] ), row2l ) );\n STOREU( &S->h[6], _mm_xor_si128( LOADU( &S->h[6] ), row2h ) );\n return 0;\n}\n\n\nint blake2b_update( blake2b_state *S, const uint8_t *in, uint64_t inlen )\n{\n while( inlen > 0 )\n {\n size_t left = S->buflen;\n size_t fill = 2 * BLAKE2B_BLOCKBYTES - left;\n\n if( inlen > fill )\n {\n memcpy( S->buf + left, in, fill ); /* Fill buffer */\n S->buflen += fill;\n blake2b_increment_counter( S, BLAKE2B_BLOCKBYTES );\n blake2b_compress( S, S->buf ); /* Compress */\n memcpy( S->buf, S->buf + BLAKE2B_BLOCKBYTES, BLAKE2B_BLOCKBYTES ); /* Shift buffer left */\n S->buflen -= BLAKE2B_BLOCKBYTES;\n in += fill;\n inlen -= fill;\n }\n else /* inlen <= fill */\n {\n memcpy( S->buf + left, in, inlen );\n S->buflen += inlen; /* Be lazy, do not compress */\n in += inlen;\n inlen -= inlen;\n }\n }\n\n return 0;\n}\n\n\nint blake2b_final( blake2b_state *S, uint8_t *out, uint8_t outlen )\n{\n if( outlen > BLAKE2B_OUTBYTES )\n return -1;\n\n if( blake2b_is_lastblock( S ) )\n return -1;\n\n if( S->buflen > BLAKE2B_BLOCKBYTES )\n {\n blake2b_increment_counter( S, BLAKE2B_BLOCKBYTES );\n blake2b_compress( S, S->buf );\n S->buflen -= BLAKE2B_BLOCKBYTES;\n memcpy( S->buf, S->buf + BLAKE2B_BLOCKBYTES, S->buflen );\n }\n\n blake2b_increment_counter( S, S->buflen );\n blake2b_set_lastblock( S );\n memset( S->buf + S->buflen, 0, 2 * BLAKE2B_BLOCKBYTES - S->buflen ); /* Padding */\n blake2b_compress( S, S->buf );\n memcpy( out, &S->h[0], outlen );\n return 0;\n}\n\n\nint blake2b( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen )\n{\n blake2b_state S[1];\n\n /* Verify parameters */\n if ( NULL == in && inlen > 0 ) return -1;\n\n if ( NULL == out ) return -1;\n\n if( NULL == key && keylen > 0 ) return -1;\n\n if( !outlen || outlen > BLAKE2B_OUTBYTES ) return -1;\n\n if( keylen > BLAKE2B_KEYBYTES ) return -1;\n\n if( keylen )\n {\n if( blake2b_init_key( S, outlen, key, keylen ) < 0 ) return -1;\n }\n else\n {\n if( blake2b_init( S, outlen ) < 0 ) return -1;\n }\n\n blake2b_update( S, ( const uint8_t * )in, inlen );\n blake2b_final( S, out, outlen );\n return 0;\n}\n\n#if defined(SUPERCOP)\nint crypto_hash( unsigned char *out, unsigned char *in, unsigned long long inlen )\n{\n return blake2b( out, in, NULL, BLAKE2B_OUTBYTES, inlen, 0 );\n}\n#endif\n\n#if defined(BLAKE2B_SELFTEST)\n#include \n#include \"blake2-kat.h\"\nint main( int argc, char **argv )\n{\n uint8_t key[BLAKE2B_KEYBYTES];\n uint8_t buf[KAT_LENGTH];\n\n for( size_t i = 0; i < BLAKE2B_KEYBYTES; ++i )\n key[i] = ( uint8_t )i;\n\n for( size_t i = 0; i < KAT_LENGTH; ++i )\n buf[i] = ( uint8_t )i;\n\n for( size_t i = 0; i < KAT_LENGTH; ++i )\n {\n uint8_t hash[BLAKE2B_OUTBYTES];\n blake2b( hash, buf, key, BLAKE2B_OUTBYTES, i, BLAKE2B_KEYBYTES );\n\n if( 0 != memcmp( hash, blake2b_keyed_kat[i], BLAKE2B_OUTBYTES ) )\n {\n puts( \"error\" );\n return -1;\n }\n }\n\n puts( \"ok\" );\n return 0;\n}\n#endif\n\n" }, { "path": "benchmarks/byte_order.c", "content": "/* byte_order.c - byte order related platform dependent routines,\n *\n * Copyright: 2008-2012 Aleksey Kravchenko \n *\n * Permission is hereby granted, free of charge, to any person obtaining a\n * copy of this software and associated documentation files (the \"Software\"),\n * to deal in the Software without restriction, including without limitation\n * the rights to use, copy, modify, merge, publish, distribute, sublicense,\n * and/or sell copies of the Software, and to permit persons to whom the\n * Software is furnished to do so.\n *\n * This program is distributed in the hope that it will be useful, but\n * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY\n * or FITNESS FOR A PARTICULAR PURPOSE. Use this program at your own risk!\n */\n#include \"byte_order.h\"\n\n#if !(__GNUC__ >= 4 || (__GNUC__ ==3 && __GNUC_MINOR__ >= 4)) /* if !GCC or GCC < 4.3 */\n\n# if _MSC_VER >= 1300 && (_M_IX86 || _M_AMD64 || _M_IA64) /* if MSVC++ >= 2002 on x86/x64 */\n# include \n# pragma intrinsic(_BitScanForward)\n\n/**\n * Returns index of the trailing bit of x.\n *\n * @param x the number to process\n * @return zero-based index of the trailing bit\n */\nunsigned rhash_ctz(unsigned x)\n{\n\tunsigned long index;\n\tunsigned char isNonzero = _BitScanForward(&index, x); /* MSVC intrinsic */\n\treturn (isNonzero ? (unsigned)index : 0);\n}\n# else /* _MSC_VER >= 1300... */\n\n/**\n * Returns index of the trailing bit of a 32-bit number.\n * This is a plain C equivalent for GCC __builtin_ctz() bit scan.\n *\n * @param x the number to process\n * @return zero-based index of the trailing bit\n */\nunsigned rhash_ctz(unsigned x)\n{\n\t/* array for conversion to bit position */\n\tstatic unsigned char bit_pos[32] = {\n\t\t0, 1, 28, 2, 29, 14, 24, 3, 30, 22, 20, 15, 25, 17, 4, 8,\n\t\t31, 27, 13, 23, 21, 19, 16, 7, 26, 12, 18, 6, 11, 5, 10, 9\n\t};\n\n\t/* The De Bruijn bit-scan was devised in 1997, according to Donald Knuth\n\t * by Martin Lauter. The constant 0x077CB531UL is a De Bruijn sequence,\n\t * which produces a unique pattern of bits into the high 5 bits for each\n\t * possible bit position that it is multiplied against.\n\t * See http://graphics.stanford.edu/~seander/bithacks.html\n\t * and http://chessprogramming.wikispaces.com/BitScan */\n\treturn (unsigned)bit_pos[((uint32_t)((x & -x) * 0x077CB531U)) >> 27];\n}\n# endif /* _MSC_VER >= 1300... */\n#endif /* !(GCC >= 4.3) */\n\n/**\n * Copy a memory block with simultaneous exchanging byte order.\n * The byte order is changed from little-endian 32-bit integers\n * to big-endian (or vice-versa).\n *\n * @param to the pointer where to copy memory block\n * @param index the index to start writing from\n * @param from the source block to copy\n * @param length length of the memory block\n */\nvoid rhash_swap_copy_str_to_u32(void* to, int index, const void* from, size_t length)\n{\n\t/* if all pointers and length are 32-bits aligned */\n\tif ( 0 == (( (int)((char*)to - (char*)0) | ((char*)from - (char*)0) | index | length ) & 3) ) {\n\t\t/* copy memory as 32-bit words */\n\t\tconst uint32_t* src = (const uint32_t*)from;\n\t\tconst uint32_t* end = (const uint32_t*)((const char*)src + length);\n\t\tuint32_t* dst = (uint32_t*)((char*)to + index);\n\t\twhile (src < end) *(dst++) = bswap_32( *(src++) );\n\t} else {\n\t\tconst char* src = (const char*)from;\n\t\tfor (length += index; (size_t)index < length; index++) ((char*)to)[index ^ 3] = *(src++);\n\t}\n}\n\n/**\n * Copy a memory block with changed byte order.\n * The byte order is changed from little-endian 64-bit integers\n * to big-endian (or vice-versa).\n *\n * @param to the pointer where to copy memory block\n * @param index the index to start writing from\n * @param from the source block to copy\n * @param length length of the memory block\n */\nvoid rhash_swap_copy_str_to_u64(void* to, int index, const void* from, size_t length)\n{\n\t/* if all pointers and length are 64-bits aligned */\n\tif ( 0 == (( (int)((char*)to - (char*)0) | ((char*)from - (char*)0) | index | length ) & 7) ) {\n\t\t/* copy aligned memory block as 64-bit integers */\n\t\tconst uint64_t* src = (const uint64_t*)from;\n\t\tconst uint64_t* end = (const uint64_t*)((const char*)src + length);\n\t\tuint64_t* dst = (uint64_t*)((char*)to + index);\n\t\twhile (src < end) *(dst++) = bswap_64( *(src++) );\n\t} else {\n\t\tconst char* src = (const char*)from;\n\t\tfor (length += index; (size_t)index < length; index++) ((char*)to)[index ^ 7] = *(src++);\n\t}\n}\n\n/**\n * Copy data from a sequence of 64-bit words to a binary string of given length,\n * while changing byte order.\n *\n * @param to the binary string to receive data\n * @param from the source sequence of 64-bit words\n * @param length the size in bytes of the data being copied\n */\nvoid rhash_swap_copy_u64_to_str(void* to, const void* from, size_t length)\n{\n\t/* if all pointers and length are 64-bits aligned */\n\tif ( 0 == (( (int)((char*)to - (char*)0) | ((char*)from - (char*)0) | length ) & 7) ) {\n\t\t/* copy aligned memory block as 64-bit integers */\n\t\tconst uint64_t* src = (const uint64_t*)from;\n\t\tconst uint64_t* end = (const uint64_t*)((const char*)src + length);\n\t\tuint64_t* dst = (uint64_t*)to;\n\t\twhile (src < end) *(dst++) = bswap_64( *(src++) );\n\t} else {\n\t\tsize_t index;\n\t\tchar* dst = (char*)to;\n\t\tfor (index = 0; index < length; index++) *(dst++) = ((char*)from)[index ^ 7];\n\t}\n}\n\n/**\n * Exchange byte order in the given array of 32-bit integers.\n *\n * @param arr the array to process\n * @param length array length\n */\nvoid rhash_u32_mem_swap(unsigned *arr, int length)\n{\n\tunsigned* end = arr + length;\n\tfor (; arr < end; arr++) {\n\t\t*arr = bswap_32(*arr);\n\t}\n}\n" }, { "path": "benchmarks/byte_order.h", "content": "/* byte_order.h */\n#ifndef BYTE_ORDER_H\n#define BYTE_ORDER_H\n#include \"ustd.h\"\n#include \n\n#ifdef IN_RHASH\n#include \"config.h\"\n#endif\n\n#ifdef __GLIBC__\n# include \n#endif\n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\n/* if x86 compatible cpu */\n#if defined(i386) || defined(__i386__) || defined(__i486__) || \\\n\tdefined(__i586__) || defined(__i686__) || defined(__pentium__) || \\\n\tdefined(__pentiumpro__) || defined(__pentium4__) || \\\n\tdefined(__nocona__) || defined(prescott) || defined(__core2__) || \\\n\tdefined(__k6__) || defined(__k8__) || defined(__athlon__) || \\\n\tdefined(__amd64) || defined(__amd64__) || \\\n\tdefined(__x86_64) || defined(__x86_64__) || defined(_M_IX86) || \\\n\tdefined(_M_AMD64) || defined(_M_IA64) || defined(_M_X64)\n/* detect if x86-64 instruction set is supported */\n# if defined(_LP64) || defined(__LP64__) || defined(__x86_64) || \\\n\tdefined(__x86_64__) || defined(_M_AMD64) || defined(_M_X64)\n# define CPU_X64\n# else\n# define CPU_IA32\n# endif\n#endif\n\n\n/* detect CPU endianness */\n#if (defined(__BYTE_ORDER) && defined(__LITTLE_ENDIAN) && \\\n\t\t__BYTE_ORDER == __LITTLE_ENDIAN) || \\\n\tdefined(CPU_IA32) || defined(CPU_X64) || \\\n\tdefined(__ia64) || defined(__ia64__) || defined(__alpha__) || defined(_M_ALPHA) || \\\n\tdefined(vax) || defined(MIPSEL) || defined(_ARM_) || defined(__arm__)\n# define CPU_LITTLE_ENDIAN\n# define IS_BIG_ENDIAN 0\n# define IS_LITTLE_ENDIAN 1\n#elif (defined(__BYTE_ORDER) && defined(__BIG_ENDIAN) && \\\n\t\t__BYTE_ORDER == __BIG_ENDIAN) || \\\n\tdefined(__sparc) || defined(__sparc__) || defined(sparc) || \\\n\tdefined(_ARCH_PPC) || defined(_ARCH_PPC64) || defined(_POWER) || \\\n\tdefined(__POWERPC__) || defined(POWERPC) || defined(__powerpc) || \\\n\tdefined(__powerpc__) || defined(__powerpc64__) || defined(__ppc__) || \\\n\tdefined(__hpux) || defined(_MIPSEB) || defined(mc68000) || \\\n\tdefined(__s390__) || defined(__s390x__) || defined(sel)\n# define CPU_BIG_ENDIAN\n# define IS_BIG_ENDIAN 1\n# define IS_LITTLE_ENDIAN 0\n#else\n# error \"Can't detect CPU architechture\"\n#endif\n\n#define IS_ALIGNED_32(p) (0 == (3 & ((const char*)(p) - (const char*)0)))\n#define IS_ALIGNED_64(p) (0 == (7 & ((const char*)(p) - (const char*)0)))\n\n#if defined(_MSC_VER)\n#define ALIGN_ATTR(n) __declspec(align(n))\n#elif defined(__GNUC__)\n#define ALIGN_ATTR(n) __attribute__((aligned (n)))\n#else\n#define ALIGN_ATTR(n) /* nothing */\n#endif\n\n\n#if defined(_MSC_VER) || defined(__BORLANDC__)\n#define I64(x) x##ui64\n#else\n#define I64(x) x##LL\n#endif\n\n/* convert a hash flag to index */\n#if __GNUC__ >= 4 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 4) /* GCC < 3.4 */\n# define rhash_ctz(x) __builtin_ctz(x)\n#else\nunsigned rhash_ctz(unsigned); /* define as function */\n#endif\n\nvoid rhash_swap_copy_str_to_u32(void* to, int index, const void* from, size_t length);\nvoid rhash_swap_copy_str_to_u64(void* to, int index, const void* from, size_t length);\nvoid rhash_swap_copy_u64_to_str(void* to, const void* from, size_t length);\nvoid rhash_u32_mem_swap(unsigned *p, int length_in_u32);\n\n/* define bswap_32 */\n#if defined(__GNUC__) && defined(CPU_IA32) && !defined(__i386__)\n/* for intel x86 CPU */\nstatic inline uint32_t bswap_32(uint32_t x) {\n\t__asm(\"bswap\\t%0\" : \"=r\" (x) : \"0\" (x));\n\treturn x;\n}\n#elif defined(__GNUC__) && (__GNUC__ >= 4) && (__GNUC__ > 4 || __GNUC_MINOR__ >= 3)\n/* for GCC >= 4.3 */\n# define bswap_32(x) __builtin_bswap32(x)\n#elif (_MSC_VER > 1300) && (defined(CPU_IA32) || defined(CPU_X64)) /* MS VC */\n# define bswap_32(x) _byteswap_ulong((unsigned long)x)\n#elif !defined(__STRICT_ANSI__)\n/* general bswap_32 definition */\nstatic inline uint32_t bswap_32(uint32_t x) {\n\tx = ((x << 8) & 0xFF00FF00) | ((x >> 8) & 0x00FF00FF);\n\treturn (x >> 16) | (x << 16);\n}\n#else\n#define bswap_32(x) ((((x) & 0xff000000) >> 24) | (((x) & 0x00ff0000) >> 8) | \\\n\t(((x) & 0x0000ff00) << 8) | (((x) & 0x000000ff) << 24))\n#endif /* bswap_32 */\n\n#if defined(__GNUC__) && (__GNUC__ >= 4) && (__GNUC__ > 4 || __GNUC_MINOR__ >= 3)\n# define bswap_64(x) __builtin_bswap64(x)\n#elif (_MSC_VER > 1300) && (defined(CPU_IA32) || defined(CPU_X64)) /* MS VC */\n# define bswap_64(x) _byteswap_uint64((__int64)x)\n#elif !defined(__STRICT_ANSI__)\nstatic inline uint64_t bswap_64(uint64_t x) {\n\tunion {\n\t\tuint64_t ll;\n\t\tuint32_t l[2];\n\t} w, r;\n\tw.ll = x;\n\tr.l[0] = bswap_32(w.l[1]);\n\tr.l[1] = bswap_32(w.l[0]);\n\treturn r.ll;\n}\n#else\n#error \"bswap_64 unsupported\"\n#endif\n\n#ifdef CPU_BIG_ENDIAN\n# define be2me_32(x) (x)\n# define be2me_64(x) (x)\n# define le2me_32(x) bswap_32(x)\n# define le2me_64(x) bswap_64(x)\n\n# define be32_copy(to, index, from, length) memcpy((to) + (index), (from), (length))\n# define le32_copy(to, index, from, length) rhash_swap_copy_str_to_u32((to), (index), (from), (length))\n# define be64_copy(to, index, from, length) memcpy((to) + (index), (from), (length))\n# define le64_copy(to, index, from, length) rhash_swap_copy_str_to_u64((to), (index), (from), (length))\n# define me64_to_be_str(to, from, length) memcpy((to), (from), (length))\n# define me64_to_le_str(to, from, length) rhash_swap_copy_u64_to_str((to), (from), (length))\n\n#else /* CPU_BIG_ENDIAN */\n# define be2me_32(x) bswap_32(x)\n# define be2me_64(x) bswap_64(x)\n# define le2me_32(x) (x)\n# define le2me_64(x) (x)\n\n# define be32_copy(to, index, from, length) rhash_swap_copy_str_to_u32((to), (index), (from), (length))\n# define le32_copy(to, index, from, length) memcpy((to) + (index), (from), (length))\n# define be64_copy(to, index, from, length) rhash_swap_copy_str_to_u64((to), (index), (from), (length))\n# define le64_copy(to, index, from, length) memcpy((to) + (index), (from), (length))\n# define me64_to_be_str(to, from, length) rhash_swap_copy_u64_to_str((to), (from), (length))\n# define me64_to_le_str(to, from, length) memcpy((to), (from), (length))\n#endif /* CPU_BIG_ENDIAN */\n\n/* ROTL/ROTR macros rotate a 32/64-bit word left/right by n bits */\n#define ROTL32(dword, n) ((dword) << (n) ^ ((dword) >> (32 - (n))))\n#define ROTR32(dword, n) ((dword) >> (n) ^ ((dword) << (32 - (n))))\n#define ROTL64(qword, n) ((qword) << (n) ^ ((qword) >> (64 - (n))))\n#define ROTR64(qword, n) ((qword) >> (n) ^ ((qword) << (64 - (n))))\n\n#ifdef __cplusplus\n} /* extern \"C\" */\n#endif /* __cplusplus */\n\n#endif /* BYTE_ORDER_H */\n" }, { "path": "benchmarks/chacha-prng.h", "content": "/* Copyright (c) 2016 Vladimir Makarov \n\n Permission is hereby granted, free of charge, to any person\n obtaining a copy of this software and associated documentation\n files (the \"Software\"), to deal in the Software without\n restriction, including without limitation the rights to use, copy,\n modify, merge, publish, distribute, sublicense, and/or sell copies\n of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n\n The above copyright notice and this permission notice shall be\n included in all copies or substantial portions of the Software.\n\n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\n MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\n BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\n ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\n CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.\n*/\n\n/* Pseudo Random Number Generator (PRNG) based on ChaCha stream cipher\n designed by D.J. Bernstein. The code is ChaCha reference code\n (please see https://cr.yp.to/chacha.html) adapted for our purposes.\n\n It is a crypto level PRNG as the stream cypher on which it is\n based.\n\n To use a generator call `init_chacha_prng` first, then call\n `get_chacha_prn` as much as you want to get a new PRN. At the end\n of the PRNG use, call `finish_chacha_prng`. You can change the\n default seed by calling `set_chacha_prng_seed`.\n\n The PRNG passes NIST Statistical Test Suite for Random and\n Pseudorandom Number Generators for Cryptographic Applications\n (version 2.2.1) with 1000 bitstreams each containing 1M bits.\n\n The generation of a new number takes about 40 CPU cycles on x86_64\n (Intel 4.2GHz i7-4790K), or speed of the generation is about 106M\n numbers per sec. */\n\n#ifndef __CHACHA_PRNG__\n#define __CHACHA_PRNG__\n\n#ifdef _MSC_VER\ntypedef unsigned __int32 uint32_t;\ntypedef unsigned __int64 uint64_t;\n#else\n#include \n#endif\n\n#include \n\nstatic inline uint32_t\n_chacha_prng_rotl (uint32_t v, int c) {\n return (v << c) | (v >> (32 - c));\n}\n\n/* ChaCha state transformation step. */\nstatic inline void\n_chacha_prng_quarter_round (uint32_t *a, uint32_t *b, uint32_t *c, uint32_t *d) {\n *a += *b; *d = _chacha_prng_rotl (*d ^ *a, 16);\n *c += *d; *b = _chacha_prng_rotl (*b ^ *c, 12);\n *a += *b; *d = _chacha_prng_rotl (*d ^ *a, 8);\n *c += *d; *b = _chacha_prng_rotl (*b ^ *c, 7);\n}\n\n/* Major ChaCha state transformation. */\nstatic inline void\n_chacha_prng_salsa20 (uint32_t output[16], const uint32_t input[16]) {\n int i;\n\n for (i = 0; i < 16; i++)\n output[i] = input[i];\n for (i = 8; i > 0; i -= 2) {\n _chacha_prng_quarter_round (&output[0], &output[4], &output[8],&output[12]);\n _chacha_prng_quarter_round (&output[1], &output[5], &output[9],&output[13]);\n _chacha_prng_quarter_round (&output[2], &output[6],&output[10],&output[14]);\n _chacha_prng_quarter_round (&output[3], &output[7],&output[11],&output[15]);\n _chacha_prng_quarter_round (&output[0], &output[5],&output[10],&output[15]);\n _chacha_prng_quarter_round (&output[1], &output[6],&output[11],&output[12]);\n _chacha_prng_quarter_round (&output[2], &output[7], &output[8],&output[13]);\n _chacha_prng_quarter_round (&output[3], &output[4], &output[9],&output[14]);\n }\n for (i = 0; i < 16; i++)\n output[i] += input[i];\n}\n\n/* Internal state of the PRNG. */\nstatic struct {\n int ind; /* position in the output */\n /* The current PRNG state and parts of the recently generated\n numbers. */\n uint32_t input[16], output[16];\n} _chacha_prng_state;\n\n/* Some random prime numbers. */\nstatic const uint32_t chacha_prng_primes[4] = {0xfa835867, 0x2086ca69, 0x1467c0fb, 0x638e2b99};\n\n/* Internal function to set ChaCha PRNG seed by K and IV. */\nstatic inline void\n_set_chacha_prng_key_iv (uint32_t k[8], uint32_t iv[2]) {\n int i;\n \n for (i = 0; i < 8; i++)\n _chacha_prng_state.input[i + 4] = k[i];\n for (i = 0; i < 4; i++)\n _chacha_prng_state.input[i] = chacha_prng_primes[i];\n _chacha_prng_state.input[12] = 0;\n _chacha_prng_state.input[13] = 0;\n _chacha_prng_state.input[14] = iv[0];\n _chacha_prng_state.input[15] = iv[1];\n}\n\n/* Internal function to initiate ChaCha PRNG with seed given by K and\n IV. */\nstatic inline void\n_init_chacha_prng_with_key_iv (uint32_t k[8], uint32_t iv[2]) {\n _set_chacha_prng_key_iv (k, iv);\n _chacha_prng_state.ind = 16;\n}\n\n/* Initiate the PRNG with some random seed. */\nstatic inline void\ninit_chacha_prng (void) {\n int i;\n uint32_t k[8], iv[2];\n\n for (i = 0; i < 8; i++)\n k[i] = rand ();\n iv[0] = rand ();\n iv[1] = rand ();\n _init_chacha_prng_with_key_iv (k, iv);\n}\n\n/* Set ChaCha PRNG SEED. */\nstatic inline void\nset_chacha_prng_seed (uint32_t seed) {\n static uint32_t k[8] = {0, 0, 0, 0};\n static uint32_t iv[2] = {0, 0};\n \n k[0] = seed;\n _set_chacha_prng_key_iv (k, iv);\n}\n\n/* Return the next pseudo-random number. */\nstatic inline uint64_t\nget_chacha_prn (void)\n{\n uint64_t res;\n\n for (;;) {\n if (_chacha_prng_state.ind < 16) {\n res = ((uint64_t) _chacha_prng_state.output[_chacha_prng_state.ind] << 32\n\t | _chacha_prng_state.output[_chacha_prng_state.ind + 1]);\n _chacha_prng_state.ind += 2;\n return res;\n }\n _chacha_prng_state.ind = 0;\n _chacha_prng_salsa20 (_chacha_prng_state.output, _chacha_prng_state.input);\n _chacha_prng_state.input[12]++;\n if (! _chacha_prng_state.input[12])\n /* If it is becomming zero we produced too many numbers by\n\t current PRNG. */\n _chacha_prng_state.input[13]++;\n }\n}\n\n/* Empty function for our PRNGs interface. */\nstatic inline void\nfinish_chacha_prng (void) {\n}\n\n#endif\n" }, { "path": "benchmarks/gen-table.rb", "content": "#!/usr/bin/ruby\n# Take stdin and output the table\nrows = []\ncols = []\ntab = {}\ncur = \"\"\nn = 0\nSTDIN.each_line do |line|\n puts line\n if md = /[+]+([0-9a-zA-Z-]+)/.match(line)\n rows.push(cur=line[md.begin(1)...md.end(1)])\n n += 1\n elsif md = /([0-9a-zA-Z-]+)\\s*:\\s*(\\d+.\\d+)s/.match(line)\n name=line[md.begin(1)...md.end(1)]\n cols.push(name) if n == 1\n tab[cur + name] = line[md.begin(2)...md.end(2)]\n end\nend\n\n\ndef print_header(cols, mr, mc)\n print \"|\".ljust(mr, \" \")\n cols.each do |e|\n print \" | \", e.ljust(mc, \" \")\n end\n print \" |\\n\", \":\".ljust(mr + 1, \"-\")\n cols.each do |e|\n print \"|\", \":\".rjust(mc + 2, \"-\")\n end\n print \"|\\n\"\nend\n\nmr = rows.map { |e| e.length}.max\nmc = cols.map { |e| e.length}.max\nmc = 11 if mc < 11\n\nprint_header(cols, mr, mc)\n\nrows.each { |r|\n print \"|\", r.ljust(mr, \" \"), \"| \"\n min = 100000.0;\n cols.each { |c|\n min = tab[r + c].to_f if tab.has_key?(r + c) && tab[r + c].to_f < min\n }\n cols.each { |c|\n if ! tab.has_key?(r + c)\n print \"-\".ljust(mc - 4, \" \"), \" | \"\n continue\n end\n v = tab[r + c]\n print v.to_f == min ? \"**\" : \" \"\n print v\n print v.to_f == min ? \"**\" : \" \"\n print \" \".ljust(mc - 4 - v.length, \" \")\n print \" | \"\n }\n print \"\\n\"\n}\n" }, { "path": "benchmarks/meow_hash.h", "content": "/* ========================================================================\n\n Meow - A Fast Non-cryptographic Hash\n (C) Copyright 2018 by Molly Rocket, Inc. (https://mollyrocket.com)\n \n See https://mollyrocket.com/meowhash for details.\n \n ========================================================================\n \n zlib License\n \n (C) Copyright 2018 Molly Rocket, Inc.\n \n This software is provided 'as-is', without any express or implied\n warranty. In no event will the authors be held liable for any damages\n arising from the use of this software.\n \n Permission is granted to anyone to use this software for any purpose,\n including commercial applications, and to alter it and redistribute it\n freely, subject to the following restrictions:\n \n 1. The origin of this software must not be misrepresented; you must not\n claim that you wrote the original software. If you use this software\n in a product, an acknowledgment in the product documentation would be\n appreciated but is not required.\n 2. Altered source versions must be plainly marked as such, and must not be\n misrepresented as being the original software.\n 3. This notice may not be removed or altered from any source distribution.\n \n ========================================================================\n \n FAQ\n \n Q: What is it?\n \n A: Meow is a 128-bit non-cryptographic hash that operates at high speeds\n on x64 and ARM processors that provide AES instructions. It is\n designed to be truncatable to 64 and 32-bit hash values and still\n retain good collision resistance.\n \n Q: What is it GOOD for?\n \n A: Quickly hashing any amount of data for comparison purposes such as\n block deduplication or change detection. It is extremely fast on\n all buffer sizes, from one byte to one gigabyte and up.\n \n Q: What is it BAD for?\n \n A: Anything security-related. It should be assumed that it provides\n no protection from adversaries whatsoever. It is also not particularly\n fast on processors that don't support AES instructions (eg., non-x64/ARM\n processors).\n \n Q: Why is it called the \"Meow hash\"?\n \n A: It is named after a character in Meow the Infinite\n (https://meowtheinfinite.com)\n \n Q: Who wrote it?\n \n A: CASEY MURATORI (https://caseymuratori.com) wrote the original\n implementation for use in processing large-footprint assets for\n the game 1935 (https://molly1935.com).\n \n After the initial version, the hash was refined via collaboration\n with several great programmers who contributed suggestions and\n modifications:\n \n JEFF ROBERTS (https://radgametools.com) provided a super slick\n way to handle the residual end-of-buffer bytes that dramatically\n improved Meow's small hash performance.\n \n MARTINS MOZEIKO (https://matrins.ninja) ported Meow to ARM and\n ANSI-C, and added the proper preprocessor dressing for clean\n compilation on a variety of compiler configurations.\n \n FABIAN GIESEN (https://fgiesen.wordpress.com) provided support\n for getting the benchmarking working properly across a number\n of platforms.\n \n ARAS PRANCKEVICIUS (https://aras-p.info) provided the allocation\n shim for compilation on Mac OS X.\n \n ========================================================================\n \n USAGE\n \n For a complete working example, see meow_example.cpp. Briefly:\n \n // Include meow_intrinsics if you want it to detect platforms\n // and define types and intrinsics for you. Omit it if you\n // want to define them yourself.\n #include \"meow_intrinsics.h\"\n \n // Include meow_hash for the Meow hash function\n #include \"meow_hash.h\"\n \n // Hash a block of data using CPU-specific acceleration\n meow_u128 MeowHash_Accelerated(u64 Seed, u64 Len, void *Source);\n \n // Check if two Meow hashes are the same\n // (returns zero if they aren't, non-zero if they are)\n int MeowHashesAreEqual(meow_u128 A, meow_u128 B)\n \n // Truncate a Meow hash to 64 bits\n meow_u64 MeowU64From(meow_u128 Hash);\n \n // Truncate a Meow hash to 32 bits\n meow_u32 MeowU32From(meow_u128 Hash);\n \n **** VERY IMPORTANT X64 COMPILATION NOTES ****\n \n On x64, Meow uses the AESDEC instruction, which comes in two flavors:\n SSE (aesdec) and AVX (vaesdec). If you are compiling _with_ AVX support,\n your compiler will probably emit the AVX variant, which means your code\n WILL NOT RUN on computers that do not have AVX. If you need to deploy\n this hash on computers that do not have AVX, you must take care to\n TURN OFF support for AVX in your compiler for the file that includes\n the Meow hash!\n \n ======================================================================== */\n\n//\n// NOTE(casey): This version is EXPERIMENTAL. The Meow hash is still\n// undergoing testing and finalization.\n//\n// **** EXPECT HASHES/APIs TO CHANGE UNTIL THE VERSION NUMBER HITS 1.0. ****\n//\n// You have been warned.\n//\n\nstatic const unsigned char MeowShiftAdjust[31] = {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,128,128,128,128,128,128,128,128,128,128,128,128,128,128,128};\nstatic const unsigned char MeowMaskLen[32] = {255,255,255,255, 255,255,255,255, 255,255,255,255, 255,255,255,255, 0,0,0,0, 0,0,0,0, 0,0,0,0, 0,0,0,0};\n\n// TODO(casey): These constants are loaded to initialize the lanes. Jacob should\n// give us some feedback on what they should _actually_ be set to.\n#define MEOW_S0_INIT { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,10,11, 12,13,14,15}\n#define MEOW_S1_INIT {16,17,18,19, 20,21,22,23, 24,25,26,27, 28,29,30,31}\n#define MEOW_S2_INIT {32,33,34,35, 36,37,38,39, 40,41,42,43, 44,45,46,47}\n#define MEOW_S3_INIT {48,49,50,51, 52,53,54,55, 56,57,58,59, 60,61,62,63}\nstatic const unsigned char MeowS0Init[] = MEOW_S0_INIT;\nstatic const unsigned char MeowS1Init[] = MEOW_S1_INIT;\nstatic const unsigned char MeowS2Init[] = MEOW_S2_INIT;\nstatic const unsigned char MeowS3Init[] = MEOW_S3_INIT;\n\n//\n// NOTE(casey): 128-wide AES-NI Meow (maximum of 16 bytes/clock single threaded)\n//\n\nstatic meow_hash\nMeowHash_Accelerated(meow_u64 Seed, meow_u64 TotalLengthInBytes, void *SourceInit)\n{\n //\n // NOTE(casey): Initialize the four AES streams and the mixer\n //\n \n meow_aes_128 S0 = Meow128_GetAESConstant(MeowS0Init);\n meow_aes_128 S1 = Meow128_GetAESConstant(MeowS1Init);\n meow_aes_128 S2 = Meow128_GetAESConstant(MeowS2Init);\n meow_aes_128 S3 = Meow128_GetAESConstant(MeowS3Init);\n \n meow_u128 Mixer = Meow128_Set64x2(Seed - TotalLengthInBytes,\n Seed + TotalLengthInBytes + 1);\n \n //\n // NOTE(casey): Handle as many full 256-byte blocks as possible\n //\n \n meow_u8 *Source = (meow_u8 *)SourceInit;\n meow_u64 Len = TotalLengthInBytes;\n int unsigned Len8 = Len & 15;\n int unsigned Len128 = Len & 48;\n \n while(Len >= 64)\n {\n S0 = Meow128_AESDEC_Mem(S0, Source);\n S1 = Meow128_AESDEC_Mem(S1, Source + 16);\n S2 = Meow128_AESDEC_Mem(S2, Source + 32);\n S3 = Meow128_AESDEC_Mem(S3, Source + 48);\n \n Len -= 64;\n Source += 64;\n }\n \n //\n // NOTE(casey): Overhanging individual bytes\n //\n \n if(Len8)\n {\n meow_u8 *Overhang = Source + Len128;\n int Align = ((int)(meow_umm)Overhang) & 15;\n if(Align)\n {\n int End = ((int)(meow_umm)Overhang) & (MEOW_PAGESIZE - 1);\n \n // NOTE(jeffr): If we are nowhere near the page end, use full unaligned load (cmov to set)\n if (End <= (MEOW_PAGESIZE - 16))\n {\n Align = 0;\n }\n \n // NOTE(jeffr): If we will read over the page end, use a full unaligned load (cmov to set)\n if ((End + Len8) > MEOW_PAGESIZE)\n {\n Align = 0;\n }\n \n meow_u128 Partial = Meow128_Shuffle_Mem(Overhang - Align, &MeowShiftAdjust[Align]);\n \n Partial = Meow128_And_Mem( Partial, &MeowMaskLen[16 - Len8] );\n S3 = Meow128_AESDEC(S3, Partial);\n }\n else\n {\n // NOTE(casey): We don't have to do Jeff's heroics when we know the\n // buffer is aligned, since we cannot span a memory page (by definition).\n meow_u128 Partial = Meow128_And_Mem(*(meow_u128 *)Overhang, &MeowMaskLen[16 - Len8]);\n S3 = Meow128_AESDEC(S3, Partial);\n }\n }\n \n //\n // NOTE(casey): Overhanging full 128-bit lanes\n //\n \n switch(Len128)\n {\n case 48: S2 = Meow128_AESDEC_Mem(S2, Source + 32);\n case 32: S1 = Meow128_AESDEC_Mem(S1, Source + 16);\n case 16: S0 = Meow128_AESDEC_Mem(S0, Source);\n }\n \n //\n // NOTE(casey): Mix the four lanes down to one 128-bit hash\n //\n \n S3 = Meow128_AESDEC(S3, Mixer);\n S2 = Meow128_AESDEC(S2, Mixer);\n S1 = Meow128_AESDEC(S1, Mixer);\n S0 = Meow128_AESDEC(S0, Mixer);\n \n S2 = Meow128_AESDEC(S2, Meow128_AESDEC_Finalize(S3));\n S0 = Meow128_AESDEC(S0, Meow128_AESDEC_Finalize(S1));\n \n S2 = Meow128_AESDEC(S2, Mixer);\n \n S0 = Meow128_AESDEC(S0, Meow128_AESDEC_Finalize(S2));\n S0 = Meow128_AESDEC(S0, Mixer);\n \n meow_hash Result;\n Meow128_CopyToHash(Meow128_AESDEC_Finalize(S0), Result);\n \n return(Result);\n}\n" }, { "path": "benchmarks/meow_intrinsics.h", "content": "/* ========================================================================\n\n meow_intrinsics.h\n (C) Copyright 2018 by Molly Rocket, Inc. (https://mollyrocket.com)\n \n See https://mollyrocket.com/meowhash for details.\n \n This is the default way to define all of the types and operations that\n meow_hash.h needs. However, if you've got your _own_ equivalent type\n definitions and intrinsics, you can _omit_ this header file and just\n #define/typedef all the Meow ops to map to your own ops, keeping things\n nice and uniform in your codebase.\n \n ======================================================================== */\n\n#if !defined(MEOW_HASH_INTRINSICS_H)\n\n//\n// NOTE(casey): Try to guess the source file for compiler intrinsics\n//\n#if _MSC_VER\n\n#if _M_AMD64 || _M_IX86\n#include \n#elif _M_ARM64\n#include \n#endif\n\n#else\n\n#if __x86_64__ || __i386__\n#include \n#elif __aarch64__\n#include \n#endif\n\n#endif\n\n//\n// NOTE(casey): Set #define's to their defaults\n//\n\n#if !defined(MEOW_HASH_INTEL) || !defined(MEOW_HASH_ARMV8)\n#if __x86_64__ || _M_AMD64\n#define MEOW_HASH_INTEL 1\n#define MEOW_64BIT 1\n#define MEOW_PAGESIZE 4096\n#elif __i386__ || _M_IX86\n#define MEOW_HASH_INTEL 1\n#define MEOW_64BIT 0\n#define MEOW_PAGESIZE 4096\n#elif __aarch64__ || _M_ARM64\n#define MEOW_HASH_ARMV8 1\n#define MEOW_64BIT 1\n#define MEOW_PAGESIZE 4096\n#else\n#error Cannot determine architecture to use!\n#endif\n#endif\n\n//\n// NOTE(casey): Define basic types\n//\n\n#define meow_u8 char unsigned\n#define meow_u16 short unsigned\n#define meow_u32 int unsigned\n#define meow_u64 long long unsigned\n\n#if MEOW_64BIT\n#define meow_umm long long unsigned\n#else\n#define meow_umm int unsigned\n#endif\n\n//\n// NOTE(casey): Operations for x64 processors\n//\n\n#if MEOW_HASH_INTEL\n\n#define meow_u128 __m128i\n#define meow_aes_128 __m128i\n#define meow_u256 __m256i\n#define meow_aes_256 __m256i\n#define meow_u512 __m512i\n#define meow_aes_512 __m512i\n\n#define MeowU32From(A, I) (_mm_extract_epi32((A), (I)))\n#define MeowU64From(A, I) (_mm_extract_epi64((A), (I)))\n#define MeowHashesAreEqual(A, B) (_mm_movemask_epi8(_mm_cmpeq_epi8((A), (B))) == 0xFFFF)\n\n#define Meow128_AESDEC(Prior, Xor) _mm_aesdec_si128((Prior), (Xor))\n#define Meow128_AESDEC_Mem(Prior, Xor) _mm_aesdec_si128((Prior), _mm_loadu_si128((meow_u128 *)(Xor)))\n#define Meow128_AESDEC_Finalize(A) (A)\n#define Meow128_Set64x2(Low64, High64) _mm_set_epi64x((High64), (Low64))\n#define Meow128_Set64x2_State(Low64, High64) Meow128_Set64x2(Low64, High64)\n#define Meow128_GetAESConstant(Ptr) (*(meow_u128 *)(Ptr))\n\n#define Meow128_And_Mem(A,B) _mm_and_si128((A),_mm_loadu_si128((meow_u128 *)(B)))\n#define Meow128_Shuffle_Mem(Mem,Control) _mm_shuffle_epi8(_mm_loadu_si128((meow_u128 *)(Mem)),_mm_loadu_si128((meow_u128 *)(Control)))\n\n// TODO(casey): Not sure if this should actually be Meow128_Zero(A) ((A) = _mm_setzero_si128()), maybe\n#define Meow128_Zero() _mm_setzero_si128()\n\n#define Meow256_AESDEC(Prior, XOr) _mm256_aesdec_epi128((Prior), (XOr))\n#define Meow256_AESDEC_Mem(Prior, XOr) _mm256_aesdec_epi128((Prior), *(meow_u256 *)(XOr))\n#define Meow256_Zero() _mm256_setzero_si256()\n#define Meow256_PartialLoad(A, B) _mm256_mask_loadu_epi8(_mm256_setzero_si256(), _cvtu32_mask32((1UL<<(B)) - 1), (A))\n#define Meow128_FromLow(A) _mm256_extracti128_si256((A), 0)\n#define Meow128_FromHigh(A) _mm256_extracti128_si256((A), 1)\n\n#define Meow512_AESDEC(Prior, XOr) _mm512_aesdec_epi128((Prior), (XOr))\n#define Meow512_AESDEC_Mem(Prior, XOr) _mm512_aesdec_epi128((Prior), *(meow_u512 *)(XOr))\n#define Meow512_Zero() _mm512_setzero_si512()\n#define Meow512_PartialLoad(A, B) _mm512_mask_loadu_epi8(_mm512_setzero_si512(), _cvtu64_mask64((1ULL<<(B)) - 1), (A))\n#define Meow256_FromLow(A) _mm512_extracti64x4_epi64((A), 0)\n#define Meow256_FromHigh(A) _mm512_extracti64x4_epi64((A), 1)\n\n//\n// NOTE(casey): Operations for ARM processors\n//\n\n#elif MEOW_HASH_ARMV8\n\n#define meow_u128 uint8x16_t\n\n// NOTE(mmozeiko): AES opcodes on ARMv8 work a bit differently than on Intel\n// On Intel the \"x = AESDEC(x, m)\" does following:\n// x = InvMixColumns(SubBytes(ShiftRows(x))) ^ m\n// But on ARMv8 the \"x = AESDEC(x, m)\" does following:\n// x = SubBytes(ShiftRows(x ^ m))\n// Thus on ARMv8 it requires extra InvMixColumns call and delay on Xor operation.\n// On iteration N it needs to use m[N-1] as input, and remeber m[N] for next iteration.\n// This structure will store memory operand in member B which will be used in\n// next AESDEC opcode. Remember to do one more XOR(A,B) when finishing AES\n// operations in a loop.\ntypedef struct {\n meow_u128 A;\n meow_u128 B;\n} meow_aes_128;\n\n#define MeowU32From(A, I) (vgetq_lane_u32(vreinterpretq_u32_u8((A)), (I)))\n#define MeowU64From(A, I) (vgetq_lane_u64(vreinterpretq_u64_u8((A)), (I)))\n\nstatic int\nMeowHashesAreEqualImpl(meow_u128 A, meow_u128 B)\n{\n uint8x16_t Powers = {\n 1, 2, 4, 8, 16, 32, 64, 128, 1, 2, 4, 8, 16, 32, 64, 128,\n };\n\n uint8x16_t Input = vceqq_u8(A, B);\n uint64x2_t Mask = vpaddlq_u32(vpaddlq_u16(vpaddlq_u8(vandq_u8(Input, Powers))));\n\n meow_u16 Output;\n vst1q_lane_u8((meow_u8*)&Output + 0, vreinterpretq_u8_u64(Mask), 0);\n vst1q_lane_u8((meow_u8*)&Output + 1, vreinterpretq_u8_u64(Mask), 8);\n return Output == 0xFFFF;\n}\n\n#define MeowHashesAreEqual(A, B) MeowHashesAreEqualImpl((A), (B))\n\nstatic meow_aes_128\nMeow128_AESDEC(meow_aes_128 Prior, meow_u128 Xor)\n{\n meow_aes_128 R;\n R.A = vaesimcq_u8(vaesdq_u8(Prior.A, Prior.B));\n R.B = Xor;\n return(R);\n}\n\nstatic meow_aes_128\nMeow128_AESDEC_Mem(meow_aes_128 Prior, void *Xor)\n{\n meow_aes_128 R;\n R.A = vaesimcq_u8(vaesdq_u8(Prior.A, Prior.B));\n R.B = vld1q_u8((meow_u8*)Xor);\n return(R);\n}\n\nstatic meow_u128\nMeow128_AESDEC_Finalize(meow_aes_128 Value)\n{\n meow_u128 R = veorq_u8(Value.A, Value.B);\n return(R);\n}\n\nstatic meow_u128\nMeow128_Zero()\n{\n meow_u128 R = vdupq_n_u8(0);\n return(R);\n}\n\nstatic meow_aes_128\nMeow128_GetAESConstant(const meow_u8 *Ptr)\n{\n meow_aes_128 R;\n R.A = vld1q_u8(Ptr);\n R.B = vdupq_n_u8(0);\n return(R);\n}\n\nstatic meow_u128\nMeow128_Set64x2(meow_u64 Low64, meow_u64 High64)\n{\n meow_u128 R = vreinterpretq_u8_u64(vcombine_u64(vcreate_u64(Low64), vcreate_u64(High64)));\n return(R);\n}\n\nstatic meow_aes_128\nMeow128_Set64x2_State(meow_u64 Low64, meow_u64 High64)\n{\n meow_aes_128 R;\n R.A = Meow128_Set64x2(Low64, High64);\n R.B = Meow128_Zero();\n return(R);\n}\n\n#define Meow128_And_Mem(A,B) vandq_u8((A), vld1q_u8((meow_u8 *)B))\n#define Meow128_Shuffle_Mem(Mem,Control) vqtbl1q_u8(vld1q_u8((meow_u8 *)(Mem)),vld1q_u8((meow_u8 *)(Control)))\n\n#endif\n\n#define MEOW_HASH_VERSION 4\n#define MEOW_HASH_VERSION_NAME \"0.4/himalayan\"\n\n#if MEOW_INCLUDE_C\n\n// NOTE(casey): Unfortunately, if you want an ANSI-C version, we have to slow everyone\n// else down because you can't return 128-bit values by register anymore (in case the\n// CPU doesn't support that)\nunion meow_hash\n{\n meow_u128 u128;\n meow_u64 u64[2];\n meow_u32 u32[4];\n};\n#define Meow128_CopyToHash(A, B) ((B).u128 = (A))\n\n#undef MeowU64From\n#undef MeowU32From\n#undef MeowHashesAreEqual\n#define MeowU32From(A, I) ((A).u32[I])\n#define MeowU64From(A, I) ((A).u64[I])\n#define MeowHashesAreEqual(A, B) (((A).u32[0] == (B).u32[0]) && ((A).u32[1] == (B).u32[1]) && ((A).u32[2] == (B).u32[2]) && ((A).u32[3] == (B).u32[3]))\n\n#else\n\ntypedef meow_u128 meow_hash;\n#define Meow128_CopyToHash(A, B) ((B) = (A))\n\n#endif\n\ntypedef struct meow_hash_state meow_hash_state;\ntypedef meow_hash meow_hash_implementation(meow_u64 Seed, meow_u64 Len, void *Source);\ntypedef void meow_absorb_implementation(struct meow_hash_state *State, meow_u64 Len, void *Source);\n\n#define MEOW_HASH_INTRINSICS_H\n#endif\n" }, { "path": "benchmarks/metrohash64.cpp", "content": "// metrohash64.cpp\n//\n// Copyright 2015-2018 J. Andrew Rogers\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"platform.h\"\n#include \"metrohash64.h\"\n\n#include \n\nconst char * MetroHash64::test_string = \"012345678901234567890123456789012345678901234567890123456789012\";\n\nconst uint8_t MetroHash64::test_seed_0[8] = { 0x6B, 0x75, 0x3D, 0xAE, 0x06, 0x70, 0x4B, 0xAD };\nconst uint8_t MetroHash64::test_seed_1[8] = { 0x3B, 0x0D, 0x48, 0x1C, 0xF4, 0xB9, 0xB8, 0xDF };\n\n\n\nMetroHash64::MetroHash64(const uint64_t seed)\n{\n Initialize(seed);\n}\n\n\nvoid MetroHash64::Initialize(const uint64_t seed)\n{\n vseed = (static_cast(seed) + k2) * k0;\n\n // initialize internal hash registers\n state.v[0] = vseed;\n state.v[1] = vseed;\n state.v[2] = vseed;\n state.v[3] = vseed;\n\n // initialize total length of input\n bytes = 0;\n}\n\n\nvoid MetroHash64::Update(const uint8_t * const buffer, const uint64_t length)\n{\n const uint8_t * ptr = reinterpret_cast(buffer);\n const uint8_t * const end = ptr + length;\n\n // input buffer may be partially filled\n if (bytes % 32)\n {\n uint64_t fill = 32 - (bytes % 32);\n if (fill > length)\n fill = length;\n\n memcpy(input.b + (bytes % 32), ptr, static_cast(fill));\n ptr += fill;\n bytes += fill;\n \n // input buffer is still partially filled\n if ((bytes % 32) != 0) return;\n\n // process full input buffer\n state.v[0] += read_u64(&input.b[ 0]) * k0; state.v[0] = rotate_right(state.v[0],29) + state.v[2];\n state.v[1] += read_u64(&input.b[ 8]) * k1; state.v[1] = rotate_right(state.v[1],29) + state.v[3];\n state.v[2] += read_u64(&input.b[16]) * k2; state.v[2] = rotate_right(state.v[2],29) + state.v[0];\n state.v[3] += read_u64(&input.b[24]) * k3; state.v[3] = rotate_right(state.v[3],29) + state.v[1];\n }\n \n // bulk update\n bytes += static_cast(end - ptr);\n while (ptr <= (end - 32))\n {\n // process directly from the source, bypassing the input buffer\n state.v[0] += read_u64(ptr) * k0; ptr += 8; state.v[0] = rotate_right(state.v[0],29) + state.v[2];\n state.v[1] += read_u64(ptr) * k1; ptr += 8; state.v[1] = rotate_right(state.v[1],29) + state.v[3];\n state.v[2] += read_u64(ptr) * k2; ptr += 8; state.v[2] = rotate_right(state.v[2],29) + state.v[0];\n state.v[3] += read_u64(ptr) * k3; ptr += 8; state.v[3] = rotate_right(state.v[3],29) + state.v[1];\n }\n \n // store remaining bytes in input buffer\n if (ptr < end)\n memcpy(input.b, ptr, static_cast(end - ptr));\n}\n\n\nvoid MetroHash64::Finalize(uint8_t * const hash)\n{\n // finalize bulk loop, if used\n if (bytes >= 32)\n {\n state.v[2] ^= rotate_right(((state.v[0] + state.v[3]) * k0) + state.v[1], 37) * k1;\n state.v[3] ^= rotate_right(((state.v[1] + state.v[2]) * k1) + state.v[0], 37) * k0;\n state.v[0] ^= rotate_right(((state.v[0] + state.v[2]) * k0) + state.v[3], 37) * k1;\n state.v[1] ^= rotate_right(((state.v[1] + state.v[3]) * k1) + state.v[2], 37) * k0;\n\n state.v[0] = vseed + (state.v[0] ^ state.v[1]);\n }\n \n // process any bytes remaining in the input buffer\n const uint8_t * ptr = reinterpret_cast(input.b);\n const uint8_t * const end = ptr + (bytes % 32);\n \n if ((end - ptr) >= 16)\n {\n state.v[1] = state.v[0] + (read_u64(ptr) * k2); ptr += 8; state.v[1] = rotate_right(state.v[1],29) * k3;\n state.v[2] = state.v[0] + (read_u64(ptr) * k2); ptr += 8; state.v[2] = rotate_right(state.v[2],29) * k3;\n state.v[1] ^= rotate_right(state.v[1] * k0, 21) + state.v[2];\n state.v[2] ^= rotate_right(state.v[2] * k3, 21) + state.v[1];\n state.v[0] += state.v[2];\n }\n\n if ((end - ptr) >= 8)\n {\n state.v[0] += read_u64(ptr) * k3; ptr += 8;\n state.v[0] ^= rotate_right(state.v[0], 55) * k1;\n }\n\n if ((end - ptr) >= 4)\n {\n state.v[0] += read_u32(ptr) * k3; ptr += 4;\n state.v[0] ^= rotate_right(state.v[0], 26) * k1;\n }\n\n if ((end - ptr) >= 2)\n {\n state.v[0] += read_u16(ptr) * k3; ptr += 2;\n state.v[0] ^= rotate_right(state.v[0], 48) * k1;\n }\n\n if ((end - ptr) >= 1)\n {\n state.v[0] += read_u8 (ptr) * k3;\n state.v[0] ^= rotate_right(state.v[0], 37) * k1;\n }\n \n state.v[0] ^= rotate_right(state.v[0], 28);\n state.v[0] *= k0;\n state.v[0] ^= rotate_right(state.v[0], 29);\n\n bytes = 0;\n\n // do any endian conversion here\n\n memcpy(hash, state.v, 8);\n}\n\n\nvoid MetroHash64::Hash(const uint8_t * buffer, const uint64_t length, uint8_t * const hash, const uint64_t seed)\n{\n const uint8_t * ptr = reinterpret_cast(buffer);\n const uint8_t * const end = ptr + length;\n\n uint64_t h = (static_cast(seed) + k2) * k0;\n\n if (length >= 32)\n {\n uint64_t v[4];\n v[0] = h;\n v[1] = h;\n v[2] = h;\n v[3] = h;\n\n do\n {\n v[0] += read_u64(ptr) * k0; ptr += 8; v[0] = rotate_right(v[0],29) + v[2];\n v[1] += read_u64(ptr) * k1; ptr += 8; v[1] = rotate_right(v[1],29) + v[3];\n v[2] += read_u64(ptr) * k2; ptr += 8; v[2] = rotate_right(v[2],29) + v[0];\n v[3] += read_u64(ptr) * k3; ptr += 8; v[3] = rotate_right(v[3],29) + v[1];\n }\n while (ptr <= (end - 32));\n\n v[2] ^= rotate_right(((v[0] + v[3]) * k0) + v[1], 37) * k1;\n v[3] ^= rotate_right(((v[1] + v[2]) * k1) + v[0], 37) * k0;\n v[0] ^= rotate_right(((v[0] + v[2]) * k0) + v[3], 37) * k1;\n v[1] ^= rotate_right(((v[1] + v[3]) * k1) + v[2], 37) * k0;\n h += v[0] ^ v[1];\n }\n\n if ((end - ptr) >= 16)\n {\n uint64_t v0 = h + (read_u64(ptr) * k2); ptr += 8; v0 = rotate_right(v0,29) * k3;\n uint64_t v1 = h + (read_u64(ptr) * k2); ptr += 8; v1 = rotate_right(v1,29) * k3;\n v0 ^= rotate_right(v0 * k0, 21) + v1;\n v1 ^= rotate_right(v1 * k3, 21) + v0;\n h += v1;\n }\n\n if ((end - ptr) >= 8)\n {\n h += read_u64(ptr) * k3; ptr += 8;\n h ^= rotate_right(h, 55) * k1;\n }\n\n if ((end - ptr) >= 4)\n {\n h += read_u32(ptr) * k3; ptr += 4;\n h ^= rotate_right(h, 26) * k1;\n }\n\n if ((end - ptr) >= 2)\n {\n h += read_u16(ptr) * k3; ptr += 2;\n h ^= rotate_right(h, 48) * k1;\n }\n\n if ((end - ptr) >= 1)\n {\n h += read_u8 (ptr) * k3;\n h ^= rotate_right(h, 37) * k1;\n }\n\n h ^= rotate_right(h, 28);\n h *= k0;\n h ^= rotate_right(h, 29);\n\n memcpy(hash, &h, 8);\n}\n\n\nbool MetroHash64::ImplementationVerified()\n{\n uint8_t hash[8];\n const uint8_t * key = reinterpret_cast(MetroHash64::test_string);\n\n // verify one-shot implementation\n MetroHash64::Hash(key, strlen(MetroHash64::test_string), hash, 0);\n if (memcmp(hash, MetroHash64::test_seed_0, 8) != 0) return false;\n\n MetroHash64::Hash(key, strlen(MetroHash64::test_string), hash, 1);\n if (memcmp(hash, MetroHash64::test_seed_1, 8) != 0) return false;\n\n // verify incremental implementation\n MetroHash64 metro;\n \n metro.Initialize(0);\n metro.Update(reinterpret_cast(MetroHash64::test_string), strlen(MetroHash64::test_string));\n metro.Finalize(hash);\n if (memcmp(hash, MetroHash64::test_seed_0, 8) != 0) return false;\n\n metro.Initialize(1);\n metro.Update(reinterpret_cast(MetroHash64::test_string), strlen(MetroHash64::test_string));\n metro.Finalize(hash);\n if (memcmp(hash, MetroHash64::test_seed_1, 8) != 0) return false;\n\n return true;\n}\n\n\nvoid metrohash64_1(const uint8_t * key, uint64_t len, uint32_t seed, uint8_t * out)\n{\n static const uint64_t k0 = 0xC83A91E1;\n static const uint64_t k1 = 0x8648DBDB;\n static const uint64_t k2 = 0x7BDEC03B;\n static const uint64_t k3 = 0x2F5870A5;\n\n const uint8_t * ptr = reinterpret_cast(key);\n const uint8_t * const end = ptr + len;\n \n uint64_t hash = ((static_cast(seed) + k2) * k0) + len;\n \n if (len >= 32)\n {\n uint64_t v[4];\n v[0] = hash;\n v[1] = hash;\n v[2] = hash;\n v[3] = hash;\n \n do\n {\n v[0] += read_u64(ptr) * k0; ptr += 8; v[0] = rotate_right(v[0],29) + v[2];\n v[1] += read_u64(ptr) * k1; ptr += 8; v[1] = rotate_right(v[1],29) + v[3];\n v[2] += read_u64(ptr) * k2; ptr += 8; v[2] = rotate_right(v[2],29) + v[0];\n v[3] += read_u64(ptr) * k3; ptr += 8; v[3] = rotate_right(v[3],29) + v[1];\n }\n while (ptr <= (end - 32));\n\n v[2] ^= rotate_right(((v[0] + v[3]) * k0) + v[1], 33) * k1;\n v[3] ^= rotate_right(((v[1] + v[2]) * k1) + v[0], 33) * k0;\n v[0] ^= rotate_right(((v[0] + v[2]) * k0) + v[3], 33) * k1;\n v[1] ^= rotate_right(((v[1] + v[3]) * k1) + v[2], 33) * k0;\n hash += v[0] ^ v[1];\n }\n \n if ((end - ptr) >= 16)\n {\n uint64_t v0 = hash + (read_u64(ptr) * k0); ptr += 8; v0 = rotate_right(v0,33) * k1;\n uint64_t v1 = hash + (read_u64(ptr) * k1); ptr += 8; v1 = rotate_right(v1,33) * k2;\n v0 ^= rotate_right(v0 * k0, 35) + v1;\n v1 ^= rotate_right(v1 * k3, 35) + v0;\n hash += v1;\n }\n \n if ((end - ptr) >= 8)\n {\n hash += read_u64(ptr) * k3; ptr += 8;\n hash ^= rotate_right(hash, 33) * k1;\n \n }\n \n if ((end - ptr) >= 4)\n {\n hash += read_u32(ptr) * k3; ptr += 4;\n hash ^= rotate_right(hash, 15) * k1;\n }\n \n if ((end - ptr) >= 2)\n {\n hash += read_u16(ptr) * k3; ptr += 2;\n hash ^= rotate_right(hash, 13) * k1;\n }\n \n if ((end - ptr) >= 1)\n {\n hash += read_u8 (ptr) * k3;\n hash ^= rotate_right(hash, 25) * k1;\n }\n \n hash ^= rotate_right(hash, 33);\n hash *= k0;\n hash ^= rotate_right(hash, 33);\n\n memcpy(out, &hash, 8);\n}\n\n\nvoid metrohash64_2(const uint8_t * key, uint64_t len, uint32_t seed, uint8_t * out)\n{\n static const uint64_t k0 = 0xD6D018F5;\n static const uint64_t k1 = 0xA2AA033B;\n static const uint64_t k2 = 0x62992FC1;\n static const uint64_t k3 = 0x30BC5B29; \n\n const uint8_t * ptr = reinterpret_cast(key);\n const uint8_t * const end = ptr + len;\n \n uint64_t hash = ((static_cast(seed) + k2) * k0) + len;\n \n if (len >= 32)\n {\n uint64_t v[4];\n v[0] = hash;\n v[1] = hash;\n v[2] = hash;\n v[3] = hash;\n \n do\n {\n v[0] += read_u64(ptr) * k0; ptr += 8; v[0] = rotate_right(v[0],29) + v[2];\n v[1] += read_u64(ptr) * k1; ptr += 8; v[1] = rotate_right(v[1],29) + v[3];\n v[2] += read_u64(ptr) * k2; ptr += 8; v[2] = rotate_right(v[2],29) + v[0];\n v[3] += read_u64(ptr) * k3; ptr += 8; v[3] = rotate_right(v[3],29) + v[1];\n }\n while (ptr <= (end - 32));\n\n v[2] ^= rotate_right(((v[0] + v[3]) * k0) + v[1], 30) * k1;\n v[3] ^= rotate_right(((v[1] + v[2]) * k1) + v[0], 30) * k0;\n v[0] ^= rotate_right(((v[0] + v[2]) * k0) + v[3], 30) * k1;\n v[1] ^= rotate_right(((v[1] + v[3]) * k1) + v[2], 30) * k0;\n hash += v[0] ^ v[1];\n }\n \n if ((end - ptr) >= 16)\n {\n uint64_t v0 = hash + (read_u64(ptr) * k2); ptr += 8; v0 = rotate_right(v0,29) * k3;\n uint64_t v1 = hash + (read_u64(ptr) * k2); ptr += 8; v1 = rotate_right(v1,29) * k3;\n v0 ^= rotate_right(v0 * k0, 34) + v1;\n v1 ^= rotate_right(v1 * k3, 34) + v0;\n hash += v1;\n }\n \n if ((end - ptr) >= 8)\n {\n hash += read_u64(ptr) * k3; ptr += 8;\n hash ^= rotate_right(hash, 36) * k1;\n }\n \n if ((end - ptr) >= 4)\n {\n hash += read_u32(ptr) * k3; ptr += 4;\n hash ^= rotate_right(hash, 15) * k1;\n }\n \n if ((end - ptr) >= 2)\n {\n hash += read_u16(ptr) * k3; ptr += 2;\n hash ^= rotate_right(hash, 15) * k1;\n }\n \n if ((end - ptr) >= 1)\n {\n hash += read_u8 (ptr) * k3;\n hash ^= rotate_right(hash, 23) * k1;\n }\n \n hash ^= rotate_right(hash, 28);\n hash *= k0;\n hash ^= rotate_right(hash, 29);\n\n memcpy(out, &hash, 8);\n}\n\n\n" }, { "path": "benchmarks/metrohash64.h", "content": "// metrohash64.h\n//\n// Copyright 2015-2018 J. Andrew Rogers\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef METROHASH_METROHASH_64_H\n#define METROHASH_METROHASH_64_H\n\n#include \n\nclass MetroHash64\n{\npublic:\n static const uint32_t bits = 64;\n\n // Constructor initializes the same as Initialize()\n MetroHash64(const uint64_t seed=0);\n \n // Initializes internal state for new hash with optional seed\n void Initialize(const uint64_t seed=0);\n \n // Update the hash state with a string of bytes. If the length\n // is sufficiently long, the implementation switches to a bulk\n // hashing algorithm directly on the argument buffer for speed.\n void Update(const uint8_t * buffer, const uint64_t length);\n \n // Constructs the final hash and writes it to the argument buffer.\n // After a hash is finalized, this instance must be Initialized()-ed\n // again or the behavior of Update() and Finalize() is undefined.\n void Finalize(uint8_t * const hash);\n \n // A non-incremental function implementation. This can be significantly\n // faster than the incremental implementation for some usage patterns.\n static void Hash(const uint8_t * buffer, const uint64_t length, uint8_t * const hash, const uint64_t seed=0);\n\n // Does implementation correctly execute test vectors?\n static bool ImplementationVerified();\n\n // test vectors -- Hash(test_string, seed=0) => test_seed_0\n static const char * test_string;\n static const uint8_t test_seed_0[8];\n static const uint8_t test_seed_1[8];\n\nprivate:\n static const uint64_t k0 = 0xD6D018F5;\n static const uint64_t k1 = 0xA2AA033B;\n static const uint64_t k2 = 0x62992FC1;\n static const uint64_t k3 = 0x30BC5B29;\n \n struct { uint64_t v[4]; } state;\n struct { uint8_t b[32]; } input;\n uint64_t bytes;\n uint64_t vseed;\n};\n\n\n// Legacy 64-bit hash functions -- do not use\nvoid metrohash64_1(const uint8_t * key, uint64_t len, uint32_t seed, uint8_t * out);\nvoid metrohash64_2(const uint8_t * key, uint64_t len, uint32_t seed, uint8_t * out);\n\n\n#endif // #ifndef METROHASH_METROHASH_64_H\n" }, { "path": "benchmarks/mum512-prng.h", "content": "/* Copyright (c) 2016 Vladimir Makarov \n\n Permission is hereby granted, free of charge, to any person\n obtaining a copy of this software and associated documentation\n files (the \"Software\"), to deal in the Software without\n restriction, including without limitation the rights to use, copy,\n modify, merge, publish, distribute, sublicense, and/or sell copies\n of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n\n The above copyright notice and this permission notice shall be\n included in all copies or substantial portions of the Software.\n\n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\n MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\n BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\n ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\n CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.\n*/\n\n/* Pseudo Random Number Generator (PRNG) based on MUM512 hash\n function. It might be a crypto level PRNG.\n\n To use a generator call `init_mum512_prng` first, then call\n `get_mum512_prn` as much as you want to get a new PRN. At the end\n of the PRNG use, call `finish_mum512_prng`. You can change the\n default seed by calling set_mum512_seed.\n\n The PRNG passes NIST Statistical Test Suite for Random and\n Pseudorandom Number Generators for Cryptographic Applications\n (version 2.2.1) with 1000 bitstreams each containing 1M bits.\n\n The generation of a new number takes about 62 CPU cycles on x86_64\n (Intel 4.2GHz i7-4790K), or speed of the generation is about 67M\n numbers per sec. */\n\n#ifndef __MUM512_PRNG__\n#define __MUM512_PRNG__\n\n#include \"mum512.h\"\n\n/* Default seed (initial state). */\nstatic uint64_t mum512_start_seed[4][2] = {\n {0Xc0e484a76bbdbf2bULL, 0Xea0a397c33eb3fafULL},\n {0Xf445f7c55f41d3e6ULL, 0X3bd8dc401433c0ddULL},\n {0Xf35d8101c6625902ULL, 0X4dfa7e49a5c0fbf9ULL},\n {0X69695bb2eec9914bULL, 0Xac3cb4a350e58879ULL},\n}; \n\nstatic struct {\n /* An index in the output used to generate the current PRN. */\n int ind;\n /* count from which the current state generated. */\n _mc_ti count;\n /* The current MUM512 state. */\n _mc_ti state[4];\n /* The digest used to generate the next 8 PRNs. */\n uint64_t output[8]; \n} mum512_prng_state;\n\nstatic inline void\ninit_mum512_prng (void) {\n int i;\n\n mum512_prng_state.ind = 0; mum512_prng_state.count = _mc_const (0, 0);\n for (i = 0; i < 4; i++)\n mum512_prng_state.state[i] = _mc_get (mum512_start_seed[i]);\n}\n\n/* Make the state equal to {SEED, 0, ..., 0}. */\nstatic inline void\nset_mum512_seed (uint32_t seed) {\n int i;\n \n mum512_prng_state.state[0] = _mc_const (seed, 0);\n for (i = 1; i < 4; i++)\n mum512_prng_state.state[i] = _mc_const (0, 0);\n}\n\n\nstatic inline uint64_t\nget_mum512_prn (void) {\n uint64_t res;\n \n if (mum512_prng_state.ind == 0) {\n mum512_prng_state.count = _mc_add (mum512_prng_state.count, _mc_const (0, 0));\n _mc_hash_aligned (mum512_prng_state.state, &mum512_prng_state.count, 16);\n _mc_mix (mum512_prng_state.state, mum512_prng_state.output);\n }\n res = mum512_prng_state.output[mum512_prng_state.ind];\n mum512_prng_state.ind = (mum512_prng_state.ind + 1) & 0x7;\n return res;\n}\n\nstatic inline void\nfinish_mum512_prng (void) {\n}\n\n#endif\n" }, { "path": "benchmarks/platform.h", "content": "// platform.h\n//\n// The MIT License (MIT)\n//\n// Copyright (c) 2015 J. Andrew Rogers\n//\n// Permission is hereby granted, free of charge, to any person obtaining a copy\n// of this software and associated documentation files (the \"Software\"), to deal\n// in the Software without restriction, including without limitation the rights\n// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n// copies of the Software, and to permit persons to whom the Software is\n// furnished to do so, subject to the following conditions:\n//\n// The above copyright notice and this permission notice shall be included in all\n// copies or substantial portions of the Software.\n//\n// THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n// SOFTWARE.\n//\n\n#ifndef METROHASH_PLATFORM_H\n#define METROHASH_PLATFORM_H\n\n#include \n\n// rotate right idiom recognized by most compilers\ninline static uint64_t rotate_right(uint64_t v, unsigned k)\n{\n return (v >> k) | (v << (64 - k));\n}\n\n// unaligned reads, fast and safe on Nehalem and later microarchitectures\ninline static uint64_t read_u64(const void * const ptr)\n{\n return static_cast(*reinterpret_cast(ptr));\n}\n\ninline static uint64_t read_u32(const void * const ptr)\n{\n return static_cast(*reinterpret_cast(ptr));\n}\n\ninline static uint64_t read_u16(const void * const ptr)\n{\n return static_cast(*reinterpret_cast(ptr));\n}\n\ninline static uint64_t read_u8 (const void * const ptr)\n{\n return static_cast(*reinterpret_cast(ptr));\n}\n\n\n#endif // #ifndef METROHASH_PLATFORM_H\n" }, { "path": "benchmarks/rapidhash.h", "content": "/*\r\n * rapidhash V3 - Very fast, high quality, platform-independent hashing algorithm.\r\n *\r\n * Based on 'wyhash', by Wang Yi \r\n * \r\n * Copyright (C) 2025 Nicolas De Carli\r\n *\r\n * Permission is hereby granted, free of charge, to any person obtaining a copy\r\n * of this software and associated documentation files (the \"Software\"), to deal\r\n * in the Software without restriction, including without limitation the rights\r\n * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\r\n * copies of the Software, and to permit persons to whom the Software is\r\n * furnished to do so, subject to the following conditions:\r\n *\r\n * The above copyright notice and this permission notice shall be included in all\r\n * copies or substantial portions of the Software.\r\n *\r\n * THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\r\n * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\r\n * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\r\n * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\r\n * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\r\n * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r\n * SOFTWARE.\r\n *\r\n * You can contact the author at:\r\n * - rapidhash source repository: https://github.com/Nicoshev/rapidhash\r\n */\r\n\r\n #pragma once\r\n \r\n/*\r\n * Includes.\r\n */\r\n #include \r\n #include \r\n #if defined(_MSC_VER)\r\n # include \r\n # if defined(_M_X64) && !defined(_M_ARM64EC)\r\n # pragma intrinsic(_umul128)\r\n # endif\r\n #endif\r\n \r\n /*\r\n * C/C++ macros.\r\n */\r\n \r\n #ifdef _MSC_VER\r\n # define RAPIDHASH_ALWAYS_INLINE __forceinline\r\n #elif defined(__GNUC__)\r\n # define RAPIDHASH_ALWAYS_INLINE inline __attribute__((__always_inline__))\r\n #else\r\n # define RAPIDHASH_ALWAYS_INLINE inline\r\n #endif\r\n \r\n #ifdef __cplusplus\r\n # define RAPIDHASH_NOEXCEPT noexcept\r\n # define RAPIDHASH_CONSTEXPR constexpr\r\n # ifndef RAPIDHASH_INLINE\r\n # define RAPIDHASH_INLINE RAPIDHASH_ALWAYS_INLINE\r\n # endif\r\n # if __cplusplus >= 201402L && !defined(_MSC_VER)\r\n # define RAPIDHASH_INLINE_CONSTEXPR RAPIDHASH_ALWAYS_INLINE constexpr\r\n # else\r\n # define RAPIDHASH_INLINE_CONSTEXPR RAPIDHASH_ALWAYS_INLINE\r\n # endif\r\n #else\r\n # define RAPIDHASH_NOEXCEPT\r\n # define RAPIDHASH_CONSTEXPR static const\r\n # ifndef RAPIDHASH_INLINE\r\n # define RAPIDHASH_INLINE static RAPIDHASH_ALWAYS_INLINE\r\n # endif\r\n # define RAPIDHASH_INLINE_CONSTEXPR RAPIDHASH_INLINE\r\n #endif\r\n\r\n /*\r\n * Unrolled macro.\r\n * Improves large input speed, but increases code size and worsens small input speed.\r\n *\r\n * RAPIDHASH_COMPACT: Normal behavior.\r\n * RAPIDHASH_UNROLLED: \r\n *\r\n */\r\n #ifndef RAPIDHASH_UNROLLED\r\n # define RAPIDHASH_COMPACT\r\n #elif defined(RAPIDHASH_COMPACT)\r\n # error \"cannot define RAPIDHASH_COMPACT and RAPIDHASH_UNROLLED simultaneously.\"\r\n #endif\r\n \r\n /*\r\n * Protection macro, alters behaviour of rapid_mum multiplication function.\r\n *\r\n * RAPIDHASH_FAST: Normal behavior, max speed.\r\n * RAPIDHASH_PROTECTED: Extra protection against entropy loss.\r\n */\r\n #ifndef RAPIDHASH_PROTECTED\r\n # define RAPIDHASH_FAST\r\n #elif defined(RAPIDHASH_FAST)\r\n # error \"cannot define RAPIDHASH_PROTECTED and RAPIDHASH_FAST simultaneously.\"\r\n #endif\r\n \r\n /*\r\n * Likely and unlikely macros.\r\n */\r\n #if defined(__GNUC__) || defined(__INTEL_COMPILER) || defined(__clang__)\r\n # define _likely_(x) __builtin_expect(x,1)\r\n # define _unlikely_(x) __builtin_expect(x,0)\r\n #else\r\n # define _likely_(x) (x)\r\n # define _unlikely_(x) (x)\r\n #endif\r\n \r\n /*\r\n * Endianness macros.\r\n */\r\n #ifndef RAPIDHASH_LITTLE_ENDIAN\r\n # if defined(_WIN32) || defined(__LITTLE_ENDIAN__) || (defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__)\r\n # define RAPIDHASH_LITTLE_ENDIAN\r\n # elif defined(__BIG_ENDIAN__) || (defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__)\r\n # define RAPIDHASH_BIG_ENDIAN\r\n # else\r\n # warning \"could not determine endianness! Falling back to little endian.\"\r\n # define RAPIDHASH_LITTLE_ENDIAN\r\n # endif\r\n #endif\r\n \r\n /*\r\n * Default secret parameters.\r\n */\r\n RAPIDHASH_CONSTEXPR uint64_t rapid_secret[8] = {\r\n 0x2d358dccaa6c78a5ull,\r\n 0x8bb84b93962eacc9ull,\r\n 0x4b33a62ed433d4a3ull,\r\n 0x4d5a2da51de1aa47ull,\r\n 0xa0761d6478bd642full,\r\n 0xe7037ed1a0b428dbull,\r\n 0x90ed1765281c388cull,\r\n 0xaaaaaaaaaaaaaaaaull};\r\n \r\n /*\r\n * 64*64 -> 128bit multiply function.\r\n *\r\n * @param A Address of 64-bit number.\r\n * @param B Address of 64-bit number.\r\n *\r\n * Calculates 128-bit C = *A * *B.\r\n *\r\n * When RAPIDHASH_FAST is defined:\r\n * Overwrites A contents with C's low 64 bits.\r\n * Overwrites B contents with C's high 64 bits.\r\n *\r\n * When RAPIDHASH_PROTECTED is defined:\r\n * Xors and overwrites A contents with C's low 64 bits.\r\n * Xors and overwrites B contents with C's high 64 bits.\r\n */\r\n RAPIDHASH_INLINE_CONSTEXPR void rapid_mum(uint64_t *A, uint64_t *B) RAPIDHASH_NOEXCEPT {\r\n #if defined(__SIZEOF_INT128__)\r\n __uint128_t r=*A; r*=*B;\r\n #ifdef RAPIDHASH_PROTECTED\r\n *A^=(uint64_t)r; *B^=(uint64_t)(r>>64);\r\n #else\r\n *A=(uint64_t)r; *B=(uint64_t)(r>>64);\r\n #endif\r\n #elif defined(_MSC_VER) && (defined(_WIN64) || defined(_M_HYBRID_CHPE_ARM64))\r\n #if defined(_M_X64)\r\n #ifdef RAPIDHASH_PROTECTED\r\n uint64_t a, b;\r\n a=_umul128(*A,*B,&b);\r\n *A^=a; *B^=b;\r\n #else\r\n *A=_umul128(*A,*B,B);\r\n #endif\r\n #else\r\n #ifdef RAPIDHASH_PROTECTED\r\n uint64_t a, b;\r\n b = __umulh(*A, *B);\r\n a = *A * *B;\r\n *A^=a; *B^=b;\r\n #else\r\n uint64_t c = __umulh(*A, *B);\r\n *A = *A * *B;\r\n *B = c;\r\n #endif\r\n #endif\r\n #else\r\n uint64_t ha=*A>>32, hb=*B>>32, la=(uint32_t)*A, lb=(uint32_t)*B;\r\n uint64_t rh=ha*hb, rm0=ha*lb, rm1=hb*la, rl=la*lb, t=rl+(rm0<<32), c=t>32)+(rm1>>32)+c;\r\n #ifdef RAPIDHASH_PROTECTED\r\n *A^=lo; *B^=hi;\r\n #else\r\n *A=lo; *B=hi;\r\n #endif\r\n #endif\r\n }\r\n \r\n /*\r\n * Multiply and xor mix function.\r\n *\r\n * @param A 64-bit number.\r\n * @param B 64-bit number.\r\n *\r\n * Calculates 128-bit C = A * B.\r\n * Returns 64-bit xor between high and low 64 bits of C.\r\n */\r\n RAPIDHASH_INLINE_CONSTEXPR uint64_t rapid_mix(uint64_t A, uint64_t B) RAPIDHASH_NOEXCEPT { rapid_mum(&A,&B); return A^B; }\r\n \r\n /*\r\n * Read functions.\r\n */\r\n #ifdef RAPIDHASH_LITTLE_ENDIAN\r\n RAPIDHASH_INLINE uint64_t rapid_read64(const uint8_t *p) RAPIDHASH_NOEXCEPT { uint64_t v; memcpy(&v, p, sizeof(uint64_t)); return v;}\r\n RAPIDHASH_INLINE uint64_t rapid_read32(const uint8_t *p) RAPIDHASH_NOEXCEPT { uint32_t v; memcpy(&v, p, sizeof(uint32_t)); return v;}\r\n #elif defined(__GNUC__) || defined(__INTEL_COMPILER) || defined(__clang__)\r\n RAPIDHASH_INLINE uint64_t rapid_read64(const uint8_t *p) RAPIDHASH_NOEXCEPT { uint64_t v; memcpy(&v, p, sizeof(uint64_t)); return __builtin_bswap64(v);}\r\n RAPIDHASH_INLINE uint64_t rapid_read32(const uint8_t *p) RAPIDHASH_NOEXCEPT { uint32_t v; memcpy(&v, p, sizeof(uint32_t)); return __builtin_bswap32(v);}\r\n #elif defined(_MSC_VER)\r\n RAPIDHASH_INLINE uint64_t rapid_read64(const uint8_t *p) RAPIDHASH_NOEXCEPT { uint64_t v; memcpy(&v, p, sizeof(uint64_t)); return _byteswap_uint64(v);}\r\n RAPIDHASH_INLINE uint64_t rapid_read32(const uint8_t *p) RAPIDHASH_NOEXCEPT { uint32_t v; memcpy(&v, p, sizeof(uint32_t)); return _byteswap_ulong(v);}\r\n #else\r\n RAPIDHASH_INLINE uint64_t rapid_read64(const uint8_t *p) RAPIDHASH_NOEXCEPT {\r\n uint64_t v; memcpy(&v, p, 8);\r\n return (((v >> 56) & 0xff)| ((v >> 40) & 0xff00)| ((v >> 24) & 0xff0000)| ((v >> 8) & 0xff000000)| ((v << 8) & 0xff00000000)| ((v << 24) & 0xff0000000000)| ((v << 40) & 0xff000000000000)| ((v << 56) & 0xff00000000000000));\r\n }\r\n RAPIDHASH_INLINE uint64_t rapid_read32(const uint8_t *p) RAPIDHASH_NOEXCEPT {\r\n uint32_t v; memcpy(&v, p, 4);\r\n return (((v >> 24) & 0xff)| ((v >> 8) & 0xff00)| ((v << 8) & 0xff0000)| ((v << 24) & 0xff000000));\r\n }\r\n #endif\r\n \r\n /*\r\n * rapidhash main function.\r\n *\r\n * @param key Buffer to be hashed.\r\n * @param len @key length, in bytes.\r\n * @param seed 64-bit seed used to alter the hash result predictably.\r\n * @param secret Triplet of 64-bit secrets used to alter hash result predictably.\r\n *\r\n * Returns a 64-bit hash.\r\n */\r\nRAPIDHASH_INLINE_CONSTEXPR uint64_t rapidhash_internal(const void *key, size_t len, uint64_t seed, const uint64_t* secret) RAPIDHASH_NOEXCEPT {\r\n const uint8_t *p=(const uint8_t *)key;\r\n seed ^= rapid_mix(seed ^ secret[2], secret[1]);\r\n uint64_t a=0, b=0;\r\n size_t i = len;\r\n if (_likely_(len <= 16)) {\r\n if (len >= 4) {\r\n seed ^= len;\r\n if (len >= 8) {\r\n const uint8_t* plast = p + len - 8;\r\n a = rapid_read64(p);\r\n b = rapid_read64(plast);\r\n } else {\r\n const uint8_t* plast = p + len - 4;\r\n a = rapid_read32(p);\r\n b = rapid_read32(plast);\r\n }\r\n } else if (len > 0) {\r\n a = (((uint64_t)p[0])<<45)|p[len-1];\r\n b = p[len>>1];\r\n } else\r\n a = b = 0;\r\n } else {\r\n if (len > 112) {\r\n uint64_t see1 = seed, see2 = seed;\r\n uint64_t see3 = seed, see4 = seed;\r\n uint64_t see5 = seed, see6 = seed;\r\n#ifdef RAPIDHASH_COMPACT\r\n do {\r\n seed = rapid_mix(rapid_read64(p) ^ secret[0], rapid_read64(p + 8) ^ seed);\r\n see1 = rapid_mix(rapid_read64(p + 16) ^ secret[1], rapid_read64(p + 24) ^ see1);\r\n see2 = rapid_mix(rapid_read64(p + 32) ^ secret[2], rapid_read64(p + 40) ^ see2);\r\n see3 = rapid_mix(rapid_read64(p + 48) ^ secret[3], rapid_read64(p + 56) ^ see3);\r\n see4 = rapid_mix(rapid_read64(p + 64) ^ secret[4], rapid_read64(p + 72) ^ see4);\r\n see5 = rapid_mix(rapid_read64(p + 80) ^ secret[5], rapid_read64(p + 88) ^ see5);\r\n see6 = rapid_mix(rapid_read64(p + 96) ^ secret[6], rapid_read64(p + 104) ^ see6);\r\n p += 112;\r\n i -= 112;\r\n } while(i > 112);\r\n#else\r\n while (i > 224) {\r\n seed = rapid_mix(rapid_read64(p) ^ secret[0], rapid_read64(p + 8) ^ seed);\r\n see1 = rapid_mix(rapid_read64(p + 16) ^ secret[1], rapid_read64(p + 24) ^ see1);\r\n see2 = rapid_mix(rapid_read64(p + 32) ^ secret[2], rapid_read64(p + 40) ^ see2);\r\n see3 = rapid_mix(rapid_read64(p + 48) ^ secret[3], rapid_read64(p + 56) ^ see3);\r\n see4 = rapid_mix(rapid_read64(p + 64) ^ secret[4], rapid_read64(p + 72) ^ see4);\r\n see5 = rapid_mix(rapid_read64(p + 80) ^ secret[5], rapid_read64(p + 88) ^ see5);\r\n see6 = rapid_mix(rapid_read64(p + 96) ^ secret[6], rapid_read64(p + 104) ^ see6);\r\n seed = rapid_mix(rapid_read64(p + 112) ^ secret[0], rapid_read64(p + 120) ^ seed);\r\n see1 = rapid_mix(rapid_read64(p + 128) ^ secret[1], rapid_read64(p + 136) ^ see1);\r\n see2 = rapid_mix(rapid_read64(p + 144) ^ secret[2], rapid_read64(p + 152) ^ see2);\r\n see3 = rapid_mix(rapid_read64(p + 160) ^ secret[3], rapid_read64(p + 168) ^ see3);\r\n see4 = rapid_mix(rapid_read64(p + 176) ^ secret[4], rapid_read64(p + 184) ^ see4);\r\n see5 = rapid_mix(rapid_read64(p + 192) ^ secret[5], rapid_read64(p + 200) ^ see5);\r\n see6 = rapid_mix(rapid_read64(p + 208) ^ secret[6], rapid_read64(p + 216) ^ see6);\r\n p += 224;\r\n i -= 224;\r\n }\r\n if (i > 112) {\r\n seed = rapid_mix(rapid_read64(p) ^ secret[0], rapid_read64(p + 8) ^ seed);\r\n see1 = rapid_mix(rapid_read64(p + 16) ^ secret[1], rapid_read64(p + 24) ^ see1);\r\n see2 = rapid_mix(rapid_read64(p + 32) ^ secret[2], rapid_read64(p + 40) ^ see2);\r\n see3 = rapid_mix(rapid_read64(p + 48) ^ secret[3], rapid_read64(p + 56) ^ see3);\r\n see4 = rapid_mix(rapid_read64(p + 64) ^ secret[4], rapid_read64(p + 72) ^ see4);\r\n see5 = rapid_mix(rapid_read64(p + 80) ^ secret[5], rapid_read64(p + 88) ^ see5);\r\n see6 = rapid_mix(rapid_read64(p + 96) ^ secret[6], rapid_read64(p + 104) ^ see6);\r\n p += 112;\r\n i -= 112;\r\n }\r\n#endif\r\n seed ^= see1;\r\n see2 ^= see3;\r\n see4 ^= see5;\r\n seed ^= see6;\r\n see2 ^= see4;\r\n seed ^= see2;\r\n }\r\n if (i > 16) {\r\n seed = rapid_mix(rapid_read64(p) ^ secret[2], rapid_read64(p + 8) ^ seed);\r\n if (i > 32) {\r\n seed = rapid_mix(rapid_read64(p + 16) ^ secret[2], rapid_read64(p + 24) ^ seed);\r\n if (i > 48) {\r\n seed = rapid_mix(rapid_read64(p + 32) ^ secret[1], rapid_read64(p + 40) ^ seed);\r\n if (i > 64) {\r\n seed = rapid_mix(rapid_read64(p + 48) ^ secret[1], rapid_read64(p + 56) ^ seed);\r\n if (i > 80) {\r\n seed = rapid_mix(rapid_read64(p + 64) ^ secret[2], rapid_read64(p + 72) ^ seed);\r\n if (i > 96) {\r\n seed = rapid_mix(rapid_read64(p + 80) ^ secret[1], rapid_read64(p + 88) ^ seed);\r\n }\r\n }\r\n }\r\n }\r\n }\r\n }\r\n a=rapid_read64(p+i-16) ^ i; b=rapid_read64(p+i-8);\r\n }\r\n a ^= secret[1];\r\n b ^= seed;\r\n rapid_mum(&a, &b);\r\n return rapid_mix(a ^ secret[7], b ^ secret[1] ^ i);\r\n}\r\n\r\n /*\r\n * rapidhashMicro main function.\r\n *\r\n * @param key Buffer to be hashed.\r\n * @param len @key length, in bytes.\r\n * @param seed 64-bit seed used to alter the hash result predictably.\r\n * @param secret Triplet of 64-bit secrets used to alter hash result predictably.\r\n *\r\n * Returns a 64-bit hash.\r\n */\r\n RAPIDHASH_INLINE_CONSTEXPR uint64_t rapidhashMicro_internal(const void *key, size_t len, uint64_t seed, const uint64_t* secret) RAPIDHASH_NOEXCEPT {\r\n const uint8_t *p=(const uint8_t *)key;\r\n seed ^= rapid_mix(seed ^ secret[2], secret[1]);\r\n uint64_t a=0, b=0;\r\n size_t i = len;\r\n if (_likely_(len <= 16)) {\r\n if (len >= 4) {\r\n seed ^= len;\r\n if (len >= 8) {\r\n const uint8_t* plast = p + len - 8;\r\n a = rapid_read64(p);\r\n b = rapid_read64(plast);\r\n } else {\r\n const uint8_t* plast = p + len - 4;\r\n a = rapid_read32(p);\r\n b = rapid_read32(plast);\r\n }\r\n } else if (len > 0) {\r\n a = (((uint64_t)p[0])<<45)|p[len-1];\r\n b = p[len>>1];\r\n } else\r\n a = b = 0;\r\n } else {\r\n if (i > 80) {\r\n uint64_t see1 = seed, see2 = seed;\r\n uint64_t see3 = seed, see4 = seed;\r\n do {\r\n seed = rapid_mix(rapid_read64(p) ^ secret[0], rapid_read64(p + 8) ^ seed);\r\n see1 = rapid_mix(rapid_read64(p + 16) ^ secret[1], rapid_read64(p + 24) ^ see1);\r\n see2 = rapid_mix(rapid_read64(p + 32) ^ secret[2], rapid_read64(p + 40) ^ see2);\r\n see3 = rapid_mix(rapid_read64(p + 48) ^ secret[3], rapid_read64(p + 56) ^ see3);\r\n see4 = rapid_mix(rapid_read64(p + 64) ^ secret[4], rapid_read64(p + 72) ^ see4);\r\n p += 80;\r\n i -= 80;\r\n } while(i > 80);\r\n seed ^= see1;\r\n see2 ^= see3;\r\n seed ^= see4;\r\n seed ^= see2;\r\n }\r\n if (i > 16) {\r\n seed = rapid_mix(rapid_read64(p) ^ secret[2], rapid_read64(p + 8) ^ seed);\r\n if (i > 32) {\r\n seed = rapid_mix(rapid_read64(p + 16) ^ secret[2], rapid_read64(p + 24) ^ seed);\r\n if (i > 48) {\r\n seed = rapid_mix(rapid_read64(p + 32) ^ secret[1], rapid_read64(p + 40) ^ seed);\r\n if (i > 64) {\r\n seed = rapid_mix(rapid_read64(p + 48) ^ secret[1], rapid_read64(p + 56) ^ seed);\r\n }\r\n }\r\n }\r\n }\r\n a=rapid_read64(p+i-16) ^ i; b=rapid_read64(p+i-8);\r\n }\r\n a ^= secret[1];\r\n b ^= seed;\r\n rapid_mum(&a, &b);\r\n return rapid_mix(a ^ secret[7], b ^ secret[1] ^ i);\r\n }\r\n\r\n /*\r\n * rapidhashNano main function.\r\n *\r\n * @param key Buffer to be hashed.\r\n * @param len @key length, in bytes.\r\n * @param seed 64-bit seed used to alter the hash result predictably.\r\n * @param secret Triplet of 64-bit secrets used to alter hash result predictably.\r\n *\r\n * Returns a 64-bit hash.\r\n */\r\n RAPIDHASH_INLINE_CONSTEXPR uint64_t rapidhashNano_internal(const void *key, size_t len, uint64_t seed, const uint64_t* secret) RAPIDHASH_NOEXCEPT {\r\n const uint8_t *p=(const uint8_t *)key;\r\n seed ^= rapid_mix(seed ^ secret[2], secret[1]);\r\n uint64_t a=0, b=0;\r\n size_t i = len;\r\n if (_likely_(len <= 16)) {\r\n if (len >= 4) {\r\n seed ^= len;\r\n if (len >= 8) {\r\n const uint8_t* plast = p + len - 8;\r\n a = rapid_read64(p);\r\n b = rapid_read64(plast);\r\n } else {\r\n const uint8_t* plast = p + len - 4;\r\n a = rapid_read32(p);\r\n b = rapid_read32(plast);\r\n }\r\n } else if (len > 0) {\r\n a = (((uint64_t)p[0])<<45)|p[len-1];\r\n b = p[len>>1];\r\n } else\r\n a = b = 0;\r\n } else {\r\n if (i > 48) {\r\n uint64_t see1 = seed, see2 = seed;\r\n do {\r\n seed = rapid_mix(rapid_read64(p) ^ secret[0], rapid_read64(p + 8) ^ seed);\r\n see1 = rapid_mix(rapid_read64(p + 16) ^ secret[1], rapid_read64(p + 24) ^ see1);\r\n see2 = rapid_mix(rapid_read64(p + 32) ^ secret[2], rapid_read64(p + 40) ^ see2);\r\n p += 48;\r\n i -= 48;\r\n } while(i > 48);\r\n seed ^= see1;\r\n seed ^= see2;\r\n }\r\n if (i > 16) {\r\n seed = rapid_mix(rapid_read64(p) ^ secret[2], rapid_read64(p + 8) ^ seed);\r\n if (i > 32) {\r\n seed = rapid_mix(rapid_read64(p + 16) ^ secret[2], rapid_read64(p + 24) ^ seed);\r\n }\r\n }\r\n a=rapid_read64(p+i-16) ^ i; b=rapid_read64(p+i-8);\r\n }\r\n a ^= secret[1];\r\n b ^= seed;\r\n rapid_mum(&a, &b);\r\n return rapid_mix(a ^ secret[7], b ^ secret[1] ^ i);\r\n }\r\n \r\n/*\r\n * rapidhash seeded hash function.\r\n *\r\n * @param key Buffer to be hashed.\r\n * @param len @key length, in bytes.\r\n * @param seed 64-bit seed used to alter the hash result predictably.\r\n *\r\n * Calls rapidhash_internal using provided parameters and default secrets.\r\n *\r\n * Returns a 64-bit hash.\r\n */\r\nRAPIDHASH_INLINE_CONSTEXPR uint64_t rapidhash_withSeed(const void *key, size_t len, uint64_t seed) RAPIDHASH_NOEXCEPT {\r\n return rapidhash_internal(key, len, seed, rapid_secret);\r\n}\r\n \r\n/*\r\n * rapidhash general purpose hash function.\r\n *\r\n * @param key Buffer to be hashed.\r\n * @param len @key length, in bytes.\r\n *\r\n * Calls rapidhash_withSeed using provided parameters and the default seed.\r\n *\r\n * Returns a 64-bit hash.\r\n */\r\nRAPIDHASH_INLINE_CONSTEXPR uint64_t rapidhash(const void *key, size_t len) RAPIDHASH_NOEXCEPT {\r\n return rapidhash_withSeed(key, len, 0);\r\n}\r\n\r\n/*\r\n * rapidhashMicro seeded hash function.\r\n *\r\n * Designed for HPC and server applications, where cache misses make a noticeable performance detriment.\r\n * Clang-18+ compiles it to ~140 instructions without stack usage, both on x86-64 and aarch64.\r\n * Faster for sizes up to 512 bytes, just 15%-20% slower for inputs above 1kb.\r\n *\r\n * @param key Buffer to be hashed.\r\n * @param len @key length, in bytes.\r\n * @param seed 64-bit seed used to alter the hash result predictably.\r\n *\r\n * Calls rapidhash_internal using provided parameters and default secrets.\r\n *\r\n * Returns a 64-bit hash.\r\n */\r\n RAPIDHASH_INLINE_CONSTEXPR uint64_t rapidhashMicro_withSeed(const void *key, size_t len, uint64_t seed) RAPIDHASH_NOEXCEPT {\r\n return rapidhashMicro_internal(key, len, seed, rapid_secret);\r\n}\r\n \r\n/*\r\n * rapidhashMicro hash function.\r\n *\r\n * @param key Buffer to be hashed.\r\n * @param len @key length, in bytes.\r\n *\r\n * Calls rapidhash_withSeed using provided parameters and the default seed.\r\n *\r\n * Returns a 64-bit hash.\r\n */\r\nRAPIDHASH_INLINE_CONSTEXPR uint64_t rapidhashMicro(const void *key, size_t len) RAPIDHASH_NOEXCEPT {\r\n return rapidhashMicro_withSeed(key, len, 0);\r\n}\r\n\r\n/*\r\n * rapidhashNano seeded hash function.\r\n *\r\n * @param key Buffer to be hashed.\r\n * @param len @key length, in bytes.\r\n * @param seed 64-bit seed used to alter the hash result predictably.\r\n *\r\n * Calls rapidhash_internal using provided parameters and default secrets.\r\n *\r\n * Returns a 64-bit hash.\r\n */\r\n RAPIDHASH_INLINE_CONSTEXPR uint64_t rapidhashNano_withSeed(const void *key, size_t len, uint64_t seed) RAPIDHASH_NOEXCEPT {\r\n return rapidhashNano_internal(key, len, seed, rapid_secret);\r\n}\r\n \r\n/*\r\n * rapidhashNano hash function.\r\n *\r\n * Designed for Mobile and embedded applications, where keeping a small code size is a top priority.\r\n * Clang-18+ compiles it to less than 100 instructions without stack usage, both on x86-64 and aarch64.\r\n * The fastest for sizes up to 48 bytes, but may be considerably slower for larger inputs.\r\n *\r\n * @param key Buffer to be hashed.\r\n * @param len @key length, in bytes.\r\n *\r\n * Calls rapidhash_withSeed using provided parameters and the default seed.\r\n *\r\n * Returns a 64-bit hash.\r\n */\r\nRAPIDHASH_INLINE_CONSTEXPR uint64_t rapidhashNano(const void *key, size_t len) RAPIDHASH_NOEXCEPT {\r\n return rapidhashNano_withSeed(key, len, 0);\r\n}\r\n" }, { "path": "benchmarks/sha3.c", "content": "/* sha3.c - an implementation of Secure Hash Algorithm 3 (Keccak).\n * based on the\n * The Keccak SHA-3 submission. Submission to NIST (Round 3), 2011\n * by Guido Bertoni, Joan Daemen, Michaël Peeters and Gilles Van Assche\n *\n * Copyright: 2013 Aleksey Kravchenko \n *\n * Permission is hereby granted, free of charge, to any person obtaining a\n * copy of this software and associated documentation files (the \"Software\"),\n * to deal in the Software without restriction, including without limitation\n * the rights to use, copy, modify, merge, publish, distribute, sublicense,\n * and/or sell copies of the Software, and to permit persons to whom the\n * Software is furnished to do so.\n *\n * This program is distributed in the hope that it will be useful, but\n * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY\n * or FITNESS FOR A PARTICULAR PURPOSE. Use this program at your own risk!\n */\n\n#include \n#include \n#include \"byte_order.h\"\n#include \"sha3.h\"\n\n/* constants */\n#define NumberOfRounds 24\n\n/* SHA3 (Keccak) constants for 24 rounds */\nstatic uint64_t keccak_round_constants[NumberOfRounds] = {\n\tI64(0x0000000000000001), I64(0x0000000000008082), I64(0x800000000000808A), I64(0x8000000080008000),\n\tI64(0x000000000000808B), I64(0x0000000080000001), I64(0x8000000080008081), I64(0x8000000000008009),\n\tI64(0x000000000000008A), I64(0x0000000000000088), I64(0x0000000080008009), I64(0x000000008000000A),\n\tI64(0x000000008000808B), I64(0x800000000000008B), I64(0x8000000000008089), I64(0x8000000000008003),\n\tI64(0x8000000000008002), I64(0x8000000000000080), I64(0x000000000000800A), I64(0x800000008000000A),\n\tI64(0x8000000080008081), I64(0x8000000000008080), I64(0x0000000080000001), I64(0x8000000080008008)\n};\n\n/* Initializing a sha3 context for given number of output bits */\nstatic void rhash_keccak_init(sha3_ctx *ctx, unsigned bits)\n{\n\t/* NB: The Keccak capacity parameter = bits * 2 */\n\tunsigned rate = 1600 - bits * 2;\n\n\tmemset(ctx, 0, sizeof(sha3_ctx));\n\tctx->block_size = rate / 8;\n\tassert(rate <= 1600 && (rate % 64) == 0);\n}\n\n/**\n * Initialize context before calculating hash.\n *\n * @param ctx context to initialize\n */\nvoid rhash_sha3_224_init(sha3_ctx *ctx)\n{\n\trhash_keccak_init(ctx, 224);\n}\n\n/**\n * Initialize context before calculating hash.\n *\n * @param ctx context to initialize\n */\nvoid rhash_sha3_256_init(sha3_ctx *ctx)\n{\n\trhash_keccak_init(ctx, 256);\n}\n\n/**\n * Initialize context before calculating hash.\n *\n * @param ctx context to initialize\n */\nvoid rhash_sha3_384_init(sha3_ctx *ctx)\n{\n\trhash_keccak_init(ctx, 384);\n}\n\n/**\n * Initialize context before calculating hash.\n *\n * @param ctx context to initialize\n */\nvoid rhash_sha3_512_init(sha3_ctx *ctx)\n{\n\trhash_keccak_init(ctx, 512);\n}\n\n/* Keccak theta() transformation */\nstatic void keccak_theta(uint64_t *A)\n{\n\tunsigned int x;\n\tuint64_t C[5], D[5];\n\n\tfor (x = 0; x < 5; x++) {\n\t\tC[x] = A[x] ^ A[x + 5] ^ A[x + 10] ^ A[x + 15] ^ A[x + 20];\n\t}\n\tD[0] = ROTL64(C[1], 1) ^ C[4];\n\tD[1] = ROTL64(C[2], 1) ^ C[0];\n\tD[2] = ROTL64(C[3], 1) ^ C[1];\n\tD[3] = ROTL64(C[4], 1) ^ C[2];\n\tD[4] = ROTL64(C[0], 1) ^ C[3];\n\n\tfor (x = 0; x < 5; x++) {\n\t\tA[x] ^= D[x];\n\t\tA[x + 5] ^= D[x];\n\t\tA[x + 10] ^= D[x];\n\t\tA[x + 15] ^= D[x];\n\t\tA[x + 20] ^= D[x];\n\t}\n}\n\n/* Keccak pi() transformation */\nstatic void keccak_pi(uint64_t *A)\n{\n\tuint64_t A1;\n\tA1 = A[1];\n\tA[ 1] = A[ 6];\n\tA[ 6] = A[ 9];\n\tA[ 9] = A[22];\n\tA[22] = A[14];\n\tA[14] = A[20];\n\tA[20] = A[ 2];\n\tA[ 2] = A[12];\n\tA[12] = A[13];\n\tA[13] = A[19];\n\tA[19] = A[23];\n\tA[23] = A[15];\n\tA[15] = A[ 4];\n\tA[ 4] = A[24];\n\tA[24] = A[21];\n\tA[21] = A[ 8];\n\tA[ 8] = A[16];\n\tA[16] = A[ 5];\n\tA[ 5] = A[ 3];\n\tA[ 3] = A[18];\n\tA[18] = A[17];\n\tA[17] = A[11];\n\tA[11] = A[ 7];\n\tA[ 7] = A[10];\n\tA[10] = A1;\n\t/* note: A[ 0] is left as is */\n}\n\n/* Keccak chi() transformation */\nstatic void keccak_chi(uint64_t *A)\n{\n\tint i;\n\tfor (i = 0; i < 25; i += 5) {\n\t\tuint64_t A0 = A[0 + i], A1 = A[1 + i];\n\t\tA[0 + i] ^= ~A1 & A[2 + i];\n\t\tA[1 + i] ^= ~A[2 + i] & A[3 + i];\n\t\tA[2 + i] ^= ~A[3 + i] & A[4 + i];\n\t\tA[3 + i] ^= ~A[4 + i] & A0;\n\t\tA[4 + i] ^= ~A0 & A1;\n\t}\n}\n\nstatic void rhash_sha3_permutation(uint64_t *state)\n{\n\tint round;\n\tfor (round = 0; round < NumberOfRounds; round++)\n\t{\n\t\tkeccak_theta(state);\n\n\t\t/* apply Keccak rho() transformation */\n\t\tstate[ 1] = ROTL64(state[ 1], 1);\n\t\tstate[ 2] = ROTL64(state[ 2], 62);\n\t\tstate[ 3] = ROTL64(state[ 3], 28);\n\t\tstate[ 4] = ROTL64(state[ 4], 27);\n\t\tstate[ 5] = ROTL64(state[ 5], 36);\n\t\tstate[ 6] = ROTL64(state[ 6], 44);\n\t\tstate[ 7] = ROTL64(state[ 7], 6);\n\t\tstate[ 8] = ROTL64(state[ 8], 55);\n\t\tstate[ 9] = ROTL64(state[ 9], 20);\n\t\tstate[10] = ROTL64(state[10], 3);\n\t\tstate[11] = ROTL64(state[11], 10);\n\t\tstate[12] = ROTL64(state[12], 43);\n\t\tstate[13] = ROTL64(state[13], 25);\n\t\tstate[14] = ROTL64(state[14], 39);\n\t\tstate[15] = ROTL64(state[15], 41);\n\t\tstate[16] = ROTL64(state[16], 45);\n\t\tstate[17] = ROTL64(state[17], 15);\n\t\tstate[18] = ROTL64(state[18], 21);\n\t\tstate[19] = ROTL64(state[19], 8);\n\t\tstate[20] = ROTL64(state[20], 18);\n\t\tstate[21] = ROTL64(state[21], 2);\n\t\tstate[22] = ROTL64(state[22], 61);\n\t\tstate[23] = ROTL64(state[23], 56);\n\t\tstate[24] = ROTL64(state[24], 14);\n\n\t\tkeccak_pi(state);\n\t\tkeccak_chi(state);\n\n\t\t/* apply iota(state, round) */\n\t\t*state ^= keccak_round_constants[round];\n\t}\n}\n\n/**\n * The core transformation. Process the specified block of data.\n *\n * @param hash the algorithm state\n * @param block the message block to process\n * @param block_size the size of the processed block in bytes\n */\nstatic void rhash_sha3_process_block(uint64_t hash[25], const uint64_t *block, size_t block_size)\n{\n\t/* expanded loop */\n\thash[ 0] ^= le2me_64(block[ 0]);\n\thash[ 1] ^= le2me_64(block[ 1]);\n\thash[ 2] ^= le2me_64(block[ 2]);\n\thash[ 3] ^= le2me_64(block[ 3]);\n\thash[ 4] ^= le2me_64(block[ 4]);\n\thash[ 5] ^= le2me_64(block[ 5]);\n\thash[ 6] ^= le2me_64(block[ 6]);\n\thash[ 7] ^= le2me_64(block[ 7]);\n\thash[ 8] ^= le2me_64(block[ 8]);\n\t/* if not sha3-512 */\n\tif (block_size > 72) {\n\t\thash[ 9] ^= le2me_64(block[ 9]);\n\t\thash[10] ^= le2me_64(block[10]);\n\t\thash[11] ^= le2me_64(block[11]);\n\t\thash[12] ^= le2me_64(block[12]);\n\t\t/* if not sha3-384 */\n\t\tif (block_size > 104) {\n\t\t\thash[13] ^= le2me_64(block[13]);\n\t\t\thash[14] ^= le2me_64(block[14]);\n\t\t\thash[15] ^= le2me_64(block[15]);\n\t\t\thash[16] ^= le2me_64(block[16]);\n\t\t\t/* if not sha3-256 */\n\t\t\tif (block_size > 136) {\n\t\t\t\thash[17] ^= le2me_64(block[17]);\n#ifdef FULL_SHA3_FAMILY_SUPPORT\n\t\t\t\t/* if not sha3-224 */\n\t\t\t\tif (block_size > 144) {\n\t\t\t\t\thash[18] ^= le2me_64(block[18]);\n\t\t\t\t\thash[19] ^= le2me_64(block[19]);\n\t\t\t\t\thash[20] ^= le2me_64(block[20]);\n\t\t\t\t\thash[21] ^= le2me_64(block[21]);\n\t\t\t\t\thash[22] ^= le2me_64(block[22]);\n\t\t\t\t\thash[23] ^= le2me_64(block[23]);\n\t\t\t\t\thash[24] ^= le2me_64(block[24]);\n\t\t\t\t}\n#endif\n\t\t\t}\n\t\t}\n\t}\n\t/* make a permutation of the hash */\n\trhash_sha3_permutation(hash);\n}\n\n#define SHA3_FINALIZED 0x80000000\n\n/**\n * Calculate message hash.\n * Can be called repeatedly with chunks of the message to be hashed.\n *\n * @param ctx the algorithm context containing current hashing state\n * @param msg message chunk\n * @param size length of the message chunk\n */\nvoid rhash_sha3_update(sha3_ctx *ctx, const unsigned char *msg, size_t size)\n{\n\tsize_t index = (size_t)ctx->rest;\n\tsize_t block_size = (size_t)ctx->block_size;\n\n\tif (ctx->rest & SHA3_FINALIZED) return; /* too late for additional input */\n\tctx->rest = (unsigned)((ctx->rest + size) % block_size);\n\n\t/* fill partial block */\n\tif (index) {\n\t\tsize_t left = block_size - index;\n\t\tmemcpy((char*)ctx->message + index, msg, (size < left ? size : left));\n\t\tif (size < left) return;\n\n\t\t/* process partial block */\n\t\trhash_sha3_process_block(ctx->hash, ctx->message, block_size);\n\t\tmsg += left;\n\t\tsize -= left;\n\t}\n\twhile (size >= block_size) {\n\t\tuint64_t* aligned_message_block;\n\t\tif (IS_ALIGNED_64(msg)) {\n\t\t\t/* the most common case is processing of an already aligned message\n\t\t\twithout copying it */\n\t\t\taligned_message_block = (uint64_t*)msg;\n\t\t} else {\n\t\t\tmemcpy(ctx->message, msg, block_size);\n\t\t\taligned_message_block = ctx->message;\n\t\t}\n\n\t\trhash_sha3_process_block(ctx->hash, aligned_message_block, block_size);\n\t\tmsg += block_size;\n\t\tsize -= block_size;\n\t}\n\tif (size) {\n\t\tmemcpy(ctx->message, msg, size); /* save leftovers */\n\t}\n}\n\n/**\n * Store calculated hash into the given array.\n *\n * @param ctx the algorithm context containing current hashing state\n * @param result calculated hash in binary form\n */\nvoid rhash_sha3_final(sha3_ctx *ctx, unsigned char* result)\n{\n\tsize_t digest_length = 100 - ctx->block_size / 2;\n\tconst size_t block_size = ctx->block_size;\n\n\tif (!(ctx->rest & SHA3_FINALIZED))\n\t{\n\t\t/* clear the rest of the data queue */\n\t\tmemset((char*)ctx->message + ctx->rest, 0, block_size - ctx->rest);\n\t\t((char*)ctx->message)[ctx->rest] |= 0x06;\n\t\t((char*)ctx->message)[block_size - 1] |= 0x80;\n\n\t\t/* process final block */\n\t\trhash_sha3_process_block(ctx->hash, ctx->message, block_size);\n\t\tctx->rest = SHA3_FINALIZED; /* mark context as finalized */\n\t}\n\n\tassert(block_size > digest_length);\n\tif (result) me64_to_le_str(result, ctx->hash, digest_length);\n}\n\n#ifdef USE_KECCAK\n/**\n* Store calculated hash into the given array.\n*\n* @param ctx the algorithm context containing current hashing state\n* @param result calculated hash in binary form\n*/\nvoid rhash_keccak_final(sha3_ctx *ctx, unsigned char* result)\n{\n\tsize_t digest_length = 100 - ctx->block_size / 2;\n\tconst size_t block_size = ctx->block_size;\n\n\tif (!(ctx->rest & SHA3_FINALIZED))\n\t{\n\t\t/* clear the rest of the data queue */\n\t\tmemset((char*)ctx->message + ctx->rest, 0, block_size - ctx->rest);\n\t\t((char*)ctx->message)[ctx->rest] |= 0x01;\n\t\t((char*)ctx->message)[block_size - 1] |= 0x80;\n\n\t\t/* process final block */\n\t\trhash_sha3_process_block(ctx->hash, ctx->message, block_size);\n\t\tctx->rest = SHA3_FINALIZED; /* mark context as finalized */\n\t}\n\n\tassert(block_size > digest_length);\n\tif (result) me64_to_le_str(result, ctx->hash, digest_length);\n}\n#endif /* USE_KECCAK */\n" }, { "path": "benchmarks/sha3.h", "content": "/* sha3.h */\n#ifndef RHASH_SHA3_H\n#define RHASH_SHA3_H\n#include \"ustd.h\"\n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\n#define sha3_224_hash_size 28\n#define sha3_256_hash_size 32\n#define sha3_384_hash_size 48\n#define sha3_512_hash_size 64\n#define sha3_max_permutation_size 25\n#define sha3_max_rate_in_qwords 24\n\n/**\n * SHA3 Algorithm context.\n */\ntypedef struct sha3_ctx\n{\n\t/* 1600 bits algorithm hashing state */\n\tuint64_t hash[sha3_max_permutation_size];\n\t/* 1536-bit buffer for leftovers */\n\tuint64_t message[sha3_max_rate_in_qwords];\n\t/* count of bytes in the message[] buffer */\n\tunsigned rest;\n\t/* size of a message block processed at once */\n\tunsigned block_size;\n} sha3_ctx;\n\n/* methods for calculating the hash function */\n\nvoid rhash_sha3_224_init(sha3_ctx *ctx);\nvoid rhash_sha3_256_init(sha3_ctx *ctx);\nvoid rhash_sha3_384_init(sha3_ctx *ctx);\nvoid rhash_sha3_512_init(sha3_ctx *ctx);\nvoid rhash_sha3_update(sha3_ctx *ctx, const unsigned char* msg, size_t size);\nvoid rhash_sha3_final(sha3_ctx *ctx, unsigned char* result);\n\n#ifdef USE_KECCAK\n#define rhash_keccak_224_init rhash_sha3_224_init\n#define rhash_keccak_256_init rhash_sha3_256_init\n#define rhash_keccak_384_init rhash_sha3_384_init\n#define rhash_keccak_512_init rhash_sha3_512_init\n#define rhash_keccak_update rhash_sha3_update\nvoid rhash_keccak_final(sha3_ctx *ctx, unsigned char* result);\n#endif\n\n#ifdef __cplusplus\n} /* extern \"C\" */\n#endif /* __cplusplus */\n\n#endif /* RHASH_SHA3_H */\n" }, { "path": "benchmarks/sha512.c", "content": "/* sha512.c - an implementation of SHA-384/512 hash functions\n * based on FIPS 180-3 (Federal Information Processing Standart).\n *\n * Copyright: 2010-2012 Aleksey Kravchenko \n *\n * Permission is hereby granted, free of charge, to any person obtaining a\n * copy of this software and associated documentation files (the \"Software\"),\n * to deal in the Software without restriction, including without limitation\n * the rights to use, copy, modify, merge, publish, distribute, sublicense,\n * and/or sell copies of the Software, and to permit persons to whom the\n * Software is furnished to do so.\n *\n * This program is distributed in the hope that it will be useful, but\n * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY\n * or FITNESS FOR A PARTICULAR PURPOSE. Use this program at your own risk!\n */\n\n#include \n#include \"sha512.h\"\n#include \"byte_order.h\"\n\n/* SHA-384 and SHA-512 constants for 80 rounds. These qwords represent\n * the first 64 bits of the fractional parts of the cube\n * roots of the first 80 prime numbers. */\nstatic const uint64_t rhash_k512[80] = {\n\tI64(0x428a2f98d728ae22), I64(0x7137449123ef65cd), I64(0xb5c0fbcfec4d3b2f),\n\tI64(0xe9b5dba58189dbbc), I64(0x3956c25bf348b538), I64(0x59f111f1b605d019),\n\tI64(0x923f82a4af194f9b), I64(0xab1c5ed5da6d8118), I64(0xd807aa98a3030242),\n\tI64(0x12835b0145706fbe), I64(0x243185be4ee4b28c), I64(0x550c7dc3d5ffb4e2),\n\tI64(0x72be5d74f27b896f), I64(0x80deb1fe3b1696b1), I64(0x9bdc06a725c71235),\n\tI64(0xc19bf174cf692694), I64(0xe49b69c19ef14ad2), I64(0xefbe4786384f25e3),\n\tI64(0x0fc19dc68b8cd5b5), I64(0x240ca1cc77ac9c65), I64(0x2de92c6f592b0275),\n\tI64(0x4a7484aa6ea6e483), I64(0x5cb0a9dcbd41fbd4), I64(0x76f988da831153b5),\n\tI64(0x983e5152ee66dfab), I64(0xa831c66d2db43210), I64(0xb00327c898fb213f),\n\tI64(0xbf597fc7beef0ee4), I64(0xc6e00bf33da88fc2), I64(0xd5a79147930aa725),\n\tI64(0x06ca6351e003826f), I64(0x142929670a0e6e70), I64(0x27b70a8546d22ffc),\n\tI64(0x2e1b21385c26c926), I64(0x4d2c6dfc5ac42aed), I64(0x53380d139d95b3df),\n\tI64(0x650a73548baf63de), I64(0x766a0abb3c77b2a8), I64(0x81c2c92e47edaee6),\n\tI64(0x92722c851482353b), I64(0xa2bfe8a14cf10364), I64(0xa81a664bbc423001),\n\tI64(0xc24b8b70d0f89791), I64(0xc76c51a30654be30), I64(0xd192e819d6ef5218),\n\tI64(0xd69906245565a910), I64(0xf40e35855771202a), I64(0x106aa07032bbd1b8),\n\tI64(0x19a4c116b8d2d0c8), I64(0x1e376c085141ab53), I64(0x2748774cdf8eeb99),\n\tI64(0x34b0bcb5e19b48a8), I64(0x391c0cb3c5c95a63), I64(0x4ed8aa4ae3418acb),\n\tI64(0x5b9cca4f7763e373), I64(0x682e6ff3d6b2b8a3), I64(0x748f82ee5defb2fc),\n\tI64(0x78a5636f43172f60), I64(0x84c87814a1f0ab72), I64(0x8cc702081a6439ec),\n\tI64(0x90befffa23631e28), I64(0xa4506cebde82bde9), I64(0xbef9a3f7b2c67915),\n\tI64(0xc67178f2e372532b), I64(0xca273eceea26619c), I64(0xd186b8c721c0c207),\n\tI64(0xeada7dd6cde0eb1e), I64(0xf57d4f7fee6ed178), I64(0x06f067aa72176fba),\n\tI64(0x0a637dc5a2c898a6), I64(0x113f9804bef90dae), I64(0x1b710b35131c471b),\n\tI64(0x28db77f523047d84), I64(0x32caab7b40c72493), I64(0x3c9ebe0a15c9bebc),\n\tI64(0x431d67c49c100d4c), I64(0x4cc5d4becb3e42b6), I64(0x597f299cfc657e2a),\n\tI64(0x5fcb6fab3ad6faec), I64(0x6c44198c4a475817)\n};\n\n/* The SHA512/384 functions defined by FIPS 180-3, 4.1.3 */\n/* Optimized version of Ch(x,y,z)=((x & y) | (~x & z)) */\n#define Ch(x,y,z) ((z) ^ ((x) & ((y) ^ (z))))\n/* Optimized version of Maj(x,y,z)=((x & y) ^ (x & z) ^ (y & z)) */\n#define Maj(x,y,z) (((x) & (y)) ^ ((z) & ((x) ^ (y))))\n\n#define Sigma0(x) (ROTR64((x), 28) ^ ROTR64((x), 34) ^ ROTR64((x), 39))\n#define Sigma1(x) (ROTR64((x), 14) ^ ROTR64((x), 18) ^ ROTR64((x), 41))\n#define sigma0(x) (ROTR64((x), 1) ^ ROTR64((x), 8) ^ ((x) >> 7))\n#define sigma1(x) (ROTR64((x), 19) ^ ROTR64((x), 61) ^ ((x) >> 6))\n\n/* Recalculate element n-th of circular buffer W using formula\n * W[n] = sigma1(W[n - 2]) + W[n - 7] + sigma0(W[n - 15]) + W[n - 16]; */\n#define RECALCULATE_W(W,n) (W[n] += \\\n\t(sigma1(W[(n - 2) & 15]) + W[(n - 7) & 15] + sigma0(W[(n - 15) & 15])))\n\n#define ROUND(a,b,c,d,e,f,g,h,k,data) { \\\n\tuint64_t T1 = h + Sigma1(e) + Ch(e,f,g) + k + (data); \\\n\td += T1, h = T1 + Sigma0(a) + Maj(a,b,c); }\n#define ROUND_1_16(a,b,c,d,e,f,g,h,n) \\\n\tROUND(a,b,c,d,e,f,g,h, rhash_k512[n], W[n] = be2me_64(block[n]))\n#define ROUND_17_80(a,b,c,d,e,f,g,h,n) \\\n\tROUND(a,b,c,d,e,f,g,h, k[n], RECALCULATE_W(W, n))\n\n/**\n * Initialize context before calculating hash.\n *\n * @param ctx context to initialize\n */\nvoid rhash_sha512_init(sha512_ctx *ctx)\n{\n\t/* Initial values. These words were obtained by taking the first 32\n\t * bits of the fractional parts of the square roots of the first\n\t * eight prime numbers. */\n\tstatic const uint64_t SHA512_H0[8] = {\n\t\tI64(0x6a09e667f3bcc908), I64(0xbb67ae8584caa73b), I64(0x3c6ef372fe94f82b),\n\t\tI64(0xa54ff53a5f1d36f1), I64(0x510e527fade682d1), I64(0x9b05688c2b3e6c1f),\n\t\tI64(0x1f83d9abfb41bd6b), I64(0x5be0cd19137e2179)\n\t};\n\n\tctx->length = 0;\n\tctx->digest_length = sha512_hash_size;\n\n\t/* initialize algorithm state */\n\tmemcpy(ctx->hash, SHA512_H0, sizeof(ctx->hash));\n}\n\n/**\n * Initialize context before calculaing hash.\n *\n * @param ctx context to initialize\n */\nvoid rhash_sha384_init(struct sha512_ctx *ctx)\n{\n\t/* Initial values from FIPS 180-3. These words were obtained by taking\n\t * the first sixty-four bits of the fractional parts of the square\n\t * roots of ninth through sixteenth prime numbers. */\n\tstatic const uint64_t SHA384_H0[8] = {\n\t\tI64(0xcbbb9d5dc1059ed8), I64(0x629a292a367cd507), I64(0x9159015a3070dd17),\n\t\tI64(0x152fecd8f70e5939), I64(0x67332667ffc00b31), I64(0x8eb44a8768581511),\n\t\tI64(0xdb0c2e0d64f98fa7), I64(0x47b5481dbefa4fa4)\n\t};\n\n\tctx->length = 0;\n\tctx->digest_length = sha384_hash_size;\n\n\tmemcpy(ctx->hash, SHA384_H0, sizeof(ctx->hash));\n}\n\n/**\n * The core transformation. Process a 512-bit block.\n *\n * @param hash algorithm state\n * @param block the message block to process\n */\nstatic void rhash_sha512_process_block(uint64_t hash[8], uint64_t block[16])\n{\n\tuint64_t A, B, C, D, E, F, G, H;\n\tuint64_t W[16];\n\tconst uint64_t *k;\n\tint i;\n\n\tA = hash[0], B = hash[1], C = hash[2], D = hash[3];\n\tE = hash[4], F = hash[5], G = hash[6], H = hash[7];\n\n\t/* Compute SHA using alternate Method: FIPS 180-3 6.1.3 */\n\tROUND_1_16(A, B, C, D, E, F, G, H, 0);\n\tROUND_1_16(H, A, B, C, D, E, F, G, 1);\n\tROUND_1_16(G, H, A, B, C, D, E, F, 2);\n\tROUND_1_16(F, G, H, A, B, C, D, E, 3);\n\tROUND_1_16(E, F, G, H, A, B, C, D, 4);\n\tROUND_1_16(D, E, F, G, H, A, B, C, 5);\n\tROUND_1_16(C, D, E, F, G, H, A, B, 6);\n\tROUND_1_16(B, C, D, E, F, G, H, A, 7);\n\tROUND_1_16(A, B, C, D, E, F, G, H, 8);\n\tROUND_1_16(H, A, B, C, D, E, F, G, 9);\n\tROUND_1_16(G, H, A, B, C, D, E, F, 10);\n\tROUND_1_16(F, G, H, A, B, C, D, E, 11);\n\tROUND_1_16(E, F, G, H, A, B, C, D, 12);\n\tROUND_1_16(D, E, F, G, H, A, B, C, 13);\n\tROUND_1_16(C, D, E, F, G, H, A, B, 14);\n\tROUND_1_16(B, C, D, E, F, G, H, A, 15);\n\n\tfor (i = 16, k = &rhash_k512[16]; i < 80; i += 16, k += 16) {\n\t\tROUND_17_80(A, B, C, D, E, F, G, H, 0);\n\t\tROUND_17_80(H, A, B, C, D, E, F, G, 1);\n\t\tROUND_17_80(G, H, A, B, C, D, E, F, 2);\n\t\tROUND_17_80(F, G, H, A, B, C, D, E, 3);\n\t\tROUND_17_80(E, F, G, H, A, B, C, D, 4);\n\t\tROUND_17_80(D, E, F, G, H, A, B, C, 5);\n\t\tROUND_17_80(C, D, E, F, G, H, A, B, 6);\n\t\tROUND_17_80(B, C, D, E, F, G, H, A, 7);\n\t\tROUND_17_80(A, B, C, D, E, F, G, H, 8);\n\t\tROUND_17_80(H, A, B, C, D, E, F, G, 9);\n\t\tROUND_17_80(G, H, A, B, C, D, E, F, 10);\n\t\tROUND_17_80(F, G, H, A, B, C, D, E, 11);\n\t\tROUND_17_80(E, F, G, H, A, B, C, D, 12);\n\t\tROUND_17_80(D, E, F, G, H, A, B, C, 13);\n\t\tROUND_17_80(C, D, E, F, G, H, A, B, 14);\n\t\tROUND_17_80(B, C, D, E, F, G, H, A, 15);\n\t}\n\n\thash[0] += A, hash[1] += B, hash[2] += C, hash[3] += D;\n\thash[4] += E, hash[5] += F, hash[6] += G, hash[7] += H;\n}\n\n/**\n * Calculate message hash.\n * Can be called repeatedly with chunks of the message to be hashed.\n *\n * @param ctx the algorithm context containing current hashing state\n * @param msg message chunk\n * @param size length of the message chunk\n */\nvoid rhash_sha512_update(sha512_ctx *ctx, const unsigned char *msg, size_t size)\n{\n\tsize_t index = (size_t)ctx->length & 127;\n\tctx->length += size;\n\n\t/* fill partial block */\n\tif (index) {\n\t\tsize_t left = sha512_block_size - index;\n\t\tmemcpy((char*)ctx->message + index, msg, (size < left ? size : left));\n\t\tif (size < left) return;\n\n\t\t/* process partial block */\n\t\trhash_sha512_process_block(ctx->hash, ctx->message);\n\t\tmsg += left;\n\t\tsize -= left;\n\t}\n\twhile (size >= sha512_block_size) {\n\t\tuint64_t* aligned_message_block;\n\t\tif (IS_ALIGNED_64(msg)) {\n\t\t\t/* the most common case is processing of an already aligned message\n\t\t\twithout copying it */\n\t\t\taligned_message_block = (uint64_t*)msg;\n\t\t} else {\n\t\t\tmemcpy(ctx->message, msg, sha512_block_size);\n\t\t\taligned_message_block = ctx->message;\n\t\t}\n\n\t\trhash_sha512_process_block(ctx->hash, aligned_message_block);\n\t\tmsg += sha512_block_size;\n\t\tsize -= sha512_block_size;\n\t}\n\tif (size) {\n\t\tmemcpy(ctx->message, msg, size); /* save leftovers */\n\t}\n}\n\n/**\n * Store calculated hash into the given array.\n *\n * @param ctx the algorithm context containing current hashing state\n * @param result calculated hash in binary form\n */\nvoid rhash_sha512_final(sha512_ctx *ctx, unsigned char* result)\n{\n\tsize_t index = ((unsigned)ctx->length & 127) >> 3;\n\tunsigned shift = ((unsigned)ctx->length & 7) * 8;\n\n\t/* pad message and process the last block */\n\n\t/* append the byte 0x80 to the message */\n\tctx->message[index] &= le2me_64( ~(I64(0xFFFFFFFFFFFFFFFF) << shift) );\n\tctx->message[index++] ^= le2me_64( I64(0x80) << shift );\n\n\t/* if no room left in the message to store 128-bit message length */\n\tif (index >= 15) {\n\t\tif (index == 15) ctx->message[index] = 0;\n\t\trhash_sha512_process_block(ctx->hash, ctx->message);\n\t\tindex = 0;\n\t}\n\twhile (index < 15) {\n\t\tctx->message[index++] = 0;\n\t}\n\tctx->message[15] = be2me_64(ctx->length << 3);\n\trhash_sha512_process_block(ctx->hash, ctx->message);\n\n\tif (result) be64_copy(result, 0, ctx->hash, ctx->digest_length);\n}\n" }, { "path": "benchmarks/sha512.h", "content": "/* sha.h sha512 and sha384 hash functions */\n#ifndef SHA512_H\n#define SHA512_H\n\n#if _MSC_VER >= 1300\n\n# define int64_t __int64\n# define int32_t __int32\n# define int16_t __int16\n# define int8_t __int8\n# define uint64_t unsigned __int64\n# define uint32_t unsigned __int32\n# define uint16_t unsigned __int16\n# define uint8_t unsigned __int8\n\n/* disable warnings: The POSIX name for this item is deprecated. Use the ISO C++ conformant name. */\n#pragma warning(disable : 4996)\n\n#else /* _MSC_VER >= 1300 */\n\n# include \n# include \n\n#endif /* _MSC_VER >= 1300 */\n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\n#define sha512_block_size 128\n#define sha512_hash_size 64\n#define sha384_hash_size 48\n\n/* algorithm context */\ntypedef struct sha512_ctx\n{\n\tuint64_t message[16]; /* 1024-bit buffer for leftovers */\n\tuint64_t length; /* number of processed bytes */\n\tuint64_t hash[8]; /* 512-bit algorithm internal hashing state */\n\tunsigned digest_length; /* length of the algorithm digest in bytes */\n} sha512_ctx;\n\nvoid rhash_sha384_init(sha512_ctx *ctx);\nvoid rhash_sha512_init(sha512_ctx *ctx);\nvoid rhash_sha512_update(sha512_ctx *ctx, const unsigned char* data, size_t length);\nvoid rhash_sha512_final(sha512_ctx *ctx, unsigned char* result);\n\n#ifdef __cplusplus\n} /* extern \"C\" */\n#endif /* __cplusplus */\n\n#endif /* SHA512_H */\n" }, { "path": "benchmarks/sip24-prng.h", "content": "/* Copyright (c) 2016 Vladimir Makarov \n\n Permission is hereby granted, free of charge, to any person\n obtaining a copy of this software and associated documentation\n files (the \"Software\"), to deal in the Software without\n restriction, including without limitation the rights to use, copy,\n modify, merge, publish, distribute, sublicense, and/or sell copies\n of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n\n The above copyright notice and this permission notice shall be\n included in all copies or substantial portions of the Software.\n\n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\n MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\n BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\n ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\n CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.\n*/\n\n/* Pseudo Random Number Generator (PRNG) based on SipHash24 designed\n by J.P. Aumasson and D.J. Bernstein. The SipHash24 reference code\n (please see https://github.com/veorq/SipHash) adapted for our\n purposes.\n\n To use a generator call `init_sip24_prng` or\n `init_sip24_prng_with_seed` first, then call `get_sip24_prn` as much\n as you want to get a new PRN. At the end of the PRNG use, call\n `finish_sip24_prng`. You can change the default seed by calling\n `set_sip24_prng_seed`.\n\n The PRNG passes NIST Statistical Test Suite for Random and\n Pseudorandom Number Generators for Cryptographic Applications\n (version 2.2.1) with 1000 bitstreams each containing 1M bits.\n\n The generation of a new number takes about 11 CPU cycles on x86_64\n (Intel 4.2GHz i7-4790K), or speed of the generation is about 380M\n numbers per sec. So it is pretty fast. */\n\n#ifndef __SIP24_PRNG__\n#define __SIP24_PRNG__\n\n#ifdef _MSC_VER\ntypedef unsigned __int32 uint32_t;\ntypedef unsigned __int64 uint64_t;\n#else\n#include \n#endif\n\n#include \n\nstatic inline uint64_t\n_sip24_prng_rotl (uint64_t v, int c) {\n return (v << c) | (v >> (64 - c));\n}\n\n/* A Sip round */\nstatic inline void\n_sip24_prng_round (uint64_t *v0, uint64_t *v1, uint64_t *v2, uint64_t *v3) {\n *v0 += *v1; *v1 = _sip24_prng_rotl (*v1, 13);\n *v1 ^= *v0; *v0 = _sip24_prng_rotl (*v0, 32);\n *v2 += *v3; *v3 = _sip24_prng_rotl (*v3, 16);\n *v3 ^= *v2;\n *v0 += *v3; *v3 = _sip24_prng_rotl (*v3, 21);\n *v3 ^= *v0;\n *v2 += *v1; *v1 = _sip24_prng_rotl (*v1, 17);\n *v1 ^= *v2; *v2 = _sip24_prng_rotl (*v2, 32);\n}\n\n/* The number of intermediate and final Sip rounds. */\n#define cROUNDS 2\n#define dROUNDS 4\n\n/* Internal state of the PRNG. */\nstatic struct {\n int ind; /* position in the output */\n uint64_t v; /* current message from which PRNs were generated */\n /* The current PRNG state and parts of the recenty generated\n numbers. */\n uint64_t init_state[4], state[4];\n} _sip24_prng_state;\n\nstatic inline void\n_sip24_prng_gen (void)\n{\n int i;\n\n for (i = 0; i < 4; i++)\n _sip24_prng_state.state[i] = _sip24_prng_state.init_state[i];\n _sip24_prng_state.state[3] ^= _sip24_prng_state.v;\n for (i = 0; i < cROUNDS; ++i)\n _sip24_prng_round (&_sip24_prng_state.state[0], &_sip24_prng_state.state[1],\n\t\t &_sip24_prng_state.state[2], &_sip24_prng_state.state[3]);\n _sip24_prng_state.state[0] ^= _sip24_prng_state.v;\n for (i = 0; i < cROUNDS; ++i)\n _sip24_prng_round (&_sip24_prng_state.state[0], &_sip24_prng_state.state[1],\n\t\t &_sip24_prng_state.state[2], &_sip24_prng_state.state[3]);\n _sip24_prng_state.state[2] ^= 0xff;\n for (i = 0; i < dROUNDS; ++i)\n _sip24_prng_round (&_sip24_prng_state.state[0], &_sip24_prng_state.state[1],\n\t\t &_sip24_prng_state.state[2], &_sip24_prng_state.state[3]);\n}\n\n/* Some random numbers from Siphash24. */\nstatic const uint64_t _sip24_prng_nums[4] = {\n 0x736f6d6570736575ULL, 0x646f72616e646f6dULL,\n 0x6c7967656e657261ULL, 0x7465646279746573ULL,\n};\n\n/* Set the PRNG seed by K. */\nstatic inline void\nset_sip24_prng_seed (uint64_t k[4]) {\n int i;\n \n for (i = 0; i < 4; i++)\n _sip24_prng_state.init_state[i] = k[i] ^ _sip24_prng_nums[i];\n}\n\n/* Initiate the PRNG with seed given by K. */\nstatic inline void\ninit_sip24_prng_with_seed (uint64_t k[4]) {\n set_sip24_prng_seed (k);\n _sip24_prng_state.v = 0;\n _sip24_prng_state.ind = 4;\n}\n\n/* Initiate the PRNG with some random seed. */\nstatic inline void\ninit_sip24_prng (void) {\n int i;\n uint64_t k[4];\n\n for (i = 0; i < 4; i++)\n k[i] = rand ();\n init_sip24_prng_with_seed (k);\n}\n\n/* Return the next pseudo-random number. */\nstatic inline uint64_t\nget_sip24_prn (void)\n{\n uint64_t res;\n \n for (;;) {\n if (_sip24_prng_state.ind < 4) {\n res = _sip24_prng_state.state[_sip24_prng_state.ind];\n _sip24_prng_state.ind++;\n return res;\n }\n _sip24_prng_state.ind = 0;\n _sip24_prng_gen ();\n _sip24_prng_state.v++; \n }\n}\n\n/* Empty function for our PRNGs interface. */\nstatic inline void\nfinish_sip24_prng (void) {\n}\n\n#endif\n" }, { "path": "benchmarks/siphash24.c", "content": "/*\n SipHash reference C implementation\n\n Copyright (c) 2012-2014 Jean-Philippe Aumasson\n \n Copyright (c) 2012-2014 Daniel J. Bernstein \n\n To the extent possible under law, the author(s) have dedicated all copyright\n and related and neighboring rights to this software to the public domain\n worldwide. This software is distributed without any warranty.\n\n You should have received a copy of the CC0 Public Domain Dedication along\n with\n this software. If not, see\n .\n */\n#include \n#include \n#include \n\n/* default: SipHash-2-4 */\n#define cROUNDS 2\n#define dROUNDS 4\n\n#define ROTL(x, b) (uint64_t)(((x) << (b)) | ((x) >> (64 - (b))))\n\n#define U32TO8_LE(p, v) \\\n (p)[0] = (uint8_t)((v)); \\\n (p)[1] = (uint8_t)((v) >> 8); \\\n (p)[2] = (uint8_t)((v) >> 16); \\\n (p)[3] = (uint8_t)((v) >> 24);\n\n#define U64TO8_LE(p, v) \\\n U32TO8_LE((p), (uint32_t)((v))); \\\n U32TO8_LE((p) + 4, (uint32_t)((v) >> 32));\n\n#define U8TO64_LE(p) \\\n (((uint64_t)((p)[0])) | ((uint64_t)((p)[1]) << 8) | \\\n ((uint64_t)((p)[2]) << 16) | ((uint64_t)((p)[3]) << 24) | \\\n ((uint64_t)((p)[4]) << 32) | ((uint64_t)((p)[5]) << 40) | \\\n ((uint64_t)((p)[6]) << 48) | ((uint64_t)((p)[7]) << 56))\n\n#define SIPROUND \\\n do { \\\n v0 += v1; \\\n v1 = ROTL(v1, 13); \\\n v1 ^= v0; \\\n v0 = ROTL(v0, 32); \\\n v2 += v3; \\\n v3 = ROTL(v3, 16); \\\n v3 ^= v2; \\\n v0 += v3; \\\n v3 = ROTL(v3, 21); \\\n v3 ^= v0; \\\n v2 += v1; \\\n v1 = ROTL(v1, 17); \\\n v1 ^= v2; \\\n v2 = ROTL(v2, 32); \\\n } while (0)\n\n#ifdef DEBUG\n#define TRACE \\\n do { \\\n printf(\"(%3d) v0 %08x %08x\\n\", (int)inlen, (uint32_t)(v0 >> 32), \\\n (uint32_t)v0); \\\n printf(\"(%3d) v1 %08x %08x\\n\", (int)inlen, (uint32_t)(v1 >> 32), \\\n (uint32_t)v1); \\\n printf(\"(%3d) v2 %08x %08x\\n\", (int)inlen, (uint32_t)(v2 >> 32), \\\n (uint32_t)v2); \\\n printf(\"(%3d) v3 %08x %08x\\n\", (int)inlen, (uint32_t)(v3 >> 32), \\\n (uint32_t)v3); \\\n } while (0)\n#else\n#define TRACE\n#endif\n\nint siphash(uint8_t *out, const uint8_t *in, uint64_t inlen, const uint8_t *k) {\n /* \"somepseudorandomlygeneratedbytes\" */\n uint64_t v0 = 0x736f6d6570736575ULL;\n uint64_t v1 = 0x646f72616e646f6dULL;\n uint64_t v2 = 0x6c7967656e657261ULL;\n uint64_t v3 = 0x7465646279746573ULL;\n uint64_t b;\n uint64_t k0 = U8TO64_LE(k);\n uint64_t k1 = U8TO64_LE(k + 8);\n uint64_t m;\n int i;\n const uint8_t *end = in + inlen - (inlen % sizeof(uint64_t));\n const int left = inlen & 7;\n b = ((uint64_t)inlen) << 56;\n v3 ^= k1;\n v2 ^= k0;\n v1 ^= k1;\n v0 ^= k0;\n\n#ifdef DOUBLE\n v1 ^= 0xee;\n#endif\n\n for (; in != end; in += 8) {\n m = U8TO64_LE(in);\n v3 ^= m;\n\n TRACE;\n for (i = 0; i < cROUNDS; ++i)\n SIPROUND;\n\n v0 ^= m;\n }\n\n switch (left) {\n case 7:\n b |= ((uint64_t)in[6]) << 48;\n case 6:\n b |= ((uint64_t)in[5]) << 40;\n case 5:\n b |= ((uint64_t)in[4]) << 32;\n case 4:\n b |= ((uint64_t)in[3]) << 24;\n case 3:\n b |= ((uint64_t)in[2]) << 16;\n case 2:\n b |= ((uint64_t)in[1]) << 8;\n case 1:\n b |= ((uint64_t)in[0]);\n break;\n case 0:\n break;\n }\n\n v3 ^= b;\n\n TRACE;\n for (i = 0; i < cROUNDS; ++i)\n SIPROUND;\n\n v0 ^= b;\n\n#ifndef DOUBLE\n v2 ^= 0xff;\n#else\n v2 ^= 0xee;\n#endif\n\n TRACE;\n for (i = 0; i < dROUNDS; ++i)\n SIPROUND;\n\n b = v0 ^ v1 ^ v2 ^ v3;\n U64TO8_LE(out, b);\n\n#ifdef DOUBLE\n v1 ^= 0xdd;\n\n TRACE;\n for (i = 0; i < dROUNDS; ++i)\n SIPROUND;\n\n b = v0 ^ v1 ^ v2 ^ v3;\n U64TO8_LE(out + 8, b);\n#endif\n\n return 0;\n}\n\n" }, { "path": "benchmarks/splitmix64.c", "content": "/* Written in 2015 by Sebastiano Vigna (vigna@acm.org)\n\nTo the extent possible under law, the author has dedicated all copyright\nand related and neighboring rights to this software to the public domain\nworldwide. This software is distributed without any warranty.\n\nSee . */\n\n#include \n\n/* This is a fixed-increment version of Java 8's SplittableRandom generator\n See http://dx.doi.org/10.1145/2714064.2660195 and \n http://docs.oracle.com/javase/8/docs/api/java/util/SplittableRandom.html\n\n It is a very fast generator passing BigCrush, and it can be useful if\n for some reason you absolutely want 64 bits of state; otherwise, we\n rather suggest to use a xoroshiro128+ (for moderately parallel\n computations) or xorshift1024* (for massively parallel computations)\n generator. */\n\nstatic uint64_t x; /* The state can be seeded with any value. */\n\nuint64_t next() {\n\tuint64_t z = (x += 0x9e3779b97f4a7c15);\n\tz = (z ^ (z >> 30)) * 0xbf58476d1ce4e5b9;\n\tz = (z ^ (z >> 27)) * 0x94d049bb133111eb;\n\treturn z ^ (z >> 31);\n}\n" }, { "path": "benchmarks/t1ha/src/t1ha0.c", "content": "/*\n * Copyright (c) 2016-2020 Positive Technologies, https://www.ptsecurity.com,\n * Fast Positive Hash.\n *\n * Portions Copyright (c) 2010-2020 Leonid Yuriev ,\n * The 1Hippeus project (t1h).\n *\n * This software is provided 'as-is', without any express or implied\n * warranty. In no event will the authors be held liable for any damages\n * arising from the use of this software.\n *\n * Permission is granted to anyone to use this software for any purpose,\n * including commercial applications, and to alter it and redistribute it\n * freely, subject to the following restrictions:\n *\n * 1. The origin of this software must not be misrepresented; you must not\n * claim that you wrote the original software. If you use this software\n * in a product, an acknowledgement in the product documentation would be\n * appreciated but is not required.\n * 2. Altered source versions must be plainly marked as such, and must not be\n * misrepresented as being the original software.\n * 3. This notice may not be removed or altered from any source distribution.\n */\n\n/*\n * t1ha = { Fast Positive Hash, aka \"Позитивный Хэш\" }\n * by [Positive Technologies](https://www.ptsecurity.ru)\n *\n * Briefly, it is a 64-bit Hash Function:\n * 1. Created for 64-bit little-endian platforms, in predominantly for x86_64,\n * but portable and without penalties it can run on any 64-bit CPU.\n * 2. In most cases up to 15% faster than City64, xxHash, mum-hash, metro-hash\n * and all others portable hash-functions (which do not use specific\n * hardware tricks).\n * 3. Not suitable for cryptography.\n *\n * The Future will (be) Positive. Всё будет хорошо.\n *\n * ACKNOWLEDGEMENT:\n * The t1ha was originally developed by Leonid Yuriev (Леонид Юрьев)\n * for The 1Hippeus project - zerocopy messaging in the spirit of Sparta!\n */\n\n#ifndef T1HA0_DISABLED\n#include \"t1ha_bits.h\"\n#include \"t1ha_selfcheck.h\"\n\nstatic __maybe_unused __always_inline uint32_t tail32_le_aligned(const void *v,\n size_t tail) {\n const uint8_t *const p = (const uint8_t *)v;\n#if T1HA_USE_FAST_ONESHOT_READ && !defined(__SANITIZE_ADDRESS__)\n /* We can perform a 'oneshot' read, which is little bit faster. */\n const unsigned shift = ((4 - tail) & 3) << 3;\n return fetch32_le_aligned(p) & ((~UINT32_C(0)) >> shift);\n#else\n uint32_t r = 0;\n switch (tail & 3) {\n default:\n unreachable();\n/* fall through */\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n /* For most CPUs this code is better when not needed\n * copying for alignment or byte reordering. */\n case 0:\n return fetch32_le_aligned(p);\n case 3:\n r = (uint32_t)p[2] << 16;\n /* fall through */\n case 2:\n return r + fetch16_le_aligned(p);\n case 1:\n return p[0];\n#else\n case 0:\n r += p[3];\n r <<= 8;\n /* fall through */\n case 3:\n r += p[2];\n r <<= 8;\n /* fall through */\n case 2:\n r += p[1];\n r <<= 8;\n /* fall through */\n case 1:\n return r + p[0];\n#endif\n }\n#endif /* T1HA_USE_FAST_ONESHOT_READ */\n}\n\nstatic __maybe_unused __always_inline uint32_t\ntail32_le_unaligned(const void *v, size_t tail) {\n const uint8_t *p = (const uint8_t *)v;\n#ifdef can_read_underside\n /* On some systems (e.g. x86) we can perform a 'oneshot' read, which\n * is little bit faster. Thanks Marcin Żukowski \n * for the reminder. */\n const unsigned offset = (4 - tail) & 3;\n const unsigned shift = offset << 3;\n if (likely(can_read_underside(p, 4))) {\n p -= offset;\n return fetch32_le_unaligned(p) >> shift;\n }\n return fetch32_le_unaligned(p) & ((~UINT32_C(0)) >> shift);\n#else\n uint32_t r = 0;\n switch (tail & 3) {\n default:\n unreachable();\n/* fall through */\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__EFFICIENT && \\\n __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n /* For most CPUs this code is better when not needed\n * copying for alignment or byte reordering. */\n case 0:\n return fetch32_le_unaligned(p);\n case 3:\n r = (uint32_t)p[2] << 16;\n /* fall through */\n case 2:\n return r + fetch16_le_unaligned(p);\n case 1:\n return p[0];\n#else\n /* For most CPUs this code is better than a\n * copying for alignment and/or byte reordering. */\n case 0:\n r += p[3];\n r <<= 8;\n /* fall through */\n case 3:\n r += p[2];\n r <<= 8;\n /* fall through */\n case 2:\n r += p[1];\n r <<= 8;\n /* fall through */\n case 1:\n return r + p[0];\n#endif\n }\n#endif /* can_read_underside */\n}\n\nstatic __maybe_unused __always_inline uint32_t tail32_be_aligned(const void *v,\n size_t tail) {\n const uint8_t *const p = (const uint8_t *)v;\n#if T1HA_USE_FAST_ONESHOT_READ && !defined(__SANITIZE_ADDRESS__)\n /* We can perform a 'oneshot' read, which is little bit faster. */\n const unsigned shift = ((4 - tail) & 3) << 3;\n return fetch32_be_aligned(p) >> shift;\n#else\n switch (tail & 3) {\n default:\n unreachable();\n/* fall through */\n#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n /* For most CPUs this code is better when not needed\n * copying for alignment or byte reordering. */\n case 1:\n return p[0];\n case 2:\n return fetch16_be_aligned(p);\n case 3:\n return fetch16_be_aligned(p) << 8 | p[2];\n case 0:\n return fetch32_be_aligned(p);\n#else\n case 1:\n return p[0];\n case 2:\n return p[1] | (uint32_t)p[0] << 8;\n case 3:\n return p[2] | (uint32_t)p[1] << 8 | (uint32_t)p[0] << 16;\n case 0:\n return p[3] | (uint32_t)p[2] << 8 | (uint32_t)p[1] << 16 |\n (uint32_t)p[0] << 24;\n#endif\n }\n#endif /* T1HA_USE_FAST_ONESHOT_READ */\n}\n\nstatic __maybe_unused __always_inline uint32_t\ntail32_be_unaligned(const void *v, size_t tail) {\n const uint8_t *p = (const uint8_t *)v;\n#ifdef can_read_underside\n /* On some systems we can perform a 'oneshot' read, which is little bit\n * faster. Thanks Marcin Żukowski for the\n * reminder. */\n const unsigned offset = (4 - tail) & 3;\n const unsigned shift = offset << 3;\n if (likely(can_read_underside(p, 4))) {\n p -= offset;\n return fetch32_be_unaligned(p) & ((~UINT32_C(0)) >> shift);\n }\n return fetch32_be_unaligned(p) >> shift;\n#else\n switch (tail & 3) {\n default:\n unreachable();\n/* fall through */\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__EFFICIENT && \\\n __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n /* For most CPUs this code is better when not needed\n * copying for alignment or byte reordering. */\n case 1:\n return p[0];\n case 2:\n return fetch16_be_unaligned(p);\n case 3:\n return fetch16_be_unaligned(p) << 8 | p[2];\n case 0:\n return fetch32_be_unaligned(p);\n#else\n /* For most CPUs this code is better than a\n * copying for alignment and/or byte reordering. */\n case 1:\n return p[0];\n case 2:\n return p[1] | (uint32_t)p[0] << 8;\n case 3:\n return p[2] | (uint32_t)p[1] << 8 | (uint32_t)p[0] << 16;\n case 0:\n return p[3] | (uint32_t)p[2] << 8 | (uint32_t)p[1] << 16 |\n (uint32_t)p[0] << 24;\n#endif\n }\n#endif /* can_read_underside */\n}\n\n/***************************************************************************/\n\n#ifndef rot32\nstatic __maybe_unused __always_inline uint32_t rot32(uint32_t v, unsigned s) {\n return (v >> s) | (v << (32 - s));\n}\n#endif /* rot32 */\n\nstatic __always_inline void mixup32(uint32_t *a, uint32_t *b, uint32_t v,\n uint32_t prime) {\n uint64_t l = mul_32x32_64(*b + v, prime);\n *a ^= (uint32_t)l;\n *b += (uint32_t)(l >> 32);\n}\n\nstatic __always_inline uint64_t final32(uint32_t a, uint32_t b) {\n uint64_t l = (b ^ rot32(a, 13)) | (uint64_t)a << 32;\n l *= prime_0;\n l ^= l >> 41;\n l *= prime_4;\n l ^= l >> 47;\n l *= prime_6;\n return l;\n}\n\n/* 32-bit 'magic' primes */\nstatic const uint32_t prime32_0 = UINT32_C(0x92D78269);\nstatic const uint32_t prime32_1 = UINT32_C(0xCA9B4735);\nstatic const uint32_t prime32_2 = UINT32_C(0xA4ABA1C3);\nstatic const uint32_t prime32_3 = UINT32_C(0xF6499843);\nstatic const uint32_t prime32_4 = UINT32_C(0x86F0FD61);\nstatic const uint32_t prime32_5 = UINT32_C(0xCA2DA6FB);\nstatic const uint32_t prime32_6 = UINT32_C(0xC4BB3575);\n\n/* TODO: C++ template in the next version */\n#define T1HA0_BODY(ENDIANNES, ALIGNESS) \\\n const uint32_t *v = (const uint32_t *)data; \\\n if (unlikely(len > 16)) { \\\n uint32_t c = ~a; \\\n uint32_t d = rot32(b, 5); \\\n const uint32_t *detent = \\\n (const uint32_t *)((const uint8_t *)data + len - 15); \\\n do { \\\n const uint32_t w0 = fetch32_##ENDIANNES##_##ALIGNESS(v + 0); \\\n const uint32_t w1 = fetch32_##ENDIANNES##_##ALIGNESS(v + 1); \\\n const uint32_t w2 = fetch32_##ENDIANNES##_##ALIGNESS(v + 2); \\\n const uint32_t w3 = fetch32_##ENDIANNES##_##ALIGNESS(v + 3); \\\n v += 4; \\\n prefetch(v); \\\n \\\n const uint32_t d13 = w1 + rot32(w3 + d, 17); \\\n const uint32_t c02 = w0 ^ rot32(w2 + c, 11); \\\n d ^= rot32(a + w0, 3); \\\n c ^= rot32(b + w1, 7); \\\n b = prime32_1 * (c02 + w3); \\\n a = prime32_0 * (d13 ^ w2); \\\n } while (likely(v < detent)); \\\n \\\n c += a; \\\n d += b; \\\n a ^= prime32_6 * (rot32(c, 16) + d); \\\n b ^= prime32_5 * (c + rot32(d, 16)); \\\n \\\n len &= 15; \\\n } \\\n \\\n switch (len) { \\\n default: \\\n mixup32(&a, &b, fetch32_##ENDIANNES##_##ALIGNESS(v++), prime32_4); \\\n /* fall through */ \\\n case 12: \\\n case 11: \\\n case 10: \\\n case 9: \\\n mixup32(&b, &a, fetch32_##ENDIANNES##_##ALIGNESS(v++), prime32_3); \\\n /* fall through */ \\\n case 8: \\\n case 7: \\\n case 6: \\\n case 5: \\\n mixup32(&a, &b, fetch32_##ENDIANNES##_##ALIGNESS(v++), prime32_2); \\\n /* fall through */ \\\n case 4: \\\n case 3: \\\n case 2: \\\n case 1: \\\n mixup32(&b, &a, tail32_##ENDIANNES##_##ALIGNESS(v, len), prime32_1); \\\n /* fall through */ \\\n case 0: \\\n return final32(a, b); \\\n }\n\nuint64_t t1ha0_32le(const void *data, size_t len, uint64_t seed) {\n uint32_t a = rot32((uint32_t)len, 17) + (uint32_t)seed;\n uint32_t b = (uint32_t)len ^ (uint32_t)(seed >> 32);\n\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__EFFICIENT\n T1HA0_BODY(le, unaligned);\n#else\n const bool misaligned = (((uintptr_t)data) & (ALIGNMENT_32 - 1)) != 0;\n if (misaligned) {\n T1HA0_BODY(le, unaligned);\n } else {\n T1HA0_BODY(le, aligned);\n }\n#endif\n}\n\nuint64_t t1ha0_32be(const void *data, size_t len, uint64_t seed) {\n uint32_t a = rot32((uint32_t)len, 17) + (uint32_t)seed;\n uint32_t b = (uint32_t)len ^ (uint32_t)(seed >> 32);\n\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__EFFICIENT\n T1HA0_BODY(be, unaligned);\n#else\n const bool misaligned = (((uintptr_t)data) & (ALIGNMENT_32 - 1)) != 0;\n if (misaligned) {\n T1HA0_BODY(be, unaligned);\n } else {\n T1HA0_BODY(be, aligned);\n }\n#endif\n}\n\n/***************************************************************************/\n\n#if T1HA0_AESNI_AVAILABLE && defined(__ia32__)\n__cold uint64_t t1ha_ia32cpu_features(void) {\n uint32_t features = 0;\n uint32_t extended = 0;\n#ifdef __GNUC__\n uint32_t eax, ebx, ecx, edx;\n const unsigned cpuid_max = __get_cpuid_max(0, NULL);\n if (cpuid_max >= 1) {\n __cpuid_count(1, 0, eax, ebx, features, edx);\n if (cpuid_max >= 7)\n __cpuid_count(7, 0, eax, extended, ecx, edx);\n }\n#elif defined(_MSC_VER)\n int info[4];\n __cpuid(info, 0);\n const unsigned cpuid_max = info[0];\n if (cpuid_max >= 1) {\n __cpuidex(info, 1, 0);\n features = info[2];\n if (cpuid_max >= 7) {\n __cpuidex(info, 7, 0);\n extended = info[1];\n }\n }\n#endif\n return features | (uint64_t)extended << 32;\n}\n#endif /* T1HA0_AESNI_AVAILABLE && __ia32__ */\n\n#if T1HA0_RUNTIME_SELECT\n\n__cold t1ha0_function_t t1ha0_resolve(void) {\n\n#if T1HA0_AESNI_AVAILABLE && defined(__ia32__)\n uint64_t features = t1ha_ia32cpu_features();\n if (t1ha_ia32_AESNI_avail(features)) {\n if (t1ha_ia32_AVX_avail(features))\n return t1ha_ia32_AVX2_avail(features) ? t1ha0_ia32aes_avx2\n : t1ha0_ia32aes_avx;\n return t1ha0_ia32aes_noavx;\n }\n#endif /* T1HA0_AESNI_AVAILABLE && __ia32__ */\n\n#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n#if (UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul) && \\\n (!defined(T1HA1_DISABLED) || !defined(T1HA2_DISABLED))\n#ifndef T1HA1_DISABLED\n return t1ha1_be;\n#else\n return t1ha2_atonce;\n#endif /* T1HA1_DISABLED */\n#else\n return t1ha0_32be;\n#endif\n#else /* __BYTE_ORDER__ != __ORDER_BIG_ENDIAN__ */\n#if (UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul) && \\\n (!defined(T1HA1_DISABLED) || !defined(T1HA2_DISABLED))\n#ifndef T1HA1_DISABLED\n return t1ha1_le;\n#else\n return t1ha2_atonce;\n#endif /* T1HA1_DISABLED */\n#else\n return t1ha0_32le;\n#endif\n#endif /* __BYTE_ORDER__ */\n}\n\n#if T1HA_USE_INDIRECT_FUNCTIONS\n/* Use IFUNC (GNU ELF indirect functions) to choice implementation at runtime.\n * For more info please see\n * https://en.wikipedia.org/wiki/Executable_and_Linkable_Format\n * and https://sourceware.org/glibc/wiki/GNU_IFUNC */\n#if __has_attribute(__ifunc__)\nuint64_t t1ha0(const void *data, size_t len, uint64_t seed)\n __attribute__((__ifunc__(\"t1ha0_resolve\")));\n#else\n__asm(\"\\t.globl\\tt1ha0\\n\\t.type\\tt1ha0, \"\n \"%gnu_indirect_function\\n\\t.set\\tt1ha0,t1ha0_resolve\");\n#endif /* __has_attribute(__ifunc__) */\n\n#elif __GNUC_PREREQ(4, 0) || __has_attribute(__constructor__)\n\nuint64_t (*t1ha0_funcptr)(const void *, size_t, uint64_t);\n\nstatic __cold void __attribute__((__constructor__)) t1ha0_init(void) {\n t1ha0_funcptr = t1ha0_resolve();\n}\n\n#else /* T1HA_USE_INDIRECT_FUNCTIONS */\n\nstatic __cold uint64_t t1ha0_proxy(const void *data, size_t len,\n uint64_t seed) {\n t1ha0_funcptr = t1ha0_resolve();\n return t1ha0_funcptr(data, len, seed);\n}\n\nuint64_t (*t1ha0_funcptr)(const void *, size_t, uint64_t) = t1ha0_proxy;\n\n#endif /* !T1HA_USE_INDIRECT_FUNCTIONS */\n#endif /* T1HA0_RUNTIME_SELECT */\n\n#endif /* T1HA0_DISABLED */\n" }, { "path": "benchmarks/t1ha/src/t1ha0_ia32aes_a.h", "content": "/*\n * Copyright (c) 2016-2020 Positive Technologies, https://www.ptsecurity.com,\n * Fast Positive Hash.\n *\n * Portions Copyright (c) 2010-2020 Leonid Yuriev ,\n * The 1Hippeus project (t1h).\n *\n * This software is provided 'as-is', without any express or implied\n * warranty. In no event will the authors be held liable for any damages\n * arising from the use of this software.\n *\n * Permission is granted to anyone to use this software for any purpose,\n * including commercial applications, and to alter it and redistribute it\n * freely, subject to the following restrictions:\n *\n * 1. The origin of this software must not be misrepresented; you must not\n * claim that you wrote the original software. If you use this software\n * in a product, an acknowledgement in the product documentation would be\n * appreciated but is not required.\n * 2. Altered source versions must be plainly marked as such, and must not be\n * misrepresented as being the original software.\n * 3. This notice may not be removed or altered from any source distribution.\n */\n\n/*\n * t1ha = { Fast Positive Hash, aka \"Позитивный Хэш\" }\n * by [Positive Technologies](https://www.ptsecurity.ru)\n *\n * Briefly, it is a 64-bit Hash Function:\n * 1. Created for 64-bit little-endian platforms, in predominantly for x86_64,\n * but portable and without penalties it can run on any 64-bit CPU.\n * 2. In most cases up to 15% faster than City64, xxHash, mum-hash, metro-hash\n * and all others portable hash-functions (which do not use specific\n * hardware tricks).\n * 3. Not suitable for cryptography.\n *\n * The Future will (be) Positive. Всё будет хорошо.\n *\n * ACKNOWLEDGEMENT:\n * The t1ha was originally developed by Leonid Yuriev (Леонид Юрьев)\n * for The 1Hippeus project - zerocopy messaging in the spirit of Sparta!\n */\n\n#include \"t1ha_bits.h\"\n#include \"t1ha_selfcheck.h\"\n\n#if T1HA0_AESNI_AVAILABLE\n\nuint64_t T1HA_IA32AES_NAME(const void *data, size_t len, uint64_t seed) {\n uint64_t a = seed;\n uint64_t b = len;\n\n if (unlikely(len > 32)) {\n __m128i x = _mm_set_epi64x(a, b);\n __m128i y = _mm_aesenc_si128(x, _mm_set_epi64x(prime_5, prime_6));\n\n const __m128i *__restrict v = (const __m128i *)data;\n const __m128i *__restrict const detent =\n (const __m128i *)((const uint8_t *)data + len - 127);\n\n while (v < detent) {\n __m128i v0 = _mm_loadu_si128(v + 0);\n __m128i v1 = _mm_loadu_si128(v + 1);\n __m128i v2 = _mm_loadu_si128(v + 2);\n __m128i v3 = _mm_loadu_si128(v + 3);\n __m128i v4 = _mm_loadu_si128(v + 4);\n __m128i v5 = _mm_loadu_si128(v + 5);\n __m128i v6 = _mm_loadu_si128(v + 6);\n __m128i v7 = _mm_loadu_si128(v + 7);\n\n __m128i v0y = _mm_aesenc_si128(v0, y);\n __m128i v2x6 = _mm_aesenc_si128(v2, _mm_xor_si128(x, v6));\n __m128i v45_67 =\n _mm_xor_si128(_mm_aesenc_si128(v4, v5), _mm_add_epi64(v6, v7));\n\n __m128i v0y7_1 = _mm_aesdec_si128(_mm_sub_epi64(v7, v0y), v1);\n __m128i v2x6_3 = _mm_aesenc_si128(v2x6, v3);\n\n x = _mm_aesenc_si128(v45_67, _mm_add_epi64(x, y));\n y = _mm_aesenc_si128(v2x6_3, _mm_xor_si128(v0y7_1, v5));\n v += 8;\n }\n\n if (len & 64) {\n __m128i v0y = _mm_add_epi64(y, _mm_loadu_si128(v++));\n __m128i v1x = _mm_sub_epi64(x, _mm_loadu_si128(v++));\n x = _mm_aesdec_si128(x, v0y);\n y = _mm_aesdec_si128(y, v1x);\n\n __m128i v2y = _mm_add_epi64(y, _mm_loadu_si128(v++));\n __m128i v3x = _mm_sub_epi64(x, _mm_loadu_si128(v++));\n x = _mm_aesdec_si128(x, v2y);\n y = _mm_aesdec_si128(y, v3x);\n }\n\n if (len & 32) {\n __m128i v0y = _mm_add_epi64(y, _mm_loadu_si128(v++));\n __m128i v1x = _mm_sub_epi64(x, _mm_loadu_si128(v++));\n x = _mm_aesdec_si128(x, v0y);\n y = _mm_aesdec_si128(y, v1x);\n }\n\n if (len & 16) {\n y = _mm_add_epi64(x, y);\n x = _mm_aesdec_si128(x, _mm_loadu_si128(v++));\n }\n\n x = _mm_add_epi64(_mm_aesdec_si128(x, _mm_aesenc_si128(y, x)), y);\n#if defined(__x86_64__) || defined(_M_X64)\n#if defined(__SSE4_1__) || defined(__AVX__)\n a = _mm_extract_epi64(x, 0);\n b = _mm_extract_epi64(x, 1);\n#else\n a = _mm_cvtsi128_si64(x);\n b = _mm_cvtsi128_si64(_mm_unpackhi_epi64(x, x));\n#endif\n#else\n#if defined(__SSE4_1__) || defined(__AVX__)\n a = (uint32_t)_mm_extract_epi32(x, 0) | (uint64_t)_mm_extract_epi32(x, 1)\n << 32;\n b = (uint32_t)_mm_extract_epi32(x, 2) | (uint64_t)_mm_extract_epi32(x, 3)\n << 32;\n#else\n a = (uint32_t)_mm_cvtsi128_si32(x);\n a |= (uint64_t)_mm_cvtsi128_si32(_mm_shuffle_epi32(x, 1)) << 32;\n x = _mm_unpackhi_epi64(x, x);\n b = (uint32_t)_mm_cvtsi128_si32(x);\n b |= (uint64_t)_mm_cvtsi128_si32(_mm_shuffle_epi32(x, 1)) << 32;\n#endif\n#endif\n#ifdef __AVX__\n _mm256_zeroupper();\n#elif !(defined(_X86_64_) || defined(__x86_64__) || defined(_M_X64) || \\\n defined(__e2k__))\n _mm_empty();\n#endif\n data = v;\n len &= 15;\n }\n\n const uint64_t *v = (const uint64_t *)data;\n switch (len) {\n default:\n mixup64(&a, &b, fetch64_le_unaligned(v++), prime_4);\n /* fall through */\n case 24:\n case 23:\n case 22:\n case 21:\n case 20:\n case 19:\n case 18:\n case 17:\n mixup64(&b, &a, fetch64_le_unaligned(v++), prime_3);\n /* fall through */\n case 16:\n case 15:\n case 14:\n case 13:\n case 12:\n case 11:\n case 10:\n case 9:\n mixup64(&a, &b, fetch64_le_unaligned(v++), prime_2);\n /* fall through */\n case 8:\n case 7:\n case 6:\n case 5:\n case 4:\n case 3:\n case 2:\n case 1:\n mixup64(&b, &a, tail64_le_unaligned(v, len), prime_1);\n /* fall through */\n case 0:\n return final64(a, b);\n }\n}\n\n#endif /* T1HA0_AESNI_AVAILABLE */\n#undef T1HA_IA32AES_NAME\n" }, { "path": "benchmarks/t1ha/src/t1ha0_ia32aes_avx.c", "content": "#ifndef T1HA0_DISABLED\n#define T1HA_IA32AES_NAME t1ha0_ia32aes_avx\n#include \"t1ha0_ia32aes_a.h\"\n#endif /* T1HA0_DISABLED */\n" }, { "path": "benchmarks/t1ha/src/t1ha0_ia32aes_avx2.c", "content": "#ifndef T1HA0_DISABLED\n#define T1HA_IA32AES_NAME t1ha0_ia32aes_avx2\n#include \"t1ha0_ia32aes_b.h\"\n#endif /* T1HA0_DISABLED */\n" }, { "path": "benchmarks/t1ha/src/t1ha0_ia32aes_b.h", "content": "/*\n * Copyright (c) 2016-2020 Positive Technologies, https://www.ptsecurity.com,\n * Fast Positive Hash.\n *\n * Portions Copyright (c) 2010-2020 Leonid Yuriev ,\n * The 1Hippeus project (t1h).\n *\n * This software is provided 'as-is', without any express or implied\n * warranty. In no event will the authors be held liable for any damages\n * arising from the use of this software.\n *\n * Permission is granted to anyone to use this software for any purpose,\n * including commercial applications, and to alter it and redistribute it\n * freely, subject to the following restrictions:\n *\n * 1. The origin of this software must not be misrepresented; you must not\n * claim that you wrote the original software. If you use this software\n * in a product, an acknowledgement in the product documentation would be\n * appreciated but is not required.\n * 2. Altered source versions must be plainly marked as such, and must not be\n * misrepresented as being the original software.\n * 3. This notice may not be removed or altered from any source distribution.\n */\n\n/*\n * t1ha = { Fast Positive Hash, aka \"Позитивный Хэш\" }\n * by [Positive Technologies](https://www.ptsecurity.ru)\n *\n * Briefly, it is a 64-bit Hash Function:\n * 1. Created for 64-bit little-endian platforms, in predominantly for x86_64,\n * but portable and without penalties it can run on any 64-bit CPU.\n * 2. In most cases up to 15% faster than City64, xxHash, mum-hash, metro-hash\n * and all others portable hash-functions (which do not use specific\n * hardware tricks).\n * 3. Not suitable for cryptography.\n *\n * The Future will (be) Positive. Всё будет хорошо.\n *\n * ACKNOWLEDGEMENT:\n * The t1ha was originally developed by Leonid Yuriev (Леонид Юрьев)\n * for The 1Hippeus project - zerocopy messaging in the spirit of Sparta!\n */\n\n#include \"t1ha_bits.h\"\n#include \"t1ha_selfcheck.h\"\n\n#if T1HA0_AESNI_AVAILABLE\n\nuint64_t T1HA_IA32AES_NAME(const void *data, size_t len, uint64_t seed) {\n uint64_t a = seed;\n uint64_t b = len;\n\n if (unlikely(len > 32)) {\n __m128i x = _mm_set_epi64x(a, b);\n __m128i y = _mm_aesenc_si128(x, _mm_set_epi64x(prime_0, prime_1));\n\n const __m128i *v = (const __m128i *)data;\n const __m128i *const detent =\n (const __m128i *)((const uint8_t *)data + (len & ~15ul));\n data = detent;\n\n if (len & 16) {\n x = _mm_add_epi64(x, _mm_loadu_si128(v++));\n y = _mm_aesenc_si128(x, y);\n }\n len &= 15;\n\n if (v + 7 < detent) {\n __m128i salt = y;\n do {\n __m128i t = _mm_aesenc_si128(_mm_loadu_si128(v++), salt);\n t = _mm_aesdec_si128(t, _mm_loadu_si128(v++));\n t = _mm_aesdec_si128(t, _mm_loadu_si128(v++));\n t = _mm_aesdec_si128(t, _mm_loadu_si128(v++));\n\n t = _mm_aesdec_si128(t, _mm_loadu_si128(v++));\n t = _mm_aesdec_si128(t, _mm_loadu_si128(v++));\n t = _mm_aesdec_si128(t, _mm_loadu_si128(v++));\n t = _mm_aesdec_si128(t, _mm_loadu_si128(v++));\n\n salt = _mm_add_epi64(salt, _mm_set_epi64x(prime_5, prime_6));\n t = _mm_aesenc_si128(x, t);\n x = _mm_add_epi64(y, x);\n y = t;\n } while (v + 7 < detent);\n }\n\n while (v < detent) {\n __m128i v0y = _mm_add_epi64(y, _mm_loadu_si128(v++));\n __m128i v1x = _mm_sub_epi64(x, _mm_loadu_si128(v++));\n x = _mm_aesdec_si128(x, v0y);\n y = _mm_aesdec_si128(y, v1x);\n }\n\n x = _mm_add_epi64(_mm_aesdec_si128(x, _mm_aesenc_si128(y, x)), y);\n#if defined(__x86_64__) || defined(_M_X64)\n#if defined(__SSE4_1__) || defined(__AVX__)\n a = _mm_extract_epi64(x, 0);\n b = _mm_extract_epi64(x, 1);\n#else\n a = _mm_cvtsi128_si64(x);\n b = _mm_cvtsi128_si64(_mm_unpackhi_epi64(x, x));\n#endif\n#else\n#if defined(__SSE4_1__) || defined(__AVX__)\n a = (uint32_t)_mm_extract_epi32(x, 0) | (uint64_t)_mm_extract_epi32(x, 1)\n << 32;\n b = (uint32_t)_mm_extract_epi32(x, 2) | (uint64_t)_mm_extract_epi32(x, 3)\n << 32;\n#else\n a = (uint32_t)_mm_cvtsi128_si32(x);\n a |= (uint64_t)_mm_cvtsi128_si32(_mm_shuffle_epi32(x, 1)) << 32;\n x = _mm_unpackhi_epi64(x, x);\n b = (uint32_t)_mm_cvtsi128_si32(x);\n b |= (uint64_t)_mm_cvtsi128_si32(_mm_shuffle_epi32(x, 1)) << 32;\n#endif\n#endif\n#ifdef __AVX__\n _mm256_zeroupper();\n#elif !(defined(_X86_64_) || defined(__x86_64__) || defined(_M_X64) || \\\n defined(__e2k__))\n _mm_empty();\n#endif\n }\n\n const uint64_t *v = (const uint64_t *)data;\n switch (len) {\n default:\n mixup64(&a, &b, fetch64_le_unaligned(v++), prime_4);\n /* fall through */\n case 24:\n case 23:\n case 22:\n case 21:\n case 20:\n case 19:\n case 18:\n case 17:\n mixup64(&b, &a, fetch64_le_unaligned(v++), prime_3);\n /* fall through */\n case 16:\n case 15:\n case 14:\n case 13:\n case 12:\n case 11:\n case 10:\n case 9:\n mixup64(&a, &b, fetch64_le_unaligned(v++), prime_2);\n /* fall through */\n case 8:\n case 7:\n case 6:\n case 5:\n case 4:\n case 3:\n case 2:\n case 1:\n mixup64(&b, &a, tail64_le_unaligned(v, len), prime_1);\n /* fall through */\n case 0:\n return final64(a, b);\n }\n}\n\n#endif /* T1HA0_AESNI_AVAILABLE */\n#undef T1HA_IA32AES_NAME\n" }, { "path": "benchmarks/t1ha/src/t1ha0_ia32aes_noavx.c", "content": "#ifndef T1HA0_DISABLED\n#define T1HA_IA32AES_NAME t1ha0_ia32aes_noavx\n#include \"t1ha0_ia32aes_a.h\"\n#endif /* T1HA0_DISABLED */\n" }, { "path": "benchmarks/t1ha/src/t1ha0_selfcheck.c", "content": "/*\n * Copyright (c) 2016-2020 Positive Technologies, https://www.ptsecurity.com,\n * Fast Positive Hash.\n *\n * Portions Copyright (c) 2010-2020 Leonid Yuriev ,\n * The 1Hippeus project (t1h).\n *\n * This software is provided 'as-is', without any express or implied\n * warranty. In no event will the authors be held liable for any damages\n * arising from the use of this software.\n *\n * Permission is granted to anyone to use this software for any purpose,\n * including commercial applications, and to alter it and redistribute it\n * freely, subject to the following restrictions:\n *\n * 1. The origin of this software must not be misrepresented; you must not\n * claim that you wrote the original software. If you use this software\n * in a product, an acknowledgement in the product documentation would be\n * appreciated but is not required.\n * 2. Altered source versions must be plainly marked as such, and must not be\n * misrepresented as being the original software.\n * 3. This notice may not be removed or altered from any source distribution.\n */\n\n/*\n * t1ha = { Fast Positive Hash, aka \"Позитивный Хэш\" }\n * by [Positive Technologies](https://www.ptsecurity.ru)\n *\n * Briefly, it is a 64-bit Hash Function:\n * 1. Created for 64-bit little-endian platforms, in predominantly for x86_64,\n * but portable and without penalties it can run on any 64-bit CPU.\n * 2. In most cases up to 15% faster than City64, xxHash, mum-hash, metro-hash\n * and all others portable hash-functions (which do not use specific\n * hardware tricks).\n * 3. Not suitable for cryptography.\n *\n * The Future will (be) Positive. Всё будет хорошо.\n *\n * ACKNOWLEDGEMENT:\n * The t1ha was originally developed by Leonid Yuriev (Леонид Юрьев)\n * for The 1Hippeus project - zerocopy messaging in the spirit of Sparta!\n */\n\n#ifndef T1HA0_DISABLED\n#include \"t1ha_bits.h\"\n#include \"t1ha_selfcheck.h\"\n\n/* *INDENT-OFF* */\n/* clang-format off */\n\nconst uint64_t t1ha_refval_32le[81] = { 0,\n 0xC92229C10FAEA50E, 0x3DF1354B0DFDC443, 0x968F016D60417BB3, 0x85AAFB50C6DA770F,\n 0x66CCE3BB6842C7D6, 0xDDAA39C11537C226, 0x35958D281F0C9C8C, 0x8C5D64B091DE608E,\n 0x4094DF680D39786B, 0x1014F4AA2A2EDF4D, 0x39D21891615AA310, 0x7EF51F67C398C7C4,\n 0x06163990DDBF319D, 0xE229CAA00C8D6F3F, 0xD2240B4B0D54E0F5, 0xEA2E7E905DDEAF94,\n 0x8D4F8A887183A5CE, 0x44337F9A63C5820C, 0x94938D1E86A9B797, 0x96E9CABA5CA210CC,\n 0x6EFBB9CC9E8F7708, 0x3D12EA0282FB8BBC, 0x5DA781EE205A2C48, 0xFA4A51A12677FE12,\n 0x81D5F04E20660B28, 0x57258D043BCD3841, 0x5C9BEB62059C1ED2, 0x57A02162F9034B33,\n 0xBA2A13E457CE19B8, 0xE593263BF9451F3A, 0x0BC1175539606BC5, 0xA3E2929E9C5F289F,\n 0x86BDBD06835E35F7, 0xA180950AB48BAADC, 0x7812C994D9924028, 0x308366011415F46B,\n 0x77FE9A9991C5F959, 0x925C340B70B0B1E3, 0xCD9C5BA4C41E2E10, 0x7CC4E7758B94CD93,\n 0x898B235962EA4625, 0xD7E3E5BF22893286, 0x396F4CDD33056C64, 0x740AB2E32F17CD9F,\n 0x60D12FF9CD15B321, 0xBEE3A6C9903A81D8, 0xB47040913B33C35E, 0x19EE8C2ACC013CFF,\n 0x5DEC94C5783B55C4, 0x78DC122D562C5F1D, 0x6520F008DA1C181E, 0x77CAF155A36EBF7C,\n 0x0A09E02BDB883CA6, 0xFD5D9ADA7E3FB895, 0xC6F5FDD9EEAB83B5, 0x84589BB29F52A92A,\n 0x9B2517F13F8E9814, 0x6F752AF6A52E31EC, 0x8E717799E324CE8A, 0x84D90AEF39262D58,\n 0x79C27B13FC28944D, 0xE6D6DF6438E0044A, 0x51B603E400D79CA4, 0x6A902B28C588B390,\n 0x8D7F8DE9E6CB1D83, 0xCF1A4DC11CA7F044, 0xEF02E43C366786F1, 0x89915BCDBCFBE30F,\n 0x5928B306F1A9CC7F, 0xA8B59092996851C5, 0x22050A20427E8B25, 0x6E6D64018941E7EE,\n 0x9798C898B81AE846, 0x80EF218CDC30124A, 0xFCE45E60D55B0284, 0x4010E735D3147C35,\n 0xEB647D999FD8DC7E, 0xD3544DCAB14FE907, 0xB588B27D8438700C, 0xA49EBFC43E057A4C\n};\n\nconst uint64_t t1ha_refval_32be[81] = { 0,\n 0xC92229C10FAEA50E, 0x0FE212630DD87E0F, 0x968F016D60417BB3, 0xE6B12B2C889913AB,\n 0xAA3787887A9DA368, 0x06EE7202D53CEF39, 0x6149AFB2C296664B, 0x86C893210F9A5805,\n 0x8379E5DA988AA04C, 0x24763AA7CE411A60, 0x9CF9C64B395A4CF8, 0xFFC192C338DDE904,\n 0x094575BAB319E5F5, 0xBBBACFE7728C6511, 0x36B8C3CEBE4EF409, 0xAA0BA8A3397BA4D0,\n 0xF9F85CF7124EE653, 0x3ADF4F7DF2A887AE, 0xAA2A0F5964AA9A7A, 0xF18B563F42D36EB8,\n 0x034366CEF8334F5C, 0xAE2E85180E330E5F, 0xA5CE9FBFDF5C65B8, 0x5E509F25A9CA9B0B,\n 0xE30D1358C2013BD2, 0xBB3A04D5EB8111FE, 0xB04234E82A15A28D, 0x87426A56D0EA0E2F,\n 0x095086668E07F9F8, 0xF4CD3A43B6A6AEA5, 0x73F9B9B674D472A6, 0x558344229A1E4DCF,\n 0x0AD4C95B2279181A, 0x5E3D19D80821CA6B, 0x652492D25BEBA258, 0xEFA84B02EAB849B1,\n 0x81AD2D253059AC2C, 0x1400CCB0DFB2F457, 0x5688DC72A839860E, 0x67CC130E0FD1B0A7,\n 0x0A851E3A94E21E69, 0x2EA0000B6A073907, 0xAE9776FF9BF1D02E, 0xC0A96B66B160631C,\n 0xA93341DE4ED7C8F0, 0x6FBADD8F5B85E141, 0xB7D295F1C21E0CBA, 0x6D6114591B8E434F,\n 0xF5B6939B63D97BE7, 0x3C80D5053F0E5DB4, 0xAC520ACC6B73F62D, 0xD1051F5841CF3966,\n 0x62245AEA644AE760, 0x0CD56BE15497C62D, 0x5BB93435C4988FB6, 0x5FADB88EB18DB512,\n 0xC897CAE2242475CC, 0xF1A094EF846DC9BB, 0x2B1D8B24924F79B6, 0xC6DF0C0E8456EB53,\n 0xE6A40128303A9B9C, 0x64D37AF5EFFA7BD9, 0x90FEB70A5AE2A598, 0xEC3BA5F126D9FF4B,\n 0x3121C8EC3AC51B29, 0x3B41C4D422166EC1, 0xB4878DDCBF48ED76, 0x5CB850D77CB762E4,\n 0x9A27A43CC1DD171F, 0x2FDFFC6F99CB424A, 0xF54A57E09FDEA7BB, 0x5F78E5EE2CAB7039,\n 0xB8BA95883DB31CBA, 0x131C61EB84AF86C3, 0x84B1F64E9C613DA7, 0xE94C1888C0C37C02,\n 0xEA08F8BFB2039CDE, 0xCCC6D04D243EC753, 0x8977D105298B0629, 0x7AAA976494A5905E\n};\n\n#if T1HA0_AESNI_AVAILABLE\nconst uint64_t t1ha_refval_ia32aes_a[81] = { 0,\n 0x772C7311BE32FF42, 0xB231AC660E5B23B5, 0x71F6DF5DA3B4F532, 0x555859635365F660,\n 0xE98808F1CD39C626, 0x2EB18FAF2163BB09, 0x7B9DD892C8019C87, 0xE2B1431C4DA4D15A,\n 0x1984E718A5477F70, 0x08DD17B266484F79, 0x4C83A05D766AD550, 0x92DCEBB131D1907D,\n 0xD67BC6FC881B8549, 0xF6A9886555FBF66B, 0x6E31616D7F33E25E, 0x36E31B7426E3049D,\n 0x4F8E4FAF46A13F5F, 0x03EB0CB3253F819F, 0x636A7769905770D2, 0x3ADF3781D16D1148,\n 0x92D19CB1818BC9C2, 0x283E68F4D459C533, 0xFA83A8A88DECAA04, 0x8C6F00368EAC538C,\n 0x7B66B0CF3797B322, 0x5131E122FDABA3FF, 0x6E59FF515C08C7A9, 0xBA2C5269B2C377B0,\n 0xA9D24FD368FE8A2B, 0x22DB13D32E33E891, 0x7B97DFC804B876E5, 0xC598BDFCD0E834F9,\n 0xB256163D3687F5A7, 0x66D7A73C6AEF50B3, 0xBB34C6A4396695D2, 0x7F46E1981C3256AD,\n 0x4B25A9B217A6C5B4, 0x7A0A6BCDD2321DA9, 0x0A1F55E690A7B44E, 0x8F451A91D7F05244,\n 0x624D5D3C9B9800A7, 0x09DDC2B6409DDC25, 0x3E155765865622B6, 0x96519FAC9511B381,\n 0x512E58482FE4FBF0, 0x1AB260EA7D54AE1C, 0x67976F12CC28BBBD, 0x0607B5B2E6250156,\n 0x7E700BEA717AD36E, 0x06A058D9D61CABB3, 0x57DA5324A824972F, 0x1193BA74DBEBF7E7,\n 0xC18DC3140E7002D4, 0x9F7CCC11DFA0EF17, 0xC487D6C20666A13A, 0xB67190E4B50EF0C8,\n 0xA53DAA608DF0B9A5, 0x7E13101DE87F9ED3, 0x7F8955AE2F05088B, 0x2DF7E5A097AD383F,\n 0xF027683A21EA14B5, 0x9BB8AEC3E3360942, 0x92BE39B54967E7FE, 0x978C6D332E7AFD27,\n 0xED512FE96A4FAE81, 0x9E1099B8140D7BA3, 0xDFD5A5BE1E6FE9A6, 0x1D82600E23B66DD4,\n 0x3FA3C3B7EE7B52CE, 0xEE84F7D2A655EF4C, 0x2A4361EC769E3BEB, 0x22E4B38916636702,\n 0x0063096F5D39A115, 0x6C51B24DAAFA5434, 0xBAFB1DB1B411E344, 0xFF529F161AE0C4B0,\n 0x1290EAE3AC0A686F, 0xA7B0D4585447D1BE, 0xAED3D18CB6CCAD53, 0xFC73D46F8B41BEC6\n};\n\nconst uint64_t t1ha_refval_ia32aes_b[81] = { 0,\n 0x772C7311BE32FF42, 0x4398F62A8CB6F72A, 0x71F6DF5DA3B4F532, 0x555859635365F660,\n 0xE98808F1CD39C626, 0x2EB18FAF2163BB09, 0x7B9DD892C8019C87, 0xE2B1431C4DA4D15A,\n 0x1984E718A5477F70, 0x08DD17B266484F79, 0x4C83A05D766AD550, 0x92DCEBB131D1907D,\n 0xD67BC6FC881B8549, 0xF6A9886555FBF66B, 0x6E31616D7F33E25E, 0x36E31B7426E3049D,\n 0x4F8E4FAF46A13F5F, 0x03EB0CB3253F819F, 0x636A7769905770D2, 0x3ADF3781D16D1148,\n 0x92D19CB1818BC9C2, 0x283E68F4D459C533, 0xFA83A8A88DECAA04, 0x8C6F00368EAC538C,\n 0x7B66B0CF3797B322, 0x5131E122FDABA3FF, 0x6E59FF515C08C7A9, 0xBA2C5269B2C377B0,\n 0xA9D24FD368FE8A2B, 0x22DB13D32E33E891, 0x7B97DFC804B876E5, 0xC598BDFCD0E834F9,\n 0xB256163D3687F5A7, 0x66D7A73C6AEF50B3, 0xE810F88E85CEA11A, 0x4814F8F3B83E4394,\n 0x9CABA22D10A2F690, 0x0D10032511F58111, 0xE9A36EF5EEA3CD58, 0xC79242DE194D9D7C,\n 0xC3871AA0435EE5C8, 0x52890BED43CCF4CD, 0x07A1D0861ACCD373, 0x227B816FF0FEE9ED,\n 0x59FFBF73AACFC0C4, 0x09AB564F2BEDAD0C, 0xC05F744F2EE38318, 0x7B50B621D547C661,\n 0x0C1F71CB4E68E5D1, 0x0E33A47881D4DBAA, 0xF5C3BF198E9A7C2E, 0x16328FD8C0F68A91,\n 0xA3E399C9AB3E9A59, 0x163AE71CBCBB18B8, 0x18F17E4A8C79F7AB, 0x9250E2EA37014B45,\n 0x7BBBB111D60B03E4, 0x3DAA4A3071A0BD88, 0xA28828D790A2D6DC, 0xBC70FC88F64BE3F1,\n 0xA3E48008BA4333C7, 0x739E435ACAFC79F7, 0x42BBB360BE007CC6, 0x4FFB6FD2AF74EC92,\n 0x2A799A2994673146, 0xBE0A045B69D48E9F, 0x549432F54FC6A278, 0x371D3C60369FC702,\n 0xDB4557D415B08CA7, 0xE8692F0A83850B37, 0x022E46AEB36E9AAB, 0x117AC9B814E4652D,\n 0xA361041267AE9048, 0x277CB51C961C3DDA, 0xAFFC96F377CB8A8D, 0x83CC79FA01DD1BA7,\n 0xA494842ACF4B802C, 0xFC6D9CDDE2C34A3F, 0x4ED6863CE455F7A7, 0x630914D0DB7AAE98\n};\n#endif /* T1HA0_AESNI_AVAILABLE */\n\n/* *INDENT-ON* */\n/* clang-format on */\n\n__cold int t1ha_selfcheck__t1ha0_32le(void) {\n return t1ha_selfcheck(t1ha0_32le, t1ha_refval_32le);\n}\n\n__cold int t1ha_selfcheck__t1ha0_32be(void) {\n return t1ha_selfcheck(t1ha0_32be, t1ha_refval_32be);\n}\n\n#if T1HA0_AESNI_AVAILABLE\n__cold int t1ha_selfcheck__t1ha0_ia32aes_noavx(void) {\n return t1ha_selfcheck(t1ha0_ia32aes_noavx, t1ha_refval_ia32aes_a);\n}\n\n__cold int t1ha_selfcheck__t1ha0_ia32aes_avx(void) {\n return t1ha_selfcheck(t1ha0_ia32aes_avx, t1ha_refval_ia32aes_a);\n}\n\n#ifndef __e2k__\n__cold int t1ha_selfcheck__t1ha0_ia32aes_avx2(void) {\n return t1ha_selfcheck(t1ha0_ia32aes_avx2, t1ha_refval_ia32aes_b);\n}\n#endif /* ! __e2k__ */\n#endif /* if T1HA0_AESNI_AVAILABLE */\n\n__cold int t1ha_selfcheck__t1ha0(void) {\n int rc = t1ha_selfcheck__t1ha0_32le() | t1ha_selfcheck__t1ha0_32be();\n\n#if (UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul) && \\\n (!defined(T1HA1_DISABLED) || !defined(T1HA2_DISABLED))\n#if defined(T1HA1_DISABLED)\n rc |= t1ha_selfcheck__t1ha2();\n#else\n rc |= t1ha_selfcheck__t1ha1();\n#endif /* T1HA1_DISABLED */\n#endif /* 32/64 */\n\n#if T1HA0_AESNI_AVAILABLE\n#ifdef __e2k__\n rc |= t1ha_selfcheck__t1ha0_ia32aes_noavx();\n rc |= t1ha_selfcheck__t1ha0_ia32aes_avx();\n#else\n uint64_t features = t1ha_ia32cpu_features();\n if (t1ha_ia32_AESNI_avail(features)) {\n rc |= t1ha_selfcheck__t1ha0_ia32aes_noavx();\n if (t1ha_ia32_AVX_avail(features)) {\n rc |= t1ha_selfcheck__t1ha0_ia32aes_avx();\n if (t1ha_ia32_AVX2_avail(features))\n rc |= t1ha_selfcheck__t1ha0_ia32aes_avx2();\n }\n }\n#endif /* __e2k__ */\n#endif /* T1HA0_AESNI_AVAILABLE */\n\n return rc;\n}\n\n#endif /* T1HA0_DISABLED */\n" }, { "path": "benchmarks/t1ha/src/t1ha1.c", "content": "/*\n * Copyright (c) 2016-2020 Positive Technologies, https://www.ptsecurity.com,\n * Fast Positive Hash.\n *\n * Portions Copyright (c) 2010-2020 Leonid Yuriev ,\n * The 1Hippeus project (t1h).\n *\n * This software is provided 'as-is', without any express or implied\n * warranty. In no event will the authors be held liable for any damages\n * arising from the use of this software.\n *\n * Permission is granted to anyone to use this software for any purpose,\n * including commercial applications, and to alter it and redistribute it\n * freely, subject to the following restrictions:\n *\n * 1. The origin of this software must not be misrepresented; you must not\n * claim that you wrote the original software. If you use this software\n * in a product, an acknowledgement in the product documentation would be\n * appreciated but is not required.\n * 2. Altered source versions must be plainly marked as such, and must not be\n * misrepresented as being the original software.\n * 3. This notice may not be removed or altered from any source distribution.\n */\n\n/*\n * t1ha = { Fast Positive Hash, aka \"Позитивный Хэш\" }\n * by [Positive Technologies](https://www.ptsecurity.ru)\n *\n * Briefly, it is a 64-bit Hash Function:\n * 1. Created for 64-bit little-endian platforms, in predominantly for x86_64,\n * but portable and without penalties it can run on any 64-bit CPU.\n * 2. In most cases up to 15% faster than City64, xxHash, mum-hash, metro-hash\n * and all others portable hash-functions (which do not use specific\n * hardware tricks).\n * 3. Not suitable for cryptography.\n *\n * The Future will (be) Positive. Всё будет хорошо.\n *\n * ACKNOWLEDGEMENT:\n * The t1ha was originally developed by Leonid Yuriev (Леонид Юрьев)\n * for The 1Hippeus project - zerocopy messaging in the spirit of Sparta!\n */\n\n#ifndef T1HA1_DISABLED\n#include \"t1ha_bits.h\"\n#include \"t1ha_selfcheck.h\"\n\n/* xor-mul-xor mixer */\nstatic __inline uint64_t mix64(uint64_t v, uint64_t p) {\n v *= p;\n return v ^ rot64(v, 41);\n}\n\nstatic __inline uint64_t final_weak_avalanche(uint64_t a, uint64_t b) {\n /* LY: for performance reason on a some not high-end CPUs\n * I replaced the second mux64() operation by mix64().\n * Unfortunately this approach fails the \"strict avalanche criteria\",\n * see test results at https://github.com/demerphq/smhasher. */\n return mux64(rot64(a + b, 17), prime_4) + mix64(a ^ b, prime_0);\n}\n\n/* TODO: C++ template in the next version */\n#define T1HA1_BODY(ENDIANNES, ALIGNESS) \\\n const uint64_t *v = (const uint64_t *)data; \\\n if (unlikely(len > 32)) { \\\n uint64_t c = rot64(len, 17) + seed; \\\n uint64_t d = len ^ rot64(seed, 17); \\\n const uint64_t *detent = \\\n (const uint64_t *)((const uint8_t *)data + len - 31); \\\n do { \\\n const uint64_t w0 = fetch64_##ENDIANNES##_##ALIGNESS(v + 0); \\\n const uint64_t w1 = fetch64_##ENDIANNES##_##ALIGNESS(v + 1); \\\n const uint64_t w2 = fetch64_##ENDIANNES##_##ALIGNESS(v + 2); \\\n const uint64_t w3 = fetch64_##ENDIANNES##_##ALIGNESS(v + 3); \\\n v += 4; \\\n prefetch(v); \\\n \\\n const uint64_t d02 = w0 ^ rot64(w2 + d, 17); \\\n const uint64_t c13 = w1 ^ rot64(w3 + c, 17); \\\n d -= b ^ rot64(w1, 31); \\\n c += a ^ rot64(w0, 41); \\\n b ^= prime_0 * (c13 + w2); \\\n a ^= prime_1 * (d02 + w3); \\\n } while (likely(v < detent)); \\\n \\\n a ^= prime_6 * (rot64(c, 17) + d); \\\n b ^= prime_5 * (c + rot64(d, 17)); \\\n len &= 31; \\\n } \\\n \\\n switch (len) { \\\n default: \\\n b += mux64(fetch64_##ENDIANNES##_##ALIGNESS(v++), prime_4); \\\n /* fall through */ \\\n case 24: \\\n case 23: \\\n case 22: \\\n case 21: \\\n case 20: \\\n case 19: \\\n case 18: \\\n case 17: \\\n a += mux64(fetch64_##ENDIANNES##_##ALIGNESS(v++), prime_3); \\\n /* fall through */ \\\n case 16: \\\n case 15: \\\n case 14: \\\n case 13: \\\n case 12: \\\n case 11: \\\n case 10: \\\n case 9: \\\n b += mux64(fetch64_##ENDIANNES##_##ALIGNESS(v++), prime_2); \\\n /* fall through */ \\\n case 8: \\\n case 7: \\\n case 6: \\\n case 5: \\\n case 4: \\\n case 3: \\\n case 2: \\\n case 1: \\\n a += mux64(tail64_##ENDIANNES##_##ALIGNESS(v, len), prime_1); \\\n /* fall through */ \\\n case 0: \\\n return final_weak_avalanche(a, b); \\\n }\n\nuint64_t t1ha1_le(const void *data, size_t len, uint64_t seed) {\n uint64_t a = seed;\n uint64_t b = len;\n\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__EFFICIENT\n T1HA1_BODY(le, unaligned);\n#else\n const bool misaligned = (((uintptr_t)data) & (ALIGNMENT_64 - 1)) != 0;\n if (misaligned) {\n T1HA1_BODY(le, unaligned);\n } else {\n T1HA1_BODY(le, aligned);\n }\n#endif\n}\n\nuint64_t t1ha1_be(const void *data, size_t len, uint64_t seed) {\n uint64_t a = seed;\n uint64_t b = len;\n\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__EFFICIENT\n T1HA1_BODY(be, unaligned);\n#else\n const bool misaligned = (((uintptr_t)data) & (ALIGNMENT_64 - 1)) != 0;\n if (misaligned) {\n T1HA1_BODY(be, unaligned);\n } else {\n T1HA1_BODY(be, aligned);\n }\n#endif\n}\n\n#endif /* T1HA1_DISABLED */\n" }, { "path": "benchmarks/t1ha/src/t1ha1_selfcheck.c", "content": "/*\n * Copyright (c) 2016-2020 Positive Technologies, https://www.ptsecurity.com,\n * Fast Positive Hash.\n *\n * Portions Copyright (c) 2010-2020 Leonid Yuriev ,\n * The 1Hippeus project (t1h).\n *\n * This software is provided 'as-is', without any express or implied\n * warranty. In no event will the authors be held liable for any damages\n * arising from the use of this software.\n *\n * Permission is granted to anyone to use this software for any purpose,\n * including commercial applications, and to alter it and redistribute it\n * freely, subject to the following restrictions:\n *\n * 1. The origin of this software must not be misrepresented; you must not\n * claim that you wrote the original software. If you use this software\n * in a product, an acknowledgement in the product documentation would be\n * appreciated but is not required.\n * 2. Altered source versions must be plainly marked as such, and must not be\n * misrepresented as being the original software.\n * 3. This notice may not be removed or altered from any source distribution.\n */\n\n/*\n * t1ha = { Fast Positive Hash, aka \"Позитивный Хэш\" }\n * by [Positive Technologies](https://www.ptsecurity.ru)\n *\n * Briefly, it is a 64-bit Hash Function:\n * 1. Created for 64-bit little-endian platforms, in predominantly for x86_64,\n * but portable and without penalties it can run on any 64-bit CPU.\n * 2. In most cases up to 15% faster than City64, xxHash, mum-hash, metro-hash\n * and all others portable hash-functions (which do not use specific\n * hardware tricks).\n * 3. Not suitable for cryptography.\n *\n * The Future will (be) Positive. Всё будет хорошо.\n *\n * ACKNOWLEDGEMENT:\n * The t1ha was originally developed by Leonid Yuriev (Леонид Юрьев)\n * for The 1Hippeus project - zerocopy messaging in the spirit of Sparta!\n */\n\n#ifndef T1HA1_DISABLED\n#include \"t1ha_bits.h\"\n#include \"t1ha_selfcheck.h\"\n\n/* *INDENT-OFF* */\n/* clang-format off */\n\nconst uint64_t t1ha_refval_64le[81] = { 0,\n 0x6A580668D6048674, 0xA2FE904AFF0D0879, 0xE3AB9C06FAF4D023, 0x6AF1C60874C95442,\n 0xB3557E561A6C5D82, 0x0AE73C696F3D37C0, 0x5EF25F7062324941, 0x9B784F3B4CE6AF33,\n 0x6993BB206A74F070, 0xF1E95DF109076C4C, 0x4E1EB70C58E48540, 0x5FDD7649D8EC44E4,\n 0x559122C706343421, 0x380133D58665E93D, 0x9CE74296C8C55AE4, 0x3556F9A5757AB6D0,\n 0xF62751F7F25C469E, 0x851EEC67F6516D94, 0xED463EE3848A8695, 0xDC8791FEFF8ED3AC,\n 0x2569C744E1A282CF, 0xF90EB7C1D70A80B9, 0x68DFA6A1B8050A4C, 0x94CCA5E8210D2134,\n 0xF5CC0BEABC259F52, 0x40DBC1F51618FDA7, 0x0807945BF0FB52C6, 0xE5EF7E09DE70848D,\n 0x63E1DF35FEBE994A, 0x2025E73769720D5A, 0xAD6120B2B8A152E1, 0x2A71D9F13959F2B7,\n 0x8A20849A27C32548, 0x0BCBC9FE3B57884E, 0x0E028D255667AEAD, 0xBE66DAD3043AB694,\n 0xB00E4C1238F9E2D4, 0x5C54BDE5AE280E82, 0x0E22B86754BC3BC4, 0x016707EBF858B84D,\n 0x990015FBC9E095EE, 0x8B9AF0A3E71F042F, 0x6AA56E88BD380564, 0xAACE57113E681A0F,\n 0x19F81514AFA9A22D, 0x80DABA3D62BEAC79, 0x715210412CABBF46, 0xD8FA0B9E9D6AA93F,\n 0x6C2FC5A4109FD3A2, 0x5B3E60EEB51DDCD8, 0x0A7C717017756FE7, 0xA73773805CA31934,\n 0x4DBD6BB7A31E85FD, 0x24F619D3D5BC2DB4, 0x3E4AF35A1678D636, 0x84A1A8DF8D609239,\n 0x359C862CD3BE4FCD, 0xCF3A39F5C27DC125, 0xC0FF62F8FD5F4C77, 0x5E9F2493DDAA166C,\n 0x17424152BE1CA266, 0xA78AFA5AB4BBE0CD, 0x7BFB2E2CEF118346, 0x647C3E0FF3E3D241,\n 0x0352E4055C13242E, 0x6F42FC70EB660E38, 0x0BEBAD4FABF523BA, 0x9269F4214414D61D,\n 0x1CA8760277E6006C, 0x7BAD25A859D87B5D, 0xAD645ADCF7414F1D, 0xB07F517E88D7AFB3,\n 0xB321C06FB5FFAB5C, 0xD50F162A1EFDD844, 0x1DFD3D1924FBE319, 0xDFAEAB2F09EF7E78,\n 0xA7603B5AF07A0B1E, 0x41CD044C0E5A4EE3, 0xF64D2F86E813BF33, 0xFF9FDB99305EB06A\n};\n\nconst uint64_t t1ha_refval_64be[81] = { 0,\n 0x6A580668D6048674, 0xDECC975A0E3B8177, 0xE3AB9C06FAF4D023, 0xE401FA8F1B6AF969,\n 0x67DB1DAE56FB94E3, 0x1106266A09B7A073, 0x550339B1EF2C7BBB, 0x290A2BAF590045BB,\n 0xA182C1258C09F54A, 0x137D53C34BE7143A, 0xF6D2B69C6F42BEDC, 0x39643EAF2CA2E4B4,\n 0x22A81F139A2C9559, 0x5B3D6AEF0AF33807, 0x56E3F80A68643C08, 0x9E423BE502378780,\n 0xCDB0986F9A5B2FD5, 0xD5B3C84E7933293F, 0xE5FB8C90399E9742, 0x5D393C1F77B2CF3D,\n 0xC8C82F5B2FF09266, 0xACA0230CA6F7B593, 0xCB5805E2960D1655, 0x7E2AD5B704D77C95,\n 0xC5E903CDB8B9EB5D, 0x4CC7D0D21CC03511, 0x8385DF382CFB3E93, 0xF17699D0564D348A,\n 0xF77EE7F8274A4C8D, 0xB9D8CEE48903BABE, 0xFE0EBD2A82B9CFE9, 0xB49FB6397270F565,\n 0x173735C8C342108E, 0xA37C7FBBEEC0A2EA, 0xC13F66F462BB0B6E, 0x0C04F3C2B551467E,\n 0x76A9CB156810C96E, 0x2038850919B0B151, 0xCEA19F2B6EED647B, 0x6746656D2FA109A4,\n 0xF05137F221007F37, 0x892FA9E13A3B4948, 0x4D57B70D37548A32, 0x1A7CFB3D566580E6,\n 0x7CB30272A45E3FAC, 0x137CCFFD9D51423F, 0xB87D96F3B82DF266, 0x33349AEE7472ED37,\n 0x5CC0D3C99555BC07, 0x4A8F4FA196D964EF, 0xE82A0D64F281FBFA, 0x38A1BAC2C36823E1,\n 0x77D197C239FD737E, 0xFB07746B4E07DF26, 0xC8A2198E967672BD, 0x5F1A146D143FA05A,\n 0x26B877A1201AB7AC, 0x74E5B145214723F8, 0xE9CE10E3C70254BC, 0x299393A0C05B79E8,\n 0xFD2D2B9822A5E7E2, 0x85424FEA50C8E50A, 0xE6839E714B1FFFE5, 0x27971CCB46F9112A,\n 0xC98695A2E0715AA9, 0x338E1CBB4F858226, 0xFC6B5C5CF7A8D806, 0x8973CAADDE8DA50C,\n 0x9C6D47AE32EBAE72, 0x1EBF1F9F21D26D78, 0x80A9704B8E153859, 0x6AFD20A939F141FB,\n 0xC35F6C2B3B553EEF, 0x59529E8B0DC94C1A, 0x1569DF036EBC4FA1, 0xDA32B88593C118F9,\n 0xF01E4155FF5A5660, 0x765A2522DCE2B185, 0xCEE95554128073EF, 0x60F072A5CA51DE2F\n};\n\n/* *INDENT-ON* */\n/* clang-format on */\n\n__cold int t1ha_selfcheck__t1ha1_le(void) {\n return t1ha_selfcheck(t1ha1_le, t1ha_refval_64le);\n}\n\n__cold int t1ha_selfcheck__t1ha1_be(void) {\n return t1ha_selfcheck(t1ha1_be, t1ha_refval_64be);\n}\n\n__cold int t1ha_selfcheck__t1ha1(void) {\n return t1ha_selfcheck__t1ha1_le() | t1ha_selfcheck__t1ha1_be();\n}\n\n#endif /* T1HA1_DISABLED */\n" }, { "path": "benchmarks/t1ha/src/t1ha2.c", "content": "/*\n * Copyright (c) 2016-2020 Positive Technologies, https://www.ptsecurity.com,\n * Fast Positive Hash.\n *\n * Portions Copyright (c) 2010-2020 Leonid Yuriev ,\n * The 1Hippeus project (t1h).\n *\n * This software is provided 'as-is', without any express or implied\n * warranty. In no event will the authors be held liable for any damages\n * arising from the use of this software.\n *\n * Permission is granted to anyone to use this software for any purpose,\n * including commercial applications, and to alter it and redistribute it\n * freely, subject to the following restrictions:\n *\n * 1. The origin of this software must not be misrepresented; you must not\n * claim that you wrote the original software. If you use this software\n * in a product, an acknowledgement in the product documentation would be\n * appreciated but is not required.\n * 2. Altered source versions must be plainly marked as such, and must not be\n * misrepresented as being the original software.\n * 3. This notice may not be removed or altered from any source distribution.\n */\n\n/*\n * t1ha = { Fast Positive Hash, aka \"Позитивный Хэш\" }\n * by [Positive Technologies](https://www.ptsecurity.ru)\n *\n * Briefly, it is a 64-bit Hash Function:\n * 1. Created for 64-bit little-endian platforms, in predominantly for x86_64,\n * but portable and without penalties it can run on any 64-bit CPU.\n * 2. In most cases up to 15% faster than City64, xxHash, mum-hash, metro-hash\n * and all others portable hash-functions (which do not use specific\n * hardware tricks).\n * 3. Not suitable for cryptography.\n *\n * The Future will (be) Positive. Всё будет хорошо.\n *\n * ACKNOWLEDGEMENT:\n * The t1ha was originally developed by Leonid Yuriev (Леонид Юрьев)\n * for The 1Hippeus project - zerocopy messaging in the spirit of Sparta!\n */\n\n#ifndef T1HA2_DISABLED\n#include \"t1ha_bits.h\"\n#include \"t1ha_selfcheck.h\"\n\nstatic __always_inline void init_ab(t1ha_state256_t *s, uint64_t x,\n uint64_t y) {\n s->n.a = x;\n s->n.b = y;\n}\n\nstatic __always_inline void init_cd(t1ha_state256_t *s, uint64_t x,\n uint64_t y) {\n s->n.c = rot64(y, 23) + ~x;\n s->n.d = ~y + rot64(x, 19);\n}\n\n/* TODO: C++ template in the next version */\n#define T1HA2_UPDATE(ENDIANNES, ALIGNESS, state, v) \\\n do { \\\n t1ha_state256_t *const s = state; \\\n const uint64_t w0 = fetch64_##ENDIANNES##_##ALIGNESS(v + 0); \\\n const uint64_t w1 = fetch64_##ENDIANNES##_##ALIGNESS(v + 1); \\\n const uint64_t w2 = fetch64_##ENDIANNES##_##ALIGNESS(v + 2); \\\n const uint64_t w3 = fetch64_##ENDIANNES##_##ALIGNESS(v + 3); \\\n \\\n const uint64_t d02 = w0 + rot64(w2 + s->n.d, 56); \\\n const uint64_t c13 = w1 + rot64(w3 + s->n.c, 19); \\\n s->n.d ^= s->n.b + rot64(w1, 38); \\\n s->n.c ^= s->n.a + rot64(w0, 57); \\\n s->n.b ^= prime_6 * (c13 + w2); \\\n s->n.a ^= prime_5 * (d02 + w3); \\\n } while (0)\n\nstatic __always_inline void squash(t1ha_state256_t *s) {\n s->n.a ^= prime_6 * (s->n.c + rot64(s->n.d, 23));\n s->n.b ^= prime_5 * (rot64(s->n.c, 19) + s->n.d);\n}\n\n/* TODO: C++ template in the next version */\n#define T1HA2_LOOP(ENDIANNES, ALIGNESS, state, data, len) \\\n do { \\\n const void *detent = (const uint8_t *)data + len - 31; \\\n do { \\\n const uint64_t *v = (const uint64_t *)data; \\\n data = (const uint64_t *)data + 4; \\\n prefetch(data); \\\n T1HA2_UPDATE(le, ALIGNESS, state, v); \\\n } while (likely(data < detent)); \\\n } while (0)\n\n/* TODO: C++ template in the next version */\n#define T1HA2_TAIL_AB(ENDIANNES, ALIGNESS, state, data, len) \\\n do { \\\n t1ha_state256_t *const s = state; \\\n const uint64_t *v = (const uint64_t *)data; \\\n switch (len) { \\\n default: \\\n mixup64(&s->n.a, &s->n.b, fetch64_##ENDIANNES##_##ALIGNESS(v++), \\\n prime_4); \\\n /* fall through */ \\\n case 24: \\\n case 23: \\\n case 22: \\\n case 21: \\\n case 20: \\\n case 19: \\\n case 18: \\\n case 17: \\\n mixup64(&s->n.b, &s->n.a, fetch64_##ENDIANNES##_##ALIGNESS(v++), \\\n prime_3); \\\n /* fall through */ \\\n case 16: \\\n case 15: \\\n case 14: \\\n case 13: \\\n case 12: \\\n case 11: \\\n case 10: \\\n case 9: \\\n mixup64(&s->n.a, &s->n.b, fetch64_##ENDIANNES##_##ALIGNESS(v++), \\\n prime_2); \\\n /* fall through */ \\\n case 8: \\\n case 7: \\\n case 6: \\\n case 5: \\\n case 4: \\\n case 3: \\\n case 2: \\\n case 1: \\\n mixup64(&s->n.b, &s->n.a, tail64_##ENDIANNES##_##ALIGNESS(v, len), \\\n prime_1); \\\n /* fall through */ \\\n case 0: \\\n return final64(s->n.a, s->n.b); \\\n } \\\n } while (0)\n\n/* TODO: C++ template in the next version */\n#define T1HA2_TAIL_ABCD(ENDIANNES, ALIGNESS, state, data, len) \\\n do { \\\n t1ha_state256_t *const s = state; \\\n const uint64_t *v = (const uint64_t *)data; \\\n switch (len) { \\\n default: \\\n mixup64(&s->n.a, &s->n.d, fetch64_##ENDIANNES##_##ALIGNESS(v++), \\\n prime_4); \\\n /* fall through */ \\\n case 24: \\\n case 23: \\\n case 22: \\\n case 21: \\\n case 20: \\\n case 19: \\\n case 18: \\\n case 17: \\\n mixup64(&s->n.b, &s->n.a, fetch64_##ENDIANNES##_##ALIGNESS(v++), \\\n prime_3); \\\n /* fall through */ \\\n case 16: \\\n case 15: \\\n case 14: \\\n case 13: \\\n case 12: \\\n case 11: \\\n case 10: \\\n case 9: \\\n mixup64(&s->n.c, &s->n.b, fetch64_##ENDIANNES##_##ALIGNESS(v++), \\\n prime_2); \\\n /* fall through */ \\\n case 8: \\\n case 7: \\\n case 6: \\\n case 5: \\\n case 4: \\\n case 3: \\\n case 2: \\\n case 1: \\\n mixup64(&s->n.d, &s->n.c, tail64_##ENDIANNES##_##ALIGNESS(v, len), \\\n prime_1); \\\n /* fall through */ \\\n case 0: \\\n return final128(s->n.a, s->n.b, s->n.c, s->n.d, extra_result); \\\n } \\\n } while (0)\n\nstatic __always_inline uint64_t final128(uint64_t a, uint64_t b, uint64_t c,\n uint64_t d, uint64_t *h) {\n mixup64(&a, &b, rot64(c, 41) ^ d, prime_0);\n mixup64(&b, &c, rot64(d, 23) ^ a, prime_6);\n mixup64(&c, &d, rot64(a, 19) ^ b, prime_5);\n mixup64(&d, &a, rot64(b, 31) ^ c, prime_4);\n *h = c + d;\n return a ^ b;\n}\n\n//------------------------------------------------------------------------------\n\nuint64_t t1ha2_atonce(const void *data, size_t length, uint64_t seed) {\n t1ha_state256_t state;\n init_ab(&state, seed, length);\n\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__EFFICIENT\n if (unlikely(length > 32)) {\n init_cd(&state, seed, length);\n#if defined(__LCC__) && __LCC__ > 123\n/* Форсирует комбинирование пар арифметических операций в двухэтажные операции\n * в ближайшем после объявления директивы цикле, даже если эвристики оптимизации\n * говорят, что это нецелесообразно */\n#pragma comb_oper\n#endif /* E2K LCC > 1.23 */\n T1HA2_LOOP(le, unaligned, &state, data, length);\n squash(&state);\n length &= 31;\n }\n T1HA2_TAIL_AB(le, unaligned, &state, data, length);\n#else\n const bool misaligned = (((uintptr_t)data) & (ALIGNMENT_64 - 1)) != 0;\n if (misaligned) {\n if (unlikely(length > 32)) {\n init_cd(&state, seed, length);\n#if defined(__LCC__) && __LCC__ > 123\n/* Форсирует комбинирование пар арифметических операций в двухэтажные операции\n * в ближайшем после объявления директивы цикле, даже если эвристики оптимизации\n * говорят, что это нецелесообразно */\n#pragma comb_oper\n#endif /* E2K LCC > 1.23 */\n T1HA2_LOOP(le, unaligned, &state, data, length);\n squash(&state);\n length &= 31;\n }\n T1HA2_TAIL_AB(le, unaligned, &state, data, length);\n } else {\n if (unlikely(length > 32)) {\n init_cd(&state, seed, length);\n#if defined(__LCC__) && __LCC__ > 123\n/* Форсирует комбинирование пар арифметических операций в двухэтажные операции\n * в ближайшем после объявления директивы цикле, даже если эвристики оптимизации\n * говорят, что это нецелесообразно */\n#pragma comb_oper\n#endif /* E2K LCC > 1.23 */\n T1HA2_LOOP(le, aligned, &state, data, length);\n squash(&state);\n length &= 31;\n }\n T1HA2_TAIL_AB(le, aligned, &state, data, length);\n }\n#endif\n}\n\nuint64_t t1ha2_atonce128(uint64_t *__restrict extra_result,\n const void *__restrict data, size_t length,\n uint64_t seed) {\n t1ha_state256_t state;\n init_ab(&state, seed, length);\n init_cd(&state, seed, length);\n\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__EFFICIENT\n if (unlikely(length > 32)) {\n#if defined(__LCC__) && __LCC__ > 123\n/* Форсирует комбинирование пар арифметических операций в двухэтажные операции\n * в ближайшем после объявления директивы цикле, даже если эвристики оптимизации\n * говорят, что это нецелесообразно */\n#pragma comb_oper\n#endif /* E2K LCC > 1.23 */\n T1HA2_LOOP(le, unaligned, &state, data, length);\n length &= 31;\n }\n T1HA2_TAIL_ABCD(le, unaligned, &state, data, length);\n#else\n const bool misaligned = (((uintptr_t)data) & (ALIGNMENT_64 - 1)) != 0;\n if (misaligned) {\n if (unlikely(length > 32)) {\n#if defined(__LCC__) && __LCC__ > 123\n/* Форсирует комбинирование пар арифметических операций в двухэтажные операции\n * в ближайшем после объявления директивы цикле, даже если эвристики оптимизации\n * говорят, что это нецелесообразно */\n#pragma comb_oper\n#endif /* E2K LCC > 1.23 */\n T1HA2_LOOP(le, unaligned, &state, data, length);\n length &= 31;\n }\n T1HA2_TAIL_ABCD(le, unaligned, &state, data, length);\n } else {\n if (unlikely(length > 32)) {\n#if defined(__LCC__) && __LCC__ > 123\n/* Форсирует комбинирование пар арифметических операций в двухэтажные операции\n * в ближайшем после объявления директивы цикле, даже если эвристики оптимизации\n * говорят, что это нецелесообразно */\n#pragma comb_oper\n#endif /* E2K LCC > 1.23 */\n T1HA2_LOOP(le, aligned, &state, data, length);\n length &= 31;\n }\n T1HA2_TAIL_ABCD(le, aligned, &state, data, length);\n }\n#endif\n}\n\n//------------------------------------------------------------------------------\n\nvoid t1ha2_init(t1ha_context_t *ctx, uint64_t seed_x, uint64_t seed_y) {\n init_ab(&ctx->state, seed_x, seed_y);\n init_cd(&ctx->state, seed_x, seed_y);\n ctx->partial = 0;\n ctx->total = 0;\n}\n\nvoid t1ha2_update(t1ha_context_t *__restrict ctx, const void *__restrict data,\n size_t length) {\n ctx->total += length;\n\n if (ctx->partial) {\n const size_t left = 32 - ctx->partial;\n const size_t chunk = (length >= left) ? left : length;\n memcpy(ctx->buffer.bytes + ctx->partial, data, chunk);\n ctx->partial += chunk;\n if (ctx->partial < 32) {\n assert(left >= length);\n return;\n }\n ctx->partial = 0;\n data = (const uint8_t *)data + chunk;\n length -= chunk;\n T1HA2_UPDATE(le, aligned, &ctx->state, ctx->buffer.u64);\n }\n\n if (length >= 32) {\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__EFFICIENT\n#if defined(__LCC__) && __LCC__ > 123\n/* Форсирует комбинирование пар арифметических операций в двухэтажные операции\n * в ближайшем после объявления директивы цикле, даже если эвристики оптимизации\n * говорят, что это нецелесообразно */\n#pragma comb_oper\n#endif /* E2K LCC > 1.23 */\n T1HA2_LOOP(le, unaligned, &ctx->state, data, length);\n#else\n const bool misaligned = (((uintptr_t)data) & (ALIGNMENT_64 - 1)) != 0;\n if (misaligned) {\n#if defined(__LCC__) && __LCC__ > 123\n/* Форсирует комбинирование пар арифметических операций в двухэтажные операции\n * в ближайшем после объявления директивы цикле, даже если эвристики оптимизации\n * говорят, что это нецелесообразно */\n#pragma comb_oper\n#endif /* E2K LCC > 1.23 */\n T1HA2_LOOP(le, unaligned, &ctx->state, data, length);\n } else {\n#if defined(__LCC__) && __LCC__ > 123\n/* Форсирует комбинирование пар арифметических операций в двухэтажные операции\n * в ближайшем после объявления директивы цикле, даже если эвристики оптимизации\n * говорят, что это нецелесообразно */\n#pragma comb_oper\n#endif /* E2K LCC > 1.23 */\n T1HA2_LOOP(le, aligned, &ctx->state, data, length);\n }\n#endif\n length &= 31;\n }\n\n if (length)\n memcpy(ctx->buffer.bytes, data, ctx->partial = length);\n}\n\nuint64_t t1ha2_final(t1ha_context_t *__restrict ctx,\n uint64_t *__restrict extra_result) {\n uint64_t bits = (ctx->total << 3) ^ (UINT64_C(1) << 63);\n#if __BYTE_ORDER__ != __ORDER_LITTLE_ENDIAN__\n bits = bswap64(bits);\n#endif\n t1ha2_update(ctx, &bits, 8);\n\n if (likely(!extra_result)) {\n squash(&ctx->state);\n T1HA2_TAIL_AB(le, aligned, &ctx->state, ctx->buffer.u64, ctx->partial);\n }\n\n T1HA2_TAIL_ABCD(le, aligned, &ctx->state, ctx->buffer.u64, ctx->partial);\n}\n\n#endif /* T1HA2_DISABLED */\n" }, { "path": "benchmarks/t1ha/src/t1ha2_selfcheck.c", "content": "/*\n * Copyright (c) 2016-2020 Positive Technologies, https://www.ptsecurity.com,\n * Fast Positive Hash.\n *\n * Portions Copyright (c) 2010-2020 Leonid Yuriev ,\n * The 1Hippeus project (t1h).\n *\n * This software is provided 'as-is', without any express or implied\n * warranty. In no event will the authors be held liable for any damages\n * arising from the use of this software.\n *\n * Permission is granted to anyone to use this software for any purpose,\n * including commercial applications, and to alter it and redistribute it\n * freely, subject to the following restrictions:\n *\n * 1. The origin of this software must not be misrepresented; you must not\n * claim that you wrote the original software. If you use this software\n * in a product, an acknowledgement in the product documentation would be\n * appreciated but is not required.\n * 2. Altered source versions must be plainly marked as such, and must not be\n * misrepresented as being the original software.\n * 3. This notice may not be removed or altered from any source distribution.\n */\n\n/*\n * t1ha = { Fast Positive Hash, aka \"Позитивный Хэш\" }\n * by [Positive Technologies](https://www.ptsecurity.ru)\n *\n * Briefly, it is a 64-bit Hash Function:\n * 1. Created for 64-bit little-endian platforms, in predominantly for x86_64,\n * but portable and without penalties it can run on any 64-bit CPU.\n * 2. In most cases up to 15% faster than City64, xxHash, mum-hash, metro-hash\n * and all others portable hash-functions (which do not use specific\n * hardware tricks).\n * 3. Not suitable for cryptography.\n *\n * The Future will (be) Positive. Всё будет хорошо.\n *\n * ACKNOWLEDGEMENT:\n * The t1ha was originally developed by Leonid Yuriev (Леонид Юрьев)\n * for The 1Hippeus project - zerocopy messaging in the spirit of Sparta!\n */\n\n#ifndef T1HA2_DISABLED\n#include \"t1ha_bits.h\"\n#include \"t1ha_selfcheck.h\"\n\n/* *INDENT-OFF* */\n/* clang-format off */\n\nconst uint64_t t1ha_refval_2atonce[81] = { 0,\n 0x772C7311BE32FF42, 0x444753D23F207E03, 0x71F6DF5DA3B4F532, 0x555859635365F660,\n 0xE98808F1CD39C626, 0x2EB18FAF2163BB09, 0x7B9DD892C8019C87, 0xE2B1431C4DA4D15A,\n 0x1984E718A5477F70, 0x08DD17B266484F79, 0x4C83A05D766AD550, 0x92DCEBB131D1907D,\n 0xD67BC6FC881B8549, 0xF6A9886555FBF66B, 0x6E31616D7F33E25E, 0x36E31B7426E3049D,\n 0x4F8E4FAF46A13F5F, 0x03EB0CB3253F819F, 0x636A7769905770D2, 0x3ADF3781D16D1148,\n 0x92D19CB1818BC9C2, 0x283E68F4D459C533, 0xFA83A8A88DECAA04, 0x8C6F00368EAC538C,\n 0x7B66B0CF3797B322, 0x5131E122FDABA3FF, 0x6E59FF515C08C7A9, 0xBA2C5269B2C377B0,\n 0xA9D24FD368FE8A2B, 0x22DB13D32E33E891, 0x7B97DFC804B876E5, 0xC598BDFCD0E834F9,\n 0xB256163D3687F5A7, 0x66D7A73C6AEF50B3, 0x25A7201C85D9E2A3, 0x911573EDA15299AA,\n 0x5C0062B669E18E4C, 0x17734ADE08D54E28, 0xFFF036E33883F43B, 0xFE0756E7777DF11E,\n 0x37972472D023F129, 0x6CFCE201B55C7F57, 0xE019D1D89F02B3E1, 0xAE5CC580FA1BB7E6,\n 0x295695FB7E59FC3A, 0x76B6C820A40DD35E, 0xB1680A1768462B17, 0x2FB6AF279137DADA,\n 0x28FB6B4366C78535, 0xEC278E53924541B1, 0x164F8AAB8A2A28B5, 0xB6C330AEAC4578AD,\n 0x7F6F371070085084, 0x94DEAD60C0F448D3, 0x99737AC232C559EF, 0x6F54A6F9CA8EDD57,\n 0x979B01E926BFCE0C, 0xF7D20BC85439C5B4, 0x64EDB27CD8087C12, 0x11488DE5F79C0BE2,\n 0x25541DDD1680B5A4, 0x8B633D33BE9D1973, 0x404A3113ACF7F6C6, 0xC59DBDEF8550CD56,\n 0x039D23C68F4F992C, 0x5BBB48E4BDD6FD86, 0x41E312248780DF5A, 0xD34791CE75D4E94F,\n 0xED523E5D04DCDCFF, 0x7A6BCE0B6182D879, 0x21FB37483CAC28D8, 0x19A1B66E8DA878AD,\n 0x6F804C5295B09ABE, 0x2A4BE5014115BA81, 0xA678ECC5FC924BE0, 0x50F7A54A99A36F59,\n 0x0FD7E63A39A66452, 0x5AB1B213DD29C4E4, 0xF3ED80D9DF6534C5, 0xC736B12EF90615FD\n};\n\nconst uint64_t t1ha_refval_2atonce128[81] = { 0x4EC7F6A48E33B00A,\n 0xB7B7FAA5BD7D8C1E, 0x3269533F66534A76, 0x6C3EC6B687923BFC, 0xC096F5E7EFA471A9,\n 0x79D8AFB550CEA471, 0xCEE0507A20FD5119, 0xFB04CFFC14A9F4BF, 0xBD4406E923807AF2,\n 0x375C02FF11010491, 0xA6EA4C2A59E173FF, 0xE0A606F0002CADDF, 0xE13BEAE6EBC07897,\n 0xF069C2463E48EA10, 0x75BEE1A97089B5FA, 0x378F22F8DE0B8085, 0x9C726FC4D53D0D8B,\n 0x71F6130A2D08F788, 0x7A9B20433FF6CF69, 0xFF49B7CD59BF6D61, 0xCCAAEE0D1CA9C6B3,\n 0xC77889D86039D2AD, 0x7B378B5BEA9B0475, 0x6520BFA79D59AD66, 0x2441490CB8A37267,\n 0xA715A66B7D5CF473, 0x9AE892C88334FD67, 0xD2FFE9AEC1D2169A, 0x790B993F18B18CBB,\n 0xA0D02FBCF6A7B1AD, 0xA90833E6F151D0C1, 0x1AC7AFA37BD79BE0, 0xD5383628B2881A24,\n 0xE5526F9D63F9F8F1, 0xC1F165A01A6D1F4D, 0x6CCEF8FF3FCFA3F2, 0x2030F18325E6DF48,\n 0x289207230E3FB17A, 0x077B66F713A3C4B9, 0x9F39843CAF871754, 0x512FDA0F808ACCF3,\n 0xF4D9801CD0CD1F14, 0x28A0C749ED323638, 0x94844CAFA671F01C, 0xD0E261876B8ACA51,\n 0x8FC2A648A4792EA2, 0x8EF87282136AF5FE, 0x5FE6A54A9FBA6B40, 0xA3CC5B8FE6223D54,\n 0xA8C3C0DD651BB01C, 0x625E9FDD534716F3, 0x1AB2604083C33AC5, 0xDE098853F8692F12,\n 0x4B0813891BD87624, 0x4AB89C4553D182AD, 0x92C15AA2A3C27ADA, 0xFF2918D68191F5D9,\n 0x06363174F641C325, 0x667112ADA74A2059, 0x4BD605D6B5E53D7D, 0xF2512C53663A14C8,\n 0x21857BCB1852667C, 0xAFBEBD0369AEE228, 0x7049340E48FBFD6B, 0x50710E1924F46954,\n 0x869A75E04A976A3F, 0x5A41ABBDD6373889, 0xA781778389B4B188, 0x21A3AFCED6C925B6,\n 0x107226192EC10B42, 0x62A862E84EC2F9B1, 0x2B15E91659606DD7, 0x613934D1F9EC5A42,\n 0x4DC3A96DC5361BAF, 0xC80BBA4CB5F12903, 0x3E3EDAE99A7D6987, 0x8F97B2D55941DCB0,\n 0x4C9787364C3E4EC1, 0xEF0A2D07BEA90CA7, 0x5FABF32C70AEEAFB, 0x3356A5CFA8F23BF4\n};\n\nconst uint64_t t1ha_refval_2stream[81] = { 0x3C8426E33CB41606,\n 0xFD74BE70EE73E617, 0xF43DE3CDD8A20486, 0x882FBCB37E8EA3BB, 0x1AA2CDD34CAA3D4B,\n 0xEE755B2BFAE07ED5, 0xD4E225250D92E213, 0xA09B49083205965B, 0xD47B21724EF9EC9E,\n 0xAC888FC3858CEE11, 0x94F820D85736F244, 0x1707951CCA920932, 0x8E0E45603F7877F0,\n 0x9FD2592C0E3A7212, 0x9A66370F3AE3D427, 0xD33382D2161DE2B7, 0x9A35BE079DA7115F,\n 0x73457C7FF58B4EC3, 0xBE8610BD53D7CE98, 0x65506DFE5CCD5371, 0x286A321AF9D5D9FA,\n 0xB81EF9A7EF3C536D, 0x2CFDB5E6825C6E86, 0xB2A58CBFDFDD303A, 0xD26094A42B950635,\n 0xA34D666A5F02AD9A, 0x0151E013EBCC72E5, 0x9254A6EA7FCB6BB5, 0x10C9361B3869DC2B,\n 0xD7EC55A060606276, 0xA2FF7F8BF8976FFD, 0xB5181BB6852DCC88, 0x0EE394BB6178BAFF,\n 0x3A8B4B400D21B89C, 0xEC270461970960FD, 0x615967FAB053877E, 0xFA51BF1CFEB4714C,\n 0x29FDA8383070F375, 0xC3B663061BC52EDA, 0x192BBAF1F1A57923, 0x6D193B52F93C53AF,\n 0x7F6F5639FE87CA1E, 0x69F7F9140B32EDC8, 0xD0F2416FB24325B6, 0x62C0E37FEDD49FF3,\n 0x57866A4B809D373D, 0x9848D24BD935E137, 0xDFC905B66734D50A, 0x9A938DD194A68529,\n 0x8276C44DF0625228, 0xA4B35D00AD67C0AB, 0x3D9CB359842DB452, 0x4241BFA8C23B267F,\n 0x650FA517BEF15952, 0x782DE2ABD8C7B1E1, 0x4EAE456166CA3E15, 0x40CDF3A02614E337,\n 0xAD84092C46102172, 0x0C68479B03F9A167, 0x7E1BA046749E181C, 0x3F3AB41A697382C1,\n 0xC5E5DD6586EBFDC4, 0xFF926CD4EB02555C, 0x035CFE67F89E709B, 0x89F06AB6464A1B9D,\n 0x8EFF58F3F7DEA758, 0x8B54AC657902089F, 0xC6C4F1F9F8DA4D64, 0xBDB729048AAAC93A,\n 0xEA76BA628F5E5CD6, 0x742159B728B8A979, 0x6D151CD3C720E53D, 0xE97FFF9368FCDC42,\n 0xCA5B38314914FBDA, 0xDD92C91D8B858EAE, 0x66E5F07CF647CBF2, 0xD4CF9B42F4985AFB,\n 0x72AE17AC7D92F6B7, 0xB8206B22AB0472E1, 0x385876B5CFD42479, 0x03294A249EBE6B26\n};\n\nconst uint64_t t1ha_refval_2stream128[81] = { 0xCD2801D3B92237D6,\n 0x10E4D47BD821546D, 0x9100704B9D65CD06, 0xD6951CB4016313EF, 0x24DB636F96F474DA,\n 0x3F4AF7DF3C49E422, 0xBFF25B8AF143459B, 0xA157EC13538BE549, 0xD3F5F52C47DBD419,\n 0x0EF3D7D735AF1575, 0x46B7B892823F7B1B, 0xEE22EA4655213289, 0x56AD76F02FE929BC,\n 0x9CF6CD1AC886546E, 0xAF45CE47AEA0B933, 0x535F9DC09F3996B7, 0x1F0C3C01694AE128,\n 0x18495069BE0766F7, 0x37E5FFB3D72A4CB1, 0x6D6C2E9299F30709, 0x4F39E693F50B41E3,\n 0xB11FC4EF0658E116, 0x48BFAACB78E5079B, 0xE1B4C89C781B3AD0, 0x81D2F34888D333A1,\n 0xF6D02270D2EA449C, 0xC884C3C2C3CE1503, 0x711AE16BA157A9B9, 0x1E6140C642558C9D,\n 0x35AB3D238F5DC55B, 0x33F07B6AEF051177, 0xE57336776EEFA71C, 0x6D445F8318BA3752,\n 0xD4F5F6631934C988, 0xD5E260085727C4A2, 0x5B54B41EC180B4FA, 0x7F5D75769C15A898,\n 0xAE5A6DB850CA33C6, 0x038CCB8044663403, 0xDA16310133DC92B8, 0x6A2FFB7AB2B7CE2B,\n 0xDC1832D9229BAE20, 0x8C62C479F5ABC9E4, 0x5EB7B617857C9CCB, 0xB79CF7D749A1E80D,\n 0xDE7FAC3798324FD3, 0x8178911813685D06, 0x6A726CBD394D4410, 0x6CBE6B3280DA1113,\n 0x6829BA4410CF1148, 0xFA7E417EB26C5BC6, 0x22ED87884D6E3A49, 0x15F1472D5115669D,\n 0x2EA0B4C8BF69D318, 0xDFE87070AA545503, 0x6B4C14B5F7144AB9, 0xC1ED49C06126551A,\n 0x351919FC425C3899, 0x7B569C0FA6F1BD3E, 0x713AC2350844CFFD, 0xE9367F9A638C2FF3,\n 0x97F17D325AEA0786, 0xBCB907CC6CF75F91, 0x0CB7517DAF247719, 0xBE16093CC45BE8A9,\n 0x786EEE97359AD6AB, 0xB7AFA4F326B97E78, 0x2694B67FE23E502E, 0x4CB492826E98E0B4,\n 0x838D119F74A416C7, 0x70D6A91E4E5677FD, 0xF3E4027AD30000E6, 0x9BDF692795807F77,\n 0x6A371F966E034A54, 0x8789CF41AE4D67EF, 0x02688755484D60AE, 0xD5834B3A4BF5CE42,\n 0x9405FC61440DE25D, 0x35EB280A157979B6, 0x48D40D6A525297AC, 0x6A87DC185054BADA\n};\n\n/* *INDENT-ON* */\n/* clang-format on */\n\n__cold int t1ha_selfcheck__t1ha2_atonce(void) {\n return t1ha_selfcheck(t1ha2_atonce, t1ha_refval_2atonce);\n}\n\n__cold static uint64_t thunk_atonce128(const void *data, size_t len,\n uint64_t seed) {\n uint64_t unused;\n return t1ha2_atonce128(&unused, data, len, seed);\n}\n\n__cold int t1ha_selfcheck__t1ha2_atonce128(void) {\n return t1ha_selfcheck(thunk_atonce128, t1ha_refval_2atonce128);\n}\n\n__cold static uint64_t thunk_stream(const void *data, size_t len,\n uint64_t seed) {\n t1ha_context_t ctx;\n t1ha2_init(&ctx, seed, seed);\n t1ha2_update(&ctx, data, len);\n return t1ha2_final(&ctx, NULL);\n}\n\n__cold static uint64_t thunk_stream128(const void *data, size_t len,\n uint64_t seed) {\n t1ha_context_t ctx;\n t1ha2_init(&ctx, seed, seed);\n t1ha2_update(&ctx, data, len);\n uint64_t unused;\n return t1ha2_final(&ctx, &unused);\n}\n\n__cold int t1ha_selfcheck__t1ha2_stream(void) {\n return t1ha_selfcheck(thunk_stream, t1ha_refval_2stream) |\n t1ha_selfcheck(thunk_stream128, t1ha_refval_2stream128);\n}\n\n__cold int t1ha_selfcheck__t1ha2(void) {\n return t1ha_selfcheck__t1ha2_atonce() | t1ha_selfcheck__t1ha2_atonce128() |\n t1ha_selfcheck__t1ha2_stream();\n}\n\n#endif /* T1HA2_DISABLED */\n" }, { "path": "benchmarks/t1ha/src/t1ha_bits.h", "content": "/*\n * Copyright (c) 2016-2020 Positive Technologies, https://www.ptsecurity.com,\n * Fast Positive Hash.\n *\n * Portions Copyright (c) 2010-2020 Leonid Yuriev ,\n * The 1Hippeus project (t1h).\n *\n * This software is provided 'as-is', without any express or implied\n * warranty. In no event will the authors be held liable for any damages\n * arising from the use of this software.\n *\n * Permission is granted to anyone to use this software for any purpose,\n * including commercial applications, and to alter it and redistribute it\n * freely, subject to the following restrictions:\n *\n * 1. The origin of this software must not be misrepresented; you must not\n * claim that you wrote the original software. If you use this software\n * in a product, an acknowledgement in the product documentation would be\n * appreciated but is not required.\n * 2. Altered source versions must be plainly marked as such, and must not be\n * misrepresented as being the original software.\n * 3. This notice may not be removed or altered from any source distribution.\n */\n\n/*\n * t1ha = { Fast Positive Hash, aka \"Позитивный Хэш\" }\n * by [Positive Technologies](https://www.ptsecurity.ru)\n *\n * Briefly, it is a 64-bit Hash Function:\n * 1. Created for 64-bit little-endian platforms, in predominantly for x86_64,\n * but portable and without penalties it can run on any 64-bit CPU.\n * 2. In most cases up to 15% faster than City64, xxHash, mum-hash, metro-hash\n * and all others portable hash-functions (which do not use specific\n * hardware tricks).\n * 3. Not suitable for cryptography.\n *\n * The Future will (be) Positive. Всё будет хорошо.\n *\n * ACKNOWLEDGEMENT:\n * The t1ha was originally developed by Leonid Yuriev (Леонид Юрьев)\n * for The 1Hippeus project - zerocopy messaging in the spirit of Sparta!\n */\n\n#pragma once\n\n#if defined(_MSC_VER)\n#pragma warning(disable : 4201) /* nameless struct/union */\n#if _MSC_VER > 1800\n#pragma warning(disable : 4464) /* relative include path contains '..' */\n#endif /* 1800 */\n#endif /* MSVC */\n#include \"../t1ha.h\"\n\n#ifndef T1HA_USE_FAST_ONESHOT_READ\n/* Define it to 1 for little bit faster code.\n * Unfortunately this may triggering a false-positive alarms from Valgrind,\n * AddressSanitizer and other similar tool.\n * So, define it to 0 for calmness if doubt. */\n#define T1HA_USE_FAST_ONESHOT_READ 1\n#endif /* T1HA_USE_FAST_ONESHOT_READ */\n\n/*****************************************************************************/\n\n#include /* for assert() */\n#include /* for bool */\n#include /* for memcpy() */\n\n#if __BYTE_ORDER__ != __ORDER_LITTLE_ENDIAN__ && \\\n __BYTE_ORDER__ != __ORDER_BIG_ENDIAN__\n#error Unsupported byte order.\n#endif\n\n#define T1HA_UNALIGNED_ACCESS__UNABLE 0\n#define T1HA_UNALIGNED_ACCESS__SLOW 1\n#define T1HA_UNALIGNED_ACCESS__EFFICIENT 2\n\n#ifndef T1HA_SYS_UNALIGNED_ACCESS\n#if defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS)\n#define T1HA_SYS_UNALIGNED_ACCESS T1HA_UNALIGNED_ACCESS__EFFICIENT\n#elif defined(__ia32__)\n#define T1HA_SYS_UNALIGNED_ACCESS T1HA_UNALIGNED_ACCESS__EFFICIENT\n#elif defined(__e2k__)\n#define T1HA_SYS_UNALIGNED_ACCESS T1HA_UNALIGNED_ACCESS__SLOW\n#else\n#define T1HA_SYS_UNALIGNED_ACCESS T1HA_UNALIGNED_ACCESS__UNABLE\n#endif\n#endif /* T1HA_SYS_UNALIGNED_ACCESS */\n\n#define ALIGNMENT_16 2\n#define ALIGNMENT_32 4\n#if UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul\n#define ALIGNMENT_64 8\n#else\n#define ALIGNMENT_64 4\n#endif\n\n#ifndef PAGESIZE\n#define PAGESIZE 4096\n#endif /* PAGESIZE */\n\n/***************************************************************************/\n\n#ifndef __has_builtin\n#define __has_builtin(x) (0)\n#endif\n\n#ifndef __has_warning\n#define __has_warning(x) (0)\n#endif\n\n#ifndef __has_feature\n#define __has_feature(x) (0)\n#endif\n\n#ifndef __has_extension\n#define __has_extension(x) (0)\n#endif\n\n#if __has_feature(address_sanitizer)\n#define __SANITIZE_ADDRESS__ 1\n#endif\n\n#ifndef __optimize\n#if defined(__clang__) && !__has_attribute(__optimize__)\n#define __optimize(ops)\n#elif defined(__GNUC__) || __has_attribute(__optimize__)\n#define __optimize(ops) __attribute__((__optimize__(ops)))\n#else\n#define __optimize(ops)\n#endif\n#endif /* __optimize */\n\n#ifndef __cold\n#if defined(__OPTIMIZE__)\n#if defined(__e2k__)\n#define __cold __optimize(1) __attribute__((__cold__))\n#elif defined(__clang__) && !__has_attribute(__cold__) && \\\n __has_attribute(__section__)\n/* just put infrequently used functions in separate section */\n#define __cold __attribute__((__section__(\"text.unlikely\"))) __optimize(\"Os\")\n#elif defined(__GNUC__) || __has_attribute(__cold__)\n#define __cold __attribute__((__cold__)) __optimize(\"Os\")\n#else\n#define __cold __optimize(\"Os\")\n#endif\n#else\n#define __cold\n#endif\n#endif /* __cold */\n\n#if __GNUC_PREREQ(4, 4) || defined(__clang__)\n\n#if defined(__ia32__) || defined(__e2k__)\n#include \n#endif\n\n#if defined(__ia32__) && !defined(__cpuid_count)\n#include \n#endif\n\n#if defined(__e2k__)\n#include \n#endif\n\n#ifndef likely\n#define likely(cond) __builtin_expect(!!(cond), 1)\n#endif\n\n#ifndef unlikely\n#define unlikely(cond) __builtin_expect(!!(cond), 0)\n#endif\n\n#if __GNUC_PREREQ(4, 5) || __has_builtin(__builtin_unreachable)\n#define unreachable() __builtin_unreachable()\n#endif\n\n#define bswap64(v) __builtin_bswap64(v)\n#define bswap32(v) __builtin_bswap32(v)\n#if __GNUC_PREREQ(4, 8) || __has_builtin(__builtin_bswap16)\n#define bswap16(v) __builtin_bswap16(v)\n#endif\n\n#if !defined(__maybe_unused) && \\\n (__GNUC_PREREQ(4, 3) || __has_attribute(__unused__))\n#define __maybe_unused __attribute__((__unused__))\n#endif\n\n#if !defined(__always_inline) && \\\n (__GNUC_PREREQ(3, 2) || __has_attribute(__always_inline__))\n#define __always_inline __inline __attribute__((__always_inline__))\n#endif\n\n#if defined(__e2k__)\n\n#if __iset__ >= 3\n#define mul_64x64_high(a, b) __builtin_e2k_umulhd(a, b)\n#endif /* __iset__ >= 3 */\n\n#if __iset__ >= 5\nstatic __maybe_unused __always_inline unsigned\ne2k_add64carry_first(uint64_t base, uint64_t addend, uint64_t *sum) {\n *sum = base + addend;\n return (unsigned)__builtin_e2k_addcd_c(base, addend, 0);\n}\n#define add64carry_first(base, addend, sum) \\\n e2k_add64carry_first(base, addend, sum)\n\nstatic __maybe_unused __always_inline unsigned\ne2k_add64carry_next(unsigned carry, uint64_t base, uint64_t addend,\n uint64_t *sum) {\n *sum = __builtin_e2k_addcd(base, addend, carry);\n return (unsigned)__builtin_e2k_addcd_c(base, addend, carry);\n}\n#define add64carry_next(carry, base, addend, sum) \\\n e2k_add64carry_next(carry, base, addend, sum)\n\nstatic __maybe_unused __always_inline void e2k_add64carry_last(unsigned carry,\n uint64_t base,\n uint64_t addend,\n uint64_t *sum) {\n *sum = __builtin_e2k_addcd(base, addend, carry);\n}\n#define add64carry_last(carry, base, addend, sum) \\\n e2k_add64carry_last(carry, base, addend, sum)\n#endif /* __iset__ >= 5 */\n\n#define fetch64_be_aligned(ptr) ((uint64_t)__builtin_e2k_ld_64s_be(ptr))\n#define fetch32_be_aligned(ptr) ((uint32_t)__builtin_e2k_ld_32u_be(ptr))\n\n#endif /* __e2k__ Elbrus */\n\n#elif defined(_MSC_VER)\n\n#if _MSC_FULL_VER < 190024234 && defined(_M_IX86)\n#pragma message( \\\n \"For AES-NI at least \\\"Microsoft C/C++ Compiler\\\" version 19.00.24234 (Visual Studio 2015 Update 3) is required.\")\n#endif\n#if _MSC_FULL_VER < 191526730\n#pragma message( \\\n \"It is recommended to use \\\"Microsoft C/C++ Compiler\\\" version 19.15.26730 (Visual Studio 2017 15.8) or newer.\")\n#endif\n#if _MSC_FULL_VER < 180040629\n#error At least \"Microsoft C/C++ Compiler\" version 18.00.40629 (Visual Studio 2013 Update 5) is required.\n#endif\n\n#pragma warning(push, 1)\n\n#include \n#include \n#define likely(cond) (cond)\n#define unlikely(cond) (cond)\n#define unreachable() __assume(0)\n#define bswap64(v) _byteswap_uint64(v)\n#define bswap32(v) _byteswap_ulong(v)\n#define bswap16(v) _byteswap_ushort(v)\n#define rot64(v, s) _rotr64(v, s)\n#define rot32(v, s) _rotr(v, s)\n#define __always_inline __forceinline\n\n#if defined(_M_X64) || defined(_M_IA64)\n#pragma intrinsic(_umul128)\n#define mul_64x64_128(a, b, ph) _umul128(a, b, ph)\n#pragma intrinsic(_addcarry_u64)\n#define add64carry_first(base, addend, sum) _addcarry_u64(0, base, addend, sum)\n#define add64carry_next(carry, base, addend, sum) \\\n _addcarry_u64(carry, base, addend, sum)\n#define add64carry_last(carry, base, addend, sum) \\\n (void)_addcarry_u64(carry, base, addend, sum)\n#endif\n\n#if defined(_M_ARM64) || defined(_M_X64) || defined(_M_IA64)\n#pragma intrinsic(__umulh)\n#define mul_64x64_high(a, b) __umulh(a, b)\n#endif\n\n#if defined(_M_IX86)\n#pragma intrinsic(__emulu)\n#define mul_32x32_64(a, b) __emulu(a, b)\n\n#if _MSC_VER >= 1915 /* LY: workaround for SSA-optimizer bug */\n#pragma intrinsic(_addcarry_u32)\n#define add32carry_first(base, addend, sum) _addcarry_u32(0, base, addend, sum)\n#define add32carry_next(carry, base, addend, sum) \\\n _addcarry_u32(carry, base, addend, sum)\n#define add32carry_last(carry, base, addend, sum) \\\n (void)_addcarry_u32(carry, base, addend, sum)\n\nstatic __forceinline char\nmsvc32_add64carry_first(uint64_t base, uint64_t addend, uint64_t *sum) {\n uint32_t *const sum32 = (uint32_t *)sum;\n const uint32_t base_32l = (uint32_t)base;\n const uint32_t base_32h = (uint32_t)(base >> 32);\n const uint32_t addend_32l = (uint32_t)addend;\n const uint32_t addend_32h = (uint32_t)(addend >> 32);\n return add32carry_next(add32carry_first(base_32l, addend_32l, sum32),\n base_32h, addend_32h, sum32 + 1);\n}\n#define add64carry_first(base, addend, sum) \\\n msvc32_add64carry_first(base, addend, sum)\n\nstatic __forceinline char msvc32_add64carry_next(char carry, uint64_t base,\n uint64_t addend,\n uint64_t *sum) {\n uint32_t *const sum32 = (uint32_t *)sum;\n const uint32_t base_32l = (uint32_t)base;\n const uint32_t base_32h = (uint32_t)(base >> 32);\n const uint32_t addend_32l = (uint32_t)addend;\n const uint32_t addend_32h = (uint32_t)(addend >> 32);\n return add32carry_next(add32carry_next(carry, base_32l, addend_32l, sum32),\n base_32h, addend_32h, sum32 + 1);\n}\n#define add64carry_next(carry, base, addend, sum) \\\n msvc32_add64carry_next(carry, base, addend, sum)\n\nstatic __forceinline void msvc32_add64carry_last(char carry, uint64_t base,\n uint64_t addend,\n uint64_t *sum) {\n uint32_t *const sum32 = (uint32_t *)sum;\n const uint32_t base_32l = (uint32_t)base;\n const uint32_t base_32h = (uint32_t)(base >> 32);\n const uint32_t addend_32l = (uint32_t)addend;\n const uint32_t addend_32h = (uint32_t)(addend >> 32);\n add32carry_last(add32carry_next(carry, base_32l, addend_32l, sum32), base_32h,\n addend_32h, sum32 + 1);\n}\n#define add64carry_last(carry, base, addend, sum) \\\n msvc32_add64carry_last(carry, base, addend, sum)\n#endif /* _MSC_FULL_VER >= 190024231 */\n\n#elif defined(_M_ARM)\n#define mul_32x32_64(a, b) _arm_umull(a, b)\n#endif\n\n#pragma warning(pop)\n#pragma warning(disable : 4514) /* 'xyz': unreferenced inline function \\\n has been removed */\n#pragma warning(disable : 4710) /* 'xyz': function not inlined */\n#pragma warning(disable : 4711) /* function 'xyz' selected for \\\n automatic inline expansion */\n#pragma warning(disable : 4127) /* conditional expression is constant */\n#pragma warning(disable : 4702) /* unreachable code */\n#endif /* Compiler */\n\n#ifndef likely\n#define likely(cond) (cond)\n#endif\n#ifndef unlikely\n#define unlikely(cond) (cond)\n#endif\n#ifndef __maybe_unused\n#define __maybe_unused\n#endif\n#ifndef __always_inline\n#define __always_inline __inline\n#endif\n#ifndef unreachable\n#define unreachable() \\\n do { \\\n } while (1)\n#endif\n\n#ifndef bswap64\n#if defined(bswap_64)\n#define bswap64 bswap_64\n#elif defined(__bswap_64)\n#define bswap64 __bswap_64\n#else\nstatic __always_inline uint64_t bswap64(uint64_t v) {\n return v << 56 | v >> 56 | ((v << 40) & UINT64_C(0x00ff000000000000)) |\n ((v << 24) & UINT64_C(0x0000ff0000000000)) |\n ((v << 8) & UINT64_C(0x000000ff00000000)) |\n ((v >> 8) & UINT64_C(0x00000000ff000000)) |\n ((v >> 24) & UINT64_C(0x0000000000ff0000)) |\n ((v >> 40) & UINT64_C(0x000000000000ff00));\n}\n#endif\n#endif /* bswap64 */\n\n#ifndef bswap32\n#if defined(bswap_32)\n#define bswap32 bswap_32\n#elif defined(__bswap_32)\n#define bswap32 __bswap_32\n#else\nstatic __always_inline uint32_t bswap32(uint32_t v) {\n return v << 24 | v >> 24 | ((v << 8) & UINT32_C(0x00ff0000)) |\n ((v >> 8) & UINT32_C(0x0000ff00));\n}\n#endif\n#endif /* bswap32 */\n\n#ifndef bswap16\n#if defined(bswap_16)\n#define bswap16 bswap_16\n#elif defined(__bswap_16)\n#define bswap16 __bswap_16\n#else\nstatic __always_inline uint16_t bswap16(uint16_t v) { return v << 8 | v >> 8; }\n#endif\n#endif /* bswap16 */\n\n#if defined(__ia32__) || \\\n T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__EFFICIENT\n/* The __builtin_assume_aligned() leads gcc/clang to load values into the\n * registers, even when it is possible to directly use an operand from memory.\n * This can lead to a shortage of registers and a significant slowdown.\n * Therefore avoid unnecessary use of __builtin_assume_aligned() for x86. */\n#define read_unaligned(ptr, bits) (*(const uint##bits##_t *__restrict)(ptr))\n#define read_aligned(ptr, bits) (*(const uint##bits##_t *__restrict)(ptr))\n#endif /* __ia32__ */\n\n#ifndef read_unaligned\n#if defined(__GNUC__) || __has_attribute(__packed__)\ntypedef struct {\n uint8_t unaligned_8;\n uint16_t unaligned_16;\n uint32_t unaligned_32;\n uint64_t unaligned_64;\n} __attribute__((__packed__)) t1ha_unaligned_proxy;\n#define read_unaligned(ptr, bits) \\\n (((const t1ha_unaligned_proxy *)((const uint8_t *)(ptr)-offsetof( \\\n t1ha_unaligned_proxy, unaligned_##bits))) \\\n ->unaligned_##bits)\n#elif defined(_MSC_VER)\n#pragma warning( \\\n disable : 4235) /* nonstandard extension used: '__unaligned' \\\n * keyword not supported on this architecture */\n#define read_unaligned(ptr, bits) (*(const __unaligned uint##bits##_t *)(ptr))\n#else\n#pragma pack(push, 1)\ntypedef struct {\n uint8_t unaligned_8;\n uint16_t unaligned_16;\n uint32_t unaligned_32;\n uint64_t unaligned_64;\n} t1ha_unaligned_proxy;\n#pragma pack(pop)\n#define read_unaligned(ptr, bits) \\\n (((const t1ha_unaligned_proxy *)((const uint8_t *)(ptr)-offsetof( \\\n t1ha_unaligned_proxy, unaligned_##bits))) \\\n ->unaligned_##bits)\n#endif\n#endif /* read_unaligned */\n\n#ifndef read_aligned\n#if __GNUC_PREREQ(4, 8) || __has_builtin(__builtin_assume_aligned)\n#define read_aligned(ptr, bits) \\\n (*(const uint##bits##_t *)__builtin_assume_aligned(ptr, ALIGNMENT_##bits))\n#elif (__GNUC_PREREQ(3, 3) || __has_attribute(__aligned__)) && \\\n !defined(__clang__)\n#define read_aligned(ptr, bits) \\\n (*(const uint##bits##_t \\\n __attribute__((__aligned__(ALIGNMENT_##bits))) *)(ptr))\n#elif __has_attribute(__assume_aligned__)\n\nstatic __always_inline const\n uint16_t *__attribute__((__assume_aligned__(ALIGNMENT_16)))\n cast_aligned_16(const void *ptr) {\n return (const uint16_t *)ptr;\n}\nstatic __always_inline const\n uint32_t *__attribute__((__assume_aligned__(ALIGNMENT_32)))\n cast_aligned_32(const void *ptr) {\n return (const uint32_t *)ptr;\n}\nstatic __always_inline const\n uint64_t *__attribute__((__assume_aligned__(ALIGNMENT_64)))\n cast_aligned_64(const void *ptr) {\n return (const uint64_t *)ptr;\n}\n\n#define read_aligned(ptr, bits) (*cast_aligned_##bits(ptr))\n\n#elif defined(_MSC_VER)\n#define read_aligned(ptr, bits) \\\n (*(const __declspec(align(ALIGNMENT_##bits)) uint##bits##_t *)(ptr))\n#else\n#define read_aligned(ptr, bits) (*(const uint##bits##_t *)(ptr))\n#endif\n#endif /* read_aligned */\n\n#ifndef prefetch\n#if (__GNUC_PREREQ(4, 0) || __has_builtin(__builtin_prefetch)) && \\\n !defined(__ia32__)\n#define prefetch(ptr) __builtin_prefetch(ptr)\n#elif defined(_M_ARM64) || defined(_M_ARM)\n#define prefetch(ptr) __prefetch(ptr)\n#else\n#define prefetch(ptr) \\\n do { \\\n (void)(ptr); \\\n } while (0)\n#endif\n#endif /* prefetch */\n\n#if __has_warning(\"-Wconstant-logical-operand\")\n#if defined(__clang__)\n#pragma clang diagnostic ignored \"-Wconstant-logical-operand\"\n#elif defined(__GNUC__)\n#pragma GCC diagnostic ignored \"-Wconstant-logical-operand\"\n#else\n#pragma warning disable \"constant-logical-operand\"\n#endif\n#endif /* -Wconstant-logical-operand */\n\n#if __has_warning(\"-Wtautological-pointer-compare\")\n#if defined(__clang__)\n#pragma clang diagnostic ignored \"-Wtautological-pointer-compare\"\n#elif defined(__GNUC__)\n#pragma GCC diagnostic ignored \"-Wtautological-pointer-compare\"\n#else\n#pragma warning disable \"tautological-pointer-compare\"\n#endif\n#endif /* -Wtautological-pointer-compare */\n\n/***************************************************************************/\n\n#if __GNUC_PREREQ(4, 0)\n#pragma GCC visibility push(hidden)\n#endif /* __GNUC_PREREQ(4,0) */\n\n/*---------------------------------------------------------- Little Endian */\n\n#ifndef fetch16_le_aligned\nstatic __maybe_unused __always_inline uint16_t\nfetch16_le_aligned(const void *v) {\n assert(((uintptr_t)v) % ALIGNMENT_16 == 0);\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n return read_aligned(v, 16);\n#else\n return bswap16(read_aligned(v, 16));\n#endif\n}\n#endif /* fetch16_le_aligned */\n\n#ifndef fetch16_le_unaligned\nstatic __maybe_unused __always_inline uint16_t\nfetch16_le_unaligned(const void *v) {\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__UNABLE && \\\n !defined(__ARM_FEATURE_UNALIGNED)\n const uint8_t *p = (const uint8_t *)v;\n return p[0] | (uint16_t)p[1] << 8;\n#elif __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n return read_unaligned(v, 16);\n#else\n return bswap16(read_unaligned(v, 16));\n#endif\n}\n#endif /* fetch16_le_unaligned */\n\n#ifndef fetch32_le_aligned\nstatic __maybe_unused __always_inline uint32_t\nfetch32_le_aligned(const void *v) {\n assert(((uintptr_t)v) % ALIGNMENT_32 == 0);\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n return read_aligned(v, 32);\n#else\n return bswap32(read_aligned(v, 32));\n#endif\n}\n#endif /* fetch32_le_aligned */\n\n#ifndef fetch32_le_unaligned\nstatic __maybe_unused __always_inline uint32_t\nfetch32_le_unaligned(const void *v) {\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__UNABLE && \\\n !defined(__ARM_FEATURE_UNALIGNED)\n return fetch16_le_unaligned(v) |\n (uint32_t)fetch16_le_unaligned((const uint8_t *)v + 2) << 16;\n#elif __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n return read_unaligned(v, 32);\n#else\n return bswap32(read_unaligned(v, 32));\n#endif\n}\n#endif /* fetch32_le_unaligned */\n\n#ifndef fetch64_le_aligned\nstatic __maybe_unused __always_inline uint64_t\nfetch64_le_aligned(const void *v) {\n assert(((uintptr_t)v) % ALIGNMENT_64 == 0);\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n return read_aligned(v, 64);\n#else\n return bswap64(read_aligned(v, 64));\n#endif\n}\n#endif /* fetch64_le_aligned */\n\n#ifndef fetch64_le_unaligned\nstatic __maybe_unused __always_inline uint64_t\nfetch64_le_unaligned(const void *v) {\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__UNABLE\n return fetch32_le_unaligned(v) |\n (uint64_t)fetch32_le_unaligned((const uint8_t *)v + 4) << 32;\n#elif __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n return read_unaligned(v, 64);\n#else\n return bswap64(read_unaligned(v, 64));\n#endif\n}\n#endif /* fetch64_le_unaligned */\n\nstatic __maybe_unused __always_inline uint64_t tail64_le_aligned(const void *v,\n size_t tail) {\n const uint8_t *const p = (const uint8_t *)v;\n#if T1HA_USE_FAST_ONESHOT_READ && !defined(__SANITIZE_ADDRESS__)\n /* We can perform a 'oneshot' read, which is little bit faster. */\n const unsigned shift = ((8 - tail) & 7) << 3;\n return fetch64_le_aligned(p) & ((~UINT64_C(0)) >> shift);\n#else\n uint64_t r = 0;\n switch (tail & 7) {\n default:\n unreachable();\n/* fall through */\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n /* For most CPUs this code is better when not needed byte reordering. */\n case 0:\n return fetch64_le_aligned(p);\n case 7:\n r = (uint64_t)p[6] << 8;\n /* fall through */\n case 6:\n r += p[5];\n r <<= 8;\n /* fall through */\n case 5:\n r += p[4];\n r <<= 32;\n /* fall through */\n case 4:\n return r + fetch32_le_aligned(p);\n case 3:\n r = (uint64_t)p[2] << 16;\n /* fall through */\n case 2:\n return r + fetch16_le_aligned(p);\n case 1:\n return p[0];\n#else\n case 0:\n r = p[7] << 8;\n /* fall through */\n case 7:\n r += p[6];\n r <<= 8;\n /* fall through */\n case 6:\n r += p[5];\n r <<= 8;\n /* fall through */\n case 5:\n r += p[4];\n r <<= 8;\n /* fall through */\n case 4:\n r += p[3];\n r <<= 8;\n /* fall through */\n case 3:\n r += p[2];\n r <<= 8;\n /* fall through */\n case 2:\n r += p[1];\n r <<= 8;\n /* fall through */\n case 1:\n return r + p[0];\n#endif\n }\n#endif /* T1HA_USE_FAST_ONESHOT_READ */\n}\n\n#if T1HA_USE_FAST_ONESHOT_READ && \\\n T1HA_SYS_UNALIGNED_ACCESS != T1HA_UNALIGNED_ACCESS__UNABLE && \\\n defined(PAGESIZE) && PAGESIZE > 42 && !defined(__SANITIZE_ADDRESS__)\n#define can_read_underside(ptr, size) \\\n (((PAGESIZE - (size)) & (uintptr_t)(ptr)) != 0)\n#endif /* T1HA_USE_FAST_ONESHOT_READ */\n\nstatic __maybe_unused __always_inline uint64_t\ntail64_le_unaligned(const void *v, size_t tail) {\n const uint8_t *p = (const uint8_t *)v;\n#if defined(can_read_underside) && \\\n (UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul)\n /* On some systems (e.g. x86_64) we can perform a 'oneshot' read, which\n * is little bit faster. Thanks Marcin Żukowski \n * for the reminder. */\n const unsigned offset = (8 - tail) & 7;\n const unsigned shift = offset << 3;\n if (likely(can_read_underside(p, 8))) {\n p -= offset;\n return fetch64_le_unaligned(p) >> shift;\n }\n return fetch64_le_unaligned(p) & ((~UINT64_C(0)) >> shift);\n#else\n uint64_t r = 0;\n switch (tail & 7) {\n default:\n unreachable();\n/* fall through */\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__EFFICIENT && \\\n __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n /* For most CPUs this code is better when not needed\n * copying for alignment or byte reordering. */\n case 0:\n return fetch64_le_unaligned(p);\n case 7:\n r = (uint64_t)p[6] << 8;\n /* fall through */\n case 6:\n r += p[5];\n r <<= 8;\n /* fall through */\n case 5:\n r += p[4];\n r <<= 32;\n /* fall through */\n case 4:\n return r + fetch32_le_unaligned(p);\n case 3:\n r = (uint64_t)p[2] << 16;\n /* fall through */\n case 2:\n return r + fetch16_le_unaligned(p);\n case 1:\n return p[0];\n#else\n /* For most CPUs this code is better than a\n * copying for alignment and/or byte reordering. */\n case 0:\n r = p[7] << 8;\n /* fall through */\n case 7:\n r += p[6];\n r <<= 8;\n /* fall through */\n case 6:\n r += p[5];\n r <<= 8;\n /* fall through */\n case 5:\n r += p[4];\n r <<= 8;\n /* fall through */\n case 4:\n r += p[3];\n r <<= 8;\n /* fall through */\n case 3:\n r += p[2];\n r <<= 8;\n /* fall through */\n case 2:\n r += p[1];\n r <<= 8;\n /* fall through */\n case 1:\n return r + p[0];\n#endif\n }\n#endif /* can_read_underside */\n}\n\n/*------------------------------------------------------------- Big Endian */\n\n#ifndef fetch16_be_aligned\nstatic __maybe_unused __always_inline uint16_t\nfetch16_be_aligned(const void *v) {\n assert(((uintptr_t)v) % ALIGNMENT_16 == 0);\n#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n return read_aligned(v, 16);\n#else\n return bswap16(read_aligned(v, 16));\n#endif\n}\n#endif /* fetch16_be_aligned */\n\n#ifndef fetch16_be_unaligned\nstatic __maybe_unused __always_inline uint16_t\nfetch16_be_unaligned(const void *v) {\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__UNABLE\n const uint8_t *p = (const uint8_t *)v;\n return (uint16_t)p[0] << 8 | p[1];\n#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n return read_unaligned(v, 16);\n#else\n return bswap16(read_unaligned(v, 16));\n#endif\n}\n#endif /* fetch16_be_unaligned */\n\n#ifndef fetch32_be_aligned\nstatic __maybe_unused __always_inline uint32_t\nfetch32_be_aligned(const void *v) {\n assert(((uintptr_t)v) % ALIGNMENT_32 == 0);\n#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n return read_aligned(v, 32);\n#else\n return bswap32(read_aligned(v, 32));\n#endif\n}\n#endif /* fetch32_be_aligned */\n\n#ifndef fetch32_be_unaligned\nstatic __maybe_unused __always_inline uint32_t\nfetch32_be_unaligned(const void *v) {\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__UNABLE\n return (uint32_t)fetch16_be_unaligned(v) << 16 |\n fetch16_be_unaligned((const uint8_t *)v + 2);\n#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n return read_unaligned(v, 32);\n#else\n return bswap32(read_unaligned(v, 32));\n#endif\n}\n#endif /* fetch32_be_unaligned */\n\n#ifndef fetch64_be_aligned\nstatic __maybe_unused __always_inline uint64_t\nfetch64_be_aligned(const void *v) {\n assert(((uintptr_t)v) % ALIGNMENT_64 == 0);\n#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n return read_aligned(v, 64);\n#else\n return bswap64(read_aligned(v, 64));\n#endif\n}\n#endif /* fetch64_be_aligned */\n\n#ifndef fetch64_be_unaligned\nstatic __maybe_unused __always_inline uint64_t\nfetch64_be_unaligned(const void *v) {\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__UNABLE\n return (uint64_t)fetch32_be_unaligned(v) << 32 |\n fetch32_be_unaligned((const uint8_t *)v + 4);\n#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n return read_unaligned(v, 64);\n#else\n return bswap64(read_unaligned(v, 64));\n#endif\n}\n#endif /* fetch64_be_unaligned */\n\nstatic __maybe_unused __always_inline uint64_t tail64_be_aligned(const void *v,\n size_t tail) {\n const uint8_t *const p = (const uint8_t *)v;\n#if T1HA_USE_FAST_ONESHOT_READ && !defined(__SANITIZE_ADDRESS__)\n /* We can perform a 'oneshot' read, which is little bit faster. */\n const unsigned shift = ((8 - tail) & 7) << 3;\n return fetch64_be_aligned(p) >> shift;\n#else\n switch (tail & 7) {\n default:\n unreachable();\n/* fall through */\n#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n /* For most CPUs this code is better when not byte reordering. */\n case 1:\n return p[0];\n case 2:\n return fetch16_be_aligned(p);\n case 3:\n return (uint32_t)fetch16_be_aligned(p) << 8 | p[2];\n case 4:\n return fetch32_be_aligned(p);\n case 5:\n return (uint64_t)fetch32_be_aligned(p) << 8 | p[4];\n case 6:\n return (uint64_t)fetch32_be_aligned(p) << 16 | fetch16_be_aligned(p + 4);\n case 7:\n return (uint64_t)fetch32_be_aligned(p) << 24 |\n (uint32_t)fetch16_be_aligned(p + 4) << 8 | p[6];\n case 0:\n return fetch64_be_aligned(p);\n#else\n case 1:\n return p[0];\n case 2:\n return p[1] | (uint32_t)p[0] << 8;\n case 3:\n return p[2] | (uint32_t)p[1] << 8 | (uint32_t)p[0] << 16;\n case 4:\n return p[3] | (uint32_t)p[2] << 8 | (uint32_t)p[1] << 16 |\n (uint32_t)p[0] << 24;\n case 5:\n return p[4] | (uint32_t)p[3] << 8 | (uint32_t)p[2] << 16 |\n (uint32_t)p[1] << 24 | (uint64_t)p[0] << 32;\n case 6:\n return p[5] | (uint32_t)p[4] << 8 | (uint32_t)p[3] << 16 |\n (uint32_t)p[2] << 24 | (uint64_t)p[1] << 32 | (uint64_t)p[0] << 40;\n case 7:\n return p[6] | (uint32_t)p[5] << 8 | (uint32_t)p[4] << 16 |\n (uint32_t)p[3] << 24 | (uint64_t)p[2] << 32 | (uint64_t)p[1] << 40 |\n (uint64_t)p[0] << 48;\n case 0:\n return p[7] | (uint32_t)p[6] << 8 | (uint32_t)p[5] << 16 |\n (uint32_t)p[4] << 24 | (uint64_t)p[3] << 32 | (uint64_t)p[2] << 40 |\n (uint64_t)p[1] << 48 | (uint64_t)p[0] << 56;\n#endif\n }\n#endif /* T1HA_USE_FAST_ONESHOT_READ */\n}\n\nstatic __maybe_unused __always_inline uint64_t\ntail64_be_unaligned(const void *v, size_t tail) {\n const uint8_t *p = (const uint8_t *)v;\n#if defined(can_read_underside) && \\\n (UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul)\n /* On some systems (e.g. x86_64) we can perform a 'oneshot' read, which\n * is little bit faster. Thanks Marcin Żukowski \n * for the reminder. */\n const unsigned offset = (8 - tail) & 7;\n const unsigned shift = offset << 3;\n if (likely(can_read_underside(p, 8))) {\n p -= offset;\n return fetch64_be_unaligned(p) & ((~UINT64_C(0)) >> shift);\n }\n return fetch64_be_unaligned(p) >> shift;\n#else\n switch (tail & 7) {\n default:\n unreachable();\n/* fall through */\n#if T1HA_SYS_UNALIGNED_ACCESS == T1HA_UNALIGNED_ACCESS__EFFICIENT && \\\n __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n /* For most CPUs this code is better when not needed\n * copying for alignment or byte reordering. */\n case 1:\n return p[0];\n case 2:\n return fetch16_be_unaligned(p);\n case 3:\n return (uint32_t)fetch16_be_unaligned(p) << 8 | p[2];\n case 4:\n return fetch32_be(p);\n case 5:\n return (uint64_t)fetch32_be_unaligned(p) << 8 | p[4];\n case 6:\n return (uint64_t)fetch32_be_unaligned(p) << 16 |\n fetch16_be_unaligned(p + 4);\n case 7:\n return (uint64_t)fetch32_be_unaligned(p) << 24 |\n (uint32_t)fetch16_be_unaligned(p + 4) << 8 | p[6];\n case 0:\n return fetch64_be_unaligned(p);\n#else\n /* For most CPUs this code is better than a\n * copying for alignment and/or byte reordering. */\n case 1:\n return p[0];\n case 2:\n return p[1] | (uint32_t)p[0] << 8;\n case 3:\n return p[2] | (uint32_t)p[1] << 8 | (uint32_t)p[0] << 16;\n case 4:\n return p[3] | (uint32_t)p[2] << 8 | (uint32_t)p[1] << 16 |\n (uint32_t)p[0] << 24;\n case 5:\n return p[4] | (uint32_t)p[3] << 8 | (uint32_t)p[2] << 16 |\n (uint32_t)p[1] << 24 | (uint64_t)p[0] << 32;\n case 6:\n return p[5] | (uint32_t)p[4] << 8 | (uint32_t)p[3] << 16 |\n (uint32_t)p[2] << 24 | (uint64_t)p[1] << 32 | (uint64_t)p[0] << 40;\n case 7:\n return p[6] | (uint32_t)p[5] << 8 | (uint32_t)p[4] << 16 |\n (uint32_t)p[3] << 24 | (uint64_t)p[2] << 32 | (uint64_t)p[1] << 40 |\n (uint64_t)p[0] << 48;\n case 0:\n return p[7] | (uint32_t)p[6] << 8 | (uint32_t)p[5] << 16 |\n (uint32_t)p[4] << 24 | (uint64_t)p[3] << 32 | (uint64_t)p[2] << 40 |\n (uint64_t)p[1] << 48 | (uint64_t)p[0] << 56;\n#endif\n }\n#endif /* can_read_underside */\n}\n\n/***************************************************************************/\n\n#ifndef rot64\nstatic __maybe_unused __always_inline uint64_t rot64(uint64_t v, unsigned s) {\n return (v >> s) | (v << (64 - s));\n}\n#endif /* rot64 */\n\n#ifndef mul_32x32_64\nstatic __maybe_unused __always_inline uint64_t mul_32x32_64(uint32_t a,\n uint32_t b) {\n return a * (uint64_t)b;\n}\n#endif /* mul_32x32_64 */\n\n#ifndef add64carry_first\nstatic __maybe_unused __always_inline unsigned\nadd64carry_first(uint64_t base, uint64_t addend, uint64_t *sum) {\n#if __has_builtin(__builtin_addcll)\n unsigned long long carryout;\n *sum = __builtin_addcll(base, addend, 0, &carryout);\n return (unsigned)carryout;\n#else\n *sum = base + addend;\n return *sum < addend;\n#endif /* __has_builtin(__builtin_addcll) */\n}\n#endif /* add64carry_fist */\n\n#ifndef add64carry_next\nstatic __maybe_unused __always_inline unsigned\nadd64carry_next(unsigned carry, uint64_t base, uint64_t addend, uint64_t *sum) {\n#if __has_builtin(__builtin_addcll)\n unsigned long long carryout;\n *sum = __builtin_addcll(base, addend, carry, &carryout);\n return (unsigned)carryout;\n#else\n *sum = base + addend + carry;\n return *sum < addend || (carry && *sum == addend);\n#endif /* __has_builtin(__builtin_addcll) */\n}\n#endif /* add64carry_next */\n\n#ifndef add64carry_last\nstatic __maybe_unused __always_inline void\nadd64carry_last(unsigned carry, uint64_t base, uint64_t addend, uint64_t *sum) {\n#if __has_builtin(__builtin_addcll)\n unsigned long long carryout;\n *sum = __builtin_addcll(base, addend, carry, &carryout);\n (void)carryout;\n#else\n *sum = base + addend + carry;\n#endif /* __has_builtin(__builtin_addcll) */\n}\n#endif /* add64carry_last */\n\n#ifndef mul_64x64_128\nstatic __maybe_unused __always_inline uint64_t mul_64x64_128(uint64_t a,\n uint64_t b,\n uint64_t *h) {\n#if (defined(__SIZEOF_INT128__) || \\\n (defined(_INTEGRAL_MAX_BITS) && _INTEGRAL_MAX_BITS >= 128)) && \\\n (!defined(__LCC__) || __LCC__ != 124)\n __uint128_t r = (__uint128_t)a * (__uint128_t)b;\n /* modern GCC could nicely optimize this */\n *h = (uint64_t)(r >> 64);\n return (uint64_t)r;\n#elif defined(mul_64x64_high)\n *h = mul_64x64_high(a, b);\n return a * b;\n#else\n /* performs 64x64 to 128 bit multiplication */\n const uint64_t ll = mul_32x32_64((uint32_t)a, (uint32_t)b);\n const uint64_t lh = mul_32x32_64(a >> 32, (uint32_t)b);\n const uint64_t hl = mul_32x32_64((uint32_t)a, b >> 32);\n const uint64_t hh = mul_32x32_64(a >> 32, b >> 32);\n\n /* Few simplification are possible here for 32-bit architectures,\n * but thus we would lost compatibility with the original 64-bit\n * version. Think is very bad idea, because then 32-bit t1ha will\n * still (relatively) very slowly and well yet not compatible. */\n uint64_t l;\n add64carry_last(add64carry_first(ll, lh << 32, &l), hh, lh >> 32, h);\n add64carry_last(add64carry_first(l, hl << 32, &l), *h, hl >> 32, h);\n return l;\n#endif\n}\n#endif /* mul_64x64_128() */\n\n#ifndef mul_64x64_high\nstatic __maybe_unused __always_inline uint64_t mul_64x64_high(uint64_t a,\n uint64_t b) {\n uint64_t h;\n mul_64x64_128(a, b, &h);\n return h;\n}\n#endif /* mul_64x64_high */\n\n/***************************************************************************/\n\n/* 'magic' primes */\nstatic const uint64_t prime_0 = UINT64_C(0xEC99BF0D8372CAAB);\nstatic const uint64_t prime_1 = UINT64_C(0x82434FE90EDCEF39);\nstatic const uint64_t prime_2 = UINT64_C(0xD4F06DB99D67BE4B);\nstatic const uint64_t prime_3 = UINT64_C(0xBD9CACC22C6E9571);\nstatic const uint64_t prime_4 = UINT64_C(0x9C06FAF4D023E3AB);\nstatic const uint64_t prime_5 = UINT64_C(0xC060724A8424F345);\nstatic const uint64_t prime_6 = UINT64_C(0xCB5AF53AE3AAAC31);\n\n/* xor high and low parts of full 128-bit product */\nstatic __maybe_unused __always_inline uint64_t mux64(uint64_t v,\n uint64_t prime) {\n uint64_t l, h;\n l = mul_64x64_128(v, prime, &h);\n return l ^ h;\n}\n\nstatic __maybe_unused __always_inline uint64_t final64(uint64_t a, uint64_t b) {\n uint64_t x = (a + rot64(b, 41)) * prime_0;\n uint64_t y = (rot64(a, 23) + b) * prime_6;\n return mux64(x ^ y, prime_5);\n}\n\nstatic __maybe_unused __always_inline void mixup64(uint64_t *__restrict a,\n uint64_t *__restrict b,\n uint64_t v, uint64_t prime) {\n uint64_t h;\n *a ^= mul_64x64_128(*b + v, prime, &h);\n *b += h;\n}\n\n/***************************************************************************/\n\ntypedef union t1ha_uint128 {\n#if defined(__SIZEOF_INT128__) || \\\n (defined(_INTEGRAL_MAX_BITS) && _INTEGRAL_MAX_BITS >= 128)\n __uint128_t v;\n#endif\n struct {\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n uint64_t l, h;\n#else\n uint64_t h, l;\n#endif\n };\n} t1ha_uint128_t;\n\nstatic __maybe_unused __always_inline t1ha_uint128_t\nnot128(const t1ha_uint128_t v) {\n t1ha_uint128_t r;\n#if defined(__SIZEOF_INT128__) || \\\n (defined(_INTEGRAL_MAX_BITS) && _INTEGRAL_MAX_BITS >= 128)\n r.v = ~v.v;\n#else\n r.l = ~v.l;\n r.h = ~v.h;\n#endif\n return r;\n}\n\nstatic __maybe_unused __always_inline t1ha_uint128_t\nleft128(const t1ha_uint128_t v, unsigned s) {\n t1ha_uint128_t r;\n assert(s < 128);\n#if defined(__SIZEOF_INT128__) || \\\n (defined(_INTEGRAL_MAX_BITS) && _INTEGRAL_MAX_BITS >= 128)\n r.v = v.v << s;\n#else\n r.l = (s < 64) ? v.l << s : 0;\n r.h = (s < 64) ? (v.h << s) | (s ? v.l >> (64 - s) : 0) : v.l << (s - 64);\n#endif\n return r;\n}\n\nstatic __maybe_unused __always_inline t1ha_uint128_t\nright128(const t1ha_uint128_t v, unsigned s) {\n t1ha_uint128_t r;\n assert(s < 128);\n#if defined(__SIZEOF_INT128__) || \\\n (defined(_INTEGRAL_MAX_BITS) && _INTEGRAL_MAX_BITS >= 128)\n r.v = v.v >> s;\n#else\n r.l = (s < 64) ? (s ? v.h << (64 - s) : 0) | (v.l >> s) : v.h >> (s - 64);\n r.h = (s < 64) ? v.h >> s : 0;\n#endif\n return r;\n}\n\nstatic __maybe_unused __always_inline t1ha_uint128_t or128(t1ha_uint128_t x,\n t1ha_uint128_t y) {\n t1ha_uint128_t r;\n#if defined(__SIZEOF_INT128__) || \\\n (defined(_INTEGRAL_MAX_BITS) && _INTEGRAL_MAX_BITS >= 128)\n r.v = x.v | y.v;\n#else\n r.l = x.l | y.l;\n r.h = x.h | y.h;\n#endif\n return r;\n}\n\nstatic __maybe_unused __always_inline t1ha_uint128_t xor128(t1ha_uint128_t x,\n t1ha_uint128_t y) {\n t1ha_uint128_t r;\n#if defined(__SIZEOF_INT128__) || \\\n (defined(_INTEGRAL_MAX_BITS) && _INTEGRAL_MAX_BITS >= 128)\n r.v = x.v ^ y.v;\n#else\n r.l = x.l ^ y.l;\n r.h = x.h ^ y.h;\n#endif\n return r;\n}\n\nstatic __maybe_unused __always_inline t1ha_uint128_t rot128(t1ha_uint128_t v,\n unsigned s) {\n s &= 127;\n#if defined(__SIZEOF_INT128__) || \\\n (defined(_INTEGRAL_MAX_BITS) && _INTEGRAL_MAX_BITS >= 128)\n v.v = (v.v << (128 - s)) | (v.v >> s);\n return v;\n#else\n return s ? or128(left128(v, 128 - s), right128(v, s)) : v;\n#endif\n}\n\nstatic __maybe_unused __always_inline t1ha_uint128_t add128(t1ha_uint128_t x,\n t1ha_uint128_t y) {\n t1ha_uint128_t r;\n#if defined(__SIZEOF_INT128__) || \\\n (defined(_INTEGRAL_MAX_BITS) && _INTEGRAL_MAX_BITS >= 128)\n r.v = x.v + y.v;\n#else\n add64carry_last(add64carry_first(x.l, y.l, &r.l), x.h, y.h, &r.h);\n#endif\n return r;\n}\n\nstatic __maybe_unused __always_inline t1ha_uint128_t mul128(t1ha_uint128_t x,\n t1ha_uint128_t y) {\n t1ha_uint128_t r;\n#if defined(__SIZEOF_INT128__) || \\\n (defined(_INTEGRAL_MAX_BITS) && _INTEGRAL_MAX_BITS >= 128)\n r.v = x.v * y.v;\n#else\n r.l = mul_64x64_128(x.l, y.l, &r.h);\n r.h += x.l * y.h + y.l * x.h;\n#endif\n return r;\n}\n\n/***************************************************************************/\n\n#if T1HA0_AESNI_AVAILABLE && defined(__ia32__)\nuint64_t t1ha_ia32cpu_features(void);\n\nstatic __maybe_unused __always_inline bool\nt1ha_ia32_AESNI_avail(uint64_t ia32cpu_features) {\n /* check for AES-NI */\n return (ia32cpu_features & UINT32_C(0x02000000)) != 0;\n}\n\nstatic __maybe_unused __always_inline bool\nt1ha_ia32_AVX_avail(uint64_t ia32cpu_features) {\n /* check for any AVX */\n return (ia32cpu_features & UINT32_C(0x1A000000)) == UINT32_C(0x1A000000);\n}\n\nstatic __maybe_unused __always_inline bool\nt1ha_ia32_AVX2_avail(uint64_t ia32cpu_features) {\n /* check for 'Advanced Vector Extensions 2' */\n return ((ia32cpu_features >> 32) & 32) != 0;\n}\n\n#endif /* T1HA0_AESNI_AVAILABLE && __ia32__ */\n" }, { "path": "benchmarks/t1ha/src/t1ha_selfcheck.c", "content": "/*\n * Copyright (c) 2016-2020 Positive Technologies, https://www.ptsecurity.com,\n * Fast Positive Hash.\n *\n * Portions Copyright (c) 2010-2020 Leonid Yuriev ,\n * The 1Hippeus project (t1h).\n *\n * This software is provided 'as-is', without any express or implied\n * warranty. In no event will the authors be held liable for any damages\n * arising from the use of this software.\n *\n * Permission is granted to anyone to use this software for any purpose,\n * including commercial applications, and to alter it and redistribute it\n * freely, subject to the following restrictions:\n *\n * 1. The origin of this software must not be misrepresented; you must not\n * claim that you wrote the original software. If you use this software\n * in a product, an acknowledgement in the product documentation would be\n * appreciated but is not required.\n * 2. Altered source versions must be plainly marked as such, and must not be\n * misrepresented as being the original software.\n * 3. This notice may not be removed or altered from any source distribution.\n */\n\n/*\n * t1ha = { Fast Positive Hash, aka \"Позитивный Хэш\" }\n * by [Positive Technologies](https://www.ptsecurity.ru)\n *\n * Briefly, it is a 64-bit Hash Function:\n * 1. Created for 64-bit little-endian platforms, in predominantly for x86_64,\n * but portable and without penalties it can run on any 64-bit CPU.\n * 2. In most cases up to 15% faster than City64, xxHash, mum-hash, metro-hash\n * and all others portable hash-functions (which do not use specific\n * hardware tricks).\n * 3. Not suitable for cryptography.\n *\n * The Future will (be) Positive. Всё будет хорошо.\n *\n * ACKNOWLEDGEMENT:\n * The t1ha was originally developed by Leonid Yuriev (Леонид Юрьев)\n * for The 1Hippeus project - zerocopy messaging in the spirit of Sparta!\n */\n\n#include \"t1ha_selfcheck.h\"\n#include \"t1ha_bits.h\"\n\nconst uint8_t t1ha_test_pattern[64] = {\n 0, 1, 2, 3, 4, 5, 6, 7, 0xFF, 0x7F, 0x3F,\n 0x1F, 0xF, 8, 16, 32, 64, 0x80, 0xFE, 0xFC, 0xF8, 0xF0,\n 0xE0, 0xC0, 0xFD, 0xFB, 0xF7, 0xEF, 0xDF, 0xBF, 0x55, 0xAA, 11,\n 17, 19, 23, 29, 37, 42, 43, 'a', 'b', 'c', 'd',\n 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o',\n 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x'};\n\nstatic __inline bool probe(uint64_t (*hash)(const void *, size_t, uint64_t),\n const uint64_t reference, const void *data,\n unsigned len, uint64_t seed) {\n const uint64_t actual = hash(data, len, seed);\n assert(actual == reference);\n return actual != reference;\n}\n\n__cold int t1ha_selfcheck(uint64_t (*hash)(const void *, size_t, uint64_t),\n const uint64_t *reference_values) {\n bool failed = false;\n\n const uint64_t zero = 0;\n failed |= probe(hash, /* empty-zero */ *reference_values++, NULL, 0, zero);\n failed |= probe(hash, /* empty-all1 */ *reference_values++, NULL, 0, ~zero);\n failed |= probe(hash, /* bin64-zero */ *reference_values++, t1ha_test_pattern,\n 64, zero);\n\n uint64_t seed = 1;\n for (int i = 1; i < 64; i++) {\n /* bin%i-1p%i */\n failed |= probe(hash, *reference_values++, t1ha_test_pattern, i, seed);\n seed <<= 1;\n }\n\n seed = ~zero;\n for (int i = 1; i <= 7; i++) {\n seed <<= 1;\n /* align%i_F%i */;\n failed |=\n probe(hash, *reference_values++, t1ha_test_pattern + i, 64 - i, seed);\n }\n\n uint8_t pattern_long[512];\n for (size_t i = 0; i < sizeof(pattern_long); ++i)\n pattern_long[i] = (uint8_t)i;\n for (int i = 0; i <= 7; i++) {\n /* long-%05i */\n failed |=\n probe(hash, *reference_values++, pattern_long + i, 128 + i * 17, seed);\n }\n\n return failed ? -1 : 0;\n}\n" }, { "path": "benchmarks/t1ha/src/t1ha_selfcheck.h", "content": "/*\n * Copyright (c) 2016-2020 Positive Technologies, https://www.ptsecurity.com,\n * Fast Positive Hash.\n *\n * Portions Copyright (c) 2010-2020 Leonid Yuriev ,\n * The 1Hippeus project (t1h).\n *\n * This software is provided 'as-is', without any express or implied\n * warranty. In no event will the authors be held liable for any damages\n * arising from the use of this software.\n *\n * Permission is granted to anyone to use this software for any purpose,\n * including commercial applications, and to alter it and redistribute it\n * freely, subject to the following restrictions:\n *\n * 1. The origin of this software must not be misrepresented; you must not\n * claim that you wrote the original software. If you use this software\n * in a product, an acknowledgement in the product documentation would be\n * appreciated but is not required.\n * 2. Altered source versions must be plainly marked as such, and must not be\n * misrepresented as being the original software.\n * 3. This notice may not be removed or altered from any source distribution.\n */\n\n/*\n * t1ha = { Fast Positive Hash, aka \"Позитивный Хэш\" }\n * by [Positive Technologies](https://www.ptsecurity.ru)\n *\n * Briefly, it is a 64-bit Hash Function:\n * 1. Created for 64-bit little-endian platforms, in predominantly for x86_64,\n * but portable and without penalties it can run on any 64-bit CPU.\n * 2. In most cases up to 15% faster than City64, xxHash, mum-hash, metro-hash\n * and all others portable hash-functions (which do not use specific\n * hardware tricks).\n * 3. Not suitable for cryptography.\n *\n * The Future will (be) Positive. Всё будет хорошо.\n *\n * ACKNOWLEDGEMENT:\n * The t1ha was originally developed by Leonid Yuriev (Леонид Юрьев)\n * for The 1Hippeus project - zerocopy messaging in the spirit of Sparta!\n */\n\n#pragma once\n#if defined(_MSC_VER) && _MSC_VER > 1800\n#pragma warning(disable : 4464) /* relative include path contains '..' */\n#endif /* MSVC */\n#include \"../t1ha.h\"\n\n/***************************************************************************/\n/* Self-checking */\n\nextern const uint8_t t1ha_test_pattern[64];\nint t1ha_selfcheck(uint64_t (*hash)(const void *, size_t, uint64_t),\n const uint64_t *reference_values);\n\n#ifndef T1HA2_DISABLED\nextern const uint64_t t1ha_refval_2atonce[81];\nextern const uint64_t t1ha_refval_2atonce128[81];\nextern const uint64_t t1ha_refval_2stream[81];\nextern const uint64_t t1ha_refval_2stream128[81];\n#endif /* T1HA2_DISABLED */\n\n#ifndef T1HA1_DISABLED\nextern const uint64_t t1ha_refval_64le[81];\nextern const uint64_t t1ha_refval_64be[81];\n#endif /* T1HA1_DISABLED */\n\n#ifndef T1HA0_DISABLED\nextern const uint64_t t1ha_refval_32le[81];\nextern const uint64_t t1ha_refval_32be[81];\n#if T1HA0_AESNI_AVAILABLE\nextern const uint64_t t1ha_refval_ia32aes_a[81];\nextern const uint64_t t1ha_refval_ia32aes_b[81];\n#endif /* T1HA0_AESNI_AVAILABLE */\n#endif /* T1HA0_DISABLED */\n" }, { "path": "benchmarks/t1ha/src/t1ha_selfcheck_all.c", "content": "/*\n * Copyright (c) 2016-2020 Positive Technologies, https://www.ptsecurity.com,\n * Fast Positive Hash.\n *\n * Portions Copyright (c) 2010-2020 Leonid Yuriev ,\n * The 1Hippeus project (t1h).\n *\n * This software is provided 'as-is', without any express or implied\n * warranty. In no event will the authors be held liable for any damages\n * arising from the use of this software.\n *\n * Permission is granted to anyone to use this software for any purpose,\n * including commercial applications, and to alter it and redistribute it\n * freely, subject to the following restrictions:\n *\n * 1. The origin of this software must not be misrepresented; you must not\n * claim that you wrote the original software. If you use this software\n * in a product, an acknowledgement in the product documentation would be\n * appreciated but is not required.\n * 2. Altered source versions must be plainly marked as such, and must not be\n * misrepresented as being the original software.\n * 3. This notice may not be removed or altered from any source distribution.\n */\n\n/*\n * t1ha = { Fast Positive Hash, aka \"Позитивный Хэш\" }\n * by [Positive Technologies](https://www.ptsecurity.ru)\n *\n * Briefly, it is a 64-bit Hash Function:\n * 1. Created for 64-bit little-endian platforms, in predominantly for x86_64,\n * but portable and without penalties it can run on any 64-bit CPU.\n * 2. In most cases up to 15% faster than City64, xxHash, mum-hash, metro-hash\n * and all others portable hash-functions (which do not use specific\n * hardware tricks).\n * 3. Not suitable for cryptography.\n *\n * The Future will (be) Positive. Всё будет хорошо.\n *\n * ACKNOWLEDGEMENT:\n * The t1ha was originally developed by Leonid Yuriev (Леонид Юрьев)\n * for The 1Hippeus project - zerocopy messaging in the spirit of Sparta!\n */\n\n#include \"t1ha_bits.h\"\n#include \"t1ha_selfcheck.h\"\n\n__cold int t1ha_selfcheck__all_enabled(void) {\n int rc = 0;\n\n#ifndef T1HA2_DISABLED\n rc |= t1ha_selfcheck__t1ha2();\n#endif /* T1HA2_DISABLED */\n\n#ifndef T1HA1_DISABLED\n rc |= t1ha_selfcheck__t1ha1();\n#endif /* T1HA1_DISABLED */\n\n#ifndef T1HA0_DISABLED\n rc |= t1ha_selfcheck__t1ha0();\n#endif /* T1HA0_DISABLED */\n\n return rc;\n}\n" }, { "path": "benchmarks/t1ha/t1ha.h", "content": "/*\n * Copyright (c) 2016-2020 Positive Technologies, https://www.ptsecurity.com,\n * Fast Positive Hash.\n *\n * Portions Copyright (c) 2010-2020 Leonid Yuriev ,\n * The 1Hippeus project (t1h).\n *\n * This software is provided 'as-is', without any express or implied\n * warranty. In no event will the authors be held liable for any damages\n * arising from the use of this software.\n *\n * Permission is granted to anyone to use this software for any purpose,\n * including commercial applications, and to alter it and redistribute it\n * freely, subject to the following restrictions:\n *\n * 1. The origin of this software must not be misrepresented; you must not\n * claim that you wrote the original software. If you use this software\n * in a product, an acknowledgement in the product documentation would be\n * appreciated but is not required.\n * 2. Altered source versions must be plainly marked as such, and must not be\n * misrepresented as being the original software.\n * 3. This notice may not be removed or altered from any source distribution.\n */\n\n/*\n * t1ha = { Fast Positive Hash, aka \"Позитивный Хэш\" }\n * by [Positive Technologies](https://www.ptsecurity.ru)\n *\n * Briefly, it is a 64-bit Hash Function:\n * 1. Created for 64-bit little-endian platforms, in predominantly for x86_64,\n * but portable and without penalties it can run on any 64-bit CPU.\n * 2. In most cases up to 15% faster than City64, xxHash, mum-hash, metro-hash\n * and all others portable hash-functions (which do not use specific\n * hardware tricks).\n * 3. Not suitable for cryptography.\n *\n * The Future will (be) Positive. Всё будет хорошо.\n *\n * ACKNOWLEDGEMENT:\n * The t1ha was originally developed by Leonid Yuriev (Леонид Юрьев)\n * for The 1Hippeus project - zerocopy messaging in the spirit of Sparta!\n */\n\n#pragma once\n\n/*****************************************************************************\n *\n * PLEASE PAY ATTENTION TO THE FOLLOWING NOTES\n * about macros definitions which controls t1ha behaviour and/or performance.\n *\n *\n * 1) T1HA_SYS_UNALIGNED_ACCESS = Defines the system/platform/CPU/architecture\n * abilities for unaligned data access.\n *\n * By default, when the T1HA_SYS_UNALIGNED_ACCESS not defined,\n * it will defined on the basis hardcoded knowledge about of capabilities\n * of most common CPU architectures. But you could override this\n * default behavior when build t1ha library itself:\n *\n * // To disable unaligned access at all.\n * #define T1HA_SYS_UNALIGNED_ACCESS 0\n *\n * // To enable unaligned access, but indicate that it significantly slow.\n * #define T1HA_SYS_UNALIGNED_ACCESS 1\n *\n * // To enable unaligned access, and indicate that it effecient.\n * #define T1HA_SYS_UNALIGNED_ACCESS 2\n *\n *\n * 2) T1HA_USE_FAST_ONESHOT_READ = Controls the data reads at the end of buffer.\n *\n * When defined to non-zero, t1ha will use 'one shot' method for reading\n * up to 8 bytes at the end of data. In this case just the one 64-bit read\n * will be performed even when the available less than 8 bytes.\n *\n * This is little bit faster that switching by length of data tail.\n * Unfortunately this will triggering a false-positive alarms from Valgrind,\n * AddressSanitizer and other similar tool.\n *\n * By default, t1ha defines it to 1, but you could override this\n * default behavior when build t1ha library itself:\n *\n * // For little bit faster and small code.\n * #define T1HA_USE_FAST_ONESHOT_READ 1\n *\n * // For calmness if doubt.\n * #define T1HA_USE_FAST_ONESHOT_READ 0\n *\n *\n * 3) T1HA0_RUNTIME_SELECT = Controls choice fastest function in runtime.\n *\n * t1ha library offers the t1ha0() function as the fastest for current CPU.\n * But actual CPU's features/capabilities and may be significantly different,\n * especially on x86 platform. Therefore, internally, t1ha0() may require\n * dynamic dispatching for choice best implementation.\n *\n * By default, t1ha enables such runtime choice and (may be) corresponding\n * indirect calls if it reasonable, but you could override this default\n * behavior when build t1ha library itself:\n *\n * // To enable runtime choice of fastest implementation.\n * #define T1HA0_RUNTIME_SELECT 1\n *\n * // To disable runtime choice of fastest implementation.\n * #define T1HA0_RUNTIME_SELECT 0\n *\n * When T1HA0_RUNTIME_SELECT is nonzero the t1ha0_resolve() function could\n * be used to get actual t1ha0() implementation address at runtime. This is\n * useful for two cases:\n * - calling by local pointer-to-function usually is little\n * bit faster (less overhead) than via a PLT thru the DSO boundary.\n * - GNU Indirect functions (see below) don't supported by environment\n * and calling by t1ha0_funcptr is not available and/or expensive.\n *\n * 4) T1HA_USE_INDIRECT_FUNCTIONS = Controls usage of GNU Indirect functions.\n *\n * In continue of T1HA0_RUNTIME_SELECT the T1HA_USE_INDIRECT_FUNCTIONS\n * controls usage of ELF indirect functions feature. In general, when\n * available, this reduces overhead of indirect function's calls though\n * a DSO-bundary (https://sourceware.org/glibc/wiki/GNU_IFUNC).\n *\n * By default, t1ha engage GNU Indirect functions when it available\n * and useful, but you could override this default behavior when build\n * t1ha library itself:\n *\n * // To enable use of GNU ELF Indirect functions.\n * #define T1HA_USE_INDIRECT_FUNCTIONS 1\n *\n * // To disable use of GNU ELF Indirect functions. This may be useful\n * // if the actual toolchain or the system's loader don't support ones.\n * #define T1HA_USE_INDIRECT_FUNCTIONS 0\n *\n * 5) T1HA0_AESNI_AVAILABLE = Controls AES-NI detection and dispatching on x86.\n *\n * In continue of T1HA0_RUNTIME_SELECT the T1HA0_AESNI_AVAILABLE controls\n * detection and usage of AES-NI CPU's feature. On the other hand, this\n * requires compiling parts of t1ha library with certain properly options,\n * and could be difficult or inconvenient in some cases.\n *\n * By default, t1ha engade AES-NI for t1ha0() on the x86 platform, but\n * you could override this default behavior when build t1ha library itself:\n *\n * // To disable detection and usage of AES-NI instructions for t1ha0().\n * // This may be useful when you unable to build t1ha library properly\n * // or known that AES-NI will be unavailable at the deploy.\n * #define T1HA0_AESNI_AVAILABLE 0\n *\n * // To force detection and usage of AES-NI instructions for t1ha0(),\n * // but I don't known reasons to anybody would need this.\n * #define T1HA0_AESNI_AVAILABLE 1\n *\n * 6) T1HA0_DISABLED, T1HA1_DISABLED, T1HA2_DISABLED = Controls availability of\n * t1ha functions.\n *\n * In some cases could be useful to import/use only few of t1ha functions\n * or just the one. So, this definitions allows disable corresponding parts\n * of t1ha library.\n *\n * // To disable t1ha0(), t1ha0_32le(), t1ha0_32be() and all AES-NI.\n * #define T1HA0_DISABLED\n *\n * // To disable t1ha1_le() and t1ha1_be().\n * #define T1HA1_DISABLED\n *\n * // To disable t1ha2_atonce(), t1ha2_atonce128() and so on.\n * #define T1HA2_DISABLED\n *\n *****************************************************************************/\n\n#define T1HA_VERSION_MAJOR 2\n#define T1HA_VERSION_MINOR 1\n#define T1HA_VERSION_RELEASE 1\n\n#ifndef __has_attribute\n#define __has_attribute(x) (0)\n#endif\n\n#ifndef __has_include\n#define __has_include(x) (0)\n#endif\n\n#ifndef __GNUC_PREREQ\n#if defined(__GNUC__) && defined(__GNUC_MINOR__)\n#define __GNUC_PREREQ(maj, min) \\\n ((__GNUC__ << 16) + __GNUC_MINOR__ >= ((maj) << 16) + (min))\n#else\n#define __GNUC_PREREQ(maj, min) 0\n#endif\n#endif /* __GNUC_PREREQ */\n\n#ifndef __CLANG_PREREQ\n#ifdef __clang__\n#define __CLANG_PREREQ(maj, min) \\\n ((__clang_major__ << 16) + __clang_minor__ >= ((maj) << 16) + (min))\n#else\n#define __CLANG_PREREQ(maj, min) (0)\n#endif\n#endif /* __CLANG_PREREQ */\n\n#ifndef __LCC_PREREQ\n#ifdef __LCC__\n#define __LCC_PREREQ(maj, min) \\\n ((__LCC__ << 16) + __LCC_MINOR__ >= ((maj) << 16) + (min))\n#else\n#define __LCC_PREREQ(maj, min) (0)\n#endif\n#endif /* __LCC_PREREQ */\n\n/*****************************************************************************/\n\n#ifdef _MSC_VER\n/* Avoid '16' bytes padding added after data member 't1ha_context::total'\n * and other warnings from std-headers if warning-level > 3. */\n#pragma warning(push, 3)\n#endif\n\n#if defined(__cplusplus) && __cplusplus >= 201103L\n#include \n#include \n#include \n#else\n#include \n#include \n#include \n#endif\n\n/*****************************************************************************/\n\n#if defined(i386) || defined(__386) || defined(__i386) || defined(__i386__) || \\\n defined(i486) || defined(__i486) || defined(__i486__) || \\\n defined(i586) | defined(__i586) || defined(__i586__) || defined(i686) || \\\n defined(__i686) || defined(__i686__) || defined(_M_IX86) || \\\n defined(_X86_) || defined(__THW_INTEL__) || defined(__I86__) || \\\n defined(__INTEL__) || defined(__x86_64) || defined(__x86_64__) || \\\n defined(__amd64__) || defined(__amd64) || defined(_M_X64) || \\\n defined(_M_AMD64) || defined(__IA32__) || defined(__INTEL__)\n#ifndef __ia32__\n/* LY: define neutral __ia32__ for x86 and x86-64 archs */\n#define __ia32__ 1\n#endif /* __ia32__ */\n#if !defined(__amd64__) && (defined(__x86_64) || defined(__x86_64__) || \\\n defined(__amd64) || defined(_M_X64))\n/* LY: define trusty __amd64__ for all AMD64/x86-64 arch */\n#define __amd64__ 1\n#endif /* __amd64__ */\n#endif /* all x86 */\n\n#if !defined(__BYTE_ORDER__) || !defined(__ORDER_LITTLE_ENDIAN__) || \\\n !defined(__ORDER_BIG_ENDIAN__)\n\n/* *INDENT-OFF* */\n/* clang-format off */\n\n#if defined(__GLIBC__) || defined(__GNU_LIBRARY__) || defined(__ANDROID__) || \\\n defined(HAVE_ENDIAN_H) || __has_include()\n#include \n#elif defined(__APPLE__) || defined(__MACH__) || defined(__OpenBSD__) || \\\n defined(HAVE_MACHINE_ENDIAN_H) || __has_include()\n#include \n#elif defined(HAVE_SYS_ISA_DEFS_H) || __has_include()\n#include \n#elif (defined(HAVE_SYS_TYPES_H) && defined(HAVE_SYS_ENDIAN_H)) || \\\n (__has_include() && __has_include())\n#include \n#include \n#elif defined(__bsdi__) || defined(__DragonFly__) || defined(__FreeBSD__) || \\\n defined(__NETBSD__) || defined(__NetBSD__) || \\\n defined(HAVE_SYS_PARAM_H) || __has_include()\n#include \n#endif /* OS */\n\n/* *INDENT-ON* */\n/* clang-format on */\n\n#if defined(__BYTE_ORDER) && defined(__LITTLE_ENDIAN) && defined(__BIG_ENDIAN)\n#define __ORDER_LITTLE_ENDIAN__ __LITTLE_ENDIAN\n#define __ORDER_BIG_ENDIAN__ __BIG_ENDIAN\n#define __BYTE_ORDER__ __BYTE_ORDER\n#elif defined(_BYTE_ORDER) && defined(_LITTLE_ENDIAN) && defined(_BIG_ENDIAN)\n#define __ORDER_LITTLE_ENDIAN__ _LITTLE_ENDIAN\n#define __ORDER_BIG_ENDIAN__ _BIG_ENDIAN\n#define __BYTE_ORDER__ _BYTE_ORDER\n#else\n#define __ORDER_LITTLE_ENDIAN__ 1234\n#define __ORDER_BIG_ENDIAN__ 4321\n\n#if defined(__LITTLE_ENDIAN__) || \\\n (defined(_LITTLE_ENDIAN) && !defined(_BIG_ENDIAN)) || \\\n defined(__ARMEL__) || defined(__THUMBEL__) || defined(__AARCH64EL__) || \\\n defined(__MIPSEL__) || defined(_MIPSEL) || defined(__MIPSEL) || \\\n defined(_M_ARM) || defined(_M_ARM64) || defined(__e2k__) || \\\n defined(__elbrus_4c__) || defined(__elbrus_8c__) || defined(__bfin__) || \\\n defined(__BFIN__) || defined(__ia64__) || defined(_IA64) || \\\n defined(__IA64__) || defined(__ia64) || defined(_M_IA64) || \\\n defined(__itanium__) || defined(__ia32__) || defined(__CYGWIN__) || \\\n defined(_WIN64) || defined(_WIN32) || defined(__TOS_WIN__) || \\\n defined(__WINDOWS__)\n#define __BYTE_ORDER__ __ORDER_LITTLE_ENDIAN__\n\n#elif defined(__BIG_ENDIAN__) || \\\n (defined(_BIG_ENDIAN) && !defined(_LITTLE_ENDIAN)) || \\\n defined(__ARMEB__) || defined(__THUMBEB__) || defined(__AARCH64EB__) || \\\n defined(__MIPSEB__) || defined(_MIPSEB) || defined(__MIPSEB) || \\\n defined(__m68k__) || defined(M68000) || defined(__hppa__) || \\\n defined(__hppa) || defined(__HPPA__) || defined(__sparc__) || \\\n defined(__sparc) || defined(__370__) || defined(__THW_370__) || \\\n defined(__s390__) || defined(__s390x__) || defined(__SYSC_ZARCH__)\n#define __BYTE_ORDER__ __ORDER_BIG_ENDIAN__\n\n#else\n#error __BYTE_ORDER__ should be defined.\n#endif /* Arch */\n\n#endif\n#endif /* __BYTE_ORDER__ || __ORDER_LITTLE_ENDIAN__ || __ORDER_BIG_ENDIAN__ */\n\n/*****************************************************************************/\n\n#ifndef __dll_export\n#if defined(_WIN32) || defined(_WIN64) || defined(__CYGWIN__)\n#if defined(__GNUC__) || __has_attribute(dllexport)\n#define __dll_export __attribute__((dllexport))\n#else\n#define __dll_export __declspec(dllexport)\n#endif\n#elif defined(__GNUC__) || __has_attribute(__visibility__)\n#define __dll_export __attribute__((__visibility__(\"default\")))\n#else\n#define __dll_export\n#endif\n#endif /* __dll_export */\n\n#ifndef __dll_import\n#if defined(_WIN32) || defined(_WIN64) || defined(__CYGWIN__)\n#if defined(__GNUC__) || __has_attribute(dllimport)\n#define __dll_import __attribute__((dllimport))\n#else\n#define __dll_import __declspec(dllimport)\n#endif\n#elif defined(__GNUC__) || __has_attribute(__visibility__)\n#define __dll_import __attribute__((__visibility__(\"default\")))\n#else\n#define __dll_import\n#endif\n#endif /* __dll_import */\n\n#ifndef __force_inline\n#ifdef _MSC_VER\n#define __force_inline __forceinline\n#elif __GNUC_PREREQ(3, 2) || __has_attribute(__always_inline__)\n#define __force_inline __inline __attribute__((__always_inline__))\n#else\n#define __force_inline __inline\n#endif\n#endif /* __force_inline */\n\n#ifndef T1HA_API\n#if defined(t1ha_EXPORTS)\n#define T1HA_API __dll_export\n#elif defined(t1ha_IMPORTS)\n#define T1HA_API __dll_import\n#else\n#define T1HA_API\n#endif\n#endif /* T1HA_API */\n\n#if defined(_MSC_VER) && defined(__ia32__)\n#define T1HA_ALIGN_PREFIX __declspec(align(32)) /* required only for SIMD */\n#else\n#define T1HA_ALIGN_PREFIX\n#endif /* _MSC_VER */\n\n#if defined(__GNUC__) && defined(__ia32__)\n#define T1HA_ALIGN_SUFFIX \\\n __attribute__((__aligned__(32))) /* required only for SIMD */\n#else\n#define T1HA_ALIGN_SUFFIX\n#endif /* GCC x86 */\n\n#ifndef T1HA_USE_INDIRECT_FUNCTIONS\n/* GNU ELF indirect functions usage control. For more info please see\n * https://en.wikipedia.org/wiki/Executable_and_Linkable_Format\n * and https://sourceware.org/glibc/wiki/GNU_IFUNC */\n#if defined(__ELF__) && defined(__amd64__) && \\\n (__has_attribute(__ifunc__) || \\\n (!defined(__clang__) && defined(__GNUC__) && __GNUC__ >= 4 && \\\n !defined(__SANITIZE_ADDRESS__) && !defined(__SSP_ALL__)))\n/* Enable gnu_indirect_function by default if :\n * - ELF AND x86_64\n * - attribute(__ifunc__) is available OR\n * GCC >= 4 WITHOUT -fsanitize=address NOR -fstack-protector-all */\n#define T1HA_USE_INDIRECT_FUNCTIONS 1\n#else\n#define T1HA_USE_INDIRECT_FUNCTIONS 0\n#endif\n#endif /* T1HA_USE_INDIRECT_FUNCTIONS */\n\n#if __GNUC_PREREQ(4, 0)\n#pragma GCC visibility push(hidden)\n#endif /* __GNUC_PREREQ(4,0) */\n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\ntypedef union T1HA_ALIGN_PREFIX t1ha_state256 {\n uint8_t bytes[32];\n uint32_t u32[8];\n uint64_t u64[4];\n struct {\n uint64_t a, b, c, d;\n } n;\n} t1ha_state256_t T1HA_ALIGN_SUFFIX;\n\ntypedef struct t1ha_context {\n t1ha_state256_t state;\n t1ha_state256_t buffer;\n size_t partial;\n uint64_t total;\n} t1ha_context_t;\n\n#ifdef _MSC_VER\n#pragma warning(pop)\n#endif\n\n/******************************************************************************\n *\n * Self-testing API.\n *\n * Unfortunately, some compilers (exactly only Microsoft Visual C/C++) has\n * a bugs which leads t1ha-functions to produce wrong results. This API allows\n * check the correctness of the actual code in runtime.\n *\n * All check-functions returns 0 on success, or -1 in case the corresponding\n * hash-function failed verification. PLEASE, always perform such checking at\n * initialization of your code, if you using MSVC or other troubleful compilers.\n */\n\nT1HA_API int t1ha_selfcheck__all_enabled(void);\n\n#ifndef T1HA2_DISABLED\nT1HA_API int t1ha_selfcheck__t1ha2_atonce(void);\nT1HA_API int t1ha_selfcheck__t1ha2_atonce128(void);\nT1HA_API int t1ha_selfcheck__t1ha2_stream(void);\nT1HA_API int t1ha_selfcheck__t1ha2(void);\n#endif /* T1HA2_DISABLED */\n\n#ifndef T1HA1_DISABLED\nT1HA_API int t1ha_selfcheck__t1ha1_le(void);\nT1HA_API int t1ha_selfcheck__t1ha1_be(void);\nT1HA_API int t1ha_selfcheck__t1ha1(void);\n#endif /* T1HA1_DISABLED */\n\n#ifndef T1HA0_DISABLED\nT1HA_API int t1ha_selfcheck__t1ha0_32le(void);\nT1HA_API int t1ha_selfcheck__t1ha0_32be(void);\nT1HA_API int t1ha_selfcheck__t1ha0(void);\n\n/* Define T1HA0_AESNI_AVAILABLE to 0 for disable AES-NI support. */\n#ifndef T1HA0_AESNI_AVAILABLE\n#if defined(__e2k__) || \\\n (defined(__ia32__) && (!defined(_M_IX86) || _MSC_VER > 1800))\n#define T1HA0_AESNI_AVAILABLE 1\n#else\n#define T1HA0_AESNI_AVAILABLE 0\n#endif\n#endif /* ifndef T1HA0_AESNI_AVAILABLE */\n\n#if T1HA0_AESNI_AVAILABLE\nT1HA_API int t1ha_selfcheck__t1ha0_ia32aes_noavx(void);\nT1HA_API int t1ha_selfcheck__t1ha0_ia32aes_avx(void);\n#ifndef __e2k__\nT1HA_API int t1ha_selfcheck__t1ha0_ia32aes_avx2(void);\n#endif\n#endif /* if T1HA0_AESNI_AVAILABLE */\n#endif /* T1HA0_DISABLED */\n\n/******************************************************************************\n *\n * t1ha2 = 64 and 128-bit, SLIGHTLY MORE ATTENTION FOR QUALITY AND STRENGTH.\n *\n * - The recommended version of \"Fast Positive Hash\" with good quality\n * for checksum, hash tables and fingerprinting.\n * - Portable and extremely efficiency on modern 64-bit CPUs.\n * Designed for 64-bit little-endian platforms,\n * in other cases will runs slowly.\n * - Great quality of hashing and still faster than other non-t1ha hashes.\n * Provides streaming mode and 128-bit result.\n *\n * Note: Due performance reason 64- and 128-bit results are completely\n * different each other, i.e. 64-bit result is NOT any part of 128-bit.\n */\n#ifndef T1HA2_DISABLED\n\n/* The at-once variant with 64-bit result */\nT1HA_API uint64_t t1ha2_atonce(const void *data, size_t length, uint64_t seed);\n\n/* The at-once variant with 128-bit result.\n * Argument `extra_result` is NOT optional and MUST be valid.\n * The high 64-bit part of 128-bit hash will be always unconditionally\n * stored to the address given by `extra_result` argument. */\nT1HA_API uint64_t t1ha2_atonce128(uint64_t *__restrict extra_result,\n const void *__restrict data, size_t length,\n uint64_t seed);\n\n/* The init/update/final trinity for streaming.\n * Return 64 or 128-bit result depentently from `extra_result` argument. */\nT1HA_API void t1ha2_init(t1ha_context_t *ctx, uint64_t seed_x, uint64_t seed_y);\nT1HA_API void t1ha2_update(t1ha_context_t *__restrict ctx,\n const void *__restrict data, size_t length);\n\n/* Argument `extra_result` is optional and MAY be NULL.\n * - If `extra_result` is NOT NULL then the 128-bit hash will be calculated,\n * and high 64-bit part of it will be stored to the address given\n * by `extra_result` argument.\n * - Otherwise the 64-bit hash will be calculated\n * and returned from function directly.\n *\n * Note: Due performance reason 64- and 128-bit results are completely\n * different each other, i.e. 64-bit result is NOT any part of 128-bit. */\nT1HA_API uint64_t t1ha2_final(t1ha_context_t *__restrict ctx,\n uint64_t *__restrict extra_result /* optional */);\n\n#endif /* T1HA2_DISABLED */\n\n/******************************************************************************\n *\n * t1ha1 = 64-bit, BASELINE FAST PORTABLE HASH:\n *\n * - Runs faster on 64-bit platforms in other cases may runs slowly.\n * - Portable and stable, returns same 64-bit result\n * on all architectures and CPUs.\n * - Unfortunately it fails the \"strict avalanche criteria\",\n * see test results at https://github.com/demerphq/smhasher.\n *\n * This flaw is insignificant for the t1ha1() purposes and imperceptible\n * from a practical point of view.\n * However, nowadays this issue has resolved in the next t1ha2(),\n * that was initially planned to providing a bit more quality.\n */\n#ifndef T1HA1_DISABLED\n\n/* The little-endian variant. */\nT1HA_API uint64_t t1ha1_le(const void *data, size_t length, uint64_t seed);\n\n/* The big-endian variant. */\nT1HA_API uint64_t t1ha1_be(const void *data, size_t length, uint64_t seed);\n\n#endif /* T1HA1_DISABLED */\n\n/******************************************************************************\n *\n * t1ha0 = 64-bit, JUST ONLY FASTER:\n *\n * - Provides fast-as-possible hashing for current CPU, including\n * 32-bit systems and engaging the available hardware acceleration.\n * - It is a facade that selects most quick-and-dirty hash\n * for the current processor. For instance, on IA32 (x86) actual function\n * will be selected in runtime, depending on current CPU capabilities\n *\n * BE CAREFUL!!! THIS IS MEANS:\n *\n * 1. The quality of hash is a subject for tradeoffs with performance.\n * So, the quality and strength of t1ha0() may be lower than t1ha1(),\n * especially on 32-bit targets, but then much faster.\n * However, guaranteed that it passes all SMHasher tests.\n *\n * 2. No warranty that the hash result will be same for particular\n * key on another machine or another version of libt1ha.\n *\n * Briefly, such hash-results and their derivatives, should be\n * used only in runtime, but should not be persist or transferred\n * over a network.\n *\n *\n * When T1HA0_RUNTIME_SELECT is nonzero the t1ha0_resolve() function could\n * be used to get actual t1ha0() implementation address at runtime. This is\n * useful for two cases:\n * - calling by local pointer-to-function usually is little\n * bit faster (less overhead) than via a PLT thru the DSO boundary.\n * - GNU Indirect functions (see below) don't supported by environment\n * and calling by t1ha0_funcptr is not available and/or expensive.\n */\n\n#ifndef T1HA0_DISABLED\n\n/* The little-endian variant for 32-bit CPU. */\nuint64_t t1ha0_32le(const void *data, size_t length, uint64_t seed);\n/* The big-endian variant for 32-bit CPU. */\nuint64_t t1ha0_32be(const void *data, size_t length, uint64_t seed);\n\n/* Define T1HA0_AESNI_AVAILABLE to 0 for disable AES-NI support. */\n#ifndef T1HA0_AESNI_AVAILABLE\n#if defined(__e2k__) || \\\n (defined(__ia32__) && (!defined(_M_IX86) || _MSC_VER > 1800))\n#define T1HA0_AESNI_AVAILABLE 1\n#else\n#define T1HA0_AESNI_AVAILABLE 0\n#endif\n#endif /* T1HA0_AESNI_AVAILABLE */\n\n/* Define T1HA0_RUNTIME_SELECT to 0 for disable dispatching t1ha0 at runtime. */\n#ifndef T1HA0_RUNTIME_SELECT\n#if T1HA0_AESNI_AVAILABLE && !defined(__e2k__)\n#define T1HA0_RUNTIME_SELECT 1\n#else\n#define T1HA0_RUNTIME_SELECT 0\n#endif\n#endif /* T1HA0_RUNTIME_SELECT */\n\n#if !T1HA0_RUNTIME_SELECT && !defined(T1HA0_USE_DEFINE)\n#if defined(__LCC__)\n#define T1HA0_USE_DEFINE 1\n#else\n#define T1HA0_USE_DEFINE 0\n#endif\n#endif /* T1HA0_USE_DEFINE */\n\n#if T1HA0_AESNI_AVAILABLE\nuint64_t t1ha0_ia32aes_noavx(const void *data, size_t length, uint64_t seed);\nuint64_t t1ha0_ia32aes_avx(const void *data, size_t length, uint64_t seed);\n#ifndef __e2k__\nuint64_t t1ha0_ia32aes_avx2(const void *data, size_t length, uint64_t seed);\n#endif\n#endif /* T1HA0_AESNI_AVAILABLE */\n\n#if T1HA0_RUNTIME_SELECT\ntypedef uint64_t (*t1ha0_function_t)(const void *, size_t, uint64_t);\nT1HA_API t1ha0_function_t t1ha0_resolve(void);\n#if T1HA_USE_INDIRECT_FUNCTIONS\nT1HA_API uint64_t t1ha0(const void *data, size_t length, uint64_t seed);\n#else\n/* Otherwise function pointer will be used.\n * Unfortunately this may cause some overhead calling. */\nT1HA_API extern uint64_t (*t1ha0_funcptr)(const void *data, size_t length,\n uint64_t seed);\nstatic __force_inline uint64_t t1ha0(const void *data, size_t length,\n uint64_t seed) {\n return t1ha0_funcptr(data, length, seed);\n}\n#endif /* T1HA_USE_INDIRECT_FUNCTIONS */\n\n#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n\n#if T1HA0_USE_DEFINE\n\n#if (UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul) && \\\n (!defined(T1HA1_DISABLED) || !defined(T1HA2_DISABLED))\n#if defined(T1HA1_DISABLED)\n#define t1ha0 t1ha2_atonce\n#else\n#define t1ha0 t1ha1_be\n#endif /* T1HA1_DISABLED */\n#else /* 32/64 */\n#define t1ha0 t1ha0_32be\n#endif /* 32/64 */\n\n#else /* T1HA0_USE_DEFINE */\n\nstatic __force_inline uint64_t t1ha0(const void *data, size_t length,\n uint64_t seed) {\n#if (UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul) && \\\n (!defined(T1HA1_DISABLED) || !defined(T1HA2_DISABLED))\n#if defined(T1HA1_DISABLED)\n return t1ha2_atonce(data, length, seed);\n#else\n return t1ha1_be(data, length, seed);\n#endif /* T1HA1_DISABLED */\n#else /* 32/64 */\n return t1ha0_32be(data, length, seed);\n#endif /* 32/64 */\n}\n\n#endif /* !T1HA0_USE_DEFINE */\n\n#else /* !T1HA0_RUNTIME_SELECT && __BYTE_ORDER__ != __ORDER_BIG_ENDIAN__ */\n\n#if T1HA0_USE_DEFINE\n\n#if (UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul) && \\\n (!defined(T1HA1_DISABLED) || !defined(T1HA2_DISABLED))\n#if defined(T1HA1_DISABLED)\n#define t1ha0 t1ha2_atonce\n#else\n#define t1ha0 t1ha1_le\n#endif /* T1HA1_DISABLED */\n#else /* 32/64 */\n#define t1ha0 t1ha0_32le\n#endif /* 32/64 */\n\n#else\n\nstatic __force_inline uint64_t t1ha0(const void *data, size_t length,\n uint64_t seed) {\n#if (UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul) && \\\n (!defined(T1HA1_DISABLED) || !defined(T1HA2_DISABLED))\n#if defined(T1HA1_DISABLED)\n return t1ha2_atonce(data, length, seed);\n#else\n return t1ha1_le(data, length, seed);\n#endif /* T1HA1_DISABLED */\n#else /* 32/64 */\n return t1ha0_32le(data, length, seed);\n#endif /* 32/64 */\n}\n\n#endif /* !T1HA0_USE_DEFINE */\n\n#endif /* !T1HA0_RUNTIME_SELECT */\n\n#endif /* T1HA0_DISABLED */\n\n#ifdef __cplusplus\n}\n#endif\n\n#if __GNUC_PREREQ(4, 0)\n#pragma GCC visibility pop\n#endif /* __GNUC_PREREQ(4,0) */\n" }, { "path": "benchmarks/ustd.h", "content": "/* ustd.h common macros and includes */\n#ifndef LIBRHASH_USTD_H\n#define LIBRHASH_USTD_H\n\n#if _MSC_VER >= 1300\n\n# define int64_t __int64\n# define int32_t __int32\n# define int16_t __int16\n# define int8_t __int8\n# define uint64_t unsigned __int64\n# define uint32_t unsigned __int32\n# define uint16_t unsigned __int16\n# define uint8_t unsigned __int8\n\n/* disable warnings: The POSIX name for this item is deprecated. Use the ISO C++ conformant name. */\n#pragma warning(disable : 4996)\n\n#else /* _MSC_VER >= 1300 */\n\n# include \n# include \n\n#endif /* _MSC_VER >= 1300 */\n\n#if _MSC_VER <= 1300\n# include /* size_t for vc6.0 */\n#endif /* _MSC_VER <= 1300 */\n\n#endif /* LIBRHASH_USTD_H */\n" }, { "path": "benchmarks/xoroshiro128plus.c", "content": "/* Written in 2016 by David Blackman and Sebastiano Vigna (vigna@acm.org)\n\nTo the extent possible under law, the author has dedicated all copyright\nand related and neighboring rights to this software to the public domain\nworldwide. This software is distributed without any warranty.\n\nSee . */\n\n#include \n\n/* This is the successor to xorshift128+. It is the fastest full-period\n generator passing BigCrush without systematic failures, but due to the\n relatively short period it is acceptable only for applications with a\n mild amount of parallelism; otherwise, use a xorshift1024* generator.\n\n Beside passing BigCrush, this generator passes the PractRand test suite\n up to (and included) 16TB, with the exception of binary rank tests,\n which fail due to the lowest bit being an LFSR; all other bits pass all\n tests. We suggest to use a sign test to extract a random Boolean value.\n \n Note that the generator uses a simulated rotate operation, which most C\n compilers will turn into a single instruction. In Java, you can use\n Long.rotateLeft(). In languages that do not make low-level rotation\n instructions accessible xorshift128+ could be faster.\n\n The state must be seeded so that it is not everywhere zero. If you have\n a 64-bit seed, we suggest to seed a splitmix64 generator and use its\n output to fill s. */\n\nuint64_t s[2];\n\nstatic inline uint64_t rotl(const uint64_t x, int k) {\n\treturn (x << k) | (x >> (64 - k));\n}\n\nuint64_t next(void) {\n\tconst uint64_t s0 = s[0];\n\tuint64_t s1 = s[1];\n\tconst uint64_t result = s0 + s1;\n\n\ts1 ^= s0;\n\ts[0] = rotl(s0, 55) ^ s1 ^ (s1 << 14); // a, b\n\ts[1] = rotl(s1, 36); // c\n\n\treturn result;\n}\n\n\n/* This is the jump function for the generator. It is equivalent\n to 2^64 calls to next(); it can be used to generate 2^64\n non-overlapping subsequences for parallel computations. */\n\nvoid jump(void) {\n\tstatic const uint64_t JUMP[] = { 0xbeac0467eba5facb, 0xd86b048b86aa9922 };\n\n\tuint64_t s0 = 0;\n\tuint64_t s1 = 0;\n\tfor(int i = 0; i < sizeof JUMP / sizeof *JUMP; i++)\n\t\tfor(int b = 0; b < 64; b++) {\n\t\t\tif (JUMP[i] & 1ULL << b) {\n\t\t\t\ts0 ^= s[0];\n\t\t\t\ts1 ^= s[1];\n\t\t\t}\n\t\t\tnext();\n\t\t}\n\n\ts[0] = s0;\n\ts[1] = s1;\n}\n" }, { "path": "benchmarks/xoroshiro128starstar.c", "content": "/* Written in 2018 by David Blackman and Sebastiano Vigna (vigna@acm.org)\n\nTo the extent possible under law, the author has dedicated all copyright\nand related and neighboring rights to this software to the public domain\nworldwide. This software is distributed without any warranty.\n\nSee . */\n\n#include \n\n/* This is xoroshiro128** 1.0, our all-purpose, rock-solid, small-state\n generator. It is extremely (sub-ns) fast and it passes all tests we are\n aware of, but its state space is large enough only for mild parallelism.\n\n For generating just floating-point numbers, xoroshiro128+ is even\n faster (but it has a very mild bias, see notes in the comments).\n\n The state must be seeded so that it is not everywhere zero. If you have\n a 64-bit seed, we suggest to seed a splitmix64 generator and use its\n output to fill s. */\n\n\nstatic inline uint64_t rotl(const uint64_t x, int k) {\n\treturn (x << k) | (x >> (64 - k));\n}\n\n\nstatic uint64_t s[2];\n\nuint64_t next(void) {\n\tconst uint64_t s0 = s[0];\n\tuint64_t s1 = s[1];\n\tconst uint64_t result = rotl(s0 * 5, 7) * 9;\n\n\ts1 ^= s0;\n\ts[0] = rotl(s0, 24) ^ s1 ^ (s1 << 16); // a, b\n\ts[1] = rotl(s1, 37); // c\n\n\treturn result;\n}\n\n\n/* This is the jump function for the generator. It is equivalent\n to 2^64 calls to next(); it can be used to generate 2^64\n non-overlapping subsequences for parallel computations. */\n\nvoid jump(void) {\n\tstatic const uint64_t JUMP[] = { 0xdf900294d8f554a5, 0x170865df4b3201fc };\n\n\tuint64_t s0 = 0;\n\tuint64_t s1 = 0;\n\tfor(int i = 0; i < sizeof JUMP / sizeof *JUMP; i++)\n\t\tfor(int b = 0; b < 64; b++) {\n\t\t\tif (JUMP[i] & UINT64_C(1) << b) {\n\t\t\t\ts0 ^= s[0];\n\t\t\t\ts1 ^= s[1];\n\t\t\t}\n\t\t\tnext();\n\t\t}\n\n\ts[0] = s0;\n\ts[1] = s1;\n}\n\n\n/* This is the long-jump function for the generator. It is equivalent to\n 2^96 calls to next(); it can be used to generate 2^32 starting points,\n from each of which jump() will generate 2^32 non-overlapping\n subsequences for parallel distributed computations. */\n\nvoid long_jump(void) {\n\tstatic const uint64_t LONG_JUMP[] = { 0xd2a98b26625eee7b, 0xdddf9b1090aa7ac1 };\n\n\tuint64_t s0 = 0;\n\tuint64_t s1 = 0;\n\tfor(int i = 0; i < sizeof LONG_JUMP / sizeof *LONG_JUMP; i++)\n\t\tfor(int b = 0; b < 64; b++) {\n\t\t\tif (LONG_JUMP[i] & UINT64_C(1) << b) {\n\t\t\t\ts0 ^= s[0];\n\t\t\t\ts1 ^= s[1];\n\t\t\t}\n\t\t\tnext();\n\t\t}\n\n\ts[0] = s0;\n\ts[1] = s1;\n}\n" }, { "path": "benchmarks/xoseed.c", "content": "#include \n#include \n#include \"splitmix64.c\"\n\nvoid main (int argc, char *argv[]) {\n int n = atoi(argv[1]);\n for (int i = 0; i < n; i++)\n printf (\" s[%d] = 0x%lx;\", i, next());\n printf (\"\\n\");\n}\n" }, { "path": "benchmarks/xoshiro256plus.c", "content": "/* Written in 2018 by David Blackman and Sebastiano Vigna (vigna@acm.org)\n\nTo the extent possible under law, the author has dedicated all copyright\nand related and neighboring rights to this software to the public domain\nworldwide. This software is distributed without any warranty.\n\nSee . */\n\n#include \n\n/* This is xoshiro256+ 1.0, our best and fastest generator for floating-point\n numbers. We suggest to use its upper bits for floating-point\n generation, as it is slightly faster than xoshiro256**. It passes all\n tests we are aware of except for the lowest three bits, which might\n fail linearity tests (and just those), so if low linear complexity is\n not considered an issue (as it is usually the case) it can be used to\n generate 64-bit outputs, too.\n\n We suggest to use a sign test to extract a random Boolean value, and\n right shifts to extract subsets of bits.\n\n The state must be seeded so that it is not everywhere zero. If you have\n a 64-bit seed, we suggest to seed a splitmix64 generator and use its\n output to fill s. */\n\n\nstatic inline uint64_t rotl(const uint64_t x, int k) {\n\treturn (x << k) | (x >> (64 - k));\n}\n\n\nstatic uint64_t s[4];\n\nuint64_t next(void) {\n\tconst uint64_t result_plus = s[0] + s[3];\n\n\tconst uint64_t t = s[1] << 17;\n\n\ts[2] ^= s[0];\n\ts[3] ^= s[1];\n\ts[1] ^= s[2];\n\ts[0] ^= s[3];\n\n\ts[2] ^= t;\n\n\ts[3] = rotl(s[3], 45);\n\n\treturn result_plus;\n}\n\n\n/* This is the jump function for the generator. It is equivalent\n to 2^128 calls to next(); it can be used to generate 2^128\n non-overlapping subsequences for parallel computations. */\n\nvoid jump(void) {\n\tstatic const uint64_t JUMP[] = { 0x180ec6d33cfd0aba, 0xd5a61266f0c9392c, 0xa9582618e03fc9aa, 0x39abdc4529b1661c };\n\n\tuint64_t s0 = 0;\n\tuint64_t s1 = 0;\n\tuint64_t s2 = 0;\n\tuint64_t s3 = 0;\n\tfor(int i = 0; i < sizeof JUMP / sizeof *JUMP; i++)\n\t\tfor(int b = 0; b < 64; b++) {\n\t\t\tif (JUMP[i] & UINT64_C(1) << b) {\n\t\t\t\ts0 ^= s[0];\n\t\t\t\ts1 ^= s[1];\n\t\t\t\ts2 ^= s[2];\n\t\t\t\ts3 ^= s[3];\n\t\t\t}\n\t\t\tnext();\t\n\t\t}\n\t\t\n\ts[0] = s0;\n\ts[1] = s1;\n\ts[2] = s2;\n\ts[3] = s3;\n}\n\n\n/* This is the long-jump function for the generator. It is equivalent to\n 2^192 calls to next(); it can be used to generate 2^64 starting points,\n from each of which jump() will generate 2^64 non-overlapping\n subsequences for parallel distributed computations. */\n\nvoid long_jump(void) {\n\tstatic const uint64_t LONG_JUMP[] = { 0x76e15d3efefdcbbf, 0xc5004e441c522fb3, 0x77710069854ee241, 0x39109bb02acbe635 };\n\n\tuint64_t s0 = 0;\n\tuint64_t s1 = 0;\n\tuint64_t s2 = 0;\n\tuint64_t s3 = 0;\n\tfor(int i = 0; i < sizeof LONG_JUMP / sizeof *LONG_JUMP; i++)\n\t\tfor(int b = 0; b < 64; b++) {\n\t\t\tif (LONG_JUMP[i] & UINT64_C(1) << b) {\n\t\t\t\ts0 ^= s[0];\n\t\t\t\ts1 ^= s[1];\n\t\t\t\ts2 ^= s[2];\n\t\t\t\ts3 ^= s[3];\n\t\t\t}\n\t\t\tnext();\t\n\t\t}\n\t\t\n\ts[0] = s0;\n\ts[1] = s1;\n\ts[2] = s2;\n\ts[3] = s3;\n}\n" }, { "path": "benchmarks/xoshiro256starstar.c", "content": "/* Written in 2018 by David Blackman and Sebastiano Vigna (vigna@acm.org)\n\nTo the extent possible under law, the author has dedicated all copyright\nand related and neighboring rights to this software to the public domain\nworldwide. This software is distributed without any warranty.\n\nSee . */\n\n#include \n\n/* This is xoshiro256** 1.0, our all-purpose, rock-solid generator. It has\n excellent (sub-ns) speed, a state (256 bits) that is large enough for\n any parallel application, and it passes all tests we are aware of.\n\n For generating just floating-point numbers, xoshiro256+ is even faster.\n\n The state must be seeded so that it is not everywhere zero. If you have\n a 64-bit seed, we suggest to seed a splitmix64 generator and use its\n output to fill s. */\n\nstatic inline uint64_t rotl(const uint64_t x, int k) {\n\treturn (x << k) | (x >> (64 - k));\n}\n\n\nstatic uint64_t s[4];\n\nuint64_t next(void) {\n\tconst uint64_t result_starstar = rotl(s[1] * 5, 7) * 9;\n\n\tconst uint64_t t = s[1] << 17;\n\n\ts[2] ^= s[0];\n\ts[3] ^= s[1];\n\ts[1] ^= s[2];\n\ts[0] ^= s[3];\n\n\ts[2] ^= t;\n\n\ts[3] = rotl(s[3], 45);\n\n\treturn result_starstar;\n}\n\n\n/* This is the jump function for the generator. It is equivalent\n to 2^128 calls to next(); it can be used to generate 2^128\n non-overlapping subsequences for parallel computations. */\n\nvoid jump(void) {\n\tstatic const uint64_t JUMP[] = { 0x180ec6d33cfd0aba, 0xd5a61266f0c9392c, 0xa9582618e03fc9aa, 0x39abdc4529b1661c };\n\n\tuint64_t s0 = 0;\n\tuint64_t s1 = 0;\n\tuint64_t s2 = 0;\n\tuint64_t s3 = 0;\n\tfor(int i = 0; i < sizeof JUMP / sizeof *JUMP; i++)\n\t\tfor(int b = 0; b < 64; b++) {\n\t\t\tif (JUMP[i] & UINT64_C(1) << b) {\n\t\t\t\ts0 ^= s[0];\n\t\t\t\ts1 ^= s[1];\n\t\t\t\ts2 ^= s[2];\n\t\t\t\ts3 ^= s[3];\n\t\t\t}\n\t\t\tnext();\t\n\t\t}\n\t\t\n\ts[0] = s0;\n\ts[1] = s1;\n\ts[2] = s2;\n\ts[3] = s3;\n}\n\n\n\n/* This is the long-jump function for the generator. It is equivalent to\n 2^192 calls to next(); it can be used to generate 2^64 starting points,\n from each of which jump() will generate 2^64 non-overlapping\n subsequences for parallel distributed computations. */\n\nvoid long_jump(void) {\n\tstatic const uint64_t LONG_JUMP[] = { 0x76e15d3efefdcbbf, 0xc5004e441c522fb3, 0x77710069854ee241, 0x39109bb02acbe635 };\n\n\tuint64_t s0 = 0;\n\tuint64_t s1 = 0;\n\tuint64_t s2 = 0;\n\tuint64_t s3 = 0;\n\tfor(int i = 0; i < sizeof LONG_JUMP / sizeof *LONG_JUMP; i++)\n\t\tfor(int b = 0; b < 64; b++) {\n\t\t\tif (LONG_JUMP[i] & UINT64_C(1) << b) {\n\t\t\t\ts0 ^= s[0];\n\t\t\t\ts1 ^= s[1];\n\t\t\t\ts2 ^= s[2];\n\t\t\t\ts3 ^= s[3];\n\t\t\t}\n\t\t\tnext();\t\n\t\t}\n\t\t\n\ts[0] = s0;\n\ts[1] = s1;\n\ts[2] = s2;\n\ts[3] = s3;\n}\n" }, { "path": "benchmarks/xoshiro512plus.c", "content": "/* Written in 2018 by David Blackman and Sebastiano Vigna (vigna@acm.org)\n\nTo the extent possible under law, the author has dedicated all copyright\nand related and neighboring rights to this software to the public domain\nworldwide. This software is distributed without any warranty.\n\nSee . */\n\n#include \n#include \n\n/* This is xoshiro512+ 1.0, our generator for floating-point numbers with\n increased state size. We suggest to use its upper bits for\n floating-point generation, as it is slightly faster than xoshiro512**.\n It passes all tests we are aware of except for the lowest three bits,\n which might fail linearity tests (and just those), so if low linear\n complexity is not considered an issue (as it is usually the case) it\n can be used to generate 64-bit outputs, too.\n\n We suggest to use a sign test to extract a random Boolean value, and\n right shifts to extract subsets of bits.\n\n The state must be seeded so that it is not everywhere zero. If you have\n a 64-bit seed, we suggest to seed a splitmix64 generator and use its\n output to fill s. */\n\nstatic inline uint64_t rotl(const uint64_t x, int k) {\n\treturn (x << k) | (x >> (64 - k));\n}\n\n\nstatic uint64_t s[8];\n\nuint64_t next(void) {\n\tconst uint64_t result_plus = s[0] + s[2];\n\n\tconst uint64_t t = s[1] << 11;\n\n\ts[2] ^= s[0];\n\ts[5] ^= s[1];\n\ts[1] ^= s[2];\n\ts[7] ^= s[3];\n\ts[3] ^= s[4];\n\ts[4] ^= s[5];\n\ts[0] ^= s[6];\n\ts[6] ^= s[7];\n\n\ts[6] ^= t;\n\n\ts[7] = rotl(s[7], 21);\n\n\treturn result_plus;\n}\n\n\n/* This is the jump function for the generator. It is equivalent\n to 2^256 calls to next(); it can be used to generate 2^256\n non-overlapping subsequences for parallel computations. */\n\nvoid jump(void) {\n\tstatic const uint64_t JUMP[] = { 0x33ed89b6e7a353f9, 0x760083d7955323be, 0x2837f2fbb5f22fae, 0x4b8c5674d309511c, 0xb11ac47a7ba28c25, 0xf1be7667092bcc1c, 0x53851efdb6df0aaf, 0x1ebbc8b23eaf25db };\n\n\tuint64_t t[sizeof s / sizeof *s];\n\tmemset(t, 0, sizeof t);\n\tfor(int i = 0; i < sizeof JUMP / sizeof *JUMP; i++)\n\t\tfor(int b = 0; b < 64; b++) {\n\t\t\tif (JUMP[i] & UINT64_C(1) << b)\n\t\t\t\tfor(int w = 0; w < sizeof s / sizeof *s; w++)\n\t\t\t\t\tt[w] ^= s[w];\n\t\t\tnext();\n\t\t}\n\t\n\tmemcpy(s, t, sizeof s);\t\n}\n" }, { "path": "benchmarks/xoshiro512starstar.c", "content": "/* Written in 2018 by David Blackman and Sebastiano Vigna (vigna@acm.org)\n\nTo the extent possible under law, the author has dedicated all copyright\nand related and neighboring rights to this software to the public domain\nworldwide. This software is distributed without any warranty.\n\nSee . */\n\n#include \n#include \n\n/* This is xoshiro512** 1.0, an all-purpose, rock-solid generator. It has\n excellent (about 1ns) speed, an increased state (512 bits) that is\n large enough for any parallel application, and it passes all tests we\n are aware of.\n\n For generating just floating-point numbers, xoshiro512+ is even faster.\n\n The state must be seeded so that it is not everywhere zero. If you have\n a 64-bit seed, we suggest to seed a splitmix64 generator and use its\n output to fill s. */\n\nstatic inline uint64_t rotl(const uint64_t x, int k) {\n\treturn (x << k) | (x >> (64 - k));\n}\n\n\nstatic uint64_t s[8];\n\nuint64_t next(void) {\n\tconst uint64_t result_starstar = rotl(s[1] * 5, 7) * 9;\n\n\tconst uint64_t t = s[1] << 11;\n\n\ts[2] ^= s[0];\n\ts[5] ^= s[1];\n\ts[1] ^= s[2];\n\ts[7] ^= s[3];\n\ts[3] ^= s[4];\n\ts[4] ^= s[5];\n\ts[0] ^= s[6];\n\ts[6] ^= s[7];\n\n\ts[6] ^= t;\n\n\ts[7] = rotl(s[7], 21);\n\n\treturn result_starstar;\n}\n\n\n/* This is the jump function for the generator. It is equivalent\n to 2^256 calls to next(); it can be used to generate 2^256\n non-overlapping subsequences for parallel computations. */\n\nvoid jump(void) {\n\tstatic const uint64_t JUMP[] = { 0x33ed89b6e7a353f9, 0x760083d7955323be, 0x2837f2fbb5f22fae, 0x4b8c5674d309511c, 0xb11ac47a7ba28c25, 0xf1be7667092bcc1c, 0x53851efdb6df0aaf, 0x1ebbc8b23eaf25db };\n\n\tuint64_t t[sizeof s / sizeof *s];\n\tmemset(t, 0, sizeof t);\n\tfor(int i = 0; i < sizeof JUMP / sizeof *JUMP; i++)\n\t\tfor(int b = 0; b < 64; b++) {\n\t\t\tif (JUMP[i] & UINT64_C(1) << b)\n\t\t\t\tfor(int w = 0; w < sizeof s / sizeof *s; w++)\n\t\t\t\t\tt[w] ^= s[w];\n\t\t\tnext();\n\t\t}\n\t\n\tmemcpy(s, t, sizeof s);\t\n}\n" }, { "path": "benchmarks/xxh3.h", "content": "/*\n * xxHash - Extremely Fast Hash algorithm\n * Development source file for `xxh3`\n * Copyright (C) 2019-2021 Yann Collet\n *\n * BSD 2-Clause License (https://www.opensource.org/licenses/bsd-license.php)\n *\n * Redistribution and use in source and binary forms, with or without\n * modification, are permitted provided that the following conditions are\n * met:\n *\n * * Redistributions of source code must retain the above copyright\n * notice, this list of conditions and the following disclaimer.\n * * Redistributions in binary form must reproduce the above\n * copyright notice, this list of conditions and the following disclaimer\n * in the documentation and/or other materials provided with the\n * distribution.\n *\n * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n *\n * You can contact the author at:\n * - xxHash homepage: https://www.xxhash.com\n * - xxHash source repository: https://github.com/Cyan4973/xxHash\n */\n\n/*\n * Note: This file used to host the source code of XXH3_* variants.\n * during the development period.\n * The source code is now properly integrated within xxhash.h.\n *\n * xxh3.h is no longer useful,\n * but it is still provided for compatibility with source code\n * which used to include it directly.\n *\n * Programs are now highly discouraged to include xxh3.h.\n * Include `xxhash.h` instead, which is the officially supported interface.\n *\n * In the future, xxh3.h will start to generate warnings, then errors,\n * then it will be removed from source package and from include directory.\n */\n\n/* Simulate the same impact as including the old xxh3.h source file */\n\n#define XXH_INLINE_ALL\n#include \"xxhash.h\"\n" }, { "path": "benchmarks/xxhash.c", "content": "/*\n * xxHash - Extremely Fast Hash algorithm\n * Copyright (C) 2012-2023 Yann Collet\n *\n * BSD 2-Clause License (https://www.opensource.org/licenses/bsd-license.php)\n *\n * Redistribution and use in source and binary forms, with or without\n * modification, are permitted provided that the following conditions are\n * met:\n *\n * * Redistributions of source code must retain the above copyright\n * notice, this list of conditions and the following disclaimer.\n * * Redistributions in binary form must reproduce the above\n * copyright notice, this list of conditions and the following disclaimer\n * in the documentation and/or other materials provided with the\n * distribution.\n *\n * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n *\n * You can contact the author at:\n * - xxHash homepage: https://www.xxhash.com\n * - xxHash source repository: https://github.com/Cyan4973/xxHash\n */\n\n/*\n * xxhash.c instantiates functions defined in xxhash.h\n */\n\n#define XXH_STATIC_LINKING_ONLY /* access advanced declarations */\n#define XXH_IMPLEMENTATION /* access definitions */\n\n#include \"xxhash.h\"\n" }, { "path": "benchmarks/xxhash.h", "content": "/*\n * xxHash - Extremely Fast Hash algorithm\n * Header File\n * Copyright (C) 2012-2023 Yann Collet\n *\n * BSD 2-Clause License (https://www.opensource.org/licenses/bsd-license.php)\n *\n * Redistribution and use in source and binary forms, with or without\n * modification, are permitted provided that the following conditions are\n * met:\n *\n * * Redistributions of source code must retain the above copyright\n * notice, this list of conditions and the following disclaimer.\n * * Redistributions in binary form must reproduce the above\n * copyright notice, this list of conditions and the following disclaimer\n * in the documentation and/or other materials provided with the\n * distribution.\n *\n * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n *\n * You can contact the author at:\n * - xxHash homepage: https://www.xxhash.com\n * - xxHash source repository: https://github.com/Cyan4973/xxHash\n */\n\n/*!\n * @mainpage xxHash\n *\n * xxHash is an extremely fast non-cryptographic hash algorithm, working at RAM speed\n * limits.\n *\n * It is proposed in four flavors, in three families:\n * 1. @ref XXH32_family\n * - Classic 32-bit hash function. Simple, compact, and runs on almost all\n * 32-bit and 64-bit systems.\n * 2. @ref XXH64_family\n * - Classic 64-bit adaptation of XXH32. Just as simple, and runs well on most\n * 64-bit systems (but _not_ 32-bit systems).\n * 3. @ref XXH3_family\n * - Modern 64-bit and 128-bit hash function family which features improved\n * strength and performance across the board, especially on smaller data.\n * It benefits greatly from SIMD and 64-bit without requiring it.\n *\n * Benchmarks\n * ---\n * The reference system uses an Intel i7-9700K CPU, and runs Ubuntu x64 20.04.\n * The open source benchmark program is compiled with clang v10.0 using -O3 flag.\n *\n * | Hash Name | ISA ext | Width | Large Data Speed | Small Data Velocity |\n * | -------------------- | ------- | ----: | ---------------: | ------------------: |\n * | XXH3_64bits() | @b AVX2 | 64 | 59.4 GB/s | 133.1 |\n * | MeowHash | AES-NI | 128 | 58.2 GB/s | 52.5 |\n * | XXH3_128bits() | @b AVX2 | 128 | 57.9 GB/s | 118.1 |\n * | CLHash | PCLMUL | 64 | 37.1 GB/s | 58.1 |\n * | XXH3_64bits() | @b SSE2 | 64 | 31.5 GB/s | 133.1 |\n * | XXH3_128bits() | @b SSE2 | 128 | 29.6 GB/s | 118.1 |\n * | RAM sequential read | | N/A | 28.0 GB/s | N/A |\n * | ahash | AES-NI | 64 | 22.5 GB/s | 107.2 |\n * | City64 | | 64 | 22.0 GB/s | 76.6 |\n * | T1ha2 | | 64 | 22.0 GB/s | 99.0 |\n * | City128 | | 128 | 21.7 GB/s | 57.7 |\n * | FarmHash | AES-NI | 64 | 21.3 GB/s | 71.9 |\n * | XXH64() | | 64 | 19.4 GB/s | 71.0 |\n * | SpookyHash | | 64 | 19.3 GB/s | 53.2 |\n * | Mum | | 64 | 18.0 GB/s | 67.0 |\n * | CRC32C | SSE4.2 | 32 | 13.0 GB/s | 57.9 |\n * | XXH32() | | 32 | 9.7 GB/s | 71.9 |\n * | City32 | | 32 | 9.1 GB/s | 66.0 |\n * | Blake3* | @b AVX2 | 256 | 4.4 GB/s | 8.1 |\n * | Murmur3 | | 32 | 3.9 GB/s | 56.1 |\n * | SipHash* | | 64 | 3.0 GB/s | 43.2 |\n * | Blake3* | @b SSE2 | 256 | 2.4 GB/s | 8.1 |\n * | HighwayHash | | 64 | 1.4 GB/s | 6.0 |\n * | FNV64 | | 64 | 1.2 GB/s | 62.7 |\n * | Blake2* | | 256 | 1.1 GB/s | 5.1 |\n * | SHA1* | | 160 | 0.8 GB/s | 5.6 |\n * | MD5* | | 128 | 0.6 GB/s | 7.8 |\n * @note\n * - Hashes which require a specific ISA extension are noted. SSE2 is also noted,\n * even though it is mandatory on x64.\n * - Hashes with an asterisk are cryptographic. Note that MD5 is non-cryptographic\n * by modern standards.\n * - Small data velocity is a rough average of algorithm's efficiency for small\n * data. For more accurate information, see the wiki.\n * - More benchmarks and strength tests are found on the wiki:\n * https://github.com/Cyan4973/xxHash/wiki\n *\n * Usage\n * ------\n * All xxHash variants use a similar API. Changing the algorithm is a trivial\n * substitution.\n *\n * @pre\n * For functions which take an input and length parameter, the following\n * requirements are assumed:\n * - The range from [`input`, `input + length`) is valid, readable memory.\n * - The only exception is if the `length` is `0`, `input` may be `NULL`.\n * - For C++, the objects must have the *TriviallyCopyable* property, as the\n * functions access bytes directly as if it was an array of `unsigned char`.\n *\n * @anchor single_shot_example\n * **Single Shot**\n *\n * These functions are stateless functions which hash a contiguous block of memory,\n * immediately returning the result. They are the easiest and usually the fastest\n * option.\n *\n * XXH32(), XXH64(), XXH3_64bits(), XXH3_128bits()\n *\n * @code{.c}\n * #include \n * #include \"xxhash.h\"\n *\n * // Example for a function which hashes a null terminated string with XXH32().\n * XXH32_hash_t hash_string(const char* string, XXH32_hash_t seed)\n * {\n * // NULL pointers are only valid if the length is zero\n * size_t length = (string == NULL) ? 0 : strlen(string);\n * return XXH32(string, length, seed);\n * }\n * @endcode\n *\n *\n * @anchor streaming_example\n * **Streaming**\n *\n * These groups of functions allow incremental hashing of unknown size, even\n * more than what would fit in a size_t.\n *\n * XXH32_reset(), XXH64_reset(), XXH3_64bits_reset(), XXH3_128bits_reset()\n *\n * @code{.c}\n * #include \n * #include \n * #include \"xxhash.h\"\n * // Example for a function which hashes a FILE incrementally with XXH3_64bits().\n * XXH64_hash_t hashFile(FILE* f)\n * {\n * // Allocate a state struct. Do not just use malloc() or new.\n * XXH3_state_t* state = XXH3_createState();\n * assert(state != NULL && \"Out of memory!\");\n * // Reset the state to start a new hashing session.\n * XXH3_64bits_reset(state);\n * char buffer[4096];\n * size_t count;\n * // Read the file in chunks\n * while ((count = fread(buffer, 1, sizeof(buffer), f)) != 0) {\n * // Run update() as many times as necessary to process the data\n * XXH3_64bits_update(state, buffer, count);\n * }\n * // Retrieve the finalized hash. This will not change the state.\n * XXH64_hash_t result = XXH3_64bits_digest(state);\n * // Free the state. Do not use free().\n * XXH3_freeState(state);\n * return result;\n * }\n * @endcode\n *\n * Streaming functions generate the xxHash value from an incremental input.\n * This method is slower than single-call functions, due to state management.\n * For small inputs, prefer `XXH32()` and `XXH64()`, which are better optimized.\n *\n * An XXH state must first be allocated using `XXH*_createState()`.\n *\n * Start a new hash by initializing the state with a seed using `XXH*_reset()`.\n *\n * Then, feed the hash state by calling `XXH*_update()` as many times as necessary.\n *\n * The function returns an error code, with 0 meaning OK, and any other value\n * meaning there is an error.\n *\n * Finally, a hash value can be produced anytime, by using `XXH*_digest()`.\n * This function returns the nn-bits hash as an int or long long.\n *\n * It's still possible to continue inserting input into the hash state after a\n * digest, and generate new hash values later on by invoking `XXH*_digest()`.\n *\n * When done, release the state using `XXH*_freeState()`.\n *\n *\n * @anchor canonical_representation_example\n * **Canonical Representation**\n *\n * The default return values from XXH functions are unsigned 32, 64 and 128 bit\n * integers.\n * This the simplest and fastest format for further post-processing.\n *\n * However, this leaves open the question of what is the order on the byte level,\n * since little and big endian conventions will store the same number differently.\n *\n * The canonical representation settles this issue by mandating big-endian\n * convention, the same convention as human-readable numbers (large digits first).\n *\n * When writing hash values to storage, sending them over a network, or printing\n * them, it's highly recommended to use the canonical representation to ensure\n * portability across a wider range of systems, present and future.\n *\n * The following functions allow transformation of hash values to and from\n * canonical format.\n *\n * XXH32_canonicalFromHash(), XXH32_hashFromCanonical(),\n * XXH64_canonicalFromHash(), XXH64_hashFromCanonical(),\n * XXH128_canonicalFromHash(), XXH128_hashFromCanonical(),\n *\n * @code{.c}\n * #include \n * #include \"xxhash.h\"\n *\n * // Example for a function which prints XXH32_hash_t in human readable format\n * void printXxh32(XXH32_hash_t hash)\n * {\n * XXH32_canonical_t cano;\n * XXH32_canonicalFromHash(&cano, hash);\n * size_t i;\n * for(i = 0; i < sizeof(cano.digest); ++i) {\n * printf(\"%02x\", cano.digest[i]);\n * }\n * printf(\"\\n\");\n * }\n *\n * // Example for a function which converts XXH32_canonical_t to XXH32_hash_t\n * XXH32_hash_t convertCanonicalToXxh32(XXH32_canonical_t cano)\n * {\n * XXH32_hash_t hash = XXH32_hashFromCanonical(&cano);\n * return hash;\n * }\n * @endcode\n *\n *\n * @file xxhash.h\n * xxHash prototypes and implementation\n */\n\n#if defined (__cplusplus)\nextern \"C\" {\n#endif\n\n/* ****************************\n * INLINE mode\n ******************************/\n/*!\n * @defgroup public Public API\n * Contains details on the public xxHash functions.\n * @{\n */\n#ifdef XXH_DOXYGEN\n/*!\n * @brief Gives access to internal state declaration, required for static allocation.\n *\n * Incompatible with dynamic linking, due to risks of ABI changes.\n *\n * Usage:\n * @code{.c}\n * #define XXH_STATIC_LINKING_ONLY\n * #include \"xxhash.h\"\n * @endcode\n */\n# define XXH_STATIC_LINKING_ONLY\n/* Do not undef XXH_STATIC_LINKING_ONLY for Doxygen */\n\n/*!\n * @brief Gives access to internal definitions.\n *\n * Usage:\n * @code{.c}\n * #define XXH_STATIC_LINKING_ONLY\n * #define XXH_IMPLEMENTATION\n * #include \"xxhash.h\"\n * @endcode\n */\n# define XXH_IMPLEMENTATION\n/* Do not undef XXH_IMPLEMENTATION for Doxygen */\n\n/*!\n * @brief Exposes the implementation and marks all functions as `inline`.\n *\n * Use these build macros to inline xxhash into the target unit.\n * Inlining improves performance on small inputs, especially when the length is\n * expressed as a compile-time constant:\n *\n * https://fastcompression.blogspot.com/2018/03/xxhash-for-small-keys-impressive-power.html\n *\n * It also keeps xxHash symbols private to the unit, so they are not exported.\n *\n * Usage:\n * @code{.c}\n * #define XXH_INLINE_ALL\n * #include \"xxhash.h\"\n * @endcode\n * Do not compile and link xxhash.o as a separate object, as it is not useful.\n */\n# define XXH_INLINE_ALL\n# undef XXH_INLINE_ALL\n/*!\n * @brief Exposes the implementation without marking functions as inline.\n */\n# define XXH_PRIVATE_API\n# undef XXH_PRIVATE_API\n/*!\n * @brief Emulate a namespace by transparently prefixing all symbols.\n *\n * If you want to include _and expose_ xxHash functions from within your own\n * library, but also want to avoid symbol collisions with other libraries which\n * may also include xxHash, you can use @ref XXH_NAMESPACE to automatically prefix\n * any public symbol from xxhash library with the value of @ref XXH_NAMESPACE\n * (therefore, avoid empty or numeric values).\n *\n * Note that no change is required within the calling program as long as it\n * includes `xxhash.h`: Regular symbol names will be automatically translated\n * by this header.\n */\n# define XXH_NAMESPACE /* YOUR NAME HERE */\n# undef XXH_NAMESPACE\n#endif\n\n#if (defined(XXH_INLINE_ALL) || defined(XXH_PRIVATE_API)) \\\n && !defined(XXH_INLINE_ALL_31684351384)\n /* this section should be traversed only once */\n# define XXH_INLINE_ALL_31684351384\n /* give access to the advanced API, required to compile implementations */\n# undef XXH_STATIC_LINKING_ONLY /* avoid macro redef */\n# define XXH_STATIC_LINKING_ONLY\n /* make all functions private */\n# undef XXH_PUBLIC_API\n# if defined(__GNUC__)\n# define XXH_PUBLIC_API static __inline __attribute__((__unused__))\n# elif defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */)\n# define XXH_PUBLIC_API static inline\n# elif defined(_MSC_VER)\n# define XXH_PUBLIC_API static __inline\n# else\n /* note: this version may generate warnings for unused static functions */\n# define XXH_PUBLIC_API static\n# endif\n\n /*\n * This part deals with the special case where a unit wants to inline xxHash,\n * but \"xxhash.h\" has previously been included without XXH_INLINE_ALL,\n * such as part of some previously included *.h header file.\n * Without further action, the new include would just be ignored,\n * and functions would effectively _not_ be inlined (silent failure).\n * The following macros solve this situation by prefixing all inlined names,\n * avoiding naming collision with previous inclusions.\n */\n /* Before that, we unconditionally #undef all symbols,\n * in case they were already defined with XXH_NAMESPACE.\n * They will then be redefined for XXH_INLINE_ALL\n */\n# undef XXH_versionNumber\n /* XXH32 */\n# undef XXH32\n# undef XXH32_createState\n# undef XXH32_freeState\n# undef XXH32_reset\n# undef XXH32_update\n# undef XXH32_digest\n# undef XXH32_copyState\n# undef XXH32_canonicalFromHash\n# undef XXH32_hashFromCanonical\n /* XXH64 */\n# undef XXH64\n# undef XXH64_createState\n# undef XXH64_freeState\n# undef XXH64_reset\n# undef XXH64_update\n# undef XXH64_digest\n# undef XXH64_copyState\n# undef XXH64_canonicalFromHash\n# undef XXH64_hashFromCanonical\n /* XXH3_64bits */\n# undef XXH3_64bits\n# undef XXH3_64bits_withSecret\n# undef XXH3_64bits_withSeed\n# undef XXH3_64bits_withSecretandSeed\n# undef XXH3_createState\n# undef XXH3_freeState\n# undef XXH3_copyState\n# undef XXH3_64bits_reset\n# undef XXH3_64bits_reset_withSeed\n# undef XXH3_64bits_reset_withSecret\n# undef XXH3_64bits_update\n# undef XXH3_64bits_digest\n# undef XXH3_generateSecret\n /* XXH3_128bits */\n# undef XXH128\n# undef XXH3_128bits\n# undef XXH3_128bits_withSeed\n# undef XXH3_128bits_withSecret\n# undef XXH3_128bits_reset\n# undef XXH3_128bits_reset_withSeed\n# undef XXH3_128bits_reset_withSecret\n# undef XXH3_128bits_reset_withSecretandSeed\n# undef XXH3_128bits_update\n# undef XXH3_128bits_digest\n# undef XXH128_isEqual\n# undef XXH128_cmp\n# undef XXH128_canonicalFromHash\n# undef XXH128_hashFromCanonical\n /* Finally, free the namespace itself */\n# undef XXH_NAMESPACE\n\n /* employ the namespace for XXH_INLINE_ALL */\n# define XXH_NAMESPACE XXH_INLINE_\n /*\n * Some identifiers (enums, type names) are not symbols,\n * but they must nonetheless be renamed to avoid redeclaration.\n * Alternative solution: do not redeclare them.\n * However, this requires some #ifdefs, and has a more dispersed impact.\n * Meanwhile, renaming can be achieved in a single place.\n */\n# define XXH_IPREF(Id) XXH_NAMESPACE ## Id\n# define XXH_OK XXH_IPREF(XXH_OK)\n# define XXH_ERROR XXH_IPREF(XXH_ERROR)\n# define XXH_errorcode XXH_IPREF(XXH_errorcode)\n# define XXH32_canonical_t XXH_IPREF(XXH32_canonical_t)\n# define XXH64_canonical_t XXH_IPREF(XXH64_canonical_t)\n# define XXH128_canonical_t XXH_IPREF(XXH128_canonical_t)\n# define XXH32_state_s XXH_IPREF(XXH32_state_s)\n# define XXH32_state_t XXH_IPREF(XXH32_state_t)\n# define XXH64_state_s XXH_IPREF(XXH64_state_s)\n# define XXH64_state_t XXH_IPREF(XXH64_state_t)\n# define XXH3_state_s XXH_IPREF(XXH3_state_s)\n# define XXH3_state_t XXH_IPREF(XXH3_state_t)\n# define XXH128_hash_t XXH_IPREF(XXH128_hash_t)\n /* Ensure the header is parsed again, even if it was previously included */\n# undef XXHASH_H_5627135585666179\n# undef XXHASH_H_STATIC_13879238742\n#endif /* XXH_INLINE_ALL || XXH_PRIVATE_API */\n\n/* ****************************************************************\n * Stable API\n *****************************************************************/\n#ifndef XXHASH_H_5627135585666179\n#define XXHASH_H_5627135585666179 1\n\n/*! @brief Marks a global symbol. */\n#if !defined(XXH_INLINE_ALL) && !defined(XXH_PRIVATE_API)\n# if defined(_WIN32) && defined(_MSC_VER) && (defined(XXH_IMPORT) || defined(XXH_EXPORT))\n# ifdef XXH_EXPORT\n# define XXH_PUBLIC_API __declspec(dllexport)\n# elif XXH_IMPORT\n# define XXH_PUBLIC_API __declspec(dllimport)\n# endif\n# else\n# define XXH_PUBLIC_API /* do nothing */\n# endif\n#endif\n\n#ifdef XXH_NAMESPACE\n# define XXH_CAT(A,B) A##B\n# define XXH_NAME2(A,B) XXH_CAT(A,B)\n# define XXH_versionNumber XXH_NAME2(XXH_NAMESPACE, XXH_versionNumber)\n/* XXH32 */\n# define XXH32 XXH_NAME2(XXH_NAMESPACE, XXH32)\n# define XXH32_createState XXH_NAME2(XXH_NAMESPACE, XXH32_createState)\n# define XXH32_freeState XXH_NAME2(XXH_NAMESPACE, XXH32_freeState)\n# define XXH32_reset XXH_NAME2(XXH_NAMESPACE, XXH32_reset)\n# define XXH32_update XXH_NAME2(XXH_NAMESPACE, XXH32_update)\n# define XXH32_digest XXH_NAME2(XXH_NAMESPACE, XXH32_digest)\n# define XXH32_copyState XXH_NAME2(XXH_NAMESPACE, XXH32_copyState)\n# define XXH32_canonicalFromHash XXH_NAME2(XXH_NAMESPACE, XXH32_canonicalFromHash)\n# define XXH32_hashFromCanonical XXH_NAME2(XXH_NAMESPACE, XXH32_hashFromCanonical)\n/* XXH64 */\n# define XXH64 XXH_NAME2(XXH_NAMESPACE, XXH64)\n# define XXH64_createState XXH_NAME2(XXH_NAMESPACE, XXH64_createState)\n# define XXH64_freeState XXH_NAME2(XXH_NAMESPACE, XXH64_freeState)\n# define XXH64_reset XXH_NAME2(XXH_NAMESPACE, XXH64_reset)\n# define XXH64_update XXH_NAME2(XXH_NAMESPACE, XXH64_update)\n# define XXH64_digest XXH_NAME2(XXH_NAMESPACE, XXH64_digest)\n# define XXH64_copyState XXH_NAME2(XXH_NAMESPACE, XXH64_copyState)\n# define XXH64_canonicalFromHash XXH_NAME2(XXH_NAMESPACE, XXH64_canonicalFromHash)\n# define XXH64_hashFromCanonical XXH_NAME2(XXH_NAMESPACE, XXH64_hashFromCanonical)\n/* XXH3_64bits */\n# define XXH3_64bits XXH_NAME2(XXH_NAMESPACE, XXH3_64bits)\n# define XXH3_64bits_withSecret XXH_NAME2(XXH_NAMESPACE, XXH3_64bits_withSecret)\n# define XXH3_64bits_withSeed XXH_NAME2(XXH_NAMESPACE, XXH3_64bits_withSeed)\n# define XXH3_64bits_withSecretandSeed XXH_NAME2(XXH_NAMESPACE, XXH3_64bits_withSecretandSeed)\n# define XXH3_createState XXH_NAME2(XXH_NAMESPACE, XXH3_createState)\n# define XXH3_freeState XXH_NAME2(XXH_NAMESPACE, XXH3_freeState)\n# define XXH3_copyState XXH_NAME2(XXH_NAMESPACE, XXH3_copyState)\n# define XXH3_64bits_reset XXH_NAME2(XXH_NAMESPACE, XXH3_64bits_reset)\n# define XXH3_64bits_reset_withSeed XXH_NAME2(XXH_NAMESPACE, XXH3_64bits_reset_withSeed)\n# define XXH3_64bits_reset_withSecret XXH_NAME2(XXH_NAMESPACE, XXH3_64bits_reset_withSecret)\n# define XXH3_64bits_reset_withSecretandSeed XXH_NAME2(XXH_NAMESPACE, XXH3_64bits_reset_withSecretandSeed)\n# define XXH3_64bits_update XXH_NAME2(XXH_NAMESPACE, XXH3_64bits_update)\n# define XXH3_64bits_digest XXH_NAME2(XXH_NAMESPACE, XXH3_64bits_digest)\n# define XXH3_generateSecret XXH_NAME2(XXH_NAMESPACE, XXH3_generateSecret)\n# define XXH3_generateSecret_fromSeed XXH_NAME2(XXH_NAMESPACE, XXH3_generateSecret_fromSeed)\n/* XXH3_128bits */\n# define XXH128 XXH_NAME2(XXH_NAMESPACE, XXH128)\n# define XXH3_128bits XXH_NAME2(XXH_NAMESPACE, XXH3_128bits)\n# define XXH3_128bits_withSeed XXH_NAME2(XXH_NAMESPACE, XXH3_128bits_withSeed)\n# define XXH3_128bits_withSecret XXH_NAME2(XXH_NAMESPACE, XXH3_128bits_withSecret)\n# define XXH3_128bits_withSecretandSeed XXH_NAME2(XXH_NAMESPACE, XXH3_128bits_withSecretandSeed)\n# define XXH3_128bits_reset XXH_NAME2(XXH_NAMESPACE, XXH3_128bits_reset)\n# define XXH3_128bits_reset_withSeed XXH_NAME2(XXH_NAMESPACE, XXH3_128bits_reset_withSeed)\n# define XXH3_128bits_reset_withSecret XXH_NAME2(XXH_NAMESPACE, XXH3_128bits_reset_withSecret)\n# define XXH3_128bits_reset_withSecretandSeed XXH_NAME2(XXH_NAMESPACE, XXH3_128bits_reset_withSecretandSeed)\n# define XXH3_128bits_update XXH_NAME2(XXH_NAMESPACE, XXH3_128bits_update)\n# define XXH3_128bits_digest XXH_NAME2(XXH_NAMESPACE, XXH3_128bits_digest)\n# define XXH128_isEqual XXH_NAME2(XXH_NAMESPACE, XXH128_isEqual)\n# define XXH128_cmp XXH_NAME2(XXH_NAMESPACE, XXH128_cmp)\n# define XXH128_canonicalFromHash XXH_NAME2(XXH_NAMESPACE, XXH128_canonicalFromHash)\n# define XXH128_hashFromCanonical XXH_NAME2(XXH_NAMESPACE, XXH128_hashFromCanonical)\n#endif\n\n\n/* *************************************\n* Compiler specifics\n***************************************/\n\n/* specific declaration modes for Windows */\n#if !defined(XXH_INLINE_ALL) && !defined(XXH_PRIVATE_API)\n# if defined(_WIN32) && defined(_MSC_VER) && (defined(XXH_IMPORT) || defined(XXH_EXPORT))\n# ifdef XXH_EXPORT\n# define XXH_PUBLIC_API __declspec(dllexport)\n# elif XXH_IMPORT\n# define XXH_PUBLIC_API __declspec(dllimport)\n# endif\n# else\n# define XXH_PUBLIC_API /* do nothing */\n# endif\n#endif\n\n#if defined (__GNUC__)\n# define XXH_CONSTF __attribute__((__const__))\n# define XXH_PUREF __attribute__((__pure__))\n# define XXH_MALLOCF __attribute__((__malloc__))\n#else\n# define XXH_CONSTF /* disable */\n# define XXH_PUREF\n# define XXH_MALLOCF\n#endif\n\n/* *************************************\n* Version\n***************************************/\n#define XXH_VERSION_MAJOR 0\n#define XXH_VERSION_MINOR 8\n#define XXH_VERSION_RELEASE 3\n/*! @brief Version number, encoded as two digits each */\n#define XXH_VERSION_NUMBER (XXH_VERSION_MAJOR *100*100 + XXH_VERSION_MINOR *100 + XXH_VERSION_RELEASE)\n\n/*!\n * @brief Obtains the xxHash version.\n *\n * This is mostly useful when xxHash is compiled as a shared library,\n * since the returned value comes from the library, as opposed to header file.\n *\n * @return @ref XXH_VERSION_NUMBER of the invoked library.\n */\nXXH_PUBLIC_API XXH_CONSTF unsigned XXH_versionNumber (void);\n\n\n/* ****************************\n* Common basic types\n******************************/\n#include /* size_t */\n/*!\n * @brief Exit code for the streaming API.\n */\ntypedef enum {\n XXH_OK = 0, /*!< OK */\n XXH_ERROR /*!< Error */\n} XXH_errorcode;\n\n\n/*-**********************************************************************\n* 32-bit hash\n************************************************************************/\n#if defined(XXH_DOXYGEN) /* Don't show include */\n/*!\n * @brief An unsigned 32-bit integer.\n *\n * Not necessarily defined to `uint32_t` but functionally equivalent.\n */\ntypedef uint32_t XXH32_hash_t;\n\n#elif !defined (__VMS) \\\n && (defined (__cplusplus) \\\n || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) )\n# ifdef _AIX\n# include \n# else\n# include \n# endif\n typedef uint32_t XXH32_hash_t;\n\n#else\n# include \n# if UINT_MAX == 0xFFFFFFFFUL\n typedef unsigned int XXH32_hash_t;\n# elif ULONG_MAX == 0xFFFFFFFFUL\n typedef unsigned long XXH32_hash_t;\n# else\n# error \"unsupported platform: need a 32-bit type\"\n# endif\n#endif\n\n/*!\n * @}\n *\n * @defgroup XXH32_family XXH32 family\n * @ingroup public\n * Contains functions used in the classic 32-bit xxHash algorithm.\n *\n * @note\n * XXH32 is useful for older platforms, with no or poor 64-bit performance.\n * Note that the @ref XXH3_family provides competitive speed for both 32-bit\n * and 64-bit systems, and offers true 64/128 bit hash results.\n *\n * @see @ref XXH64_family, @ref XXH3_family : Other xxHash families\n * @see @ref XXH32_impl for implementation details\n * @{\n */\n\n/*!\n * @brief Calculates the 32-bit hash of @p input using xxHash32.\n *\n * @param input The block of data to be hashed, at least @p length bytes in size.\n * @param length The length of @p input, in bytes.\n * @param seed The 32-bit seed to alter the hash's output predictably.\n *\n * @pre\n * The memory between @p input and @p input + @p length must be valid,\n * readable, contiguous memory. However, if @p length is `0`, @p input may be\n * `NULL`. In C++, this also must be *TriviallyCopyable*.\n *\n * @return The calculated 32-bit xxHash32 value.\n *\n * @see @ref single_shot_example \"Single Shot Example\" for an example.\n */\nXXH_PUBLIC_API XXH_PUREF XXH32_hash_t XXH32 (const void* input, size_t length, XXH32_hash_t seed);\n\n#ifndef XXH_NO_STREAM\n/*!\n * @typedef struct XXH32_state_s XXH32_state_t\n * @brief The opaque state struct for the XXH32 streaming API.\n *\n * @see XXH32_state_s for details.\n * @see @ref streaming_example \"Streaming Example\"\n */\ntypedef struct XXH32_state_s XXH32_state_t;\n\n/*!\n * @brief Allocates an @ref XXH32_state_t.\n *\n * @return An allocated pointer of @ref XXH32_state_t on success.\n * @return `NULL` on failure.\n *\n * @note Must be freed with XXH32_freeState().\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_MALLOCF XXH32_state_t* XXH32_createState(void);\n/*!\n * @brief Frees an @ref XXH32_state_t.\n *\n * @param statePtr A pointer to an @ref XXH32_state_t allocated with @ref XXH32_createState().\n *\n * @return @ref XXH_OK.\n *\n * @note @p statePtr must be allocated with XXH32_createState().\n *\n * @see @ref streaming_example \"Streaming Example\"\n *\n */\nXXH_PUBLIC_API XXH_errorcode XXH32_freeState(XXH32_state_t* statePtr);\n/*!\n * @brief Copies one @ref XXH32_state_t to another.\n *\n * @param dst_state The state to copy to.\n * @param src_state The state to copy from.\n * @pre\n * @p dst_state and @p src_state must not be `NULL` and must not overlap.\n */\nXXH_PUBLIC_API void XXH32_copyState(XXH32_state_t* dst_state, const XXH32_state_t* src_state);\n\n/*!\n * @brief Resets an @ref XXH32_state_t to begin a new hash.\n *\n * @param statePtr The state struct to reset.\n * @param seed The 32-bit seed to alter the hash result predictably.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @note This function resets and seeds a state. Call it before @ref XXH32_update().\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_errorcode XXH32_reset (XXH32_state_t* statePtr, XXH32_hash_t seed);\n\n/*!\n * @brief Consumes a block of @p input to an @ref XXH32_state_t.\n *\n * @param statePtr The state struct to update.\n * @param input The block of data to be hashed, at least @p length bytes in size.\n * @param length The length of @p input, in bytes.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n * @pre\n * The memory between @p input and @p input + @p length must be valid,\n * readable, contiguous memory. However, if @p length is `0`, @p input may be\n * `NULL`. In C++, this also must be *TriviallyCopyable*.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @note Call this to incrementally consume blocks of data.\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_errorcode XXH32_update (XXH32_state_t* statePtr, const void* input, size_t length);\n\n/*!\n * @brief Returns the calculated hash value from an @ref XXH32_state_t.\n *\n * @param statePtr The state struct to calculate the hash from.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n *\n * @return The calculated 32-bit xxHash32 value from that state.\n *\n * @note\n * Calling XXH32_digest() will not affect @p statePtr, so you can update,\n * digest, and update again.\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_PUREF XXH32_hash_t XXH32_digest (const XXH32_state_t* statePtr);\n#endif /* !XXH_NO_STREAM */\n\n/******* Canonical representation *******/\n\n/*!\n * @brief Canonical (big endian) representation of @ref XXH32_hash_t.\n */\ntypedef struct {\n unsigned char digest[4]; /*!< Hash bytes, big endian */\n} XXH32_canonical_t;\n\n/*!\n * @brief Converts an @ref XXH32_hash_t to a big endian @ref XXH32_canonical_t.\n *\n * @param dst The @ref XXH32_canonical_t pointer to be stored to.\n * @param hash The @ref XXH32_hash_t to be converted.\n *\n * @pre\n * @p dst must not be `NULL`.\n *\n * @see @ref canonical_representation_example \"Canonical Representation Example\"\n */\nXXH_PUBLIC_API void XXH32_canonicalFromHash(XXH32_canonical_t* dst, XXH32_hash_t hash);\n\n/*!\n * @brief Converts an @ref XXH32_canonical_t to a native @ref XXH32_hash_t.\n *\n * @param src The @ref XXH32_canonical_t to convert.\n *\n * @pre\n * @p src must not be `NULL`.\n *\n * @return The converted hash.\n *\n * @see @ref canonical_representation_example \"Canonical Representation Example\"\n */\nXXH_PUBLIC_API XXH_PUREF XXH32_hash_t XXH32_hashFromCanonical(const XXH32_canonical_t* src);\n\n\n/*! @cond Doxygen ignores this part */\n#ifdef __has_attribute\n# define XXH_HAS_ATTRIBUTE(x) __has_attribute(x)\n#else\n# define XXH_HAS_ATTRIBUTE(x) 0\n#endif\n/*! @endcond */\n\n/*! @cond Doxygen ignores this part */\n/*\n * C23 __STDC_VERSION__ number hasn't been specified yet. For now\n * leave as `201711L` (C17 + 1).\n * TODO: Update to correct value when its been specified.\n */\n#define XXH_C23_VN 201711L\n/*! @endcond */\n\n/*! @cond Doxygen ignores this part */\n/* C-language Attributes are added in C23. */\n#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= XXH_C23_VN) && defined(__has_c_attribute)\n# define XXH_HAS_C_ATTRIBUTE(x) __has_c_attribute(x)\n#else\n# define XXH_HAS_C_ATTRIBUTE(x) 0\n#endif\n/*! @endcond */\n\n/*! @cond Doxygen ignores this part */\n#if defined(__cplusplus) && defined(__has_cpp_attribute)\n# define XXH_HAS_CPP_ATTRIBUTE(x) __has_cpp_attribute(x)\n#else\n# define XXH_HAS_CPP_ATTRIBUTE(x) 0\n#endif\n/*! @endcond */\n\n/*! @cond Doxygen ignores this part */\n/*\n * Define XXH_FALLTHROUGH macro for annotating switch case with the 'fallthrough' attribute\n * introduced in CPP17 and C23.\n * CPP17 : https://en.cppreference.com/w/cpp/language/attributes/fallthrough\n * C23 : https://en.cppreference.com/w/c/language/attributes/fallthrough\n */\n#if XXH_HAS_C_ATTRIBUTE(fallthrough) || XXH_HAS_CPP_ATTRIBUTE(fallthrough)\n# define XXH_FALLTHROUGH [[fallthrough]]\n#elif XXH_HAS_ATTRIBUTE(__fallthrough__)\n# define XXH_FALLTHROUGH __attribute__ ((__fallthrough__))\n#else\n# define XXH_FALLTHROUGH /* fallthrough */\n#endif\n/*! @endcond */\n\n/*! @cond Doxygen ignores this part */\n/*\n * Define XXH_NOESCAPE for annotated pointers in public API.\n * https://clang.llvm.org/docs/AttributeReference.html#noescape\n * As of writing this, only supported by clang.\n */\n#if XXH_HAS_ATTRIBUTE(noescape)\n# define XXH_NOESCAPE __attribute__((__noescape__))\n#else\n# define XXH_NOESCAPE\n#endif\n/*! @endcond */\n\n\n/*!\n * @}\n * @ingroup public\n * @{\n */\n\n#ifndef XXH_NO_LONG_LONG\n/*-**********************************************************************\n* 64-bit hash\n************************************************************************/\n#if defined(XXH_DOXYGEN) /* don't include */\n/*!\n * @brief An unsigned 64-bit integer.\n *\n * Not necessarily defined to `uint64_t` but functionally equivalent.\n */\ntypedef uint64_t XXH64_hash_t;\n#elif !defined (__VMS) \\\n && (defined (__cplusplus) \\\n || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) )\n# ifdef _AIX\n# include \n# else\n# include \n# endif\n typedef uint64_t XXH64_hash_t;\n#else\n# include \n# if defined(__LP64__) && ULONG_MAX == 0xFFFFFFFFFFFFFFFFULL\n /* LP64 ABI says uint64_t is unsigned long */\n typedef unsigned long XXH64_hash_t;\n# else\n /* the following type must have a width of 64-bit */\n typedef unsigned long long XXH64_hash_t;\n# endif\n#endif\n\n/*!\n * @}\n *\n * @defgroup XXH64_family XXH64 family\n * @ingroup public\n * @{\n * Contains functions used in the classic 64-bit xxHash algorithm.\n *\n * @note\n * XXH3 provides competitive speed for both 32-bit and 64-bit systems,\n * and offers true 64/128 bit hash results.\n * It provides better speed for systems with vector processing capabilities.\n */\n\n/*!\n * @brief Calculates the 64-bit hash of @p input using xxHash64.\n *\n * @param input The block of data to be hashed, at least @p length bytes in size.\n * @param length The length of @p input, in bytes.\n * @param seed The 64-bit seed to alter the hash's output predictably.\n *\n * @pre\n * The memory between @p input and @p input + @p length must be valid,\n * readable, contiguous memory. However, if @p length is `0`, @p input may be\n * `NULL`. In C++, this also must be *TriviallyCopyable*.\n *\n * @return The calculated 64-bit xxHash64 value.\n *\n * @see @ref single_shot_example \"Single Shot Example\" for an example.\n */\nXXH_PUBLIC_API XXH_PUREF XXH64_hash_t XXH64(XXH_NOESCAPE const void* input, size_t length, XXH64_hash_t seed);\n\n/******* Streaming *******/\n#ifndef XXH_NO_STREAM\n/*!\n * @brief The opaque state struct for the XXH64 streaming API.\n *\n * @see XXH64_state_s for details.\n * @see @ref streaming_example \"Streaming Example\"\n */\ntypedef struct XXH64_state_s XXH64_state_t; /* incomplete type */\n\n/*!\n * @brief Allocates an @ref XXH64_state_t.\n *\n * @return An allocated pointer of @ref XXH64_state_t on success.\n * @return `NULL` on failure.\n *\n * @note Must be freed with XXH64_freeState().\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_MALLOCF XXH64_state_t* XXH64_createState(void);\n\n/*!\n * @brief Frees an @ref XXH64_state_t.\n *\n * @param statePtr A pointer to an @ref XXH64_state_t allocated with @ref XXH64_createState().\n *\n * @return @ref XXH_OK.\n *\n * @note @p statePtr must be allocated with XXH64_createState().\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_errorcode XXH64_freeState(XXH64_state_t* statePtr);\n\n/*!\n * @brief Copies one @ref XXH64_state_t to another.\n *\n * @param dst_state The state to copy to.\n * @param src_state The state to copy from.\n * @pre\n * @p dst_state and @p src_state must not be `NULL` and must not overlap.\n */\nXXH_PUBLIC_API void XXH64_copyState(XXH_NOESCAPE XXH64_state_t* dst_state, const XXH64_state_t* src_state);\n\n/*!\n * @brief Resets an @ref XXH64_state_t to begin a new hash.\n *\n * @param statePtr The state struct to reset.\n * @param seed The 64-bit seed to alter the hash result predictably.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @note This function resets and seeds a state. Call it before @ref XXH64_update().\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_errorcode XXH64_reset (XXH_NOESCAPE XXH64_state_t* statePtr, XXH64_hash_t seed);\n\n/*!\n * @brief Consumes a block of @p input to an @ref XXH64_state_t.\n *\n * @param statePtr The state struct to update.\n * @param input The block of data to be hashed, at least @p length bytes in size.\n * @param length The length of @p input, in bytes.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n * @pre\n * The memory between @p input and @p input + @p length must be valid,\n * readable, contiguous memory. However, if @p length is `0`, @p input may be\n * `NULL`. In C++, this also must be *TriviallyCopyable*.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @note Call this to incrementally consume blocks of data.\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_errorcode XXH64_update (XXH_NOESCAPE XXH64_state_t* statePtr, XXH_NOESCAPE const void* input, size_t length);\n\n/*!\n * @brief Returns the calculated hash value from an @ref XXH64_state_t.\n *\n * @param statePtr The state struct to calculate the hash from.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n *\n * @return The calculated 64-bit xxHash64 value from that state.\n *\n * @note\n * Calling XXH64_digest() will not affect @p statePtr, so you can update,\n * digest, and update again.\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_PUREF XXH64_hash_t XXH64_digest (XXH_NOESCAPE const XXH64_state_t* statePtr);\n#endif /* !XXH_NO_STREAM */\n/******* Canonical representation *******/\n\n/*!\n * @brief Canonical (big endian) representation of @ref XXH64_hash_t.\n */\ntypedef struct { unsigned char digest[sizeof(XXH64_hash_t)]; } XXH64_canonical_t;\n\n/*!\n * @brief Converts an @ref XXH64_hash_t to a big endian @ref XXH64_canonical_t.\n *\n * @param dst The @ref XXH64_canonical_t pointer to be stored to.\n * @param hash The @ref XXH64_hash_t to be converted.\n *\n * @pre\n * @p dst must not be `NULL`.\n *\n * @see @ref canonical_representation_example \"Canonical Representation Example\"\n */\nXXH_PUBLIC_API void XXH64_canonicalFromHash(XXH_NOESCAPE XXH64_canonical_t* dst, XXH64_hash_t hash);\n\n/*!\n * @brief Converts an @ref XXH64_canonical_t to a native @ref XXH64_hash_t.\n *\n * @param src The @ref XXH64_canonical_t to convert.\n *\n * @pre\n * @p src must not be `NULL`.\n *\n * @return The converted hash.\n *\n * @see @ref canonical_representation_example \"Canonical Representation Example\"\n */\nXXH_PUBLIC_API XXH_PUREF XXH64_hash_t XXH64_hashFromCanonical(XXH_NOESCAPE const XXH64_canonical_t* src);\n\n#ifndef XXH_NO_XXH3\n\n/*!\n * @}\n * ************************************************************************\n * @defgroup XXH3_family XXH3 family\n * @ingroup public\n * @{\n *\n * XXH3 is a more recent hash algorithm featuring:\n * - Improved speed for both small and large inputs\n * - True 64-bit and 128-bit outputs\n * - SIMD acceleration\n * - Improved 32-bit viability\n *\n * Speed analysis methodology is explained here:\n *\n * https://fastcompression.blogspot.com/2019/03/presenting-xxh3.html\n *\n * Compared to XXH64, expect XXH3 to run approximately\n * ~2x faster on large inputs and >3x faster on small ones,\n * exact differences vary depending on platform.\n *\n * XXH3's speed benefits greatly from SIMD and 64-bit arithmetic,\n * but does not require it.\n * Most 32-bit and 64-bit targets that can run XXH32 smoothly can run XXH3\n * at competitive speeds, even without vector support. Further details are\n * explained in the implementation.\n *\n * XXH3 has a fast scalar implementation, but it also includes accelerated SIMD\n * implementations for many common platforms:\n * - AVX512\n * - AVX2\n * - SSE2\n * - ARM NEON\n * - WebAssembly SIMD128\n * - POWER8 VSX\n * - s390x ZVector\n * This can be controlled via the @ref XXH_VECTOR macro, but it automatically\n * selects the best version according to predefined macros. For the x86 family, an\n * automatic runtime dispatcher is included separately in @ref xxh_x86dispatch.c.\n *\n * XXH3 implementation is portable:\n * it has a generic C90 formulation that can be compiled on any platform,\n * all implementations generate exactly the same hash value on all platforms.\n * Starting from v0.8.0, it's also labelled \"stable\", meaning that\n * any future version will also generate the same hash value.\n *\n * XXH3 offers 2 variants, _64bits and _128bits.\n *\n * When only 64 bits are needed, prefer invoking the _64bits variant, as it\n * reduces the amount of mixing, resulting in faster speed on small inputs.\n * It's also generally simpler to manipulate a scalar return type than a struct.\n *\n * The API supports one-shot hashing, streaming mode, and custom secrets.\n */\n\n/*!\n * @ingroup tuning\n * @brief Possible values for @ref XXH_VECTOR.\n *\n * Unless set explicitly, determined automatically.\n */\n# define XXH_SCALAR 0 /*!< Portable scalar version */\n# define XXH_SSE2 1 /*!< SSE2 for Pentium 4, Opteron, all x86_64. */\n# define XXH_AVX2 2 /*!< AVX2 for Haswell and Bulldozer */\n# define XXH_AVX512 3 /*!< AVX512 for Skylake and Icelake */\n# define XXH_NEON 4 /*!< NEON for most ARMv7-A, all AArch64, and WASM SIMD128 */\n# define XXH_VSX 5 /*!< VSX and ZVector for POWER8/z13 (64-bit) */\n# define XXH_SVE 6 /*!< SVE for some ARMv8-A and ARMv9-A */\n# define XXH_LSX 7 /*!< LSX (128-bit SIMD) for LoongArch64 */\n\n\n/*-**********************************************************************\n* XXH3 64-bit variant\n************************************************************************/\n\n/*!\n * @brief Calculates 64-bit unseeded variant of XXH3 hash of @p input.\n *\n * @param input The block of data to be hashed, at least @p length bytes in size.\n * @param length The length of @p input, in bytes.\n *\n * @pre\n * The memory between @p input and @p input + @p length must be valid,\n * readable, contiguous memory. However, if @p length is `0`, @p input may be\n * `NULL`. In C++, this also must be *TriviallyCopyable*.\n *\n * @return The calculated 64-bit XXH3 hash value.\n *\n * @note\n * This is equivalent to @ref XXH3_64bits_withSeed() with a seed of `0`, however\n * it may have slightly better performance due to constant propagation of the\n * defaults.\n *\n * @see\n * XXH3_64bits_withSeed(), XXH3_64bits_withSecret(): other seeding variants\n * @see @ref single_shot_example \"Single Shot Example\" for an example.\n */\nXXH_PUBLIC_API XXH_PUREF XXH64_hash_t XXH3_64bits(XXH_NOESCAPE const void* input, size_t length);\n\n/*!\n * @brief Calculates 64-bit seeded variant of XXH3 hash of @p input.\n *\n * @param input The block of data to be hashed, at least @p length bytes in size.\n * @param length The length of @p input, in bytes.\n * @param seed The 64-bit seed to alter the hash result predictably.\n *\n * @pre\n * The memory between @p input and @p input + @p length must be valid,\n * readable, contiguous memory. However, if @p length is `0`, @p input may be\n * `NULL`. In C++, this also must be *TriviallyCopyable*.\n *\n * @return The calculated 64-bit XXH3 hash value.\n *\n * @note\n * seed == 0 produces the same results as @ref XXH3_64bits().\n *\n * This variant generates a custom secret on the fly based on default secret\n * altered using the @p seed value.\n *\n * While this operation is decently fast, note that it's not completely free.\n *\n * @see @ref single_shot_example \"Single Shot Example\" for an example.\n */\nXXH_PUBLIC_API XXH_PUREF XXH64_hash_t XXH3_64bits_withSeed(XXH_NOESCAPE const void* input, size_t length, XXH64_hash_t seed);\n\n/*!\n * The bare minimum size for a custom secret.\n *\n * @see\n * XXH3_64bits_withSecret(), XXH3_64bits_reset_withSecret(),\n * XXH3_128bits_withSecret(), XXH3_128bits_reset_withSecret().\n */\n#define XXH3_SECRET_SIZE_MIN 136\n\n/*!\n * @brief Calculates 64-bit variant of XXH3 with a custom \"secret\".\n *\n * @param data The block of data to be hashed, at least @p len bytes in size.\n * @param len The length of @p data, in bytes.\n * @param secret The secret data.\n * @param secretSize The length of @p secret, in bytes.\n *\n * @return The calculated 64-bit XXH3 hash value.\n *\n * @pre\n * The memory between @p data and @p data + @p len must be valid,\n * readable, contiguous memory. However, if @p length is `0`, @p data may be\n * `NULL`. In C++, this also must be *TriviallyCopyable*.\n *\n * It's possible to provide any blob of bytes as a \"secret\" to generate the hash.\n * This makes it more difficult for an external actor to prepare an intentional collision.\n * The main condition is that @p secretSize *must* be large enough (>= @ref XXH3_SECRET_SIZE_MIN).\n * However, the quality of the secret impacts the dispersion of the hash algorithm.\n * Therefore, the secret _must_ look like a bunch of random bytes.\n * Avoid \"trivial\" or structured data such as repeated sequences or a text document.\n * Whenever in doubt about the \"randomness\" of the blob of bytes,\n * consider employing @ref XXH3_generateSecret() instead (see below).\n * It will generate a proper high entropy secret derived from the blob of bytes.\n * Another advantage of using XXH3_generateSecret() is that\n * it guarantees that all bits within the initial blob of bytes\n * will impact every bit of the output.\n * This is not necessarily the case when using the blob of bytes directly\n * because, when hashing _small_ inputs, only a portion of the secret is employed.\n *\n * @see @ref single_shot_example \"Single Shot Example\" for an example.\n */\nXXH_PUBLIC_API XXH_PUREF XXH64_hash_t XXH3_64bits_withSecret(XXH_NOESCAPE const void* data, size_t len, XXH_NOESCAPE const void* secret, size_t secretSize);\n\n\n/******* Streaming *******/\n#ifndef XXH_NO_STREAM\n/*\n * Streaming requires state maintenance.\n * This operation costs memory and CPU.\n * As a consequence, streaming is slower than one-shot hashing.\n * For better performance, prefer one-shot functions whenever applicable.\n */\n\n/*!\n * @brief The opaque state struct for the XXH3 streaming API.\n *\n * @see XXH3_state_s for details.\n * @see @ref streaming_example \"Streaming Example\"\n */\ntypedef struct XXH3_state_s XXH3_state_t;\nXXH_PUBLIC_API XXH_MALLOCF XXH3_state_t* XXH3_createState(void);\nXXH_PUBLIC_API XXH_errorcode XXH3_freeState(XXH3_state_t* statePtr);\n\n/*!\n * @brief Copies one @ref XXH3_state_t to another.\n *\n * @param dst_state The state to copy to.\n * @param src_state The state to copy from.\n * @pre\n * @p dst_state and @p src_state must not be `NULL` and must not overlap.\n */\nXXH_PUBLIC_API void XXH3_copyState(XXH_NOESCAPE XXH3_state_t* dst_state, XXH_NOESCAPE const XXH3_state_t* src_state);\n\n/*!\n * @brief Resets an @ref XXH3_state_t to begin a new hash.\n *\n * @param statePtr The state struct to reset.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @note\n * - This function resets `statePtr` and generate a secret with default parameters.\n * - Call this function before @ref XXH3_64bits_update().\n * - Digest will be equivalent to `XXH3_64bits()`.\n *\n * @see @ref streaming_example \"Streaming Example\"\n *\n */\nXXH_PUBLIC_API XXH_errorcode XXH3_64bits_reset(XXH_NOESCAPE XXH3_state_t* statePtr);\n\n/*!\n * @brief Resets an @ref XXH3_state_t with 64-bit seed to begin a new hash.\n *\n * @param statePtr The state struct to reset.\n * @param seed The 64-bit seed to alter the hash result predictably.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @note\n * - This function resets `statePtr` and generate a secret from `seed`.\n * - Call this function before @ref XXH3_64bits_update().\n * - Digest will be equivalent to `XXH3_64bits_withSeed()`.\n *\n * @see @ref streaming_example \"Streaming Example\"\n *\n */\nXXH_PUBLIC_API XXH_errorcode XXH3_64bits_reset_withSeed(XXH_NOESCAPE XXH3_state_t* statePtr, XXH64_hash_t seed);\n\n/*!\n * @brief Resets an @ref XXH3_state_t with secret data to begin a new hash.\n *\n * @param statePtr The state struct to reset.\n * @param secret The secret data.\n * @param secretSize The length of @p secret, in bytes.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @note\n * `secret` is referenced, it _must outlive_ the hash streaming session.\n *\n * Similar to one-shot API, `secretSize` must be >= @ref XXH3_SECRET_SIZE_MIN,\n * and the quality of produced hash values depends on secret's entropy\n * (secret's content should look like a bunch of random bytes).\n * When in doubt about the randomness of a candidate `secret`,\n * consider employing `XXH3_generateSecret()` instead (see below).\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_errorcode XXH3_64bits_reset_withSecret(XXH_NOESCAPE XXH3_state_t* statePtr, XXH_NOESCAPE const void* secret, size_t secretSize);\n\n/*!\n * @brief Consumes a block of @p input to an @ref XXH3_state_t.\n *\n * @param statePtr The state struct to update.\n * @param input The block of data to be hashed, at least @p length bytes in size.\n * @param length The length of @p input, in bytes.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n * @pre\n * The memory between @p input and @p input + @p length must be valid,\n * readable, contiguous memory. However, if @p length is `0`, @p input may be\n * `NULL`. In C++, this also must be *TriviallyCopyable*.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @note Call this to incrementally consume blocks of data.\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_errorcode XXH3_64bits_update (XXH_NOESCAPE XXH3_state_t* statePtr, XXH_NOESCAPE const void* input, size_t length);\n\n/*!\n * @brief Returns the calculated XXH3 64-bit hash value from an @ref XXH3_state_t.\n *\n * @param statePtr The state struct to calculate the hash from.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n *\n * @return The calculated XXH3 64-bit hash value from that state.\n *\n * @note\n * Calling XXH3_64bits_digest() will not affect @p statePtr, so you can update,\n * digest, and update again.\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_PUREF XXH64_hash_t XXH3_64bits_digest (XXH_NOESCAPE const XXH3_state_t* statePtr);\n#endif /* !XXH_NO_STREAM */\n\n/* note : canonical representation of XXH3 is the same as XXH64\n * since they both produce XXH64_hash_t values */\n\n\n/*-**********************************************************************\n* XXH3 128-bit variant\n************************************************************************/\n\n/*!\n * @brief The return value from 128-bit hashes.\n *\n * Stored in little endian order, although the fields themselves are in native\n * endianness.\n */\ntypedef struct {\n XXH64_hash_t low64; /*!< `value & 0xFFFFFFFFFFFFFFFF` */\n XXH64_hash_t high64; /*!< `value >> 64` */\n} XXH128_hash_t;\n\n/*!\n * @brief Calculates 128-bit unseeded variant of XXH3 of @p data.\n *\n * @param data The block of data to be hashed, at least @p length bytes in size.\n * @param len The length of @p data, in bytes.\n *\n * @return The calculated 128-bit variant of XXH3 value.\n *\n * The 128-bit variant of XXH3 has more strength, but it has a bit of overhead\n * for shorter inputs.\n *\n * This is equivalent to @ref XXH3_128bits_withSeed() with a seed of `0`, however\n * it may have slightly better performance due to constant propagation of the\n * defaults.\n *\n * @see XXH3_128bits_withSeed(), XXH3_128bits_withSecret(): other seeding variants\n * @see @ref single_shot_example \"Single Shot Example\" for an example.\n */\nXXH_PUBLIC_API XXH_PUREF XXH128_hash_t XXH3_128bits(XXH_NOESCAPE const void* data, size_t len);\n/*! @brief Calculates 128-bit seeded variant of XXH3 hash of @p data.\n *\n * @param data The block of data to be hashed, at least @p length bytes in size.\n * @param len The length of @p data, in bytes.\n * @param seed The 64-bit seed to alter the hash result predictably.\n *\n * @return The calculated 128-bit variant of XXH3 value.\n *\n * @note\n * seed == 0 produces the same results as @ref XXH3_64bits().\n *\n * This variant generates a custom secret on the fly based on default secret\n * altered using the @p seed value.\n *\n * While this operation is decently fast, note that it's not completely free.\n *\n * @see XXH3_128bits(), XXH3_128bits_withSecret(): other seeding variants\n * @see @ref single_shot_example \"Single Shot Example\" for an example.\n */\nXXH_PUBLIC_API XXH_PUREF XXH128_hash_t XXH3_128bits_withSeed(XXH_NOESCAPE const void* data, size_t len, XXH64_hash_t seed);\n/*!\n * @brief Calculates 128-bit variant of XXH3 with a custom \"secret\".\n *\n * @param data The block of data to be hashed, at least @p len bytes in size.\n * @param len The length of @p data, in bytes.\n * @param secret The secret data.\n * @param secretSize The length of @p secret, in bytes.\n *\n * @return The calculated 128-bit variant of XXH3 value.\n *\n * It's possible to provide any blob of bytes as a \"secret\" to generate the hash.\n * This makes it more difficult for an external actor to prepare an intentional collision.\n * The main condition is that @p secretSize *must* be large enough (>= @ref XXH3_SECRET_SIZE_MIN).\n * However, the quality of the secret impacts the dispersion of the hash algorithm.\n * Therefore, the secret _must_ look like a bunch of random bytes.\n * Avoid \"trivial\" or structured data such as repeated sequences or a text document.\n * Whenever in doubt about the \"randomness\" of the blob of bytes,\n * consider employing @ref XXH3_generateSecret() instead (see below).\n * It will generate a proper high entropy secret derived from the blob of bytes.\n * Another advantage of using XXH3_generateSecret() is that\n * it guarantees that all bits within the initial blob of bytes\n * will impact every bit of the output.\n * This is not necessarily the case when using the blob of bytes directly\n * because, when hashing _small_ inputs, only a portion of the secret is employed.\n *\n * @see @ref single_shot_example \"Single Shot Example\" for an example.\n */\nXXH_PUBLIC_API XXH_PUREF XXH128_hash_t XXH3_128bits_withSecret(XXH_NOESCAPE const void* data, size_t len, XXH_NOESCAPE const void* secret, size_t secretSize);\n\n/******* Streaming *******/\n#ifndef XXH_NO_STREAM\n/*\n * Streaming requires state maintenance.\n * This operation costs memory and CPU.\n * As a consequence, streaming is slower than one-shot hashing.\n * For better performance, prefer one-shot functions whenever applicable.\n *\n * XXH3_128bits uses the same XXH3_state_t as XXH3_64bits().\n * Use already declared XXH3_createState() and XXH3_freeState().\n *\n * All reset and streaming functions have same meaning as their 64-bit counterpart.\n */\n\n/*!\n * @brief Resets an @ref XXH3_state_t to begin a new hash.\n *\n * @param statePtr The state struct to reset.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @note\n * - This function resets `statePtr` and generate a secret with default parameters.\n * - Call it before @ref XXH3_128bits_update().\n * - Digest will be equivalent to `XXH3_128bits()`.\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_errorcode XXH3_128bits_reset(XXH_NOESCAPE XXH3_state_t* statePtr);\n\n/*!\n * @brief Resets an @ref XXH3_state_t with 64-bit seed to begin a new hash.\n *\n * @param statePtr The state struct to reset.\n * @param seed The 64-bit seed to alter the hash result predictably.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @note\n * - This function resets `statePtr` and generate a secret from `seed`.\n * - Call it before @ref XXH3_128bits_update().\n * - Digest will be equivalent to `XXH3_128bits_withSeed()`.\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_errorcode XXH3_128bits_reset_withSeed(XXH_NOESCAPE XXH3_state_t* statePtr, XXH64_hash_t seed);\n/*!\n * @brief Resets an @ref XXH3_state_t with secret data to begin a new hash.\n *\n * @param statePtr The state struct to reset.\n * @param secret The secret data.\n * @param secretSize The length of @p secret, in bytes.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * `secret` is referenced, it _must outlive_ the hash streaming session.\n * Similar to one-shot API, `secretSize` must be >= @ref XXH3_SECRET_SIZE_MIN,\n * and the quality of produced hash values depends on secret's entropy\n * (secret's content should look like a bunch of random bytes).\n * When in doubt about the randomness of a candidate `secret`,\n * consider employing `XXH3_generateSecret()` instead (see below).\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_errorcode XXH3_128bits_reset_withSecret(XXH_NOESCAPE XXH3_state_t* statePtr, XXH_NOESCAPE const void* secret, size_t secretSize);\n\n/*!\n * @brief Consumes a block of @p input to an @ref XXH3_state_t.\n *\n * Call this to incrementally consume blocks of data.\n *\n * @param statePtr The state struct to update.\n * @param input The block of data to be hashed, at least @p length bytes in size.\n * @param length The length of @p input, in bytes.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @note\n * The memory between @p input and @p input + @p length must be valid,\n * readable, contiguous memory. However, if @p length is `0`, @p input may be\n * `NULL`. In C++, this also must be *TriviallyCopyable*.\n *\n */\nXXH_PUBLIC_API XXH_errorcode XXH3_128bits_update (XXH_NOESCAPE XXH3_state_t* statePtr, XXH_NOESCAPE const void* input, size_t length);\n\n/*!\n * @brief Returns the calculated XXH3 128-bit hash value from an @ref XXH3_state_t.\n *\n * @param statePtr The state struct to calculate the hash from.\n *\n * @pre\n * @p statePtr must not be `NULL`.\n *\n * @return The calculated XXH3 128-bit hash value from that state.\n *\n * @note\n * Calling XXH3_128bits_digest() will not affect @p statePtr, so you can update,\n * digest, and update again.\n *\n */\nXXH_PUBLIC_API XXH_PUREF XXH128_hash_t XXH3_128bits_digest (XXH_NOESCAPE const XXH3_state_t* statePtr);\n#endif /* !XXH_NO_STREAM */\n\n/* Following helper functions make it possible to compare XXH128_hast_t values.\n * Since XXH128_hash_t is a structure, this capability is not offered by the language.\n * Note: For better performance, these functions can be inlined using XXH_INLINE_ALL */\n\n/*!\n * @brief Check equality of two XXH128_hash_t values\n *\n * @param h1 The 128-bit hash value.\n * @param h2 Another 128-bit hash value.\n *\n * @return `1` if `h1` and `h2` are equal.\n * @return `0` if they are not.\n */\nXXH_PUBLIC_API XXH_PUREF int XXH128_isEqual(XXH128_hash_t h1, XXH128_hash_t h2);\n\n/*!\n * @brief Compares two @ref XXH128_hash_t\n *\n * This comparator is compatible with stdlib's `qsort()`/`bsearch()`.\n *\n * @param h128_1 Left-hand side value\n * @param h128_2 Right-hand side value\n *\n * @return >0 if @p h128_1 > @p h128_2\n * @return =0 if @p h128_1 == @p h128_2\n * @return <0 if @p h128_1 < @p h128_2\n */\nXXH_PUBLIC_API XXH_PUREF int XXH128_cmp(XXH_NOESCAPE const void* h128_1, XXH_NOESCAPE const void* h128_2);\n\n\n/******* Canonical representation *******/\ntypedef struct { unsigned char digest[sizeof(XXH128_hash_t)]; } XXH128_canonical_t;\n\n\n/*!\n * @brief Converts an @ref XXH128_hash_t to a big endian @ref XXH128_canonical_t.\n *\n * @param dst The @ref XXH128_canonical_t pointer to be stored to.\n * @param hash The @ref XXH128_hash_t to be converted.\n *\n * @pre\n * @p dst must not be `NULL`.\n * @see @ref canonical_representation_example \"Canonical Representation Example\"\n */\nXXH_PUBLIC_API void XXH128_canonicalFromHash(XXH_NOESCAPE XXH128_canonical_t* dst, XXH128_hash_t hash);\n\n/*!\n * @brief Converts an @ref XXH128_canonical_t to a native @ref XXH128_hash_t.\n *\n * @param src The @ref XXH128_canonical_t to convert.\n *\n * @pre\n * @p src must not be `NULL`.\n *\n * @return The converted hash.\n * @see @ref canonical_representation_example \"Canonical Representation Example\"\n */\nXXH_PUBLIC_API XXH_PUREF XXH128_hash_t XXH128_hashFromCanonical(XXH_NOESCAPE const XXH128_canonical_t* src);\n\n\n#endif /* !XXH_NO_XXH3 */\n#endif /* XXH_NO_LONG_LONG */\n\n/*!\n * @}\n */\n#endif /* XXHASH_H_5627135585666179 */\n\n\n\n#if defined(XXH_STATIC_LINKING_ONLY) && !defined(XXHASH_H_STATIC_13879238742)\n#define XXHASH_H_STATIC_13879238742\n/* ****************************************************************************\n * This section contains declarations which are not guaranteed to remain stable.\n * They may change in future versions, becoming incompatible with a different\n * version of the library.\n * These declarations should only be used with static linking.\n * Never use them in association with dynamic linking!\n ***************************************************************************** */\n\n/*\n * These definitions are only present to allow static allocation\n * of XXH states, on stack or in a struct, for example.\n * Never **ever** access their members directly.\n */\n\n/*!\n * @internal\n * @brief Structure for XXH32 streaming API.\n *\n * @note This is only defined when @ref XXH_STATIC_LINKING_ONLY,\n * @ref XXH_INLINE_ALL, or @ref XXH_IMPLEMENTATION is defined. Otherwise it is\n * an opaque type. This allows fields to safely be changed.\n *\n * Typedef'd to @ref XXH32_state_t.\n * Do not access the members of this struct directly.\n * @see XXH64_state_s, XXH3_state_s\n */\nstruct XXH32_state_s {\n XXH32_hash_t total_len_32; /*!< Total length hashed, modulo 2^32 */\n XXH32_hash_t large_len; /*!< Whether the hash is >= 16 (handles @ref total_len_32 overflow) */\n XXH32_hash_t acc[4]; /*!< Accumulator lanes */\n unsigned char buffer[16]; /*!< Internal buffer for partial reads. */\n XXH32_hash_t bufferedSize; /*!< Amount of data in @ref buffer */\n XXH32_hash_t reserved; /*!< Reserved field. Do not read nor write to it. */\n}; /* typedef'd to XXH32_state_t */\n\n\n#ifndef XXH_NO_LONG_LONG /* defined when there is no 64-bit support */\n\n/*!\n * @internal\n * @brief Structure for XXH64 streaming API.\n *\n * @note This is only defined when @ref XXH_STATIC_LINKING_ONLY,\n * @ref XXH_INLINE_ALL, or @ref XXH_IMPLEMENTATION is defined. Otherwise it is\n * an opaque type. This allows fields to safely be changed.\n *\n * Typedef'd to @ref XXH64_state_t.\n * Do not access the members of this struct directly.\n * @see XXH32_state_s, XXH3_state_s\n */\nstruct XXH64_state_s {\n XXH64_hash_t total_len; /*!< Total length hashed. This is always 64-bit. */\n XXH64_hash_t acc[4]; /*!< Accumulator lanes */\n unsigned char buffer[32]; /*!< Internal buffer for partial reads.. */\n XXH32_hash_t bufferedSize; /*!< Amount of data in @ref buffer */\n XXH32_hash_t reserved32; /*!< Reserved field, needed for padding anyways*/\n XXH64_hash_t reserved64; /*!< Reserved field. Do not read or write to it. */\n}; /* typedef'd to XXH64_state_t */\n\n#ifndef XXH_NO_XXH3\n\n#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 201112L) /* >= C11 */\n# define XXH_ALIGN(n) _Alignas(n)\n#elif defined(__cplusplus) && (__cplusplus >= 201103L) /* >= C++11 */\n/* In C++ alignas() is a keyword */\n# define XXH_ALIGN(n) alignas(n)\n#elif defined(__GNUC__)\n# define XXH_ALIGN(n) __attribute__ ((aligned(n)))\n#elif defined(_MSC_VER)\n# define XXH_ALIGN(n) __declspec(align(n))\n#else\n# define XXH_ALIGN(n) /* disabled */\n#endif\n\n/* Old GCC versions only accept the attribute after the type in structures. */\n#if !(defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 201112L)) /* C11+ */ \\\n && ! (defined(__cplusplus) && (__cplusplus >= 201103L)) /* >= C++11 */ \\\n && defined(__GNUC__)\n# define XXH_ALIGN_MEMBER(align, type) type XXH_ALIGN(align)\n#else\n# define XXH_ALIGN_MEMBER(align, type) XXH_ALIGN(align) type\n#endif\n\n/*!\n * @brief The size of the internal XXH3 buffer.\n *\n * This is the optimal update size for incremental hashing.\n *\n * @see XXH3_64b_update(), XXH3_128b_update().\n */\n#define XXH3_INTERNALBUFFER_SIZE 256\n\n/*!\n * @internal\n * @brief Default size of the secret buffer (and @ref XXH3_kSecret).\n *\n * This is the size used in @ref XXH3_kSecret and the seeded functions.\n *\n * Not to be confused with @ref XXH3_SECRET_SIZE_MIN.\n */\n#define XXH3_SECRET_DEFAULT_SIZE 192\n\n/*!\n * @internal\n * @brief Structure for XXH3 streaming API.\n *\n * @note This is only defined when @ref XXH_STATIC_LINKING_ONLY,\n * @ref XXH_INLINE_ALL, or @ref XXH_IMPLEMENTATION is defined.\n * Otherwise it is an opaque type.\n * Never use this definition in combination with dynamic library.\n * This allows fields to safely be changed in the future.\n *\n * @note ** This structure has a strict alignment requirement of 64 bytes!! **\n * Do not allocate this with `malloc()` or `new`,\n * it will not be sufficiently aligned.\n * Use @ref XXH3_createState() and @ref XXH3_freeState(), or stack allocation.\n *\n * Typedef'd to @ref XXH3_state_t.\n * Do never access the members of this struct directly.\n *\n * @see XXH3_INITSTATE() for stack initialization.\n * @see XXH3_createState(), XXH3_freeState().\n * @see XXH32_state_s, XXH64_state_s\n */\nstruct XXH3_state_s {\n XXH_ALIGN_MEMBER(64, XXH64_hash_t acc[8]);\n /*!< The 8 accumulators. See @ref XXH32_state_s::v and @ref XXH64_state_s::v */\n XXH_ALIGN_MEMBER(64, unsigned char customSecret[XXH3_SECRET_DEFAULT_SIZE]);\n /*!< Used to store a custom secret generated from a seed. */\n XXH_ALIGN_MEMBER(64, unsigned char buffer[XXH3_INTERNALBUFFER_SIZE]);\n /*!< The internal buffer. @see XXH32_state_s::mem32 */\n XXH32_hash_t bufferedSize;\n /*!< The amount of memory in @ref buffer, @see XXH32_state_s::memsize */\n XXH32_hash_t useSeed;\n /*!< Reserved field. Needed for padding on 64-bit. */\n size_t nbStripesSoFar;\n /*!< Number or stripes processed. */\n XXH64_hash_t totalLen;\n /*!< Total length hashed. 64-bit even on 32-bit targets. */\n size_t nbStripesPerBlock;\n /*!< Number of stripes per block. */\n size_t secretLimit;\n /*!< Size of @ref customSecret or @ref extSecret */\n XXH64_hash_t seed;\n /*!< Seed for _withSeed variants. Must be zero otherwise, @see XXH3_INITSTATE() */\n XXH64_hash_t reserved64;\n /*!< Reserved field. */\n const unsigned char* extSecret;\n /*!< Reference to an external secret for the _withSecret variants, NULL\n * for other variants. */\n /* note: there may be some padding at the end due to alignment on 64 bytes */\n}; /* typedef'd to XXH3_state_t */\n\n#undef XXH_ALIGN_MEMBER\n\n/*!\n * @brief Initializes a stack-allocated `XXH3_state_s`.\n *\n * When the @ref XXH3_state_t structure is merely emplaced on stack,\n * it should be initialized with XXH3_INITSTATE() or a memset()\n * in case its first reset uses XXH3_NNbits_reset_withSeed().\n * This init can be omitted if the first reset uses default or _withSecret mode.\n * This operation isn't necessary when the state is created with XXH3_createState().\n * Note that this doesn't prepare the state for a streaming operation,\n * it's still necessary to use XXH3_NNbits_reset*() afterwards.\n */\n#define XXH3_INITSTATE(XXH3_state_ptr) \\\n do { \\\n XXH3_state_t* tmp_xxh3_state_ptr = (XXH3_state_ptr); \\\n tmp_xxh3_state_ptr->seed = 0; \\\n tmp_xxh3_state_ptr->extSecret = NULL; \\\n } while(0)\n\n\n/*!\n * @brief Calculates the 128-bit hash of @p data using XXH3.\n *\n * @param data The block of data to be hashed, at least @p len bytes in size.\n * @param len The length of @p data, in bytes.\n * @param seed The 64-bit seed to alter the hash's output predictably.\n *\n * @pre\n * The memory between @p data and @p data + @p len must be valid,\n * readable, contiguous memory. However, if @p len is `0`, @p data may be\n * `NULL`. In C++, this also must be *TriviallyCopyable*.\n *\n * @return The calculated 128-bit XXH3 value.\n *\n * @see @ref single_shot_example \"Single Shot Example\" for an example.\n */\nXXH_PUBLIC_API XXH_PUREF XXH128_hash_t XXH128(XXH_NOESCAPE const void* data, size_t len, XXH64_hash_t seed);\n\n\n/* === Experimental API === */\n/* Symbols defined below must be considered tied to a specific library version. */\n\n/*!\n * @brief Derive a high-entropy secret from any user-defined content, named customSeed.\n *\n * @param secretBuffer A writable buffer for derived high-entropy secret data.\n * @param secretSize Size of secretBuffer, in bytes. Must be >= XXH3_SECRET_SIZE_MIN.\n * @param customSeed A user-defined content.\n * @param customSeedSize Size of customSeed, in bytes.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * The generated secret can be used in combination with `*_withSecret()` functions.\n * The `_withSecret()` variants are useful to provide a higher level of protection\n * than 64-bit seed, as it becomes much more difficult for an external actor to\n * guess how to impact the calculation logic.\n *\n * The function accepts as input a custom seed of any length and any content,\n * and derives from it a high-entropy secret of length @p secretSize into an\n * already allocated buffer @p secretBuffer.\n *\n * The generated secret can then be used with any `*_withSecret()` variant.\n * The functions @ref XXH3_128bits_withSecret(), @ref XXH3_64bits_withSecret(),\n * @ref XXH3_128bits_reset_withSecret() and @ref XXH3_64bits_reset_withSecret()\n * are part of this list. They all accept a `secret` parameter\n * which must be large enough for implementation reasons (>= @ref XXH3_SECRET_SIZE_MIN)\n * _and_ feature very high entropy (consist of random-looking bytes).\n * These conditions can be a high bar to meet, so @ref XXH3_generateSecret() can\n * be employed to ensure proper quality.\n *\n * @p customSeed can be anything. It can have any size, even small ones,\n * and its content can be anything, even \"poor entropy\" sources such as a bunch\n * of zeroes. The resulting `secret` will nonetheless provide all required qualities.\n *\n * @pre\n * - @p secretSize must be >= @ref XXH3_SECRET_SIZE_MIN\n * - When @p customSeedSize > 0, supplying NULL as customSeed is undefined behavior.\n *\n * Example code:\n * @code{.c}\n * #include \n * #include \n * #include \n * #define XXH_STATIC_LINKING_ONLY // expose unstable API\n * #include \"xxhash.h\"\n * // Hashes argv[2] using the entropy from argv[1].\n * int main(int argc, char* argv[])\n * {\n * char secret[XXH3_SECRET_SIZE_MIN];\n * if (argv != 3) { return 1; }\n * XXH3_generateSecret(secret, sizeof(secret), argv[1], strlen(argv[1]));\n * XXH64_hash_t h = XXH3_64bits_withSecret(\n * argv[2], strlen(argv[2]),\n * secret, sizeof(secret)\n * );\n * printf(\"%016llx\\n\", (unsigned long long) h);\n * }\n * @endcode\n */\nXXH_PUBLIC_API XXH_errorcode XXH3_generateSecret(XXH_NOESCAPE void* secretBuffer, size_t secretSize, XXH_NOESCAPE const void* customSeed, size_t customSeedSize);\n\n/*!\n * @brief Generate the same secret as the _withSeed() variants.\n *\n * @param secretBuffer A writable buffer of @ref XXH3_SECRET_DEFAULT_SIZE bytes\n * @param seed The 64-bit seed to alter the hash result predictably.\n *\n * The generated secret can be used in combination with\n *`*_withSecret()` and `_withSecretandSeed()` variants.\n *\n * Example C++ `std::string` hash class:\n * @code{.cpp}\n * #include \n * #define XXH_STATIC_LINKING_ONLY // expose unstable API\n * #include \"xxhash.h\"\n * // Slow, seeds each time\n * class HashSlow {\n * XXH64_hash_t seed;\n * public:\n * HashSlow(XXH64_hash_t s) : seed{s} {}\n * size_t operator()(const std::string& x) const {\n * return size_t{XXH3_64bits_withSeed(x.c_str(), x.length(), seed)};\n * }\n * };\n * // Fast, caches the seeded secret for future uses.\n * class HashFast {\n * unsigned char secret[XXH3_SECRET_DEFAULT_SIZE];\n * public:\n * HashFast(XXH64_hash_t s) {\n * XXH3_generateSecret_fromSeed(secret, seed);\n * }\n * size_t operator()(const std::string& x) const {\n * return size_t{\n * XXH3_64bits_withSecret(x.c_str(), x.length(), secret, sizeof(secret))\n * };\n * }\n * };\n * @endcode\n */\nXXH_PUBLIC_API void XXH3_generateSecret_fromSeed(XXH_NOESCAPE void* secretBuffer, XXH64_hash_t seed);\n\n/*!\n * @brief Maximum size of \"short\" key in bytes.\n */\n#define XXH3_MIDSIZE_MAX 240\n\n/*!\n * @brief Calculates 64/128-bit seeded variant of XXH3 hash of @p data.\n *\n * @param data The block of data to be hashed, at least @p len bytes in size.\n * @param len The length of @p data, in bytes.\n * @param secret The secret data.\n * @param secretSize The length of @p secret, in bytes.\n * @param seed The 64-bit seed to alter the hash result predictably.\n *\n * These variants generate hash values using either:\n * - @p seed for \"short\" keys (< @ref XXH3_MIDSIZE_MAX = 240 bytes)\n * - @p secret for \"large\" keys (>= @ref XXH3_MIDSIZE_MAX).\n *\n * This generally benefits speed, compared to `_withSeed()` or `_withSecret()`.\n * `_withSeed()` has to generate the secret on the fly for \"large\" keys.\n * It's fast, but can be perceptible for \"not so large\" keys (< 1 KB).\n * `_withSecret()` has to generate the masks on the fly for \"small\" keys,\n * which requires more instructions than _withSeed() variants.\n * Therefore, _withSecretandSeed variant combines the best of both worlds.\n *\n * When @p secret has been generated by XXH3_generateSecret_fromSeed(),\n * this variant produces *exactly* the same results as `_withSeed()` variant,\n * hence offering only a pure speed benefit on \"large\" input,\n * by skipping the need to regenerate the secret for every large input.\n *\n * Another usage scenario is to hash the secret to a 64-bit hash value,\n * for example with XXH3_64bits(), which then becomes the seed,\n * and then employ both the seed and the secret in _withSecretandSeed().\n * On top of speed, an added benefit is that each bit in the secret\n * has a 50% chance to swap each bit in the output, via its impact to the seed.\n *\n * This is not guaranteed when using the secret directly in \"small data\" scenarios,\n * because only portions of the secret are employed for small data.\n */\nXXH_PUBLIC_API XXH_PUREF XXH64_hash_t\nXXH3_64bits_withSecretandSeed(XXH_NOESCAPE const void* data, size_t len,\n XXH_NOESCAPE const void* secret, size_t secretSize,\n XXH64_hash_t seed);\n\n/*!\n * @brief Calculates 128-bit seeded variant of XXH3 hash of @p data.\n *\n * @param data The memory segment to be hashed, at least @p len bytes in size.\n * @param length The length of @p data, in bytes.\n * @param secret The secret used to alter hash result predictably.\n * @param secretSize The length of @p secret, in bytes (must be >= XXH3_SECRET_SIZE_MIN)\n * @param seed64 The 64-bit seed to alter the hash result predictably.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @see XXH3_64bits_withSecretandSeed(): contract is the same.\n */\nXXH_PUBLIC_API XXH_PUREF XXH128_hash_t\nXXH3_128bits_withSecretandSeed(XXH_NOESCAPE const void* input, size_t length,\n XXH_NOESCAPE const void* secret, size_t secretSize,\n XXH64_hash_t seed64);\n\n#ifndef XXH_NO_STREAM\n/*!\n * @brief Resets an @ref XXH3_state_t with secret data to begin a new hash.\n *\n * @param statePtr A pointer to an @ref XXH3_state_t allocated with @ref XXH3_createState().\n * @param secret The secret data.\n * @param secretSize The length of @p secret, in bytes.\n * @param seed64 The 64-bit seed to alter the hash result predictably.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @see XXH3_64bits_withSecretandSeed(). Contract is identical.\n */\nXXH_PUBLIC_API XXH_errorcode\nXXH3_64bits_reset_withSecretandSeed(XXH_NOESCAPE XXH3_state_t* statePtr,\n XXH_NOESCAPE const void* secret, size_t secretSize,\n XXH64_hash_t seed64);\n\n/*!\n * @brief Resets an @ref XXH3_state_t with secret data to begin a new hash.\n *\n * @param statePtr A pointer to an @ref XXH3_state_t allocated with @ref XXH3_createState().\n * @param secret The secret data.\n * @param secretSize The length of @p secret, in bytes.\n * @param seed64 The 64-bit seed to alter the hash result predictably.\n *\n * @return @ref XXH_OK on success.\n * @return @ref XXH_ERROR on failure.\n *\n * @see XXH3_64bits_withSecretandSeed(). Contract is identical.\n *\n * Note: there was a bug in an earlier version of this function (<= v0.8.2)\n * that would make it generate an incorrect hash value\n * when @p seed == 0 and @p length < XXH3_MIDSIZE_MAX\n * and @p secret is different from XXH3_generateSecret_fromSeed().\n * As stated in the contract, the correct hash result must be\n * the same as XXH3_128bits_withSeed() when @p length <= XXH3_MIDSIZE_MAX.\n * Results generated by this older version are wrong, hence not comparable.\n */\nXXH_PUBLIC_API XXH_errorcode\nXXH3_128bits_reset_withSecretandSeed(XXH_NOESCAPE XXH3_state_t* statePtr,\n XXH_NOESCAPE const void* secret, size_t secretSize,\n XXH64_hash_t seed64);\n\n#endif /* !XXH_NO_STREAM */\n\n#endif /* !XXH_NO_XXH3 */\n#endif /* XXH_NO_LONG_LONG */\n#if defined(XXH_INLINE_ALL) || defined(XXH_PRIVATE_API)\n# define XXH_IMPLEMENTATION\n#endif\n\n#endif /* defined(XXH_STATIC_LINKING_ONLY) && !defined(XXHASH_H_STATIC_13879238742) */\n\n\n/* ======================================================================== */\n/* ======================================================================== */\n/* ======================================================================== */\n\n\n/*-**********************************************************************\n * xxHash implementation\n *-**********************************************************************\n * xxHash's implementation used to be hosted inside xxhash.c.\n *\n * However, inlining requires implementation to be visible to the compiler,\n * hence be included alongside the header.\n * Previously, implementation was hosted inside xxhash.c,\n * which was then #included when inlining was activated.\n * This construction created issues with a few build and install systems,\n * as it required xxhash.c to be stored in /include directory.\n *\n * xxHash implementation is now directly integrated within xxhash.h.\n * As a consequence, xxhash.c is no longer needed in /include.\n *\n * xxhash.c is still available and is still useful.\n * In a \"normal\" setup, when xxhash is not inlined,\n * xxhash.h only exposes the prototypes and public symbols,\n * while xxhash.c can be built into an object file xxhash.o\n * which can then be linked into the final binary.\n ************************************************************************/\n\n#if ( defined(XXH_INLINE_ALL) || defined(XXH_PRIVATE_API) \\\n || defined(XXH_IMPLEMENTATION) ) && !defined(XXH_IMPLEM_13a8737387)\n# define XXH_IMPLEM_13a8737387\n\n/* *************************************\n* Tuning parameters\n***************************************/\n\n/*!\n * @defgroup tuning Tuning parameters\n * @{\n *\n * Various macros to control xxHash's behavior.\n */\n#ifdef XXH_DOXYGEN\n/*!\n * @brief Define this to disable 64-bit code.\n *\n * Useful if only using the @ref XXH32_family and you have a strict C90 compiler.\n */\n# define XXH_NO_LONG_LONG\n# undef XXH_NO_LONG_LONG /* don't actually */\n/*!\n * @brief Controls how unaligned memory is accessed.\n *\n * By default, access to unaligned memory is controlled by `memcpy()`, which is\n * safe and portable.\n *\n * Unfortunately, on some target/compiler combinations, the generated assembly\n * is sub-optimal.\n *\n * The below switch allow selection of a different access method\n * in the search for improved performance.\n *\n * @par Possible options:\n *\n * - `XXH_FORCE_MEMORY_ACCESS=0` (default): `memcpy`\n * @par\n * Use `memcpy()`. Safe and portable. Note that most modern compilers will\n * eliminate the function call and treat it as an unaligned access.\n *\n * - `XXH_FORCE_MEMORY_ACCESS=1`: `__attribute__((aligned(1)))`\n * @par\n * Depends on compiler extensions and is therefore not portable.\n * This method is safe _if_ your compiler supports it,\n * and *generally* as fast or faster than `memcpy`.\n *\n * - `XXH_FORCE_MEMORY_ACCESS=2`: Direct cast\n * @par\n * Casts directly and dereferences. This method doesn't depend on the\n * compiler, but it violates the C standard as it directly dereferences an\n * unaligned pointer. It can generate buggy code on targets which do not\n * support unaligned memory accesses, but in some circumstances, it's the\n * only known way to get the most performance.\n *\n * - `XXH_FORCE_MEMORY_ACCESS=3`: Byteshift\n * @par\n * Also portable. This can generate the best code on old compilers which don't\n * inline small `memcpy()` calls, and it might also be faster on big-endian\n * systems which lack a native byteswap instruction. However, some compilers\n * will emit literal byteshifts even if the target supports unaligned access.\n *\n *\n * @warning\n * Methods 1 and 2 rely on implementation-defined behavior. Use these with\n * care, as what works on one compiler/platform/optimization level may cause\n * another to read garbage data or even crash.\n *\n * See https://fastcompression.blogspot.com/2015/08/accessing-unaligned-memory.html for details.\n *\n * Prefer these methods in priority order (0 > 3 > 1 > 2)\n */\n# define XXH_FORCE_MEMORY_ACCESS 0\n\n/*!\n * @def XXH_SIZE_OPT\n * @brief Controls how much xxHash optimizes for size.\n *\n * xxHash, when compiled, tends to result in a rather large binary size. This\n * is mostly due to heavy usage to forced inlining and constant folding of the\n * @ref XXH3_family to increase performance.\n *\n * However, some developers prefer size over speed. This option can\n * significantly reduce the size of the generated code. When using the `-Os`\n * or `-Oz` options on GCC or Clang, this is defined to 1 by default,\n * otherwise it is defined to 0.\n *\n * Most of these size optimizations can be controlled manually.\n *\n * This is a number from 0-2.\n * - `XXH_SIZE_OPT` == 0: Default. xxHash makes no size optimizations. Speed\n * comes first.\n * - `XXH_SIZE_OPT` == 1: Default for `-Os` and `-Oz`. xxHash is more\n * conservative and disables hacks that increase code size. It implies the\n * options @ref XXH_NO_INLINE_HINTS == 1, @ref XXH_FORCE_ALIGN_CHECK == 0,\n * and @ref XXH3_NEON_LANES == 8 if they are not already defined.\n * - `XXH_SIZE_OPT` == 2: xxHash tries to make itself as small as possible.\n * Performance may cry. For example, the single shot functions just use the\n * streaming API.\n */\n# define XXH_SIZE_OPT 0\n\n/*!\n * @def XXH_FORCE_ALIGN_CHECK\n * @brief If defined to non-zero, adds a special path for aligned inputs (XXH32()\n * and XXH64() only).\n *\n * This is an important performance trick for architectures without decent\n * unaligned memory access performance.\n *\n * It checks for input alignment, and when conditions are met, uses a \"fast\n * path\" employing direct 32-bit/64-bit reads, resulting in _dramatically\n * faster_ read speed.\n *\n * The check costs one initial branch per hash, which is generally negligible,\n * but not zero.\n *\n * Moreover, it's not useful to generate an additional code path if memory\n * access uses the same instruction for both aligned and unaligned\n * addresses (e.g. x86 and aarch64).\n *\n * In these cases, the alignment check can be removed by setting this macro to 0.\n * Then the code will always use unaligned memory access.\n * Align check is automatically disabled on x86, x64, ARM64, and some ARM chips\n * which are platforms known to offer good unaligned memory accesses performance.\n *\n * It is also disabled by default when @ref XXH_SIZE_OPT >= 1.\n *\n * This option does not affect XXH3 (only XXH32 and XXH64).\n */\n# define XXH_FORCE_ALIGN_CHECK 0\n\n/*!\n * @def XXH_NO_INLINE_HINTS\n * @brief When non-zero, sets all functions to `static`.\n *\n * By default, xxHash tries to force the compiler to inline almost all internal\n * functions.\n *\n * This can usually improve performance due to reduced jumping and improved\n * constant folding, but significantly increases the size of the binary which\n * might not be favorable.\n *\n * Additionally, sometimes the forced inlining can be detrimental to performance,\n * depending on the architecture.\n *\n * XXH_NO_INLINE_HINTS marks all internal functions as static, giving the\n * compiler full control on whether to inline or not.\n *\n * When not optimizing (-O0), using `-fno-inline` with GCC or Clang, or if\n * @ref XXH_SIZE_OPT >= 1, this will automatically be defined.\n */\n# define XXH_NO_INLINE_HINTS 0\n\n/*!\n * @def XXH3_INLINE_SECRET\n * @brief Determines whether to inline the XXH3 withSecret code.\n *\n * When the secret size is known, the compiler can improve the performance\n * of XXH3_64bits_withSecret() and XXH3_128bits_withSecret().\n *\n * However, if the secret size is not known, it doesn't have any benefit. This\n * happens when xxHash is compiled into a global symbol. Therefore, if\n * @ref XXH_INLINE_ALL is *not* defined, this will be defined to 0.\n *\n * Additionally, this defaults to 0 on GCC 12+, which has an issue with function pointers\n * that are *sometimes* force inline on -Og, and it is impossible to automatically\n * detect this optimization level.\n */\n# define XXH3_INLINE_SECRET 0\n\n/*!\n * @def XXH32_ENDJMP\n * @brief Whether to use a jump for `XXH32_finalize`.\n *\n * For performance, `XXH32_finalize` uses multiple branches in the finalizer.\n * This is generally preferable for performance,\n * but depending on exact architecture, a jmp may be preferable.\n *\n * This setting is only possibly making a difference for very small inputs.\n */\n# define XXH32_ENDJMP 0\n\n/*!\n * @internal\n * @brief Redefines old internal names.\n *\n * For compatibility with code that uses xxHash's internals before the names\n * were changed to improve namespacing. There is no other reason to use this.\n */\n# define XXH_OLD_NAMES\n# undef XXH_OLD_NAMES /* don't actually use, it is ugly. */\n\n/*!\n * @def XXH_NO_STREAM\n * @brief Disables the streaming API.\n *\n * When xxHash is not inlined and the streaming functions are not used, disabling\n * the streaming functions can improve code size significantly, especially with\n * the @ref XXH3_family which tends to make constant folded copies of itself.\n */\n# define XXH_NO_STREAM\n# undef XXH_NO_STREAM /* don't actually */\n#endif /* XXH_DOXYGEN */\n/*!\n * @}\n */\n\n#ifndef XXH_FORCE_MEMORY_ACCESS /* can be defined externally, on command line for example */\n /* prefer __packed__ structures (method 1) for GCC\n * < ARMv7 with unaligned access (e.g. Raspbian armhf) still uses byte shifting, so we use memcpy\n * which for some reason does unaligned loads. */\n# if defined(__GNUC__) && !(defined(__ARM_ARCH) && __ARM_ARCH < 7 && defined(__ARM_FEATURE_UNALIGNED))\n# define XXH_FORCE_MEMORY_ACCESS 1\n# endif\n#endif\n\n#ifndef XXH_SIZE_OPT\n /* default to 1 for -Os or -Oz */\n# if (defined(__GNUC__) || defined(__clang__)) && defined(__OPTIMIZE_SIZE__)\n# define XXH_SIZE_OPT 1\n# else\n# define XXH_SIZE_OPT 0\n# endif\n#endif\n\n#ifndef XXH_FORCE_ALIGN_CHECK /* can be defined externally */\n /* don't check on sizeopt, x86, aarch64, or arm when unaligned access is available */\n# if XXH_SIZE_OPT >= 1 || \\\n defined(__i386) || defined(__x86_64__) || defined(__aarch64__) || defined(__ARM_FEATURE_UNALIGNED) \\\n || defined(_M_IX86) || defined(_M_X64) || defined(_M_ARM64) || defined(_M_ARM) /* visual */\n# define XXH_FORCE_ALIGN_CHECK 0\n# else\n# define XXH_FORCE_ALIGN_CHECK 1\n# endif\n#endif\n\n#ifndef XXH_NO_INLINE_HINTS\n# if XXH_SIZE_OPT >= 1 || defined(__NO_INLINE__) /* -O0, -fno-inline */\n# define XXH_NO_INLINE_HINTS 1\n# else\n# define XXH_NO_INLINE_HINTS 0\n# endif\n#endif\n\n#ifndef XXH3_INLINE_SECRET\n# if (defined(__GNUC__) && !defined(__clang__) && __GNUC__ >= 12) \\\n || !defined(XXH_INLINE_ALL)\n# define XXH3_INLINE_SECRET 0\n# else\n# define XXH3_INLINE_SECRET 1\n# endif\n#endif\n\n#ifndef XXH32_ENDJMP\n/* generally preferable for performance */\n# define XXH32_ENDJMP 0\n#endif\n\n/*!\n * @defgroup impl Implementation\n * @{\n */\n\n\n/* *************************************\n* Includes & Memory related functions\n***************************************/\n#if defined(XXH_NO_STREAM)\n/* nothing */\n#elif defined(XXH_NO_STDLIB)\n\n/* When requesting to disable any mention of stdlib,\n * the library loses the ability to invoked malloc / free.\n * In practice, it means that functions like `XXH*_createState()`\n * will always fail, and return NULL.\n * This flag is useful in situations where\n * xxhash.h is integrated into some kernel, embedded or limited environment\n * without access to dynamic allocation.\n */\n\nstatic XXH_CONSTF void* XXH_malloc(size_t s) { (void)s; return NULL; }\nstatic void XXH_free(void* p) { (void)p; }\n\n#else\n\n/*\n * Modify the local functions below should you wish to use\n * different memory routines for malloc() and free()\n */\n#include \n\n/*!\n * @internal\n * @brief Modify this function to use a different routine than malloc().\n */\nstatic XXH_MALLOCF void* XXH_malloc(size_t s) { return malloc(s); }\n\n/*!\n * @internal\n * @brief Modify this function to use a different routine than free().\n */\nstatic void XXH_free(void* p) { free(p); }\n\n#endif /* XXH_NO_STDLIB */\n\n#include \n\n/*!\n * @internal\n * @brief Modify this function to use a different routine than memcpy().\n */\nstatic void* XXH_memcpy(void* dest, const void* src, size_t size)\n{\n return memcpy(dest,src,size);\n}\n\n#include /* ULLONG_MAX */\n\n\n/* *************************************\n* Compiler Specific Options\n***************************************/\n#ifdef _MSC_VER /* Visual Studio warning fix */\n# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */\n#endif\n\n#if XXH_NO_INLINE_HINTS /* disable inlining hints */\n# if defined(__GNUC__) || defined(__clang__)\n# define XXH_FORCE_INLINE static __attribute__((__unused__))\n# else\n# define XXH_FORCE_INLINE static\n# endif\n# define XXH_NO_INLINE static\n/* enable inlining hints */\n#elif defined(__GNUC__) || defined(__clang__)\n# define XXH_FORCE_INLINE static __inline__ __attribute__((__always_inline__, __unused__))\n# define XXH_NO_INLINE static __attribute__((__noinline__))\n#elif defined(_MSC_VER) /* Visual Studio */\n# define XXH_FORCE_INLINE static __forceinline\n# define XXH_NO_INLINE static __declspec(noinline)\n#elif defined (__cplusplus) \\\n || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L)) /* C99 */\n# define XXH_FORCE_INLINE static inline\n# define XXH_NO_INLINE static\n#else\n# define XXH_FORCE_INLINE static\n# define XXH_NO_INLINE static\n#endif\n\n#if defined(XXH_INLINE_ALL)\n# define XXH_STATIC XXH_FORCE_INLINE\n#else\n# define XXH_STATIC static\n#endif\n\n#if XXH3_INLINE_SECRET\n# define XXH3_WITH_SECRET_INLINE XXH_FORCE_INLINE\n#else\n# define XXH3_WITH_SECRET_INLINE XXH_NO_INLINE\n#endif\n\n#if ((defined(sun) || defined(__sun)) && __cplusplus) /* Solaris includes __STDC_VERSION__ with C++. Tested with GCC 5.5 */\n# define XXH_RESTRICT /* disable */\n#elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L /* >= C99 */\n# define XXH_RESTRICT restrict\n#elif (defined (__GNUC__) && ((__GNUC__ > 3) || (__GNUC__ == 3 && __GNUC_MINOR__ >= 1))) \\\n || (defined (__clang__)) \\\n || (defined (_MSC_VER) && (_MSC_VER >= 1400)) \\\n || (defined (__INTEL_COMPILER) && (__INTEL_COMPILER >= 1300))\n/*\n * There are a LOT more compilers that recognize __restrict but this\n * covers the major ones.\n */\n# define XXH_RESTRICT __restrict\n#else\n# define XXH_RESTRICT /* disable */\n#endif\n\n/* *************************************\n* Debug\n***************************************/\n/*!\n * @ingroup tuning\n * @def XXH_DEBUGLEVEL\n * @brief Sets the debugging level.\n *\n * XXH_DEBUGLEVEL is expected to be defined externally, typically via the\n * compiler's command line options. The value must be a number.\n */\n#ifndef XXH_DEBUGLEVEL\n# ifdef DEBUGLEVEL /* backwards compat */\n# define XXH_DEBUGLEVEL DEBUGLEVEL\n# else\n# define XXH_DEBUGLEVEL 0\n# endif\n#endif\n\n#if (XXH_DEBUGLEVEL>=1)\n# include /* note: can still be disabled with NDEBUG */\n# define XXH_ASSERT(c) assert(c)\n#else\n# if defined(__INTEL_COMPILER)\n# define XXH_ASSERT(c) XXH_ASSUME((unsigned char) (c))\n# else\n# define XXH_ASSERT(c) XXH_ASSUME(c)\n# endif\n#endif\n\n/* note: use after variable declarations */\n#ifndef XXH_STATIC_ASSERT\n# if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 201112L) /* C11 */\n# define XXH_STATIC_ASSERT_WITH_MESSAGE(c,m) do { _Static_assert((c),m); } while(0)\n# elif defined(__cplusplus) && (__cplusplus >= 201103L) /* C++11 */\n# define XXH_STATIC_ASSERT_WITH_MESSAGE(c,m) do { static_assert((c),m); } while(0)\n# else\n# define XXH_STATIC_ASSERT_WITH_MESSAGE(c,m) do { struct xxh_sa { char x[(c) ? 1 : -1]; }; } while(0)\n# endif\n# define XXH_STATIC_ASSERT(c) XXH_STATIC_ASSERT_WITH_MESSAGE((c),#c)\n#endif\n\n/*!\n * @internal\n * @def XXH_COMPILER_GUARD(var)\n * @brief Used to prevent unwanted optimizations for @p var.\n *\n * It uses an empty GCC inline assembly statement with a register constraint\n * which forces @p var into a general purpose register (eg eax, ebx, ecx\n * on x86) and marks it as modified.\n *\n * This is used in a few places to avoid unwanted autovectorization (e.g.\n * XXH32_round()). All vectorization we want is explicit via intrinsics,\n * and _usually_ isn't wanted elsewhere.\n *\n * We also use it to prevent unwanted constant folding for AArch64 in\n * XXH3_initCustomSecret_scalar().\n */\n#if defined(__GNUC__) || defined(__clang__)\n# define XXH_COMPILER_GUARD(var) __asm__(\"\" : \"+r\" (var))\n#else\n# define XXH_COMPILER_GUARD(var) ((void)0)\n#endif\n\n/* Specifically for NEON vectors which use the \"w\" constraint, on\n * Clang. */\n#if defined(__clang__) && defined(__ARM_ARCH) && !defined(__wasm__)\n# define XXH_COMPILER_GUARD_CLANG_NEON(var) __asm__(\"\" : \"+w\" (var))\n#else\n# define XXH_COMPILER_GUARD_CLANG_NEON(var) ((void)0)\n#endif\n\n/* *************************************\n* Basic Types\n***************************************/\n#if !defined (__VMS) \\\n && (defined (__cplusplus) \\\n || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) )\n# ifdef _AIX\n# include \n# else\n# include \n# endif\n typedef uint8_t xxh_u8;\n#else\n typedef unsigned char xxh_u8;\n#endif\ntypedef XXH32_hash_t xxh_u32;\n\n#ifdef XXH_OLD_NAMES\n# warning \"XXH_OLD_NAMES is planned to be removed starting v0.9. If the program depends on it, consider moving away from it by employing newer type names directly\"\n# define BYTE xxh_u8\n# define U8 xxh_u8\n# define U32 xxh_u32\n#endif\n\n/* *** Memory access *** */\n\n/*!\n * @internal\n * @fn xxh_u32 XXH_read32(const void* ptr)\n * @brief Reads an unaligned 32-bit integer from @p ptr in native endianness.\n *\n * Affected by @ref XXH_FORCE_MEMORY_ACCESS.\n *\n * @param ptr The pointer to read from.\n * @return The 32-bit native endian integer from the bytes at @p ptr.\n */\n\n/*!\n * @internal\n * @fn xxh_u32 XXH_readLE32(const void* ptr)\n * @brief Reads an unaligned 32-bit little endian integer from @p ptr.\n *\n * Affected by @ref XXH_FORCE_MEMORY_ACCESS.\n *\n * @param ptr The pointer to read from.\n * @return The 32-bit little endian integer from the bytes at @p ptr.\n */\n\n/*!\n * @internal\n * @fn xxh_u32 XXH_readBE32(const void* ptr)\n * @brief Reads an unaligned 32-bit big endian integer from @p ptr.\n *\n * Affected by @ref XXH_FORCE_MEMORY_ACCESS.\n *\n * @param ptr The pointer to read from.\n * @return The 32-bit big endian integer from the bytes at @p ptr.\n */\n\n/*!\n * @internal\n * @fn xxh_u32 XXH_readLE32_align(const void* ptr, XXH_alignment align)\n * @brief Like @ref XXH_readLE32(), but has an option for aligned reads.\n *\n * Affected by @ref XXH_FORCE_MEMORY_ACCESS.\n * Note that when @ref XXH_FORCE_ALIGN_CHECK == 0, the @p align parameter is\n * always @ref XXH_alignment::XXH_unaligned.\n *\n * @param ptr The pointer to read from.\n * @param align Whether @p ptr is aligned.\n * @pre\n * If @p align == @ref XXH_alignment::XXH_aligned, @p ptr must be 4 byte\n * aligned.\n * @return The 32-bit little endian integer from the bytes at @p ptr.\n */\n\n#if (defined(XXH_FORCE_MEMORY_ACCESS) && (XXH_FORCE_MEMORY_ACCESS==3))\n/*\n * Manual byteshift. Best for old compilers which don't inline memcpy.\n * We actually directly use XXH_readLE32 and XXH_readBE32.\n */\n#elif (defined(XXH_FORCE_MEMORY_ACCESS) && (XXH_FORCE_MEMORY_ACCESS==2))\n\n/*\n * Force direct memory access. Only works on CPU which support unaligned memory\n * access in hardware.\n */\nstatic xxh_u32 XXH_read32(const void* memPtr) { return *(const xxh_u32*) memPtr; }\n\n#elif (defined(XXH_FORCE_MEMORY_ACCESS) && (XXH_FORCE_MEMORY_ACCESS==1))\n\n/*\n * __attribute__((aligned(1))) is supported by gcc and clang. Originally the\n * documentation claimed that it only increased the alignment, but actually it\n * can decrease it on gcc, clang, and icc:\n * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69502,\n * https://gcc.godbolt.org/z/xYez1j67Y.\n */\n#ifdef XXH_OLD_NAMES\ntypedef union { xxh_u32 u32; } __attribute__((__packed__)) unalign;\n#endif\nstatic xxh_u32 XXH_read32(const void* ptr)\n{\n typedef __attribute__((__aligned__(1))) xxh_u32 xxh_unalign32;\n return *((const xxh_unalign32*)ptr);\n}\n\n#else\n\n/*\n * Portable and safe solution. Generally efficient.\n * see: https://fastcompression.blogspot.com/2015/08/accessing-unaligned-memory.html\n */\nstatic xxh_u32 XXH_read32(const void* memPtr)\n{\n xxh_u32 val;\n XXH_memcpy(&val, memPtr, sizeof(val));\n return val;\n}\n\n#endif /* XXH_FORCE_DIRECT_MEMORY_ACCESS */\n\n\n/* *** Endianness *** */\n\n/*!\n * @ingroup tuning\n * @def XXH_CPU_LITTLE_ENDIAN\n * @brief Whether the target is little endian.\n *\n * Defined to 1 if the target is little endian, or 0 if it is big endian.\n * It can be defined externally, for example on the compiler command line.\n *\n * If it is not defined,\n * a runtime check (which is usually constant folded) is used instead.\n *\n * @note\n * This is not necessarily defined to an integer constant.\n *\n * @see XXH_isLittleEndian() for the runtime check.\n */\n#ifndef XXH_CPU_LITTLE_ENDIAN\n/*\n * Try to detect endianness automatically, to avoid the nonstandard behavior\n * in `XXH_isLittleEndian()`\n */\n# if defined(_WIN32) /* Windows is always little endian */ \\\n || defined(__LITTLE_ENDIAN__) \\\n || (defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__)\n# define XXH_CPU_LITTLE_ENDIAN 1\n# elif defined(__BIG_ENDIAN__) \\\n || (defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__)\n# define XXH_CPU_LITTLE_ENDIAN 0\n# else\n/*!\n * @internal\n * @brief Runtime check for @ref XXH_CPU_LITTLE_ENDIAN.\n *\n * Most compilers will constant fold this.\n */\nstatic int XXH_isLittleEndian(void)\n{\n /*\n * Portable and well-defined behavior.\n * Don't use static: it is detrimental to performance.\n */\n const union { xxh_u32 u; xxh_u8 c[4]; } one = { 1 };\n return one.c[0];\n}\n# define XXH_CPU_LITTLE_ENDIAN XXH_isLittleEndian()\n# endif\n#endif\n\n\n\n\n/* ****************************************\n* Compiler-specific Functions and Macros\n******************************************/\n#define XXH_GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__)\n\n#ifdef __has_builtin\n# define XXH_HAS_BUILTIN(x) __has_builtin(x)\n#else\n# define XXH_HAS_BUILTIN(x) 0\n#endif\n\n\n\n/*\n * C23 and future versions have standard \"unreachable()\".\n * Once it has been implemented reliably we can add it as an\n * additional case:\n *\n * ```\n * #if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= XXH_C23_VN)\n * # include \n * # ifdef unreachable\n * # define XXH_UNREACHABLE() unreachable()\n * # endif\n * #endif\n * ```\n *\n * Note C++23 also has std::unreachable() which can be detected\n * as follows:\n * ```\n * #if defined(__cpp_lib_unreachable) && (__cpp_lib_unreachable >= 202202L)\n * # include \n * # define XXH_UNREACHABLE() std::unreachable()\n * #endif\n * ```\n * NB: `__cpp_lib_unreachable` is defined in the `` header.\n * We don't use that as including `` in `extern \"C\"` blocks\n * doesn't work on GCC12\n */\n\n#if XXH_HAS_BUILTIN(__builtin_unreachable)\n# define XXH_UNREACHABLE() __builtin_unreachable()\n\n#elif defined(_MSC_VER)\n# define XXH_UNREACHABLE() __assume(0)\n\n#else\n# define XXH_UNREACHABLE()\n#endif\n\n#if XXH_HAS_BUILTIN(__builtin_assume)\n# define XXH_ASSUME(c) __builtin_assume(c)\n#else\n# define XXH_ASSUME(c) if (!(c)) { XXH_UNREACHABLE(); }\n#endif\n\n/*!\n * @internal\n * @def XXH_rotl32(x,r)\n * @brief 32-bit rotate left.\n *\n * @param x The 32-bit integer to be rotated.\n * @param r The number of bits to rotate.\n * @pre\n * @p r > 0 && @p r < 32\n * @note\n * @p x and @p r may be evaluated multiple times.\n * @return The rotated result.\n */\n#if !defined(NO_CLANG_BUILTIN) && XXH_HAS_BUILTIN(__builtin_rotateleft32) \\\n && XXH_HAS_BUILTIN(__builtin_rotateleft64)\n# define XXH_rotl32 __builtin_rotateleft32\n# define XXH_rotl64 __builtin_rotateleft64\n#elif XXH_HAS_BUILTIN(__builtin_stdc_rotate_left)\n# define XXH_rotl32 __builtin_stdc_rotate_left\n# define XXH_rotl64 __builtin_stdc_rotate_left\n/* Note: although _rotl exists for minGW (GCC under windows), performance seems poor */\n#elif defined(_MSC_VER)\n# define XXH_rotl32(x,r) _rotl(x,r)\n# define XXH_rotl64(x,r) _rotl64(x,r)\n#else\n# define XXH_rotl32(x,r) (((x) << (r)) | ((x) >> (32 - (r))))\n# define XXH_rotl64(x,r) (((x) << (r)) | ((x) >> (64 - (r))))\n#endif\n\n/*!\n * @internal\n * @fn xxh_u32 XXH_swap32(xxh_u32 x)\n * @brief A 32-bit byteswap.\n *\n * @param x The 32-bit integer to byteswap.\n * @return @p x, byteswapped.\n */\n#if defined(_MSC_VER) /* Visual Studio */\n# define XXH_swap32 _byteswap_ulong\n#elif XXH_GCC_VERSION >= 403\n# define XXH_swap32 __builtin_bswap32\n#else\nstatic xxh_u32 XXH_swap32 (xxh_u32 x)\n{\n return ((x << 24) & 0xff000000 ) |\n ((x << 8) & 0x00ff0000 ) |\n ((x >> 8) & 0x0000ff00 ) |\n ((x >> 24) & 0x000000ff );\n}\n#endif\n\n\n/* ***************************\n* Memory reads\n*****************************/\n\n/*!\n * @internal\n * @brief Enum to indicate whether a pointer is aligned.\n */\ntypedef enum {\n XXH_aligned, /*!< Aligned */\n XXH_unaligned /*!< Possibly unaligned */\n} XXH_alignment;\n\n/*\n * XXH_FORCE_MEMORY_ACCESS==3 is an endian-independent byteshift load.\n *\n * This is ideal for older compilers which don't inline memcpy.\n */\n#if (defined(XXH_FORCE_MEMORY_ACCESS) && (XXH_FORCE_MEMORY_ACCESS==3))\n\nXXH_FORCE_INLINE xxh_u32 XXH_readLE32(const void* memPtr)\n{\n const xxh_u8* bytePtr = (const xxh_u8 *)memPtr;\n return bytePtr[0]\n | ((xxh_u32)bytePtr[1] << 8)\n | ((xxh_u32)bytePtr[2] << 16)\n | ((xxh_u32)bytePtr[3] << 24);\n}\n\nXXH_FORCE_INLINE xxh_u32 XXH_readBE32(const void* memPtr)\n{\n const xxh_u8* bytePtr = (const xxh_u8 *)memPtr;\n return bytePtr[3]\n | ((xxh_u32)bytePtr[2] << 8)\n | ((xxh_u32)bytePtr[1] << 16)\n | ((xxh_u32)bytePtr[0] << 24);\n}\n\n#else\nXXH_FORCE_INLINE xxh_u32 XXH_readLE32(const void* ptr)\n{\n return XXH_CPU_LITTLE_ENDIAN ? XXH_read32(ptr) : XXH_swap32(XXH_read32(ptr));\n}\n\nstatic xxh_u32 XXH_readBE32(const void* ptr)\n{\n return XXH_CPU_LITTLE_ENDIAN ? XXH_swap32(XXH_read32(ptr)) : XXH_read32(ptr);\n}\n#endif\n\nXXH_FORCE_INLINE xxh_u32\nXXH_readLE32_align(const void* ptr, XXH_alignment align)\n{\n if (align==XXH_unaligned) {\n return XXH_readLE32(ptr);\n } else {\n return XXH_CPU_LITTLE_ENDIAN ? *(const xxh_u32*)ptr : XXH_swap32(*(const xxh_u32*)ptr);\n }\n}\n\n\n/* *************************************\n* Misc\n***************************************/\n/*! @ingroup public */\nXXH_PUBLIC_API unsigned XXH_versionNumber (void) { return XXH_VERSION_NUMBER; }\n\n\n/* *******************************************************************\n* 32-bit hash functions\n*********************************************************************/\n/*!\n * @}\n * @defgroup XXH32_impl XXH32 implementation\n * @ingroup impl\n *\n * Details on the XXH32 implementation.\n * @{\n */\n /* #define instead of static const, to be used as initializers */\n#define XXH_PRIME32_1 0x9E3779B1U /*!< 0b10011110001101110111100110110001 */\n#define XXH_PRIME32_2 0x85EBCA77U /*!< 0b10000101111010111100101001110111 */\n#define XXH_PRIME32_3 0xC2B2AE3DU /*!< 0b11000010101100101010111000111101 */\n#define XXH_PRIME32_4 0x27D4EB2FU /*!< 0b00100111110101001110101100101111 */\n#define XXH_PRIME32_5 0x165667B1U /*!< 0b00010110010101100110011110110001 */\n\n#ifdef XXH_OLD_NAMES\n# define PRIME32_1 XXH_PRIME32_1\n# define PRIME32_2 XXH_PRIME32_2\n# define PRIME32_3 XXH_PRIME32_3\n# define PRIME32_4 XXH_PRIME32_4\n# define PRIME32_5 XXH_PRIME32_5\n#endif\n\n/*!\n * @internal\n * @brief Normal stripe processing routine.\n *\n * This shuffles the bits so that any bit from @p input impacts several bits in\n * @p acc.\n *\n * @param acc The accumulator lane.\n * @param input The stripe of input to mix.\n * @return The mixed accumulator lane.\n */\nstatic xxh_u32 XXH32_round(xxh_u32 acc, xxh_u32 input)\n{\n acc += input * XXH_PRIME32_2;\n acc = XXH_rotl32(acc, 13);\n acc *= XXH_PRIME32_1;\n#if (defined(__SSE4_1__) || defined(__aarch64__) || defined(__wasm_simd128__)) && !defined(XXH_ENABLE_AUTOVECTORIZE)\n /*\n * UGLY HACK:\n * A compiler fence is used to prevent GCC and Clang from\n * autovectorizing the XXH32 loop (pragmas and attributes don't work for some\n * reason) without globally disabling SSE4.1.\n *\n * The reason we want to avoid vectorization is because despite working on\n * 4 integers at a time, there are multiple factors slowing XXH32 down on\n * SSE4:\n * - There's a ridiculous amount of lag from pmulld (10 cycles of latency on\n * newer chips!) making it slightly slower to multiply four integers at\n * once compared to four integers independently. Even when pmulld was\n * fastest, Sandy/Ivy Bridge, it is still not worth it to go into SSE\n * just to multiply unless doing a long operation.\n *\n * - Four instructions are required to rotate,\n * movqda tmp, v // not required with VEX encoding\n * pslld tmp, 13 // tmp <<= 13\n * psrld v, 19 // x >>= 19\n * por v, tmp // x |= tmp\n * compared to one for scalar:\n * roll v, 13 // reliably fast across the board\n * shldl v, v, 13 // Sandy Bridge and later prefer this for some reason\n *\n * - Instruction level parallelism is actually more beneficial here because\n * the SIMD actually serializes this operation: While v1 is rotating, v2\n * can load data, while v3 can multiply. SSE forces them to operate\n * together.\n *\n * This is also enabled on AArch64, as Clang is *very aggressive* in vectorizing\n * the loop. NEON is only faster on the A53, and with the newer cores, it is less\n * than half the speed.\n *\n * Additionally, this is used on WASM SIMD128 because it JITs to the same\n * SIMD instructions and has the same issue.\n */\n XXH_COMPILER_GUARD(acc);\n#endif\n return acc;\n}\n\n/*!\n * @internal\n * @brief Mixes all bits to finalize the hash.\n *\n * The final mix ensures that all input bits have a chance to impact any bit in\n * the output digest, resulting in an unbiased distribution.\n *\n * @param hash The hash to avalanche.\n * @return The avalanched hash.\n */\nstatic xxh_u32 XXH32_avalanche(xxh_u32 hash)\n{\n hash ^= hash >> 15;\n hash *= XXH_PRIME32_2;\n hash ^= hash >> 13;\n hash *= XXH_PRIME32_3;\n hash ^= hash >> 16;\n return hash;\n}\n\n#define XXH_get32bits(p) XXH_readLE32_align(p, align)\n\n/*!\n * @internal\n * @brief Sets up the initial accumulator state for XXH32().\n */\nXXH_FORCE_INLINE void\nXXH32_initAccs(xxh_u32 *acc, xxh_u32 seed)\n{\n XXH_ASSERT(acc != NULL);\n acc[0] = seed + XXH_PRIME32_1 + XXH_PRIME32_2;\n acc[1] = seed + XXH_PRIME32_2;\n acc[2] = seed + 0;\n acc[3] = seed - XXH_PRIME32_1;\n}\n\n/*!\n * @internal\n * @brief Consumes a block of data for XXH32().\n *\n * @return the end input pointer.\n */\nXXH_FORCE_INLINE const xxh_u8 *\nXXH32_consumeLong(\n xxh_u32 *XXH_RESTRICT acc,\n xxh_u8 const *XXH_RESTRICT input,\n size_t len,\n XXH_alignment align\n)\n{\n const xxh_u8* const bEnd = input + len;\n const xxh_u8* const limit = bEnd - 15;\n XXH_ASSERT(acc != NULL);\n XXH_ASSERT(input != NULL);\n XXH_ASSERT(len >= 16);\n do {\n acc[0] = XXH32_round(acc[0], XXH_get32bits(input)); input += 4;\n acc[1] = XXH32_round(acc[1], XXH_get32bits(input)); input += 4;\n acc[2] = XXH32_round(acc[2], XXH_get32bits(input)); input += 4;\n acc[3] = XXH32_round(acc[3], XXH_get32bits(input)); input += 4;\n } while (input < limit);\n\n return input;\n}\n\n/*!\n * @internal\n * @brief Merges the accumulator lanes together for XXH32()\n */\nXXH_FORCE_INLINE XXH_PUREF xxh_u32\nXXH32_mergeAccs(const xxh_u32 *acc)\n{\n XXH_ASSERT(acc != NULL);\n return XXH_rotl32(acc[0], 1) + XXH_rotl32(acc[1], 7)\n + XXH_rotl32(acc[2], 12) + XXH_rotl32(acc[3], 18);\n}\n\n/*!\n * @internal\n * @brief Processes the last 0-15 bytes of @p ptr.\n *\n * There may be up to 15 bytes remaining to consume from the input.\n * This final stage will digest them to ensure that all input bytes are present\n * in the final mix.\n *\n * @param hash The hash to finalize.\n * @param ptr The pointer to the remaining input.\n * @param len The remaining length, modulo 16.\n * @param align Whether @p ptr is aligned.\n * @return The finalized hash.\n * @see XXH64_finalize().\n */\nstatic XXH_PUREF xxh_u32\nXXH32_finalize(xxh_u32 hash, const xxh_u8* ptr, size_t len, XXH_alignment align)\n{\n#define XXH_PROCESS1 do { \\\n hash += (*ptr++) * XXH_PRIME32_5; \\\n hash = XXH_rotl32(hash, 11) * XXH_PRIME32_1; \\\n} while (0)\n\n#define XXH_PROCESS4 do { \\\n hash += XXH_get32bits(ptr) * XXH_PRIME32_3; \\\n ptr += 4; \\\n hash = XXH_rotl32(hash, 17) * XXH_PRIME32_4; \\\n} while (0)\n\n if (ptr==NULL) XXH_ASSERT(len == 0);\n\n /* Compact rerolled version; generally faster */\n if (!XXH32_ENDJMP) {\n len &= 15;\n while (len >= 4) {\n XXH_PROCESS4;\n len -= 4;\n }\n while (len > 0) {\n XXH_PROCESS1;\n --len;\n }\n return XXH32_avalanche(hash);\n } else {\n switch(len&15) /* or switch(bEnd - p) */ {\n case 12: XXH_PROCESS4;\n XXH_FALLTHROUGH; /* fallthrough */\n case 8: XXH_PROCESS4;\n XXH_FALLTHROUGH; /* fallthrough */\n case 4: XXH_PROCESS4;\n return XXH32_avalanche(hash);\n\n case 13: XXH_PROCESS4;\n XXH_FALLTHROUGH; /* fallthrough */\n case 9: XXH_PROCESS4;\n XXH_FALLTHROUGH; /* fallthrough */\n case 5: XXH_PROCESS4;\n XXH_PROCESS1;\n return XXH32_avalanche(hash);\n\n case 14: XXH_PROCESS4;\n XXH_FALLTHROUGH; /* fallthrough */\n case 10: XXH_PROCESS4;\n XXH_FALLTHROUGH; /* fallthrough */\n case 6: XXH_PROCESS4;\n XXH_PROCESS1;\n XXH_PROCESS1;\n return XXH32_avalanche(hash);\n\n case 15: XXH_PROCESS4;\n XXH_FALLTHROUGH; /* fallthrough */\n case 11: XXH_PROCESS4;\n XXH_FALLTHROUGH; /* fallthrough */\n case 7: XXH_PROCESS4;\n XXH_FALLTHROUGH; /* fallthrough */\n case 3: XXH_PROCESS1;\n XXH_FALLTHROUGH; /* fallthrough */\n case 2: XXH_PROCESS1;\n XXH_FALLTHROUGH; /* fallthrough */\n case 1: XXH_PROCESS1;\n XXH_FALLTHROUGH; /* fallthrough */\n case 0: return XXH32_avalanche(hash);\n }\n XXH_ASSERT(0);\n return hash; /* reaching this point is deemed impossible */\n }\n}\n\n#ifdef XXH_OLD_NAMES\n# define PROCESS1 XXH_PROCESS1\n# define PROCESS4 XXH_PROCESS4\n#else\n# undef XXH_PROCESS1\n# undef XXH_PROCESS4\n#endif\n\n/*!\n * @internal\n * @brief The implementation for @ref XXH32().\n *\n * @param input , len , seed Directly passed from @ref XXH32().\n * @param align Whether @p input is aligned.\n * @return The calculated hash.\n */\nXXH_FORCE_INLINE XXH_PUREF xxh_u32\nXXH32_endian_align(const xxh_u8* input, size_t len, xxh_u32 seed, XXH_alignment align)\n{\n xxh_u32 h32;\n\n if (input==NULL) XXH_ASSERT(len == 0);\n\n if (len>=16) {\n xxh_u32 acc[4];\n XXH32_initAccs(acc, seed);\n\n input = XXH32_consumeLong(acc, input, len, align);\n\n h32 = XXH32_mergeAccs(acc);\n } else {\n h32 = seed + XXH_PRIME32_5;\n }\n\n h32 += (xxh_u32)len;\n\n return XXH32_finalize(h32, input, len&15, align);\n}\n\n/*! @ingroup XXH32_family */\nXXH_PUBLIC_API XXH32_hash_t XXH32 (const void* input, size_t len, XXH32_hash_t seed)\n{\n#if !defined(XXH_NO_STREAM) && XXH_SIZE_OPT >= 2\n /* Simple version, good for code maintenance, but unfortunately slow for small inputs */\n XXH32_state_t state;\n XXH32_reset(&state, seed);\n XXH32_update(&state, (const xxh_u8*)input, len);\n return XXH32_digest(&state);\n#else\n if (XXH_FORCE_ALIGN_CHECK) {\n if ((((size_t)input) & 3) == 0) { /* Input is 4-bytes aligned, leverage the speed benefit */\n return XXH32_endian_align((const xxh_u8*)input, len, seed, XXH_aligned);\n } }\n\n return XXH32_endian_align((const xxh_u8*)input, len, seed, XXH_unaligned);\n#endif\n}\n\n\n\n/******* Hash streaming *******/\n#ifndef XXH_NO_STREAM\n/*! @ingroup XXH32_family */\nXXH_PUBLIC_API XXH32_state_t* XXH32_createState(void)\n{\n return (XXH32_state_t*)XXH_malloc(sizeof(XXH32_state_t));\n}\n/*! @ingroup XXH32_family */\nXXH_PUBLIC_API XXH_errorcode XXH32_freeState(XXH32_state_t* statePtr)\n{\n XXH_free(statePtr);\n return XXH_OK;\n}\n\n/*! @ingroup XXH32_family */\nXXH_PUBLIC_API void XXH32_copyState(XXH32_state_t* dstState, const XXH32_state_t* srcState)\n{\n XXH_memcpy(dstState, srcState, sizeof(*dstState));\n}\n\n/*! @ingroup XXH32_family */\nXXH_PUBLIC_API XXH_errorcode XXH32_reset(XXH32_state_t* statePtr, XXH32_hash_t seed)\n{\n XXH_ASSERT(statePtr != NULL);\n memset(statePtr, 0, sizeof(*statePtr));\n XXH32_initAccs(statePtr->acc, seed);\n return XXH_OK;\n}\n\n\n/*! @ingroup XXH32_family */\nXXH_PUBLIC_API XXH_errorcode\nXXH32_update(XXH32_state_t* state, const void* input, size_t len)\n{\n if (input==NULL) {\n XXH_ASSERT(len == 0);\n return XXH_OK;\n }\n\n state->total_len_32 += (XXH32_hash_t)len;\n state->large_len |= (XXH32_hash_t)((len>=16) | (state->total_len_32>=16));\n\n XXH_ASSERT(state->bufferedSize < sizeof(state->buffer));\n if (len < sizeof(state->buffer) - state->bufferedSize) { /* fill in tmp buffer */\n XXH_memcpy(state->buffer + state->bufferedSize, input, len);\n state->bufferedSize += (XXH32_hash_t)len;\n return XXH_OK;\n }\n\n { const xxh_u8* xinput = (const xxh_u8*)input;\n const xxh_u8* const bEnd = xinput + len;\n\n if (state->bufferedSize) { /* non-empty buffer: complete first */\n XXH_memcpy(state->buffer + state->bufferedSize, xinput, sizeof(state->buffer) - state->bufferedSize);\n xinput += sizeof(state->buffer) - state->bufferedSize;\n /* then process one round */\n (void)XXH32_consumeLong(state->acc, state->buffer, sizeof(state->buffer), XXH_aligned);\n state->bufferedSize = 0;\n }\n\n XXH_ASSERT(xinput <= bEnd);\n if ((size_t)(bEnd - xinput) >= sizeof(state->buffer)) {\n /* Process the remaining data */\n xinput = XXH32_consumeLong(state->acc, xinput, (size_t)(bEnd - xinput), XXH_unaligned);\n }\n\n if (xinput < bEnd) {\n /* Copy the leftover to the tmp buffer */\n XXH_memcpy(state->buffer, xinput, (size_t)(bEnd-xinput));\n state->bufferedSize = (unsigned)(bEnd-xinput);\n }\n }\n\n return XXH_OK;\n}\n\n\n/*! @ingroup XXH32_family */\nXXH_PUBLIC_API XXH32_hash_t XXH32_digest(const XXH32_state_t* state)\n{\n xxh_u32 h32;\n\n if (state->large_len) {\n h32 = XXH32_mergeAccs(state->acc);\n } else {\n h32 = state->acc[2] /* == seed */ + XXH_PRIME32_5;\n }\n\n h32 += state->total_len_32;\n\n return XXH32_finalize(h32, state->buffer, state->bufferedSize, XXH_aligned);\n}\n#endif /* !XXH_NO_STREAM */\n\n/******* Canonical representation *******/\n\n/*! @ingroup XXH32_family */\nXXH_PUBLIC_API void XXH32_canonicalFromHash(XXH32_canonical_t* dst, XXH32_hash_t hash)\n{\n XXH_STATIC_ASSERT(sizeof(XXH32_canonical_t) == sizeof(XXH32_hash_t));\n if (XXH_CPU_LITTLE_ENDIAN) hash = XXH_swap32(hash);\n XXH_memcpy(dst, &hash, sizeof(*dst));\n}\n/*! @ingroup XXH32_family */\nXXH_PUBLIC_API XXH32_hash_t XXH32_hashFromCanonical(const XXH32_canonical_t* src)\n{\n return XXH_readBE32(src);\n}\n\n\n#ifndef XXH_NO_LONG_LONG\n\n/* *******************************************************************\n* 64-bit hash functions\n*********************************************************************/\n/*!\n * @}\n * @ingroup impl\n * @{\n */\n/******* Memory access *******/\n\ntypedef XXH64_hash_t xxh_u64;\n\n#ifdef XXH_OLD_NAMES\n# define U64 xxh_u64\n#endif\n\n#if (defined(XXH_FORCE_MEMORY_ACCESS) && (XXH_FORCE_MEMORY_ACCESS==3))\n/*\n * Manual byteshift. Best for old compilers which don't inline memcpy.\n * We actually directly use XXH_readLE64 and XXH_readBE64.\n */\n#elif (defined(XXH_FORCE_MEMORY_ACCESS) && (XXH_FORCE_MEMORY_ACCESS==2))\n\n/* Force direct memory access. Only works on CPU which support unaligned memory access in hardware */\nstatic xxh_u64 XXH_read64(const void* memPtr)\n{\n return *(const xxh_u64*) memPtr;\n}\n\n#elif (defined(XXH_FORCE_MEMORY_ACCESS) && (XXH_FORCE_MEMORY_ACCESS==1))\n\n/*\n * __attribute__((aligned(1))) is supported by gcc and clang. Originally the\n * documentation claimed that it only increased the alignment, but actually it\n * can decrease it on gcc, clang, and icc:\n * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69502,\n * https://gcc.godbolt.org/z/xYez1j67Y.\n */\n#ifdef XXH_OLD_NAMES\ntypedef union { xxh_u32 u32; xxh_u64 u64; } __attribute__((__packed__)) unalign64;\n#endif\nstatic xxh_u64 XXH_read64(const void* ptr)\n{\n typedef __attribute__((__aligned__(1))) xxh_u64 xxh_unalign64;\n return *((const xxh_unalign64*)ptr);\n}\n\n#else\n\n/*\n * Portable and safe solution. Generally efficient.\n * see: https://fastcompression.blogspot.com/2015/08/accessing-unaligned-memory.html\n */\nstatic xxh_u64 XXH_read64(const void* memPtr)\n{\n xxh_u64 val;\n XXH_memcpy(&val, memPtr, sizeof(val));\n return val;\n}\n\n#endif /* XXH_FORCE_DIRECT_MEMORY_ACCESS */\n\n#if defined(_MSC_VER) /* Visual Studio */\n# define XXH_swap64 _byteswap_uint64\n#elif XXH_GCC_VERSION >= 403\n# define XXH_swap64 __builtin_bswap64\n#else\nstatic xxh_u64 XXH_swap64(xxh_u64 x)\n{\n return ((x << 56) & 0xff00000000000000ULL) |\n ((x << 40) & 0x00ff000000000000ULL) |\n ((x << 24) & 0x0000ff0000000000ULL) |\n ((x << 8) & 0x000000ff00000000ULL) |\n ((x >> 8) & 0x00000000ff000000ULL) |\n ((x >> 24) & 0x0000000000ff0000ULL) |\n ((x >> 40) & 0x000000000000ff00ULL) |\n ((x >> 56) & 0x00000000000000ffULL);\n}\n#endif\n\n\n/* XXH_FORCE_MEMORY_ACCESS==3 is an endian-independent byteshift load. */\n#if (defined(XXH_FORCE_MEMORY_ACCESS) && (XXH_FORCE_MEMORY_ACCESS==3))\n\nXXH_FORCE_INLINE xxh_u64 XXH_readLE64(const void* memPtr)\n{\n const xxh_u8* bytePtr = (const xxh_u8 *)memPtr;\n return bytePtr[0]\n | ((xxh_u64)bytePtr[1] << 8)\n | ((xxh_u64)bytePtr[2] << 16)\n | ((xxh_u64)bytePtr[3] << 24)\n | ((xxh_u64)bytePtr[4] << 32)\n | ((xxh_u64)bytePtr[5] << 40)\n | ((xxh_u64)bytePtr[6] << 48)\n | ((xxh_u64)bytePtr[7] << 56);\n}\n\nXXH_FORCE_INLINE xxh_u64 XXH_readBE64(const void* memPtr)\n{\n const xxh_u8* bytePtr = (const xxh_u8 *)memPtr;\n return bytePtr[7]\n | ((xxh_u64)bytePtr[6] << 8)\n | ((xxh_u64)bytePtr[5] << 16)\n | ((xxh_u64)bytePtr[4] << 24)\n | ((xxh_u64)bytePtr[3] << 32)\n | ((xxh_u64)bytePtr[2] << 40)\n | ((xxh_u64)bytePtr[1] << 48)\n | ((xxh_u64)bytePtr[0] << 56);\n}\n\n#else\nXXH_FORCE_INLINE xxh_u64 XXH_readLE64(const void* ptr)\n{\n return XXH_CPU_LITTLE_ENDIAN ? XXH_read64(ptr) : XXH_swap64(XXH_read64(ptr));\n}\n\nstatic xxh_u64 XXH_readBE64(const void* ptr)\n{\n return XXH_CPU_LITTLE_ENDIAN ? XXH_swap64(XXH_read64(ptr)) : XXH_read64(ptr);\n}\n#endif\n\nXXH_FORCE_INLINE xxh_u64\nXXH_readLE64_align(const void* ptr, XXH_alignment align)\n{\n if (align==XXH_unaligned)\n return XXH_readLE64(ptr);\n else\n return XXH_CPU_LITTLE_ENDIAN ? *(const xxh_u64*)ptr : XXH_swap64(*(const xxh_u64*)ptr);\n}\n\n\n/******* xxh64 *******/\n/*!\n * @}\n * @defgroup XXH64_impl XXH64 implementation\n * @ingroup impl\n *\n * Details on the XXH64 implementation.\n * @{\n */\n/* #define rather that static const, to be used as initializers */\n#define XXH_PRIME64_1 0x9E3779B185EBCA87ULL /*!< 0b1001111000110111011110011011000110000101111010111100101010000111 */\n#define XXH_PRIME64_2 0xC2B2AE3D27D4EB4FULL /*!< 0b1100001010110010101011100011110100100111110101001110101101001111 */\n#define XXH_PRIME64_3 0x165667B19E3779F9ULL /*!< 0b0001011001010110011001111011000110011110001101110111100111111001 */\n#define XXH_PRIME64_4 0x85EBCA77C2B2AE63ULL /*!< 0b1000010111101011110010100111011111000010101100101010111001100011 */\n#define XXH_PRIME64_5 0x27D4EB2F165667C5ULL /*!< 0b0010011111010100111010110010111100010110010101100110011111000101 */\n\n#ifdef XXH_OLD_NAMES\n# define PRIME64_1 XXH_PRIME64_1\n# define PRIME64_2 XXH_PRIME64_2\n# define PRIME64_3 XXH_PRIME64_3\n# define PRIME64_4 XXH_PRIME64_4\n# define PRIME64_5 XXH_PRIME64_5\n#endif\n\n/*! @copydoc XXH32_round */\nstatic xxh_u64 XXH64_round(xxh_u64 acc, xxh_u64 input)\n{\n acc += input * XXH_PRIME64_2;\n acc = XXH_rotl64(acc, 31);\n acc *= XXH_PRIME64_1;\n#if (defined(__AVX512F__)) && !defined(XXH_ENABLE_AUTOVECTORIZE)\n /*\n * DISABLE AUTOVECTORIZATION:\n * A compiler fence is used to prevent GCC and Clang from\n * autovectorizing the XXH64 loop (pragmas and attributes don't work for some\n * reason) without globally disabling AVX512.\n *\n * Autovectorization of XXH64 tends to be detrimental,\n * though the exact outcome may change depending on exact cpu and compiler version.\n * For information, it has been reported as detrimental for Skylake-X,\n * but possibly beneficial for Zen4.\n *\n * The default is to disable auto-vectorization,\n * but you can select to enable it instead using `XXH_ENABLE_AUTOVECTORIZE` build variable.\n */\n XXH_COMPILER_GUARD(acc);\n#endif\n return acc;\n}\n\nstatic xxh_u64 XXH64_mergeRound(xxh_u64 acc, xxh_u64 val)\n{\n val = XXH64_round(0, val);\n acc ^= val;\n acc = acc * XXH_PRIME64_1 + XXH_PRIME64_4;\n return acc;\n}\n\n/*! @copydoc XXH32_avalanche */\nstatic xxh_u64 XXH64_avalanche(xxh_u64 hash)\n{\n hash ^= hash >> 33;\n hash *= XXH_PRIME64_2;\n hash ^= hash >> 29;\n hash *= XXH_PRIME64_3;\n hash ^= hash >> 32;\n return hash;\n}\n\n\n#define XXH_get64bits(p) XXH_readLE64_align(p, align)\n\n/*!\n * @internal\n * @brief Sets up the initial accumulator state for XXH64().\n */\nXXH_FORCE_INLINE void\nXXH64_initAccs(xxh_u64 *acc, xxh_u64 seed)\n{\n XXH_ASSERT(acc != NULL);\n acc[0] = seed + XXH_PRIME64_1 + XXH_PRIME64_2;\n acc[1] = seed + XXH_PRIME64_2;\n acc[2] = seed + 0;\n acc[3] = seed - XXH_PRIME64_1;\n}\n\n/*!\n * @internal\n * @brief Consumes a block of data for XXH64().\n *\n * @return the end input pointer.\n */\nXXH_FORCE_INLINE const xxh_u8 *\nXXH64_consumeLong(\n xxh_u64 *XXH_RESTRICT acc,\n xxh_u8 const *XXH_RESTRICT input,\n size_t len,\n XXH_alignment align\n)\n{\n const xxh_u8* const bEnd = input + len;\n const xxh_u8* const limit = bEnd - 31;\n XXH_ASSERT(acc != NULL);\n XXH_ASSERT(input != NULL);\n XXH_ASSERT(len >= 32);\n do {\n /* reroll on 32-bit */\n if (sizeof(void *) < sizeof(xxh_u64)) {\n size_t i;\n for (i = 0; i < 4; i++) {\n acc[i] = XXH64_round(acc[i], XXH_get64bits(input));\n input += 8;\n }\n } else {\n acc[0] = XXH64_round(acc[0], XXH_get64bits(input)); input += 8;\n acc[1] = XXH64_round(acc[1], XXH_get64bits(input)); input += 8;\n acc[2] = XXH64_round(acc[2], XXH_get64bits(input)); input += 8;\n acc[3] = XXH64_round(acc[3], XXH_get64bits(input)); input += 8;\n }\n } while (input < limit);\n\n return input;\n}\n\n/*!\n * @internal\n * @brief Merges the accumulator lanes together for XXH64()\n */\nXXH_FORCE_INLINE XXH_PUREF xxh_u64\nXXH64_mergeAccs(const xxh_u64 *acc)\n{\n XXH_ASSERT(acc != NULL);\n {\n xxh_u64 h64 = XXH_rotl64(acc[0], 1) + XXH_rotl64(acc[1], 7)\n + XXH_rotl64(acc[2], 12) + XXH_rotl64(acc[3], 18);\n /* reroll on 32-bit */\n if (sizeof(void *) < sizeof(xxh_u64)) {\n size_t i;\n for (i = 0; i < 4; i++) {\n h64 = XXH64_mergeRound(h64, acc[i]);\n }\n } else {\n h64 = XXH64_mergeRound(h64, acc[0]);\n h64 = XXH64_mergeRound(h64, acc[1]);\n h64 = XXH64_mergeRound(h64, acc[2]);\n h64 = XXH64_mergeRound(h64, acc[3]);\n }\n return h64;\n }\n}\n\n/*!\n * @internal\n * @brief Processes the last 0-31 bytes of @p ptr.\n *\n * There may be up to 31 bytes remaining to consume from the input.\n * This final stage will digest them to ensure that all input bytes are present\n * in the final mix.\n *\n * @param hash The hash to finalize.\n * @param ptr The pointer to the remaining input.\n * @param len The remaining length, modulo 32.\n * @param align Whether @p ptr is aligned.\n * @return The finalized hash\n * @see XXH32_finalize().\n */\nXXH_STATIC XXH_PUREF xxh_u64\nXXH64_finalize(xxh_u64 hash, const xxh_u8* ptr, size_t len, XXH_alignment align)\n{\n if (ptr==NULL) XXH_ASSERT(len == 0);\n len &= 31;\n while (len >= 8) {\n xxh_u64 const k1 = XXH64_round(0, XXH_get64bits(ptr));\n ptr += 8;\n hash ^= k1;\n hash = XXH_rotl64(hash,27) * XXH_PRIME64_1 + XXH_PRIME64_4;\n len -= 8;\n }\n if (len >= 4) {\n hash ^= (xxh_u64)(XXH_get32bits(ptr)) * XXH_PRIME64_1;\n ptr += 4;\n hash = XXH_rotl64(hash, 23) * XXH_PRIME64_2 + XXH_PRIME64_3;\n len -= 4;\n }\n while (len > 0) {\n hash ^= (*ptr++) * XXH_PRIME64_5;\n hash = XXH_rotl64(hash, 11) * XXH_PRIME64_1;\n --len;\n }\n return XXH64_avalanche(hash);\n}\n\n#ifdef XXH_OLD_NAMES\n# define PROCESS1_64 XXH_PROCESS1_64\n# define PROCESS4_64 XXH_PROCESS4_64\n# define PROCESS8_64 XXH_PROCESS8_64\n#else\n# undef XXH_PROCESS1_64\n# undef XXH_PROCESS4_64\n# undef XXH_PROCESS8_64\n#endif\n\n/*!\n * @internal\n * @brief The implementation for @ref XXH64().\n *\n * @param input , len , seed Directly passed from @ref XXH64().\n * @param align Whether @p input is aligned.\n * @return The calculated hash.\n */\nXXH_FORCE_INLINE XXH_PUREF xxh_u64\nXXH64_endian_align(const xxh_u8* input, size_t len, xxh_u64 seed, XXH_alignment align)\n{\n xxh_u64 h64;\n if (input==NULL) XXH_ASSERT(len == 0);\n\n if (len>=32) { /* Process a large block of data */\n xxh_u64 acc[4];\n XXH64_initAccs(acc, seed);\n\n input = XXH64_consumeLong(acc, input, len, align);\n\n h64 = XXH64_mergeAccs(acc);\n } else {\n h64 = seed + XXH_PRIME64_5;\n }\n\n h64 += (xxh_u64) len;\n\n return XXH64_finalize(h64, input, len, align);\n}\n\n\n/*! @ingroup XXH64_family */\nXXH_PUBLIC_API XXH64_hash_t XXH64 (XXH_NOESCAPE const void* input, size_t len, XXH64_hash_t seed)\n{\n#if !defined(XXH_NO_STREAM) && XXH_SIZE_OPT >= 2\n /* Simple version, good for code maintenance, but unfortunately slow for small inputs */\n XXH64_state_t state;\n XXH64_reset(&state, seed);\n XXH64_update(&state, (const xxh_u8*)input, len);\n return XXH64_digest(&state);\n#else\n if (XXH_FORCE_ALIGN_CHECK) {\n if ((((size_t)input) & 7)==0) { /* Input is aligned, let's leverage the speed advantage */\n return XXH64_endian_align((const xxh_u8*)input, len, seed, XXH_aligned);\n } }\n\n return XXH64_endian_align((const xxh_u8*)input, len, seed, XXH_unaligned);\n\n#endif\n}\n\n/******* Hash Streaming *******/\n#ifndef XXH_NO_STREAM\n/*! @ingroup XXH64_family*/\nXXH_PUBLIC_API XXH64_state_t* XXH64_createState(void)\n{\n return (XXH64_state_t*)XXH_malloc(sizeof(XXH64_state_t));\n}\n/*! @ingroup XXH64_family */\nXXH_PUBLIC_API XXH_errorcode XXH64_freeState(XXH64_state_t* statePtr)\n{\n XXH_free(statePtr);\n return XXH_OK;\n}\n\n/*! @ingroup XXH64_family */\nXXH_PUBLIC_API void XXH64_copyState(XXH_NOESCAPE XXH64_state_t* dstState, const XXH64_state_t* srcState)\n{\n XXH_memcpy(dstState, srcState, sizeof(*dstState));\n}\n\n/*! @ingroup XXH64_family */\nXXH_PUBLIC_API XXH_errorcode XXH64_reset(XXH_NOESCAPE XXH64_state_t* statePtr, XXH64_hash_t seed)\n{\n XXH_ASSERT(statePtr != NULL);\n memset(statePtr, 0, sizeof(*statePtr));\n XXH64_initAccs(statePtr->acc, seed);\n return XXH_OK;\n}\n\n/*! @ingroup XXH64_family */\nXXH_PUBLIC_API XXH_errorcode\nXXH64_update (XXH_NOESCAPE XXH64_state_t* state, XXH_NOESCAPE const void* input, size_t len)\n{\n if (input==NULL) {\n XXH_ASSERT(len == 0);\n return XXH_OK;\n }\n\n state->total_len += len;\n\n XXH_ASSERT(state->bufferedSize <= sizeof(state->buffer));\n if (len < sizeof(state->buffer) - state->bufferedSize) { /* fill in tmp buffer */\n XXH_memcpy(state->buffer + state->bufferedSize, input, len);\n state->bufferedSize += (XXH32_hash_t)len;\n return XXH_OK;\n }\n\n { const xxh_u8* xinput = (const xxh_u8*)input;\n const xxh_u8* const bEnd = xinput + len;\n\n if (state->bufferedSize) { /* non-empty buffer => complete first */\n XXH_memcpy(state->buffer + state->bufferedSize, xinput, sizeof(state->buffer) - state->bufferedSize);\n xinput += sizeof(state->buffer) - state->bufferedSize;\n /* and process one round */\n (void)XXH64_consumeLong(state->acc, state->buffer, sizeof(state->buffer), XXH_aligned);\n state->bufferedSize = 0;\n }\n\n XXH_ASSERT(xinput <= bEnd);\n if ((size_t)(bEnd - xinput) >= sizeof(state->buffer)) {\n /* Process the remaining data */\n xinput = XXH64_consumeLong(state->acc, xinput, (size_t)(bEnd - xinput), XXH_unaligned);\n }\n\n if (xinput < bEnd) {\n /* Copy the leftover to the tmp buffer */\n XXH_memcpy(state->buffer, xinput, (size_t)(bEnd-xinput));\n state->bufferedSize = (unsigned)(bEnd-xinput);\n }\n }\n\n return XXH_OK;\n}\n\n\n/*! @ingroup XXH64_family */\nXXH_PUBLIC_API XXH64_hash_t XXH64_digest(XXH_NOESCAPE const XXH64_state_t* state)\n{\n xxh_u64 h64;\n\n if (state->total_len >= 32) {\n h64 = XXH64_mergeAccs(state->acc);\n } else {\n h64 = state->acc[2] /*seed*/ + XXH_PRIME64_5;\n }\n\n h64 += (xxh_u64) state->total_len;\n\n return XXH64_finalize(h64, state->buffer, (size_t)state->total_len, XXH_aligned);\n}\n#endif /* !XXH_NO_STREAM */\n\n/******* Canonical representation *******/\n\n/*! @ingroup XXH64_family */\nXXH_PUBLIC_API void XXH64_canonicalFromHash(XXH_NOESCAPE XXH64_canonical_t* dst, XXH64_hash_t hash)\n{\n XXH_STATIC_ASSERT(sizeof(XXH64_canonical_t) == sizeof(XXH64_hash_t));\n if (XXH_CPU_LITTLE_ENDIAN) hash = XXH_swap64(hash);\n XXH_memcpy(dst, &hash, sizeof(*dst));\n}\n\n/*! @ingroup XXH64_family */\nXXH_PUBLIC_API XXH64_hash_t XXH64_hashFromCanonical(XXH_NOESCAPE const XXH64_canonical_t* src)\n{\n return XXH_readBE64(src);\n}\n\n#ifndef XXH_NO_XXH3\n\n/* *********************************************************************\n* XXH3\n* New generation hash designed for speed on small keys and vectorization\n************************************************************************ */\n/*!\n * @}\n * @defgroup XXH3_impl XXH3 implementation\n * @ingroup impl\n * @{\n */\n\n/* === Compiler specifics === */\n\n\n#if (defined(__GNUC__) && (__GNUC__ >= 3)) \\\n || (defined(__INTEL_COMPILER) && (__INTEL_COMPILER >= 800)) \\\n || defined(__clang__)\n# define XXH_likely(x) __builtin_expect(x, 1)\n# define XXH_unlikely(x) __builtin_expect(x, 0)\n#else\n# define XXH_likely(x) (x)\n# define XXH_unlikely(x) (x)\n#endif\n\n#ifndef XXH_HAS_INCLUDE\n# ifdef __has_include\n/*\n * Not defined as XXH_HAS_INCLUDE(x) (function-like) because\n * this causes segfaults in Apple Clang 4.2 (on Mac OS X 10.7 Lion)\n */\n# define XXH_HAS_INCLUDE __has_include\n# else\n# define XXH_HAS_INCLUDE(x) 0\n# endif\n#endif\n\n#if defined(__GNUC__) || defined(__clang__)\n# if defined(__ARM_FEATURE_SVE)\n# include \n# endif\n# if defined(__ARM_NEON__) || defined(__ARM_NEON) \\\n || (defined(_M_ARM) && _M_ARM >= 7) \\\n || defined(_M_ARM64) || defined(_M_ARM64EC) \\\n || (defined(__wasm_simd128__) && XXH_HAS_INCLUDE()) /* WASM SIMD128 via SIMDe */\n# define inline __inline__ /* circumvent a clang bug */\n# include \n# undef inline\n# elif defined(__AVX2__)\n# include \n# elif defined(__SSE2__)\n# include \n# elif defined(__loongarch_sx)\n# include \n# endif\n#endif\n\n#if defined(_MSC_VER)\n# include \n#endif\n\n/*\n * One goal of XXH3 is to make it fast on both 32-bit and 64-bit, while\n * remaining a true 64-bit/128-bit hash function.\n *\n * This is done by prioritizing a subset of 64-bit operations that can be\n * emulated without too many steps on the average 32-bit machine.\n *\n * For example, these two lines seem similar, and run equally fast on 64-bit:\n *\n * xxh_u64 x;\n * x ^= (x >> 47); // good\n * x ^= (x >> 13); // bad\n *\n * However, to a 32-bit machine, there is a major difference.\n *\n * x ^= (x >> 47) looks like this:\n *\n * x.lo ^= (x.hi >> (47 - 32));\n *\n * while x ^= (x >> 13) looks like this:\n *\n * // note: funnel shifts are not usually cheap.\n * x.lo ^= (x.lo >> 13) | (x.hi << (32 - 13));\n * x.hi ^= (x.hi >> 13);\n *\n * The first one is significantly faster than the second, simply because the\n * shift is larger than 32. This means:\n * - All the bits we need are in the upper 32 bits, so we can ignore the lower\n * 32 bits in the shift.\n * - The shift result will always fit in the lower 32 bits, and therefore,\n * we can ignore the upper 32 bits in the xor.\n *\n * Thanks to this optimization, XXH3 only requires these features to be efficient:\n *\n * - Usable unaligned access\n * - A 32-bit or 64-bit ALU\n * - If 32-bit, a decent ADC instruction\n * - A 32 or 64-bit multiply with a 64-bit result\n * - For the 128-bit variant, a decent byteswap helps short inputs.\n *\n * The first two are already required by XXH32, and almost all 32-bit and 64-bit\n * platforms which can run XXH32 can run XXH3 efficiently.\n *\n * Thumb-1, the classic 16-bit only subset of ARM's instruction set, is one\n * notable exception.\n *\n * First of all, Thumb-1 lacks support for the UMULL instruction which\n * performs the important long multiply. This means numerous __aeabi_lmul\n * calls.\n *\n * Second of all, the 8 functional registers are just not enough.\n * Setup for __aeabi_lmul, byteshift loads, pointers, and all arithmetic need\n * Lo registers, and this shuffling results in thousands more MOVs than A32.\n *\n * A32 and T32 don't have this limitation. They can access all 14 registers,\n * do a 32->64 multiply with UMULL, and the flexible operand allowing free\n * shifts is helpful, too.\n *\n * Therefore, we do a quick sanity check.\n *\n * If compiling Thumb-1 for a target which supports ARM instructions, we will\n * emit a warning, as it is not a \"sane\" platform to compile for.\n *\n * Usually, if this happens, it is because of an accident and you probably need\n * to specify -march, as you likely meant to compile for a newer architecture.\n *\n * Credit: large sections of the vectorial and asm source code paths\n * have been contributed by @easyaspi314\n */\n#if defined(__thumb__) && !defined(__thumb2__) && defined(__ARM_ARCH_ISA_ARM)\n# warning \"XXH3 is highly inefficient without ARM or Thumb-2.\"\n#endif\n\n/* ==========================================\n * Vectorization detection\n * ========================================== */\n\n#ifdef XXH_DOXYGEN\n/*!\n * @ingroup tuning\n * @brief Overrides the vectorization implementation chosen for XXH3.\n *\n * Can be defined to 0 to disable SIMD or any of the values mentioned in\n * @ref XXH_VECTOR_TYPE.\n *\n * If this is not defined, it uses predefined macros to determine the best\n * implementation.\n */\n# define XXH_VECTOR XXH_SCALAR\n/*!\n * @ingroup tuning\n * @brief Selects the minimum alignment for XXH3's accumulators.\n *\n * When using SIMD, this should match the alignment required for said vector\n * type, so, for example, 32 for AVX2.\n *\n * Default: Auto detected.\n */\n# define XXH_ACC_ALIGN 8\n#endif\n\n/* Actual definition */\n#ifndef XXH_DOXYGEN\n#endif\n\n#ifndef XXH_VECTOR /* can be defined on command line */\n# if defined(__ARM_FEATURE_SVE)\n# define XXH_VECTOR XXH_SVE\n# elif ( \\\n defined(__ARM_NEON__) || defined(__ARM_NEON) /* gcc */ \\\n || defined(_M_ARM) || defined(_M_ARM64) || defined(_M_ARM64EC) /* msvc */ \\\n || (defined(__wasm_simd128__) && XXH_HAS_INCLUDE()) /* wasm simd128 via SIMDe */ \\\n ) && ( \\\n defined(_WIN32) || defined(__LITTLE_ENDIAN__) /* little endian only */ \\\n || (defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__) \\\n )\n# define XXH_VECTOR XXH_NEON\n# elif defined(__AVX512F__)\n# define XXH_VECTOR XXH_AVX512\n# elif defined(__AVX2__)\n# define XXH_VECTOR XXH_AVX2\n# elif defined(__SSE2__) || defined(_M_AMD64) || defined(_M_X64) || (defined(_M_IX86_FP) && (_M_IX86_FP == 2))\n# define XXH_VECTOR XXH_SSE2\n# elif (defined(__PPC64__) && defined(__POWER8_VECTOR__)) \\\n || (defined(__s390x__) && defined(__VEC__)) \\\n && defined(__GNUC__) /* TODO: IBM XL */\n# define XXH_VECTOR XXH_VSX\n# elif defined(__loongarch_sx)\n# define XXH_VECTOR XXH_LSX\n# else\n# define XXH_VECTOR XXH_SCALAR\n# endif\n#endif\n\n/* __ARM_FEATURE_SVE is only supported by GCC & Clang. */\n#if (XXH_VECTOR == XXH_SVE) && !defined(__ARM_FEATURE_SVE)\n# ifdef _MSC_VER\n# pragma warning(once : 4606)\n# else\n# warning \"__ARM_FEATURE_SVE isn't supported. Use SCALAR instead.\"\n# endif\n# undef XXH_VECTOR\n# define XXH_VECTOR XXH_SCALAR\n#endif\n\n/*\n * Controls the alignment of the accumulator,\n * for compatibility with aligned vector loads, which are usually faster.\n */\n#ifndef XXH_ACC_ALIGN\n# if defined(XXH_X86DISPATCH)\n# define XXH_ACC_ALIGN 64 /* for compatibility with avx512 */\n# elif XXH_VECTOR == XXH_SCALAR /* scalar */\n# define XXH_ACC_ALIGN 8\n# elif XXH_VECTOR == XXH_SSE2 /* sse2 */\n# define XXH_ACC_ALIGN 16\n# elif XXH_VECTOR == XXH_AVX2 /* avx2 */\n# define XXH_ACC_ALIGN 32\n# elif XXH_VECTOR == XXH_NEON /* neon */\n# define XXH_ACC_ALIGN 16\n# elif XXH_VECTOR == XXH_VSX /* vsx */\n# define XXH_ACC_ALIGN 16\n# elif XXH_VECTOR == XXH_AVX512 /* avx512 */\n# define XXH_ACC_ALIGN 64\n# elif XXH_VECTOR == XXH_SVE /* sve */\n# define XXH_ACC_ALIGN 64\n# elif XXH_VECTOR == XXH_LSX /* lsx */\n# define XXH_ACC_ALIGN 64\n# endif\n#endif\n\n#if defined(XXH_X86DISPATCH) || XXH_VECTOR == XXH_SSE2 \\\n || XXH_VECTOR == XXH_AVX2 || XXH_VECTOR == XXH_AVX512\n# define XXH_SEC_ALIGN XXH_ACC_ALIGN\n#elif XXH_VECTOR == XXH_SVE\n# define XXH_SEC_ALIGN XXH_ACC_ALIGN\n#else\n# define XXH_SEC_ALIGN 8\n#endif\n\n#if defined(__GNUC__) || defined(__clang__)\n# define XXH_ALIASING __attribute__((__may_alias__))\n#else\n# define XXH_ALIASING /* nothing */\n#endif\n\n/*\n * UGLY HACK:\n * GCC usually generates the best code with -O3 for xxHash.\n *\n * However, when targeting AVX2, it is overzealous in its unrolling resulting\n * in code roughly 3/4 the speed of Clang.\n *\n * There are other issues, such as GCC splitting _mm256_loadu_si256 into\n * _mm_loadu_si128 + _mm256_inserti128_si256. This is an optimization which\n * only applies to Sandy and Ivy Bridge... which don't even support AVX2.\n *\n * That is why when compiling the AVX2 version, it is recommended to use either\n * -O2 -mavx2 -march=haswell\n * or\n * -O2 -mavx2 -mno-avx256-split-unaligned-load\n * for decent performance, or to use Clang instead.\n *\n * Fortunately, we can control the first one with a pragma that forces GCC into\n * -O2, but the other one we can't control without \"failed to inline always\n * inline function due to target mismatch\" warnings.\n */\n#if XXH_VECTOR == XXH_AVX2 /* AVX2 */ \\\n && defined(__GNUC__) && !defined(__clang__) /* GCC, not Clang */ \\\n && defined(__OPTIMIZE__) && XXH_SIZE_OPT <= 0 /* respect -O0 and -Os */\n# pragma GCC push_options\n# pragma GCC optimize(\"-O2\")\n#endif\n\n#if XXH_VECTOR == XXH_NEON\n\n/*\n * UGLY HACK: While AArch64 GCC on Linux does not seem to care, on macOS, GCC -O3\n * optimizes out the entire hashLong loop because of the aliasing violation.\n *\n * However, GCC is also inefficient at load-store optimization with vld1q/vst1q,\n * so the only option is to mark it as aliasing.\n */\ntypedef uint64x2_t xxh_aliasing_uint64x2_t XXH_ALIASING;\n\n/*!\n * @internal\n * @brief `vld1q_u64` but faster and alignment-safe.\n *\n * On AArch64, unaligned access is always safe, but on ARMv7-a, it is only\n * *conditionally* safe (`vld1` has an alignment bit like `movdq[ua]` in x86).\n *\n * GCC for AArch64 sees `vld1q_u8` as an intrinsic instead of a load, so it\n * prohibits load-store optimizations. Therefore, a direct dereference is used.\n *\n * Otherwise, `vld1q_u8` is used with `vreinterpretq_u8_u64` to do a safe\n * unaligned load.\n */\n#if defined(__aarch64__) && defined(__GNUC__) && !defined(__clang__)\nXXH_FORCE_INLINE uint64x2_t XXH_vld1q_u64(void const* ptr) /* silence -Wcast-align */\n{\n return *(xxh_aliasing_uint64x2_t const *)ptr;\n}\n#else\nXXH_FORCE_INLINE uint64x2_t XXH_vld1q_u64(void const* ptr)\n{\n return vreinterpretq_u64_u8(vld1q_u8((uint8_t const*)ptr));\n}\n#endif\n\n/*!\n * @internal\n * @brief `vmlal_u32` on low and high halves of a vector.\n *\n * This is a workaround for AArch64 GCC < 11 which implemented arm_neon.h with\n * inline assembly and were therefore incapable of merging the `vget_{low, high}_u32`\n * with `vmlal_u32`.\n */\n#if defined(__aarch64__) && defined(__GNUC__) && !defined(__clang__) && __GNUC__ < 11\nXXH_FORCE_INLINE uint64x2_t\nXXH_vmlal_low_u32(uint64x2_t acc, uint32x4_t lhs, uint32x4_t rhs)\n{\n /* Inline assembly is the only way */\n __asm__(\"umlal %0.2d, %1.2s, %2.2s\" : \"+w\" (acc) : \"w\" (lhs), \"w\" (rhs));\n return acc;\n}\nXXH_FORCE_INLINE uint64x2_t\nXXH_vmlal_high_u32(uint64x2_t acc, uint32x4_t lhs, uint32x4_t rhs)\n{\n /* This intrinsic works as expected */\n return vmlal_high_u32(acc, lhs, rhs);\n}\n#else\n/* Portable intrinsic versions */\nXXH_FORCE_INLINE uint64x2_t\nXXH_vmlal_low_u32(uint64x2_t acc, uint32x4_t lhs, uint32x4_t rhs)\n{\n return vmlal_u32(acc, vget_low_u32(lhs), vget_low_u32(rhs));\n}\n/*! @copydoc XXH_vmlal_low_u32\n * Assume the compiler converts this to vmlal_high_u32 on aarch64 */\nXXH_FORCE_INLINE uint64x2_t\nXXH_vmlal_high_u32(uint64x2_t acc, uint32x4_t lhs, uint32x4_t rhs)\n{\n return vmlal_u32(acc, vget_high_u32(lhs), vget_high_u32(rhs));\n}\n#endif\n\n/*!\n * @ingroup tuning\n * @brief Controls the NEON to scalar ratio for XXH3\n *\n * This can be set to 2, 4, 6, or 8.\n *\n * ARM Cortex CPUs are _very_ sensitive to how their pipelines are used.\n *\n * For example, the Cortex-A73 can dispatch 3 micro-ops per cycle, but only 2 of those\n * can be NEON. If you are only using NEON instructions, you are only using 2/3 of the CPU\n * bandwidth.\n *\n * This is even more noticeable on the more advanced cores like the Cortex-A76 which\n * can dispatch 8 micro-ops per cycle, but still only 2 NEON micro-ops at once.\n *\n * Therefore, to make the most out of the pipeline, it is beneficial to run 6 NEON lanes\n * and 2 scalar lanes, which is chosen by default.\n *\n * This does not apply to Apple processors or 32-bit processors, which run better with\n * full NEON. These will default to 8. Additionally, size-optimized builds run 8 lanes.\n *\n * This change benefits CPUs with large micro-op buffers without negatively affecting\n * most other CPUs:\n *\n * | Chipset | Dispatch type | NEON only | 6:2 hybrid | Diff. |\n * |:----------------------|:--------------------|----------:|-----------:|------:|\n * | Snapdragon 730 (A76) | 2 NEON/8 micro-ops | 8.8 GB/s | 10.1 GB/s | ~16% |\n * | Snapdragon 835 (A73) | 2 NEON/3 micro-ops | 5.1 GB/s | 5.3 GB/s | ~5% |\n * | Marvell PXA1928 (A53) | In-order dual-issue | 1.9 GB/s | 1.9 GB/s | 0% |\n * | Apple M1 | 4 NEON/8 micro-ops | 37.3 GB/s | 36.1 GB/s | ~-3% |\n *\n * It also seems to fix some bad codegen on GCC, making it almost as fast as clang.\n *\n * When using WASM SIMD128, if this is 2 or 6, SIMDe will scalarize 2 of the lanes meaning\n * it effectively becomes worse 4.\n *\n * @see XXH3_accumulate_512_neon()\n */\n# ifndef XXH3_NEON_LANES\n# if (defined(__aarch64__) || defined(__arm64__) || defined(_M_ARM64) || defined(_M_ARM64EC)) \\\n && !defined(__APPLE__) && XXH_SIZE_OPT <= 0\n# define XXH3_NEON_LANES 6\n# else\n# define XXH3_NEON_LANES XXH_ACC_NB\n# endif\n# endif\n#endif /* XXH_VECTOR == XXH_NEON */\n\n/*\n * VSX and Z Vector helpers.\n *\n * This is very messy, and any pull requests to clean this up are welcome.\n *\n * There are a lot of problems with supporting VSX and s390x, due to\n * inconsistent intrinsics, spotty coverage, and multiple endiannesses.\n */\n#if XXH_VECTOR == XXH_VSX\n/* Annoyingly, these headers _may_ define three macros: `bool`, `vector`,\n * and `pixel`. This is a problem for obvious reasons.\n *\n * These keywords are unnecessary; the spec literally says they are\n * equivalent to `__bool`, `__vector`, and `__pixel` and may be undef'd\n * after including the header.\n *\n * We use pragma push_macro/pop_macro to keep the namespace clean. */\n# pragma push_macro(\"bool\")\n# pragma push_macro(\"vector\")\n# pragma push_macro(\"pixel\")\n/* silence potential macro redefined warnings */\n# undef bool\n# undef vector\n# undef pixel\n\n# if defined(__s390x__)\n# include \n# else\n# include \n# endif\n\n/* Restore the original macro values, if applicable. */\n# pragma pop_macro(\"pixel\")\n# pragma pop_macro(\"vector\")\n# pragma pop_macro(\"bool\")\n\ntypedef __vector unsigned long long xxh_u64x2;\ntypedef __vector unsigned char xxh_u8x16;\ntypedef __vector unsigned xxh_u32x4;\n\n/*\n * UGLY HACK: Similar to aarch64 macOS GCC, s390x GCC has the same aliasing issue.\n */\ntypedef xxh_u64x2 xxh_aliasing_u64x2 XXH_ALIASING;\n\n# ifndef XXH_VSX_BE\n# if defined(__BIG_ENDIAN__) \\\n || (defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__)\n# define XXH_VSX_BE 1\n# elif defined(__VEC_ELEMENT_REG_ORDER__) && __VEC_ELEMENT_REG_ORDER__ == __ORDER_BIG_ENDIAN__\n# warning \"-maltivec=be is not recommended. Please use native endianness.\"\n# define XXH_VSX_BE 1\n# else\n# define XXH_VSX_BE 0\n# endif\n# endif /* !defined(XXH_VSX_BE) */\n\n# if XXH_VSX_BE\n# if defined(__POWER9_VECTOR__) || (defined(__clang__) && defined(__s390x__))\n# define XXH_vec_revb vec_revb\n# else\n/*!\n * A polyfill for POWER9's vec_revb().\n */\nXXH_FORCE_INLINE xxh_u64x2 XXH_vec_revb(xxh_u64x2 val)\n{\n xxh_u8x16 const vByteSwap = { 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00,\n 0x0F, 0x0E, 0x0D, 0x0C, 0x0B, 0x0A, 0x09, 0x08 };\n return vec_perm(val, val, vByteSwap);\n}\n# endif\n# endif /* XXH_VSX_BE */\n\n/*!\n * Performs an unaligned vector load and byte swaps it on big endian.\n */\nXXH_FORCE_INLINE xxh_u64x2 XXH_vec_loadu(const void *ptr)\n{\n xxh_u64x2 ret;\n XXH_memcpy(&ret, ptr, sizeof(xxh_u64x2));\n# if XXH_VSX_BE\n ret = XXH_vec_revb(ret);\n# endif\n return ret;\n}\n\n/*\n * vec_mulo and vec_mule are very problematic intrinsics on PowerPC\n *\n * These intrinsics weren't added until GCC 8, despite existing for a while,\n * and they are endian dependent. Also, their meaning swap depending on version.\n * */\n# if defined(__s390x__)\n /* s390x is always big endian, no issue on this platform */\n# define XXH_vec_mulo vec_mulo\n# define XXH_vec_mule vec_mule\n# elif defined(__clang__) && XXH_HAS_BUILTIN(__builtin_altivec_vmuleuw) && !defined(__ibmxl__)\n/* Clang has a better way to control this, we can just use the builtin which doesn't swap. */\n /* The IBM XL Compiler (which defined __clang__) only implements the vec_* operations */\n# define XXH_vec_mulo __builtin_altivec_vmulouw\n# define XXH_vec_mule __builtin_altivec_vmuleuw\n# else\n/* gcc needs inline assembly */\n/* Adapted from https://github.com/google/highwayhash/blob/master/highwayhash/hh_vsx.h. */\nXXH_FORCE_INLINE xxh_u64x2 XXH_vec_mulo(xxh_u32x4 a, xxh_u32x4 b)\n{\n xxh_u64x2 result;\n __asm__(\"vmulouw %0, %1, %2\" : \"=v\" (result) : \"v\" (a), \"v\" (b));\n return result;\n}\nXXH_FORCE_INLINE xxh_u64x2 XXH_vec_mule(xxh_u32x4 a, xxh_u32x4 b)\n{\n xxh_u64x2 result;\n __asm__(\"vmuleuw %0, %1, %2\" : \"=v\" (result) : \"v\" (a), \"v\" (b));\n return result;\n}\n# endif /* XXH_vec_mulo, XXH_vec_mule */\n#endif /* XXH_VECTOR == XXH_VSX */\n\n#if XXH_VECTOR == XXH_SVE\n#define ACCRND(acc, offset) \\\ndo { \\\n svuint64_t input_vec = svld1_u64(mask, xinput + offset); \\\n svuint64_t secret_vec = svld1_u64(mask, xsecret + offset); \\\n svuint64_t mixed = sveor_u64_x(mask, secret_vec, input_vec); \\\n svuint64_t swapped = svtbl_u64(input_vec, kSwap); \\\n svuint64_t mixed_lo = svextw_u64_x(mask, mixed); \\\n svuint64_t mixed_hi = svlsr_n_u64_x(mask, mixed, 32); \\\n svuint64_t mul = svmad_u64_x(mask, mixed_lo, mixed_hi, swapped); \\\n acc = svadd_u64_x(mask, acc, mul); \\\n} while (0)\n#endif /* XXH_VECTOR == XXH_SVE */\n\n/* prefetch\n * can be disabled, by declaring XXH_NO_PREFETCH build macro */\n#if defined(XXH_NO_PREFETCH)\n# define XXH_PREFETCH(ptr) (void)(ptr) /* disabled */\n#else\n# if XXH_SIZE_OPT >= 1\n# define XXH_PREFETCH(ptr) (void)(ptr)\n# elif defined(_MSC_VER) && (defined(_M_X64) || defined(_M_IX86)) /* _mm_prefetch() not defined outside of x86/x64 */\n# include /* https://msdn.microsoft.com/fr-fr/library/84szxsww(v=vs.90).aspx */\n# define XXH_PREFETCH(ptr) _mm_prefetch((const char*)(ptr), _MM_HINT_T0)\n# elif defined(__GNUC__) && ( (__GNUC__ >= 4) || ( (__GNUC__ == 3) && (__GNUC_MINOR__ >= 1) ) )\n# define XXH_PREFETCH(ptr) __builtin_prefetch((ptr), 0 /* rw==read */, 3 /* locality */)\n# else\n# define XXH_PREFETCH(ptr) (void)(ptr) /* disabled */\n# endif\n#endif /* XXH_NO_PREFETCH */\n\n\n/* ==========================================\n * XXH3 default settings\n * ========================================== */\n\n#define XXH_SECRET_DEFAULT_SIZE 192 /* minimum XXH3_SECRET_SIZE_MIN */\n\n#if (XXH_SECRET_DEFAULT_SIZE < XXH3_SECRET_SIZE_MIN)\n# error \"default keyset is not large enough\"\n#endif\n\n/*! Pseudorandom secret taken directly from FARSH. */\nXXH_ALIGN(64) static const xxh_u8 XXH3_kSecret[XXH_SECRET_DEFAULT_SIZE] = {\n 0xb8, 0xfe, 0x6c, 0x39, 0x23, 0xa4, 0x4b, 0xbe, 0x7c, 0x01, 0x81, 0x2c, 0xf7, 0x21, 0xad, 0x1c,\n 0xde, 0xd4, 0x6d, 0xe9, 0x83, 0x90, 0x97, 0xdb, 0x72, 0x40, 0xa4, 0xa4, 0xb7, 0xb3, 0x67, 0x1f,\n 0xcb, 0x79, 0xe6, 0x4e, 0xcc, 0xc0, 0xe5, 0x78, 0x82, 0x5a, 0xd0, 0x7d, 0xcc, 0xff, 0x72, 0x21,\n 0xb8, 0x08, 0x46, 0x74, 0xf7, 0x43, 0x24, 0x8e, 0xe0, 0x35, 0x90, 0xe6, 0x81, 0x3a, 0x26, 0x4c,\n 0x3c, 0x28, 0x52, 0xbb, 0x91, 0xc3, 0x00, 0xcb, 0x88, 0xd0, 0x65, 0x8b, 0x1b, 0x53, 0x2e, 0xa3,\n 0x71, 0x64, 0x48, 0x97, 0xa2, 0x0d, 0xf9, 0x4e, 0x38, 0x19, 0xef, 0x46, 0xa9, 0xde, 0xac, 0xd8,\n 0xa8, 0xfa, 0x76, 0x3f, 0xe3, 0x9c, 0x34, 0x3f, 0xf9, 0xdc, 0xbb, 0xc7, 0xc7, 0x0b, 0x4f, 0x1d,\n 0x8a, 0x51, 0xe0, 0x4b, 0xcd, 0xb4, 0x59, 0x31, 0xc8, 0x9f, 0x7e, 0xc9, 0xd9, 0x78, 0x73, 0x64,\n 0xea, 0xc5, 0xac, 0x83, 0x34, 0xd3, 0xeb, 0xc3, 0xc5, 0x81, 0xa0, 0xff, 0xfa, 0x13, 0x63, 0xeb,\n 0x17, 0x0d, 0xdd, 0x51, 0xb7, 0xf0, 0xda, 0x49, 0xd3, 0x16, 0x55, 0x26, 0x29, 0xd4, 0x68, 0x9e,\n 0x2b, 0x16, 0xbe, 0x58, 0x7d, 0x47, 0xa1, 0xfc, 0x8f, 0xf8, 0xb8, 0xd1, 0x7a, 0xd0, 0x31, 0xce,\n 0x45, 0xcb, 0x3a, 0x8f, 0x95, 0x16, 0x04, 0x28, 0xaf, 0xd7, 0xfb, 0xca, 0xbb, 0x4b, 0x40, 0x7e,\n};\n\nstatic const xxh_u64 PRIME_MX1 = 0x165667919E3779F9ULL; /*!< 0b0001011001010110011001111001000110011110001101110111100111111001 */\nstatic const xxh_u64 PRIME_MX2 = 0x9FB21C651E98DF25ULL; /*!< 0b1001111110110010000111000110010100011110100110001101111100100101 */\n\n#ifdef XXH_OLD_NAMES\n# define kSecret XXH3_kSecret\n#endif\n\n#ifdef XXH_DOXYGEN\n/*!\n * @brief Calculates a 32-bit to 64-bit long multiply.\n *\n * Implemented as a macro.\n *\n * Wraps `__emulu` on MSVC x86 because it tends to call `__allmul` when it doesn't\n * need to (but it shouldn't need to anyways, it is about 7 instructions to do\n * a 64x64 multiply...). Since we know that this will _always_ emit `MULL`, we\n * use that instead of the normal method.\n *\n * If you are compiling for platforms like Thumb-1 and don't have a better option,\n * you may also want to write your own long multiply routine here.\n *\n * @param x, y Numbers to be multiplied\n * @return 64-bit product of the low 32 bits of @p x and @p y.\n */\nXXH_FORCE_INLINE xxh_u64\nXXH_mult32to64(xxh_u64 x, xxh_u64 y)\n{\n return (x & 0xFFFFFFFF) * (y & 0xFFFFFFFF);\n}\n#elif defined(_MSC_VER) && defined(_M_IX86)\n# define XXH_mult32to64(x, y) __emulu((unsigned)(x), (unsigned)(y))\n#else\n/*\n * Downcast + upcast is usually better than masking on older compilers like\n * GCC 4.2 (especially 32-bit ones), all without affecting newer compilers.\n *\n * The other method, (x & 0xFFFFFFFF) * (y & 0xFFFFFFFF), will AND both operands\n * and perform a full 64x64 multiply -- entirely redundant on 32-bit.\n */\n# define XXH_mult32to64(x, y) ((xxh_u64)(xxh_u32)(x) * (xxh_u64)(xxh_u32)(y))\n#endif\n\n/*!\n * @brief Calculates a 64->128-bit long multiply.\n *\n * Uses `__uint128_t` and `_umul128` if available, otherwise uses a scalar\n * version.\n *\n * @param lhs , rhs The 64-bit integers to be multiplied\n * @return The 128-bit result represented in an @ref XXH128_hash_t.\n */\nstatic XXH128_hash_t\nXXH_mult64to128(xxh_u64 lhs, xxh_u64 rhs)\n{\n /*\n * GCC/Clang __uint128_t method.\n *\n * On most 64-bit targets, GCC and Clang define a __uint128_t type.\n * This is usually the best way as it usually uses a native long 64-bit\n * multiply, such as MULQ on x86_64 or MUL + UMULH on aarch64.\n *\n * Usually.\n *\n * Despite being a 32-bit platform, Clang (and emscripten) define this type\n * despite not having the arithmetic for it. This results in a laggy\n * compiler builtin call which calculates a full 128-bit multiply.\n * In that case it is best to use the portable one.\n * https://github.com/Cyan4973/xxHash/issues/211#issuecomment-515575677\n */\n#if (defined(__GNUC__) || defined(__clang__)) && !defined(__wasm__) \\\n && defined(__SIZEOF_INT128__) \\\n || (defined(_INTEGRAL_MAX_BITS) && _INTEGRAL_MAX_BITS >= 128)\n\n __uint128_t const product = (__uint128_t)lhs * (__uint128_t)rhs;\n XXH128_hash_t r128;\n r128.low64 = (xxh_u64)(product);\n r128.high64 = (xxh_u64)(product >> 64);\n return r128;\n\n /*\n * MSVC for x64's _umul128 method.\n *\n * xxh_u64 _umul128(xxh_u64 Multiplier, xxh_u64 Multiplicand, xxh_u64 *HighProduct);\n *\n * This compiles to single operand MUL on x64.\n */\n#elif (defined(_M_X64) || defined(_M_IA64)) && !defined(_M_ARM64EC)\n\n#ifndef _MSC_VER\n# pragma intrinsic(_umul128)\n#endif\n xxh_u64 product_high;\n xxh_u64 const product_low = _umul128(lhs, rhs, &product_high);\n XXH128_hash_t r128;\n r128.low64 = product_low;\n r128.high64 = product_high;\n return r128;\n\n /*\n * MSVC for ARM64's __umulh method.\n *\n * This compiles to the same MUL + UMULH as GCC/Clang's __uint128_t method.\n */\n#elif defined(_M_ARM64) || defined(_M_ARM64EC)\n\n#ifndef _MSC_VER\n# pragma intrinsic(__umulh)\n#endif\n XXH128_hash_t r128;\n r128.low64 = lhs * rhs;\n r128.high64 = __umulh(lhs, rhs);\n return r128;\n\n#else\n /*\n * Portable scalar method. Optimized for 32-bit and 64-bit ALUs.\n *\n * This is a fast and simple grade school multiply, which is shown below\n * with base 10 arithmetic instead of base 0x100000000.\n *\n * 9 3 // D2 lhs = 93\n * x 7 5 // D2 rhs = 75\n * ----------\n * 1 5 // D2 lo_lo = (93 % 10) * (75 % 10) = 15\n * 4 5 | // D2 hi_lo = (93 / 10) * (75 % 10) = 45\n * 2 1 | // D2 lo_hi = (93 % 10) * (75 / 10) = 21\n * + 6 3 | | // D2 hi_hi = (93 / 10) * (75 / 10) = 63\n * ---------\n * 2 7 | // D2 cross = (15 / 10) + (45 % 10) + 21 = 27\n * + 6 7 | | // D2 upper = (27 / 10) + (45 / 10) + 63 = 67\n * ---------\n * 6 9 7 5 // D4 res = (27 * 10) + (15 % 10) + (67 * 100) = 6975\n *\n * The reasons for adding the products like this are:\n * 1. It avoids manual carry tracking. Just like how\n * (9 * 9) + 9 + 9 = 99, the same applies with this for UINT64_MAX.\n * This avoids a lot of complexity.\n *\n * 2. It hints for, and on Clang, compiles to, the powerful UMAAL\n * instruction available in ARM's Digital Signal Processing extension\n * in 32-bit ARMv6 and later, which is shown below:\n *\n * void UMAAL(xxh_u32 *RdLo, xxh_u32 *RdHi, xxh_u32 Rn, xxh_u32 Rm)\n * {\n * xxh_u64 product = (xxh_u64)*RdLo * (xxh_u64)*RdHi + Rn + Rm;\n * *RdLo = (xxh_u32)(product & 0xFFFFFFFF);\n * *RdHi = (xxh_u32)(product >> 32);\n * }\n *\n * This instruction was designed for efficient long multiplication, and\n * allows this to be calculated in only 4 instructions at speeds\n * comparable to some 64-bit ALUs.\n *\n * 3. It isn't terrible on other platforms. Usually this will be a couple\n * of 32-bit ADD/ADCs.\n */\n\n /* First calculate all of the cross products. */\n xxh_u64 const lo_lo = XXH_mult32to64(lhs & 0xFFFFFFFF, rhs & 0xFFFFFFFF);\n xxh_u64 const hi_lo = XXH_mult32to64(lhs >> 32, rhs & 0xFFFFFFFF);\n xxh_u64 const lo_hi = XXH_mult32to64(lhs & 0xFFFFFFFF, rhs >> 32);\n xxh_u64 const hi_hi = XXH_mult32to64(lhs >> 32, rhs >> 32);\n\n /* Now add the products together. These will never overflow. */\n xxh_u64 const cross = (lo_lo >> 32) + (hi_lo & 0xFFFFFFFF) + lo_hi;\n xxh_u64 const upper = (hi_lo >> 32) + (cross >> 32) + hi_hi;\n xxh_u64 const lower = (cross << 32) | (lo_lo & 0xFFFFFFFF);\n\n XXH128_hash_t r128;\n r128.low64 = lower;\n r128.high64 = upper;\n return r128;\n#endif\n}\n\n/*!\n * @brief Calculates a 64-bit to 128-bit multiply, then XOR folds it.\n *\n * The reason for the separate function is to prevent passing too many structs\n * around by value. This will hopefully inline the multiply, but we don't force it.\n *\n * @param lhs , rhs The 64-bit integers to multiply\n * @return The low 64 bits of the product XOR'd by the high 64 bits.\n * @see XXH_mult64to128()\n */\nstatic xxh_u64\nXXH3_mul128_fold64(xxh_u64 lhs, xxh_u64 rhs)\n{\n XXH128_hash_t product = XXH_mult64to128(lhs, rhs);\n return product.low64 ^ product.high64;\n}\n\n/*! Seems to produce slightly better code on GCC for some reason. */\nXXH_FORCE_INLINE XXH_CONSTF xxh_u64 XXH_xorshift64(xxh_u64 v64, int shift)\n{\n XXH_ASSERT(0 <= shift && shift < 64);\n return v64 ^ (v64 >> shift);\n}\n\n/*\n * This is a fast avalanche stage,\n * suitable when input bits are already partially mixed\n */\nstatic XXH64_hash_t XXH3_avalanche(xxh_u64 h64)\n{\n h64 = XXH_xorshift64(h64, 37);\n h64 *= PRIME_MX1;\n h64 = XXH_xorshift64(h64, 32);\n return h64;\n}\n\n/*\n * This is a stronger avalanche,\n * inspired by Pelle Evensen's rrmxmx\n * preferable when input has not been previously mixed\n */\nstatic XXH64_hash_t XXH3_rrmxmx(xxh_u64 h64, xxh_u64 len)\n{\n /* this mix is inspired by Pelle Evensen's rrmxmx */\n h64 ^= XXH_rotl64(h64, 49) ^ XXH_rotl64(h64, 24);\n h64 *= PRIME_MX2;\n h64 ^= (h64 >> 35) + len ;\n h64 *= PRIME_MX2;\n return XXH_xorshift64(h64, 28);\n}\n\n\n/* ==========================================\n * Short keys\n * ==========================================\n * One of the shortcomings of XXH32 and XXH64 was that their performance was\n * sub-optimal on short lengths. It used an iterative algorithm which strongly\n * favored lengths that were a multiple of 4 or 8.\n *\n * Instead of iterating over individual inputs, we use a set of single shot\n * functions which piece together a range of lengths and operate in constant time.\n *\n * Additionally, the number of multiplies has been significantly reduced. This\n * reduces latency, especially when emulating 64-bit multiplies on 32-bit.\n *\n * Depending on the platform, this may or may not be faster than XXH32, but it\n * is almost guaranteed to be faster than XXH64.\n */\n\n/*\n * At very short lengths, there isn't enough input to fully hide secrets, or use\n * the entire secret.\n *\n * There is also only a limited amount of mixing we can do before significantly\n * impacting performance.\n *\n * Therefore, we use different sections of the secret and always mix two secret\n * samples with an XOR. This should have no effect on performance on the\n * seedless or withSeed variants because everything _should_ be constant folded\n * by modern compilers.\n *\n * The XOR mixing hides individual parts of the secret and increases entropy.\n *\n * This adds an extra layer of strength for custom secrets.\n */\nXXH_FORCE_INLINE XXH_PUREF XXH64_hash_t\nXXH3_len_1to3_64b(const xxh_u8* input, size_t len, const xxh_u8* secret, XXH64_hash_t seed)\n{\n XXH_ASSERT(input != NULL);\n XXH_ASSERT(1 <= len && len <= 3);\n XXH_ASSERT(secret != NULL);\n /*\n * len = 1: combined = { input[0], 0x01, input[0], input[0] }\n * len = 2: combined = { input[1], 0x02, input[0], input[1] }\n * len = 3: combined = { input[2], 0x03, input[0], input[1] }\n */\n { xxh_u8 const c1 = input[0];\n xxh_u8 const c2 = input[len >> 1];\n xxh_u8 const c3 = input[len - 1];\n xxh_u32 const combined = ((xxh_u32)c1 << 16) | ((xxh_u32)c2 << 24)\n | ((xxh_u32)c3 << 0) | ((xxh_u32)len << 8);\n xxh_u64 const bitflip = (XXH_readLE32(secret) ^ XXH_readLE32(secret+4)) + seed;\n xxh_u64 const keyed = (xxh_u64)combined ^ bitflip;\n return XXH64_avalanche(keyed);\n }\n}\n\nXXH_FORCE_INLINE XXH_PUREF XXH64_hash_t\nXXH3_len_4to8_64b(const xxh_u8* input, size_t len, const xxh_u8* secret, XXH64_hash_t seed)\n{\n XXH_ASSERT(input != NULL);\n XXH_ASSERT(secret != NULL);\n XXH_ASSERT(4 <= len && len <= 8);\n seed ^= (xxh_u64)XXH_swap32((xxh_u32)seed) << 32;\n { xxh_u32 const input1 = XXH_readLE32(input);\n xxh_u32 const input2 = XXH_readLE32(input + len - 4);\n xxh_u64 const bitflip = (XXH_readLE64(secret+8) ^ XXH_readLE64(secret+16)) - seed;\n xxh_u64 const input64 = input2 + (((xxh_u64)input1) << 32);\n xxh_u64 const keyed = input64 ^ bitflip;\n return XXH3_rrmxmx(keyed, len);\n }\n}\n\nXXH_FORCE_INLINE XXH_PUREF XXH64_hash_t\nXXH3_len_9to16_64b(const xxh_u8* input, size_t len, const xxh_u8* secret, XXH64_hash_t seed)\n{\n XXH_ASSERT(input != NULL);\n XXH_ASSERT(secret != NULL);\n XXH_ASSERT(9 <= len && len <= 16);\n { xxh_u64 const bitflip1 = (XXH_readLE64(secret+24) ^ XXH_readLE64(secret+32)) + seed;\n xxh_u64 const bitflip2 = (XXH_readLE64(secret+40) ^ XXH_readLE64(secret+48)) - seed;\n xxh_u64 const input_lo = XXH_readLE64(input) ^ bitflip1;\n xxh_u64 const input_hi = XXH_readLE64(input + len - 8) ^ bitflip2;\n xxh_u64 const acc = len\n + XXH_swap64(input_lo) + input_hi\n + XXH3_mul128_fold64(input_lo, input_hi);\n return XXH3_avalanche(acc);\n }\n}\n\nXXH_FORCE_INLINE XXH_PUREF XXH64_hash_t\nXXH3_len_0to16_64b(const xxh_u8* input, size_t len, const xxh_u8* secret, XXH64_hash_t seed)\n{\n XXH_ASSERT(len <= 16);\n { if (XXH_likely(len > 8)) return XXH3_len_9to16_64b(input, len, secret, seed);\n if (XXH_likely(len >= 4)) return XXH3_len_4to8_64b(input, len, secret, seed);\n if (len) return XXH3_len_1to3_64b(input, len, secret, seed);\n return XXH64_avalanche(seed ^ (XXH_readLE64(secret+56) ^ XXH_readLE64(secret+64)));\n }\n}\n\n/*\n * DISCLAIMER: There are known *seed-dependent* multicollisions here due to\n * multiplication by zero, affecting hashes of lengths 17 to 240.\n *\n * However, they are very unlikely.\n *\n * Keep this in mind when using the unseeded XXH3_64bits() variant: As with all\n * unseeded non-cryptographic hashes, it does not attempt to defend itself\n * against specially crafted inputs, only random inputs.\n *\n * Compared to classic UMAC where a 1 in 2^31 chance of 4 consecutive bytes\n * cancelling out the secret is taken an arbitrary number of times (addressed\n * in XXH3_accumulate_512), this collision is very unlikely with random inputs\n * and/or proper seeding:\n *\n * This only has a 1 in 2^63 chance of 8 consecutive bytes cancelling out, in a\n * function that is only called up to 16 times per hash with up to 240 bytes of\n * input.\n *\n * This is not too bad for a non-cryptographic hash function, especially with\n * only 64 bit outputs.\n *\n * The 128-bit variant (which trades some speed for strength) is NOT affected\n * by this, although it is always a good idea to use a proper seed if you care\n * about strength.\n */\nXXH_FORCE_INLINE xxh_u64 XXH3_mix16B(const xxh_u8* XXH_RESTRICT input,\n const xxh_u8* XXH_RESTRICT secret, xxh_u64 seed64)\n{\n#if defined(__GNUC__) && !defined(__clang__) /* GCC, not Clang */ \\\n && defined(__i386__) && defined(__SSE2__) /* x86 + SSE2 */ \\\n && !defined(XXH_ENABLE_AUTOVECTORIZE) /* Define to disable like XXH32 hack */\n /*\n * UGLY HACK:\n * GCC for x86 tends to autovectorize the 128-bit multiply, resulting in\n * slower code.\n *\n * By forcing seed64 into a register, we disrupt the cost model and\n * cause it to scalarize. See `XXH32_round()`\n *\n * FIXME: Clang's output is still _much_ faster -- On an AMD Ryzen 3600,\n * XXH3_64bits @ len=240 runs at 4.6 GB/s with Clang 9, but 3.3 GB/s on\n * GCC 9.2, despite both emitting scalar code.\n *\n * GCC generates much better scalar code than Clang for the rest of XXH3,\n * which is why finding a more optimal codepath is an interest.\n */\n XXH_COMPILER_GUARD(seed64);\n#endif\n { xxh_u64 const input_lo = XXH_readLE64(input);\n xxh_u64 const input_hi = XXH_readLE64(input+8);\n return XXH3_mul128_fold64(\n input_lo ^ (XXH_readLE64(secret) + seed64),\n input_hi ^ (XXH_readLE64(secret+8) - seed64)\n );\n }\n}\n\n/* For mid range keys, XXH3 uses a Mum-hash variant. */\nXXH_FORCE_INLINE XXH_PUREF XXH64_hash_t\nXXH3_len_17to128_64b(const xxh_u8* XXH_RESTRICT input, size_t len,\n const xxh_u8* XXH_RESTRICT secret, size_t secretSize,\n XXH64_hash_t seed)\n{\n XXH_ASSERT(secretSize >= XXH3_SECRET_SIZE_MIN); (void)secretSize;\n XXH_ASSERT(16 < len && len <= 128);\n\n { xxh_u64 acc = len * XXH_PRIME64_1;\n#if XXH_SIZE_OPT >= 1\n /* Smaller and cleaner, but slightly slower. */\n unsigned int i = (unsigned int)(len - 1) / 32;\n do {\n acc += XXH3_mix16B(input+16 * i, secret+32*i, seed);\n acc += XXH3_mix16B(input+len-16*(i+1), secret+32*i+16, seed);\n } while (i-- != 0);\n#else\n if (len > 32) {\n if (len > 64) {\n if (len > 96) {\n acc += XXH3_mix16B(input+48, secret+96, seed);\n acc += XXH3_mix16B(input+len-64, secret+112, seed);\n }\n acc += XXH3_mix16B(input+32, secret+64, seed);\n acc += XXH3_mix16B(input+len-48, secret+80, seed);\n }\n acc += XXH3_mix16B(input+16, secret+32, seed);\n acc += XXH3_mix16B(input+len-32, secret+48, seed);\n }\n acc += XXH3_mix16B(input+0, secret+0, seed);\n acc += XXH3_mix16B(input+len-16, secret+16, seed);\n#endif\n return XXH3_avalanche(acc);\n }\n}\n\nXXH_NO_INLINE XXH_PUREF XXH64_hash_t\nXXH3_len_129to240_64b(const xxh_u8* XXH_RESTRICT input, size_t len,\n const xxh_u8* XXH_RESTRICT secret, size_t secretSize,\n XXH64_hash_t seed)\n{\n XXH_ASSERT(secretSize >= XXH3_SECRET_SIZE_MIN); (void)secretSize;\n XXH_ASSERT(128 < len && len <= XXH3_MIDSIZE_MAX);\n\n #define XXH3_MIDSIZE_STARTOFFSET 3\n #define XXH3_MIDSIZE_LASTOFFSET 17\n\n { xxh_u64 acc = len * XXH_PRIME64_1;\n xxh_u64 acc_end;\n unsigned int const nbRounds = (unsigned int)len / 16;\n unsigned int i;\n XXH_ASSERT(128 < len && len <= XXH3_MIDSIZE_MAX);\n for (i=0; i<8; i++) {\n acc += XXH3_mix16B(input+(16*i), secret+(16*i), seed);\n }\n /* last bytes */\n acc_end = XXH3_mix16B(input + len - 16, secret + XXH3_SECRET_SIZE_MIN - XXH3_MIDSIZE_LASTOFFSET, seed);\n XXH_ASSERT(nbRounds >= 8);\n acc = XXH3_avalanche(acc);\n#if defined(__clang__) /* Clang */ \\\n && (defined(__ARM_NEON) || defined(__ARM_NEON__)) /* NEON */ \\\n && !defined(XXH_ENABLE_AUTOVECTORIZE) /* Define to disable */\n /*\n * UGLY HACK:\n * Clang for ARMv7-A tries to vectorize this loop, similar to GCC x86.\n * In everywhere else, it uses scalar code.\n *\n * For 64->128-bit multiplies, even if the NEON was 100% optimal, it\n * would still be slower than UMAAL (see XXH_mult64to128).\n *\n * Unfortunately, Clang doesn't handle the long multiplies properly and\n * converts them to the nonexistent \"vmulq_u64\" intrinsic, which is then\n * scalarized into an ugly mess of VMOV.32 instructions.\n *\n * This mess is difficult to avoid without turning autovectorization\n * off completely, but they are usually relatively minor and/or not\n * worth it to fix.\n *\n * This loop is the easiest to fix, as unlike XXH32, this pragma\n * _actually works_ because it is a loop vectorization instead of an\n * SLP vectorization.\n */\n #pragma clang loop vectorize(disable)\n#endif\n for (i=8 ; i < nbRounds; i++) {\n /*\n * Prevents clang for unrolling the acc loop and interleaving with this one.\n */\n XXH_COMPILER_GUARD(acc);\n acc_end += XXH3_mix16B(input+(16*i), secret+(16*(i-8)) + XXH3_MIDSIZE_STARTOFFSET, seed);\n }\n return XXH3_avalanche(acc + acc_end);\n }\n}\n\n\n/* ======= Long Keys ======= */\n\n#define XXH_STRIPE_LEN 64\n#define XXH_SECRET_CONSUME_RATE 8 /* nb of secret bytes consumed at each accumulation */\n#define XXH_ACC_NB (XXH_STRIPE_LEN / sizeof(xxh_u64))\n\n#ifdef XXH_OLD_NAMES\n# define STRIPE_LEN XXH_STRIPE_LEN\n# define ACC_NB XXH_ACC_NB\n#endif\n\n#ifndef XXH_PREFETCH_DIST\n# ifdef __clang__\n# define XXH_PREFETCH_DIST 320\n# else\n# if (XXH_VECTOR == XXH_AVX512)\n# define XXH_PREFETCH_DIST 512\n# else\n# define XXH_PREFETCH_DIST 384\n# endif\n# endif /* __clang__ */\n#endif /* XXH_PREFETCH_DIST */\n\n/*\n * These macros are to generate an XXH3_accumulate() function.\n * The two arguments select the name suffix and target attribute.\n *\n * The name of this symbol is XXH3_accumulate_() and it calls\n * XXH3_accumulate_512_().\n *\n * It may be useful to hand implement this function if the compiler fails to\n * optimize the inline function.\n */\n#define XXH3_ACCUMULATE_TEMPLATE(name) \\\nvoid \\\nXXH3_accumulate_##name(xxh_u64* XXH_RESTRICT acc, \\\n const xxh_u8* XXH_RESTRICT input, \\\n const xxh_u8* XXH_RESTRICT secret, \\\n size_t nbStripes) \\\n{ \\\n size_t n; \\\n for (n = 0; n < nbStripes; n++ ) { \\\n const xxh_u8* const in = input + n*XXH_STRIPE_LEN; \\\n XXH_PREFETCH(in + XXH_PREFETCH_DIST); \\\n XXH3_accumulate_512_##name( \\\n acc, \\\n in, \\\n secret + n*XXH_SECRET_CONSUME_RATE); \\\n } \\\n}\n\n\nXXH_FORCE_INLINE void XXH_writeLE64(void* dst, xxh_u64 v64)\n{\n if (!XXH_CPU_LITTLE_ENDIAN) v64 = XXH_swap64(v64);\n XXH_memcpy(dst, &v64, sizeof(v64));\n}\n\n/* Several intrinsic functions below are supposed to accept __int64 as argument,\n * as documented in https://software.intel.com/sites/landingpage/IntrinsicsGuide/ .\n * However, several environments do not define __int64 type,\n * requiring a workaround.\n */\n#if !defined (__VMS) \\\n && (defined (__cplusplus) \\\n || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) )\n typedef int64_t xxh_i64;\n#else\n /* the following type must have a width of 64-bit */\n typedef long long xxh_i64;\n#endif\n\n\n/*\n * XXH3_accumulate_512 is the tightest loop for long inputs, and it is the most optimized.\n *\n * It is a hardened version of UMAC, based off of FARSH's implementation.\n *\n * This was chosen because it adapts quite well to 32-bit, 64-bit, and SIMD\n * implementations, and it is ridiculously fast.\n *\n * We harden it by mixing the original input to the accumulators as well as the product.\n *\n * This means that in the (relatively likely) case of a multiply by zero, the\n * original input is preserved.\n *\n * On 128-bit inputs, we swap 64-bit pairs when we add the input to improve\n * cross-pollination, as otherwise the upper and lower halves would be\n * essentially independent.\n *\n * This doesn't matter on 64-bit hashes since they all get merged together in\n * the end, so we skip the extra step.\n *\n * Both XXH3_64bits and XXH3_128bits use this subroutine.\n */\n\n#if (XXH_VECTOR == XXH_AVX512) \\\n || (defined(XXH_DISPATCH_AVX512) && XXH_DISPATCH_AVX512 != 0)\n\n#ifndef XXH_TARGET_AVX512\n# define XXH_TARGET_AVX512 /* disable attribute target */\n#endif\n\nXXH_FORCE_INLINE XXH_TARGET_AVX512 void\nXXH3_accumulate_512_avx512(void* XXH_RESTRICT acc,\n const void* XXH_RESTRICT input,\n const void* XXH_RESTRICT secret)\n{\n __m512i* const xacc = (__m512i *) acc;\n XXH_ASSERT((((size_t)acc) & 63) == 0);\n XXH_STATIC_ASSERT(XXH_STRIPE_LEN == sizeof(__m512i));\n\n {\n /* data_vec = input[0]; */\n __m512i const data_vec = _mm512_loadu_si512 (input);\n /* key_vec = secret[0]; */\n __m512i const key_vec = _mm512_loadu_si512 (secret);\n /* data_key = data_vec ^ key_vec; */\n __m512i const data_key = _mm512_xor_si512 (data_vec, key_vec);\n /* data_key_lo = data_key >> 32; */\n __m512i const data_key_lo = _mm512_srli_epi64 (data_key, 32);\n /* product = (data_key & 0xffffffff) * (data_key_lo & 0xffffffff); */\n __m512i const product = _mm512_mul_epu32 (data_key, data_key_lo);\n /* xacc[0] += swap(data_vec); */\n __m512i const data_swap = _mm512_shuffle_epi32(data_vec, (_MM_PERM_ENUM)_MM_SHUFFLE(1, 0, 3, 2));\n __m512i const sum = _mm512_add_epi64(*xacc, data_swap);\n /* xacc[0] += product; */\n *xacc = _mm512_add_epi64(product, sum);\n }\n}\nXXH_FORCE_INLINE XXH_TARGET_AVX512 XXH3_ACCUMULATE_TEMPLATE(avx512)\n\n/*\n * XXH3_scrambleAcc: Scrambles the accumulators to improve mixing.\n *\n * Multiplication isn't perfect, as explained by Google in HighwayHash:\n *\n * // Multiplication mixes/scrambles bytes 0-7 of the 64-bit result to\n * // varying degrees. In descending order of goodness, bytes\n * // 3 4 2 5 1 6 0 7 have quality 228 224 164 160 100 96 36 32.\n * // As expected, the upper and lower bytes are much worse.\n *\n * Source: https://github.com/google/highwayhash/blob/0aaf66b/highwayhash/hh_avx2.h#L291\n *\n * Since our algorithm uses a pseudorandom secret to add some variance into the\n * mix, we don't need to (or want to) mix as often or as much as HighwayHash does.\n *\n * This isn't as tight as XXH3_accumulate, but still written in SIMD to avoid\n * extraction.\n *\n * Both XXH3_64bits and XXH3_128bits use this subroutine.\n */\n\nXXH_FORCE_INLINE XXH_TARGET_AVX512 void\nXXH3_scrambleAcc_avx512(void* XXH_RESTRICT acc, const void* XXH_RESTRICT secret)\n{\n XXH_ASSERT((((size_t)acc) & 63) == 0);\n XXH_STATIC_ASSERT(XXH_STRIPE_LEN == sizeof(__m512i));\n { __m512i* const xacc = (__m512i*) acc;\n const __m512i prime32 = _mm512_set1_epi32((int)XXH_PRIME32_1);\n\n /* xacc[0] ^= (xacc[0] >> 47) */\n __m512i const acc_vec = *xacc;\n __m512i const shifted = _mm512_srli_epi64 (acc_vec, 47);\n /* xacc[0] ^= secret; */\n __m512i const key_vec = _mm512_loadu_si512 (secret);\n __m512i const data_key = _mm512_ternarylogic_epi32(key_vec, acc_vec, shifted, 0x96 /* key_vec ^ acc_vec ^ shifted */);\n\n /* xacc[0] *= XXH_PRIME32_1; */\n __m512i const data_key_hi = _mm512_srli_epi64 (data_key, 32);\n __m512i const prod_lo = _mm512_mul_epu32 (data_key, prime32);\n __m512i const prod_hi = _mm512_mul_epu32 (data_key_hi, prime32);\n *xacc = _mm512_add_epi64(prod_lo, _mm512_slli_epi64(prod_hi, 32));\n }\n}\n\nXXH_FORCE_INLINE XXH_TARGET_AVX512 void\nXXH3_initCustomSecret_avx512(void* XXH_RESTRICT customSecret, xxh_u64 seed64)\n{\n XXH_STATIC_ASSERT((XXH_SECRET_DEFAULT_SIZE & 63) == 0);\n XXH_STATIC_ASSERT(XXH_SEC_ALIGN == 64);\n XXH_ASSERT(((size_t)customSecret & 63) == 0);\n (void)(&XXH_writeLE64);\n { int const nbRounds = XXH_SECRET_DEFAULT_SIZE / sizeof(__m512i);\n __m512i const seed_pos = _mm512_set1_epi64((xxh_i64)seed64);\n __m512i const seed = _mm512_mask_sub_epi64(seed_pos, 0xAA, _mm512_set1_epi8(0), seed_pos);\n\n const __m512i* const src = (const __m512i*) ((const void*) XXH3_kSecret);\n __m512i* const dest = ( __m512i*) customSecret;\n int i;\n XXH_ASSERT(((size_t)src & 63) == 0); /* control alignment */\n XXH_ASSERT(((size_t)dest & 63) == 0);\n for (i=0; i < nbRounds; ++i) {\n dest[i] = _mm512_add_epi64(_mm512_load_si512(src + i), seed);\n } }\n}\n\n#endif\n\n#if (XXH_VECTOR == XXH_AVX2) \\\n || (defined(XXH_DISPATCH_AVX2) && XXH_DISPATCH_AVX2 != 0)\n\n#ifndef XXH_TARGET_AVX2\n# define XXH_TARGET_AVX2 /* disable attribute target */\n#endif\n\nXXH_FORCE_INLINE XXH_TARGET_AVX2 void\nXXH3_accumulate_512_avx2( void* XXH_RESTRICT acc,\n const void* XXH_RESTRICT input,\n const void* XXH_RESTRICT secret)\n{\n XXH_ASSERT((((size_t)acc) & 31) == 0);\n { __m256i* const xacc = (__m256i *) acc;\n /* Unaligned. This is mainly for pointer arithmetic, and because\n * _mm256_loadu_si256 requires a const __m256i * pointer for some reason. */\n const __m256i* const xinput = (const __m256i *) input;\n /* Unaligned. This is mainly for pointer arithmetic, and because\n * _mm256_loadu_si256 requires a const __m256i * pointer for some reason. */\n const __m256i* const xsecret = (const __m256i *) secret;\n\n size_t i;\n for (i=0; i < XXH_STRIPE_LEN/sizeof(__m256i); i++) {\n /* data_vec = xinput[i]; */\n __m256i const data_vec = _mm256_loadu_si256 (xinput+i);\n /* key_vec = xsecret[i]; */\n __m256i const key_vec = _mm256_loadu_si256 (xsecret+i);\n /* data_key = data_vec ^ key_vec; */\n __m256i const data_key = _mm256_xor_si256 (data_vec, key_vec);\n /* data_key_lo = data_key >> 32; */\n __m256i const data_key_lo = _mm256_srli_epi64 (data_key, 32);\n /* product = (data_key & 0xffffffff) * (data_key_lo & 0xffffffff); */\n __m256i const product = _mm256_mul_epu32 (data_key, data_key_lo);\n /* xacc[i] += swap(data_vec); */\n __m256i const data_swap = _mm256_shuffle_epi32(data_vec, _MM_SHUFFLE(1, 0, 3, 2));\n __m256i const sum = _mm256_add_epi64(xacc[i], data_swap);\n /* xacc[i] += product; */\n xacc[i] = _mm256_add_epi64(product, sum);\n } }\n}\nXXH_FORCE_INLINE XXH_TARGET_AVX2 XXH3_ACCUMULATE_TEMPLATE(avx2)\n\nXXH_FORCE_INLINE XXH_TARGET_AVX2 void\nXXH3_scrambleAcc_avx2(void* XXH_RESTRICT acc, const void* XXH_RESTRICT secret)\n{\n XXH_ASSERT((((size_t)acc) & 31) == 0);\n { __m256i* const xacc = (__m256i*) acc;\n /* Unaligned. This is mainly for pointer arithmetic, and because\n * _mm256_loadu_si256 requires a const __m256i * pointer for some reason. */\n const __m256i* const xsecret = (const __m256i *) secret;\n const __m256i prime32 = _mm256_set1_epi32((int)XXH_PRIME32_1);\n\n size_t i;\n for (i=0; i < XXH_STRIPE_LEN/sizeof(__m256i); i++) {\n /* xacc[i] ^= (xacc[i] >> 47) */\n __m256i const acc_vec = xacc[i];\n __m256i const shifted = _mm256_srli_epi64 (acc_vec, 47);\n __m256i const data_vec = _mm256_xor_si256 (acc_vec, shifted);\n /* xacc[i] ^= xsecret; */\n __m256i const key_vec = _mm256_loadu_si256 (xsecret+i);\n __m256i const data_key = _mm256_xor_si256 (data_vec, key_vec);\n\n /* xacc[i] *= XXH_PRIME32_1; */\n __m256i const data_key_hi = _mm256_srli_epi64 (data_key, 32);\n __m256i const prod_lo = _mm256_mul_epu32 (data_key, prime32);\n __m256i const prod_hi = _mm256_mul_epu32 (data_key_hi, prime32);\n xacc[i] = _mm256_add_epi64(prod_lo, _mm256_slli_epi64(prod_hi, 32));\n }\n }\n}\n\nXXH_FORCE_INLINE XXH_TARGET_AVX2 void XXH3_initCustomSecret_avx2(void* XXH_RESTRICT customSecret, xxh_u64 seed64)\n{\n XXH_STATIC_ASSERT((XXH_SECRET_DEFAULT_SIZE & 31) == 0);\n XXH_STATIC_ASSERT((XXH_SECRET_DEFAULT_SIZE / sizeof(__m256i)) == 6);\n XXH_STATIC_ASSERT(XXH_SEC_ALIGN <= 64);\n (void)(&XXH_writeLE64);\n XXH_PREFETCH(customSecret);\n { __m256i const seed = _mm256_set_epi64x((xxh_i64)(0U - seed64), (xxh_i64)seed64, (xxh_i64)(0U - seed64), (xxh_i64)seed64);\n\n const __m256i* const src = (const __m256i*) ((const void*) XXH3_kSecret);\n __m256i* dest = ( __m256i*) customSecret;\n\n# if defined(__GNUC__) || defined(__clang__)\n /*\n * On GCC & Clang, marking 'dest' as modified will cause the compiler:\n * - do not extract the secret from sse registers in the internal loop\n * - use less common registers, and avoid pushing these reg into stack\n */\n XXH_COMPILER_GUARD(dest);\n# endif\n XXH_ASSERT(((size_t)src & 31) == 0); /* control alignment */\n XXH_ASSERT(((size_t)dest & 31) == 0);\n\n /* GCC -O2 need unroll loop manually */\n dest[0] = _mm256_add_epi64(_mm256_load_si256(src+0), seed);\n dest[1] = _mm256_add_epi64(_mm256_load_si256(src+1), seed);\n dest[2] = _mm256_add_epi64(_mm256_load_si256(src+2), seed);\n dest[3] = _mm256_add_epi64(_mm256_load_si256(src+3), seed);\n dest[4] = _mm256_add_epi64(_mm256_load_si256(src+4), seed);\n dest[5] = _mm256_add_epi64(_mm256_load_si256(src+5), seed);\n }\n}\n\n#endif\n\n/* x86dispatch always generates SSE2 */\n#if (XXH_VECTOR == XXH_SSE2) || defined(XXH_X86DISPATCH)\n\n#ifndef XXH_TARGET_SSE2\n# define XXH_TARGET_SSE2 /* disable attribute target */\n#endif\n\nXXH_FORCE_INLINE XXH_TARGET_SSE2 void\nXXH3_accumulate_512_sse2( void* XXH_RESTRICT acc,\n const void* XXH_RESTRICT input,\n const void* XXH_RESTRICT secret)\n{\n /* SSE2 is just a half-scale version of the AVX2 version. */\n XXH_ASSERT((((size_t)acc) & 15) == 0);\n { __m128i* const xacc = (__m128i *) acc;\n /* Unaligned. This is mainly for pointer arithmetic, and because\n * _mm_loadu_si128 requires a const __m128i * pointer for some reason. */\n const __m128i* const xinput = (const __m128i *) input;\n /* Unaligned. This is mainly for pointer arithmetic, and because\n * _mm_loadu_si128 requires a const __m128i * pointer for some reason. */\n const __m128i* const xsecret = (const __m128i *) secret;\n\n size_t i;\n for (i=0; i < XXH_STRIPE_LEN/sizeof(__m128i); i++) {\n /* data_vec = xinput[i]; */\n __m128i const data_vec = _mm_loadu_si128 (xinput+i);\n /* key_vec = xsecret[i]; */\n __m128i const key_vec = _mm_loadu_si128 (xsecret+i);\n /* data_key = data_vec ^ key_vec; */\n __m128i const data_key = _mm_xor_si128 (data_vec, key_vec);\n /* data_key_lo = data_key >> 32; */\n __m128i const data_key_lo = _mm_shuffle_epi32 (data_key, _MM_SHUFFLE(0, 3, 0, 1));\n /* product = (data_key & 0xffffffff) * (data_key_lo & 0xffffffff); */\n __m128i const product = _mm_mul_epu32 (data_key, data_key_lo);\n /* xacc[i] += swap(data_vec); */\n __m128i const data_swap = _mm_shuffle_epi32(data_vec, _MM_SHUFFLE(1,0,3,2));\n __m128i const sum = _mm_add_epi64(xacc[i], data_swap);\n /* xacc[i] += product; */\n xacc[i] = _mm_add_epi64(product, sum);\n } }\n}\nXXH_FORCE_INLINE XXH_TARGET_SSE2 XXH3_ACCUMULATE_TEMPLATE(sse2)\n\nXXH_FORCE_INLINE XXH_TARGET_SSE2 void\nXXH3_scrambleAcc_sse2(void* XXH_RESTRICT acc, const void* XXH_RESTRICT secret)\n{\n XXH_ASSERT((((size_t)acc) & 15) == 0);\n { __m128i* const xacc = (__m128i*) acc;\n /* Unaligned. This is mainly for pointer arithmetic, and because\n * _mm_loadu_si128 requires a const __m128i * pointer for some reason. */\n const __m128i* const xsecret = (const __m128i *) secret;\n const __m128i prime32 = _mm_set1_epi32((int)XXH_PRIME32_1);\n\n size_t i;\n for (i=0; i < XXH_STRIPE_LEN/sizeof(__m128i); i++) {\n /* xacc[i] ^= (xacc[i] >> 47) */\n __m128i const acc_vec = xacc[i];\n __m128i const shifted = _mm_srli_epi64 (acc_vec, 47);\n __m128i const data_vec = _mm_xor_si128 (acc_vec, shifted);\n /* xacc[i] ^= xsecret[i]; */\n __m128i const key_vec = _mm_loadu_si128 (xsecret+i);\n __m128i const data_key = _mm_xor_si128 (data_vec, key_vec);\n\n /* xacc[i] *= XXH_PRIME32_1; */\n __m128i const data_key_hi = _mm_shuffle_epi32 (data_key, _MM_SHUFFLE(0, 3, 0, 1));\n __m128i const prod_lo = _mm_mul_epu32 (data_key, prime32);\n __m128i const prod_hi = _mm_mul_epu32 (data_key_hi, prime32);\n xacc[i] = _mm_add_epi64(prod_lo, _mm_slli_epi64(prod_hi, 32));\n }\n }\n}\n\nXXH_FORCE_INLINE XXH_TARGET_SSE2 void XXH3_initCustomSecret_sse2(void* XXH_RESTRICT customSecret, xxh_u64 seed64)\n{\n XXH_STATIC_ASSERT((XXH_SECRET_DEFAULT_SIZE & 15) == 0);\n (void)(&XXH_writeLE64);\n { int const nbRounds = XXH_SECRET_DEFAULT_SIZE / sizeof(__m128i);\n\n# if defined(_MSC_VER) && defined(_M_IX86) && _MSC_VER < 1900\n /* MSVC 32bit mode does not support _mm_set_epi64x before 2015 */\n XXH_ALIGN(16) const xxh_i64 seed64x2[2] = { (xxh_i64)seed64, (xxh_i64)(0U - seed64) };\n __m128i const seed = _mm_load_si128((__m128i const*)seed64x2);\n# else\n __m128i const seed = _mm_set_epi64x((xxh_i64)(0U - seed64), (xxh_i64)seed64);\n# endif\n int i;\n\n const void* const src16 = XXH3_kSecret;\n __m128i* dst16 = (__m128i*) customSecret;\n# if defined(__GNUC__) || defined(__clang__)\n /*\n * On GCC & Clang, marking 'dest' as modified will cause the compiler:\n * - do not extract the secret from sse registers in the internal loop\n * - use less common registers, and avoid pushing these reg into stack\n */\n XXH_COMPILER_GUARD(dst16);\n# endif\n XXH_ASSERT(((size_t)src16 & 15) == 0); /* control alignment */\n XXH_ASSERT(((size_t)dst16 & 15) == 0);\n\n for (i=0; i < nbRounds; ++i) {\n dst16[i] = _mm_add_epi64(_mm_load_si128((const __m128i *)src16+i), seed);\n } }\n}\n\n#endif\n\n#if (XXH_VECTOR == XXH_NEON)\n\n/* forward declarations for the scalar routines */\nXXH_FORCE_INLINE void\nXXH3_scalarRound(void* XXH_RESTRICT acc, void const* XXH_RESTRICT input,\n void const* XXH_RESTRICT secret, size_t lane);\n\nXXH_FORCE_INLINE void\nXXH3_scalarScrambleRound(void* XXH_RESTRICT acc,\n void const* XXH_RESTRICT secret, size_t lane);\n\n/*!\n * @internal\n * @brief The bulk processing loop for NEON and WASM SIMD128.\n *\n * The NEON code path is actually partially scalar when running on AArch64. This\n * is to optimize the pipelining and can have up to 15% speedup depending on the\n * CPU, and it also mitigates some GCC codegen issues.\n *\n * @see XXH3_NEON_LANES for configuring this and details about this optimization.\n *\n * NEON's 32-bit to 64-bit long multiply takes a half vector of 32-bit\n * integers instead of the other platforms which mask full 64-bit vectors,\n * so the setup is more complicated than just shifting right.\n *\n * Additionally, there is an optimization for 4 lanes at once noted below.\n *\n * Since, as stated, the most optimal amount of lanes for Cortexes is 6,\n * there needs to be *three* versions of the accumulate operation used\n * for the remaining 2 lanes.\n *\n * WASM's SIMD128 uses SIMDe's arm_neon.h polyfill because the intrinsics overlap\n * nearly perfectly.\n */\n\nXXH_FORCE_INLINE void\nXXH3_accumulate_512_neon( void* XXH_RESTRICT acc,\n const void* XXH_RESTRICT input,\n const void* XXH_RESTRICT secret)\n{\n XXH_ASSERT((((size_t)acc) & 15) == 0);\n XXH_STATIC_ASSERT(XXH3_NEON_LANES > 0 && XXH3_NEON_LANES <= XXH_ACC_NB && XXH3_NEON_LANES % 2 == 0);\n { /* GCC for darwin arm64 does not like aliasing here */\n xxh_aliasing_uint64x2_t* const xacc = (xxh_aliasing_uint64x2_t*) acc;\n /* We don't use a uint32x4_t pointer because it causes bus errors on ARMv7. */\n uint8_t const* xinput = (const uint8_t *) input;\n uint8_t const* xsecret = (const uint8_t *) secret;\n\n size_t i;\n#ifdef __wasm_simd128__\n /*\n * On WASM SIMD128, Clang emits direct address loads when XXH3_kSecret\n * is constant propagated, which results in it converting it to this\n * inside the loop:\n *\n * a = v128.load(XXH3_kSecret + 0 + $secret_offset, offset = 0)\n * b = v128.load(XXH3_kSecret + 16 + $secret_offset, offset = 0)\n * ...\n *\n * This requires a full 32-bit address immediate (and therefore a 6 byte\n * instruction) as well as an add for each offset.\n *\n * Putting an asm guard prevents it from folding (at the cost of losing\n * the alignment hint), and uses the free offset in `v128.load` instead\n * of adding secret_offset each time which overall reduces code size by\n * about a kilobyte and improves performance.\n */\n XXH_COMPILER_GUARD(xsecret);\n#endif\n /* Scalar lanes use the normal scalarRound routine */\n for (i = XXH3_NEON_LANES; i < XXH_ACC_NB; i++) {\n XXH3_scalarRound(acc, input, secret, i);\n }\n i = 0;\n /* 4 NEON lanes at a time. */\n for (; i+1 < XXH3_NEON_LANES / 2; i+=2) {\n /* data_vec = xinput[i]; */\n uint64x2_t data_vec_1 = XXH_vld1q_u64(xinput + (i * 16));\n uint64x2_t data_vec_2 = XXH_vld1q_u64(xinput + ((i+1) * 16));\n /* key_vec = xsecret[i]; */\n uint64x2_t key_vec_1 = XXH_vld1q_u64(xsecret + (i * 16));\n uint64x2_t key_vec_2 = XXH_vld1q_u64(xsecret + ((i+1) * 16));\n /* data_swap = swap(data_vec) */\n uint64x2_t data_swap_1 = vextq_u64(data_vec_1, data_vec_1, 1);\n uint64x2_t data_swap_2 = vextq_u64(data_vec_2, data_vec_2, 1);\n /* data_key = data_vec ^ key_vec; */\n uint64x2_t data_key_1 = veorq_u64(data_vec_1, key_vec_1);\n uint64x2_t data_key_2 = veorq_u64(data_vec_2, key_vec_2);\n\n /*\n * If we reinterpret the 64x2 vectors as 32x4 vectors, we can use a\n * de-interleave operation for 4 lanes in 1 step with `vuzpq_u32` to\n * get one vector with the low 32 bits of each lane, and one vector\n * with the high 32 bits of each lane.\n *\n * The intrinsic returns a double vector because the original ARMv7-a\n * instruction modified both arguments in place. AArch64 and SIMD128 emit\n * two instructions from this intrinsic.\n *\n * [ dk11L | dk11H | dk12L | dk12H ] -> [ dk11L | dk12L | dk21L | dk22L ]\n * [ dk21L | dk21H | dk22L | dk22H ] -> [ dk11H | dk12H | dk21H | dk22H ]\n */\n uint32x4x2_t unzipped = vuzpq_u32(\n vreinterpretq_u32_u64(data_key_1),\n vreinterpretq_u32_u64(data_key_2)\n );\n /* data_key_lo = data_key & 0xFFFFFFFF */\n uint32x4_t data_key_lo = unzipped.val[0];\n /* data_key_hi = data_key >> 32 */\n uint32x4_t data_key_hi = unzipped.val[1];\n /*\n * Then, we can split the vectors horizontally and multiply which, as for most\n * widening intrinsics, have a variant that works on both high half vectors\n * for free on AArch64. A similar instruction is available on SIMD128.\n *\n * sum = data_swap + (u64x2) data_key_lo * (u64x2) data_key_hi\n */\n uint64x2_t sum_1 = XXH_vmlal_low_u32(data_swap_1, data_key_lo, data_key_hi);\n uint64x2_t sum_2 = XXH_vmlal_high_u32(data_swap_2, data_key_lo, data_key_hi);\n /*\n * Clang reorders\n * a += b * c; // umlal swap.2d, dkl.2s, dkh.2s\n * c += a; // add acc.2d, acc.2d, swap.2d\n * to\n * c += a; // add acc.2d, acc.2d, swap.2d\n * c += b * c; // umlal acc.2d, dkl.2s, dkh.2s\n *\n * While it would make sense in theory since the addition is faster,\n * for reasons likely related to umlal being limited to certain NEON\n * pipelines, this is worse. A compiler guard fixes this.\n */\n XXH_COMPILER_GUARD_CLANG_NEON(sum_1);\n XXH_COMPILER_GUARD_CLANG_NEON(sum_2);\n /* xacc[i] = acc_vec + sum; */\n xacc[i] = vaddq_u64(xacc[i], sum_1);\n xacc[i+1] = vaddq_u64(xacc[i+1], sum_2);\n }\n /* Operate on the remaining NEON lanes 2 at a time. */\n for (; i < XXH3_NEON_LANES / 2; i++) {\n /* data_vec = xinput[i]; */\n uint64x2_t data_vec = XXH_vld1q_u64(xinput + (i * 16));\n /* key_vec = xsecret[i]; */\n uint64x2_t key_vec = XXH_vld1q_u64(xsecret + (i * 16));\n /* acc_vec_2 = swap(data_vec) */\n uint64x2_t data_swap = vextq_u64(data_vec, data_vec, 1);\n /* data_key = data_vec ^ key_vec; */\n uint64x2_t data_key = veorq_u64(data_vec, key_vec);\n /* For two lanes, just use VMOVN and VSHRN. */\n /* data_key_lo = data_key & 0xFFFFFFFF; */\n uint32x2_t data_key_lo = vmovn_u64(data_key);\n /* data_key_hi = data_key >> 32; */\n uint32x2_t data_key_hi = vshrn_n_u64(data_key, 32);\n /* sum = data_swap + (u64x2) data_key_lo * (u64x2) data_key_hi; */\n uint64x2_t sum = vmlal_u32(data_swap, data_key_lo, data_key_hi);\n /* Same Clang workaround as before */\n XXH_COMPILER_GUARD_CLANG_NEON(sum);\n /* xacc[i] = acc_vec + sum; */\n xacc[i] = vaddq_u64 (xacc[i], sum);\n }\n }\n}\nXXH_FORCE_INLINE XXH3_ACCUMULATE_TEMPLATE(neon)\n\nXXH_FORCE_INLINE void\nXXH3_scrambleAcc_neon(void* XXH_RESTRICT acc, const void* XXH_RESTRICT secret)\n{\n XXH_ASSERT((((size_t)acc) & 15) == 0);\n\n { xxh_aliasing_uint64x2_t* xacc = (xxh_aliasing_uint64x2_t*) acc;\n uint8_t const* xsecret = (uint8_t const*) secret;\n\n size_t i;\n /* WASM uses operator overloads and doesn't need these. */\n#ifndef __wasm_simd128__\n /* { prime32_1, prime32_1 } */\n uint32x2_t const kPrimeLo = vdup_n_u32(XXH_PRIME32_1);\n /* { 0, prime32_1, 0, prime32_1 } */\n uint32x4_t const kPrimeHi = vreinterpretq_u32_u64(vdupq_n_u64((xxh_u64)XXH_PRIME32_1 << 32));\n#endif\n\n /* AArch64 uses both scalar and neon at the same time */\n for (i = XXH3_NEON_LANES; i < XXH_ACC_NB; i++) {\n XXH3_scalarScrambleRound(acc, secret, i);\n }\n for (i=0; i < XXH3_NEON_LANES / 2; i++) {\n /* xacc[i] ^= (xacc[i] >> 47); */\n uint64x2_t acc_vec = xacc[i];\n uint64x2_t shifted = vshrq_n_u64(acc_vec, 47);\n uint64x2_t data_vec = veorq_u64(acc_vec, shifted);\n\n /* xacc[i] ^= xsecret[i]; */\n uint64x2_t key_vec = XXH_vld1q_u64(xsecret + (i * 16));\n uint64x2_t data_key = veorq_u64(data_vec, key_vec);\n /* xacc[i] *= XXH_PRIME32_1 */\n#ifdef __wasm_simd128__\n /* SIMD128 has multiply by u64x2, use it instead of expanding and scalarizing */\n xacc[i] = data_key * XXH_PRIME32_1;\n#else\n /*\n * Expanded version with portable NEON intrinsics\n *\n * lo(x) * lo(y) + (hi(x) * lo(y) << 32)\n *\n * prod_hi = hi(data_key) * lo(prime) << 32\n *\n * Since we only need 32 bits of this multiply a trick can be used, reinterpreting the vector\n * as a uint32x4_t and multiplying by { 0, prime, 0, prime } to cancel out the unwanted bits\n * and avoid the shift.\n */\n uint32x4_t prod_hi = vmulq_u32 (vreinterpretq_u32_u64(data_key), kPrimeHi);\n /* Extract low bits for vmlal_u32 */\n uint32x2_t data_key_lo = vmovn_u64(data_key);\n /* xacc[i] = prod_hi + lo(data_key) * XXH_PRIME32_1; */\n xacc[i] = vmlal_u32(vreinterpretq_u64_u32(prod_hi), data_key_lo, kPrimeLo);\n#endif\n }\n }\n}\n#endif\n\n#if (XXH_VECTOR == XXH_VSX)\n\nXXH_FORCE_INLINE void\nXXH3_accumulate_512_vsx( void* XXH_RESTRICT acc,\n const void* XXH_RESTRICT input,\n const void* XXH_RESTRICT secret)\n{\n /* presumed aligned */\n xxh_aliasing_u64x2* const xacc = (xxh_aliasing_u64x2*) acc;\n xxh_u8 const* const xinput = (xxh_u8 const*) input; /* no alignment restriction */\n xxh_u8 const* const xsecret = (xxh_u8 const*) secret; /* no alignment restriction */\n xxh_u64x2 const v32 = { 32, 32 };\n size_t i;\n for (i = 0; i < XXH_STRIPE_LEN / sizeof(xxh_u64x2); i++) {\n /* data_vec = xinput[i]; */\n xxh_u64x2 const data_vec = XXH_vec_loadu(xinput + 16*i);\n /* key_vec = xsecret[i]; */\n xxh_u64x2 const key_vec = XXH_vec_loadu(xsecret + 16*i);\n xxh_u64x2 const data_key = data_vec ^ key_vec;\n /* shuffled = (data_key << 32) | (data_key >> 32); */\n xxh_u32x4 const shuffled = (xxh_u32x4)vec_rl(data_key, v32);\n /* product = ((xxh_u64x2)data_key & 0xFFFFFFFF) * ((xxh_u64x2)shuffled & 0xFFFFFFFF); */\n xxh_u64x2 const product = XXH_vec_mulo((xxh_u32x4)data_key, shuffled);\n /* acc_vec = xacc[i]; */\n xxh_u64x2 acc_vec = xacc[i];\n acc_vec += product;\n\n /* swap high and low halves */\n#ifdef __s390x__\n acc_vec += vec_permi(data_vec, data_vec, 2);\n#else\n acc_vec += vec_xxpermdi(data_vec, data_vec, 2);\n#endif\n xacc[i] = acc_vec;\n }\n}\nXXH_FORCE_INLINE XXH3_ACCUMULATE_TEMPLATE(vsx)\n\nXXH_FORCE_INLINE void\nXXH3_scrambleAcc_vsx(void* XXH_RESTRICT acc, const void* XXH_RESTRICT secret)\n{\n XXH_ASSERT((((size_t)acc) & 15) == 0);\n\n { xxh_aliasing_u64x2* const xacc = (xxh_aliasing_u64x2*) acc;\n const xxh_u8* const xsecret = (const xxh_u8*) secret;\n /* constants */\n xxh_u64x2 const v32 = { 32, 32 };\n xxh_u64x2 const v47 = { 47, 47 };\n xxh_u32x4 const prime = { XXH_PRIME32_1, XXH_PRIME32_1, XXH_PRIME32_1, XXH_PRIME32_1 };\n size_t i;\n for (i = 0; i < XXH_STRIPE_LEN / sizeof(xxh_u64x2); i++) {\n /* xacc[i] ^= (xacc[i] >> 47); */\n xxh_u64x2 const acc_vec = xacc[i];\n xxh_u64x2 const data_vec = acc_vec ^ (acc_vec >> v47);\n\n /* xacc[i] ^= xsecret[i]; */\n xxh_u64x2 const key_vec = XXH_vec_loadu(xsecret + 16*i);\n xxh_u64x2 const data_key = data_vec ^ key_vec;\n\n /* xacc[i] *= XXH_PRIME32_1 */\n /* prod_lo = ((xxh_u64x2)data_key & 0xFFFFFFFF) * ((xxh_u64x2)prime & 0xFFFFFFFF); */\n xxh_u64x2 const prod_even = XXH_vec_mule((xxh_u32x4)data_key, prime);\n /* prod_hi = ((xxh_u64x2)data_key >> 32) * ((xxh_u64x2)prime >> 32); */\n xxh_u64x2 const prod_odd = XXH_vec_mulo((xxh_u32x4)data_key, prime);\n xacc[i] = prod_odd + (prod_even << v32);\n } }\n}\n\n#endif\n\n#if (XXH_VECTOR == XXH_SVE)\n\nXXH_FORCE_INLINE void\nXXH3_accumulate_512_sve( void* XXH_RESTRICT acc,\n const void* XXH_RESTRICT input,\n const void* XXH_RESTRICT secret)\n{\n uint64_t *xacc = (uint64_t *)acc;\n const uint64_t *xinput = (const uint64_t *)(const void *)input;\n const uint64_t *xsecret = (const uint64_t *)(const void *)secret;\n svuint64_t kSwap = sveor_n_u64_z(svptrue_b64(), svindex_u64(0, 1), 1);\n uint64_t element_count = svcntd();\n if (element_count >= 8) {\n svbool_t mask = svptrue_pat_b64(SV_VL8);\n svuint64_t vacc = svld1_u64(mask, xacc);\n ACCRND(vacc, 0);\n svst1_u64(mask, xacc, vacc);\n } else if (element_count == 2) { /* sve128 */\n svbool_t mask = svptrue_pat_b64(SV_VL2);\n svuint64_t acc0 = svld1_u64(mask, xacc + 0);\n svuint64_t acc1 = svld1_u64(mask, xacc + 2);\n svuint64_t acc2 = svld1_u64(mask, xacc + 4);\n svuint64_t acc3 = svld1_u64(mask, xacc + 6);\n ACCRND(acc0, 0);\n ACCRND(acc1, 2);\n ACCRND(acc2, 4);\n ACCRND(acc3, 6);\n svst1_u64(mask, xacc + 0, acc0);\n svst1_u64(mask, xacc + 2, acc1);\n svst1_u64(mask, xacc + 4, acc2);\n svst1_u64(mask, xacc + 6, acc3);\n } else {\n svbool_t mask = svptrue_pat_b64(SV_VL4);\n svuint64_t acc0 = svld1_u64(mask, xacc + 0);\n svuint64_t acc1 = svld1_u64(mask, xacc + 4);\n ACCRND(acc0, 0);\n ACCRND(acc1, 4);\n svst1_u64(mask, xacc + 0, acc0);\n svst1_u64(mask, xacc + 4, acc1);\n }\n}\n\nXXH_FORCE_INLINE void\nXXH3_accumulate_sve(xxh_u64* XXH_RESTRICT acc,\n const xxh_u8* XXH_RESTRICT input,\n const xxh_u8* XXH_RESTRICT secret,\n size_t nbStripes)\n{\n if (nbStripes != 0) {\n uint64_t *xacc = (uint64_t *)acc;\n const uint64_t *xinput = (const uint64_t *)(const void *)input;\n const uint64_t *xsecret = (const uint64_t *)(const void *)secret;\n svuint64_t kSwap = sveor_n_u64_z(svptrue_b64(), svindex_u64(0, 1), 1);\n uint64_t element_count = svcntd();\n if (element_count >= 8) {\n svbool_t mask = svptrue_pat_b64(SV_VL8);\n svuint64_t vacc = svld1_u64(mask, xacc + 0);\n do {\n /* svprfd(svbool_t, void *, enum svfprop); */\n svprfd(mask, xinput + 128, SV_PLDL1STRM);\n ACCRND(vacc, 0);\n xinput += 8;\n xsecret += 1;\n nbStripes--;\n } while (nbStripes != 0);\n\n svst1_u64(mask, xacc + 0, vacc);\n } else if (element_count == 2) { /* sve128 */\n svbool_t mask = svptrue_pat_b64(SV_VL2);\n svuint64_t acc0 = svld1_u64(mask, xacc + 0);\n svuint64_t acc1 = svld1_u64(mask, xacc + 2);\n svuint64_t acc2 = svld1_u64(mask, xacc + 4);\n svuint64_t acc3 = svld1_u64(mask, xacc + 6);\n do {\n svprfd(mask, xinput + 128, SV_PLDL1STRM);\n ACCRND(acc0, 0);\n ACCRND(acc1, 2);\n ACCRND(acc2, 4);\n ACCRND(acc3, 6);\n xinput += 8;\n xsecret += 1;\n nbStripes--;\n } while (nbStripes != 0);\n\n svst1_u64(mask, xacc + 0, acc0);\n svst1_u64(mask, xacc + 2, acc1);\n svst1_u64(mask, xacc + 4, acc2);\n svst1_u64(mask, xacc + 6, acc3);\n } else {\n svbool_t mask = svptrue_pat_b64(SV_VL4);\n svuint64_t acc0 = svld1_u64(mask, xacc + 0);\n svuint64_t acc1 = svld1_u64(mask, xacc + 4);\n do {\n svprfd(mask, xinput + 128, SV_PLDL1STRM);\n ACCRND(acc0, 0);\n ACCRND(acc1, 4);\n xinput += 8;\n xsecret += 1;\n nbStripes--;\n } while (nbStripes != 0);\n\n svst1_u64(mask, xacc + 0, acc0);\n svst1_u64(mask, xacc + 4, acc1);\n }\n }\n}\n\n#endif\n\n#if (XXH_VECTOR == XXH_LSX)\n#define _LSX_SHUFFLE(z, y, x, w) (((z) << 6) | ((y) << 4) | ((x) << 2) | (w))\n\nXXH_FORCE_INLINE void\nXXH3_accumulate_512_lsx( void* XXH_RESTRICT acc,\n const void* XXH_RESTRICT input,\n const void* XXH_RESTRICT secret)\n{\n XXH_ASSERT((((size_t)acc) & 15) == 0);\n {\n __m128i* const xacc = (__m128i *) acc;\n const __m128i* const xinput = (const __m128i *) input;\n const __m128i* const xsecret = (const __m128i *) secret;\n\n for (size_t i = 0; i < XXH_STRIPE_LEN / sizeof(__m128i); i++) {\n /* data_vec = xinput[i]; */\n __m128i const data_vec = __lsx_vld(xinput + i, 0);\n /* key_vec = xsecret[i]; */\n __m128i const key_vec = __lsx_vld(xsecret + i, 0);\n /* data_key = data_vec ^ key_vec; */\n __m128i const data_key = __lsx_vxor_v(data_vec, key_vec);\n /* data_key_lo = data_key >> 32; */\n __m128i const data_key_lo = __lsx_vsrli_d(data_key, 32);\n // __m128i const data_key_lo = __lsx_vsrli_d(data_key, 32);\n /* product = (data_key & 0xffffffff) * (data_key_lo & 0xffffffff); */\n __m128i const product = __lsx_vmulwev_d_wu(data_key, data_key_lo);\n /* xacc[i] += swap(data_vec); */\n __m128i const data_swap = __lsx_vshuf4i_w(data_vec, _LSX_SHUFFLE(1, 0, 3, 2));\n __m128i const sum = __lsx_vadd_d(xacc[i], data_swap);\n /* xacc[i] += product; */\n xacc[i] = __lsx_vadd_d(product, sum);\n }\n }\n}\nXXH_FORCE_INLINE XXH3_ACCUMULATE_TEMPLATE(lsx)\n\nXXH_FORCE_INLINE void\nXXH3_scrambleAcc_lsx(void* XXH_RESTRICT acc, const void* XXH_RESTRICT secret)\n{\n XXH_ASSERT((((size_t)acc) & 15) == 0);\n {\n __m128i* const xacc = (__m128i*) acc;\n const __m128i* const xsecret = (const __m128i *) secret;\n const __m128i prime32 = __lsx_vreplgr2vr_w((int)XXH_PRIME32_1);\n\n for (size_t i = 0; i < XXH_STRIPE_LEN / sizeof(__m128i); i++) {\n /* xacc[i] ^= (xacc[i] >> 47) */\n __m128i const acc_vec = xacc[i];\n __m128i const shifted = __lsx_vsrli_d(acc_vec, 47);\n __m128i const data_vec = __lsx_vxor_v(acc_vec, shifted);\n /* xacc[i] ^= xsecret[i]; */\n __m128i const key_vec = __lsx_vld(xsecret + i, 0);\n __m128i const data_key = __lsx_vxor_v(data_vec, key_vec);\n\n /* xacc[i] *= XXH_PRIME32_1; */\n __m128i const data_key_hi = __lsx_vsrli_d(data_key, 32);\n __m128i const prod_lo = __lsx_vmulwev_d_wu(data_key, prime32);\n __m128i const prod_hi = __lsx_vmulwev_d_wu(data_key_hi, prime32);\n xacc[i] = __lsx_vadd_d(prod_lo, __lsx_vslli_d(prod_hi, 32));\n }\n }\n}\n\n#endif\n\n/* scalar variants - universal */\n\n#if defined(__aarch64__) && (defined(__GNUC__) || defined(__clang__))\n/*\n * In XXH3_scalarRound(), GCC and Clang have a similar codegen issue, where they\n * emit an excess mask and a full 64-bit multiply-add (MADD X-form).\n *\n * While this might not seem like much, as AArch64 is a 64-bit architecture, only\n * big Cortex designs have a full 64-bit multiplier.\n *\n * On the little cores, the smaller 32-bit multiplier is used, and full 64-bit\n * multiplies expand to 2-3 multiplies in microcode. This has a major penalty\n * of up to 4 latency cycles and 2 stall cycles in the multiply pipeline.\n *\n * Thankfully, AArch64 still provides the 32-bit long multiply-add (UMADDL) which does\n * not have this penalty and does the mask automatically.\n */\nXXH_FORCE_INLINE xxh_u64\nXXH_mult32to64_add64(xxh_u64 lhs, xxh_u64 rhs, xxh_u64 acc)\n{\n xxh_u64 ret;\n /* note: %x = 64-bit register, %w = 32-bit register */\n __asm__(\"umaddl %x0, %w1, %w2, %x3\" : \"=r\" (ret) : \"r\" (lhs), \"r\" (rhs), \"r\" (acc));\n return ret;\n}\n#else\nXXH_FORCE_INLINE xxh_u64\nXXH_mult32to64_add64(xxh_u64 lhs, xxh_u64 rhs, xxh_u64 acc)\n{\n return XXH_mult32to64((xxh_u32)lhs, (xxh_u32)rhs) + acc;\n}\n#endif\n\n/*!\n * @internal\n * @brief Scalar round for @ref XXH3_accumulate_512_scalar().\n *\n * This is extracted to its own function because the NEON path uses a combination\n * of NEON and scalar.\n */\nXXH_FORCE_INLINE void\nXXH3_scalarRound(void* XXH_RESTRICT acc,\n void const* XXH_RESTRICT input,\n void const* XXH_RESTRICT secret,\n size_t lane)\n{\n xxh_u64* xacc = (xxh_u64*) acc;\n xxh_u8 const* xinput = (xxh_u8 const*) input;\n xxh_u8 const* xsecret = (xxh_u8 const*) secret;\n XXH_ASSERT(lane < XXH_ACC_NB);\n XXH_ASSERT(((size_t)acc & (XXH_ACC_ALIGN-1)) == 0);\n {\n xxh_u64 const data_val = XXH_readLE64(xinput + lane * 8);\n xxh_u64 const data_key = data_val ^ XXH_readLE64(xsecret + lane * 8);\n xacc[lane ^ 1] += data_val; /* swap adjacent lanes */\n xacc[lane] = XXH_mult32to64_add64(data_key /* & 0xFFFFFFFF */, data_key >> 32, xacc[lane]);\n }\n}\n\n/*!\n * @internal\n * @brief Processes a 64 byte block of data using the scalar path.\n */\nXXH_FORCE_INLINE void\nXXH3_accumulate_512_scalar(void* XXH_RESTRICT acc,\n const void* XXH_RESTRICT input,\n const void* XXH_RESTRICT secret)\n{\n size_t i;\n /* ARM GCC refuses to unroll this loop, resulting in a 24% slowdown on ARMv6. */\n#if defined(__GNUC__) && !defined(__clang__) \\\n && (defined(__arm__) || defined(__thumb2__)) \\\n && defined(__ARM_FEATURE_UNALIGNED) /* no unaligned access just wastes bytes */ \\\n && XXH_SIZE_OPT <= 0\n# pragma GCC unroll 8\n#endif\n for (i=0; i < XXH_ACC_NB; i++) {\n XXH3_scalarRound(acc, input, secret, i);\n }\n}\nXXH_FORCE_INLINE XXH3_ACCUMULATE_TEMPLATE(scalar)\n\n/*!\n * @internal\n * @brief Scalar scramble step for @ref XXH3_scrambleAcc_scalar().\n *\n * This is extracted to its own function because the NEON path uses a combination\n * of NEON and scalar.\n */\nXXH_FORCE_INLINE void\nXXH3_scalarScrambleRound(void* XXH_RESTRICT acc,\n void const* XXH_RESTRICT secret,\n size_t lane)\n{\n xxh_u64* const xacc = (xxh_u64*) acc; /* presumed aligned */\n const xxh_u8* const xsecret = (const xxh_u8*) secret; /* no alignment restriction */\n XXH_ASSERT((((size_t)acc) & (XXH_ACC_ALIGN-1)) == 0);\n XXH_ASSERT(lane < XXH_ACC_NB);\n {\n xxh_u64 const key64 = XXH_readLE64(xsecret + lane * 8);\n xxh_u64 acc64 = xacc[lane];\n acc64 = XXH_xorshift64(acc64, 47);\n acc64 ^= key64;\n acc64 *= XXH_PRIME32_1;\n xacc[lane] = acc64;\n }\n}\n\n/*!\n * @internal\n * @brief Scrambles the accumulators after a large chunk has been read\n */\nXXH_FORCE_INLINE void\nXXH3_scrambleAcc_scalar(void* XXH_RESTRICT acc, const void* XXH_RESTRICT secret)\n{\n size_t i;\n for (i=0; i < XXH_ACC_NB; i++) {\n XXH3_scalarScrambleRound(acc, secret, i);\n }\n}\n\nXXH_FORCE_INLINE void\nXXH3_initCustomSecret_scalar(void* XXH_RESTRICT customSecret, xxh_u64 seed64)\n{\n /*\n * We need a separate pointer for the hack below,\n * which requires a non-const pointer.\n * Any decent compiler will optimize this out otherwise.\n */\n const xxh_u8* kSecretPtr = XXH3_kSecret;\n XXH_STATIC_ASSERT((XXH_SECRET_DEFAULT_SIZE & 15) == 0);\n\n#if defined(__GNUC__) && defined(__aarch64__)\n /*\n * UGLY HACK:\n * GCC and Clang generate a bunch of MOV/MOVK pairs for aarch64, and they are\n * placed sequentially, in order, at the top of the unrolled loop.\n *\n * While MOVK is great for generating constants (2 cycles for a 64-bit\n * constant compared to 4 cycles for LDR), it fights for bandwidth with\n * the arithmetic instructions.\n *\n * I L S\n * MOVK\n * MOVK\n * MOVK\n * MOVK\n * ADD\n * SUB STR\n * STR\n * By forcing loads from memory (as the asm line causes the compiler to assume\n * that XXH3_kSecretPtr has been changed), the pipelines are used more\n * efficiently:\n * I L S\n * LDR\n * ADD LDR\n * SUB STR\n * STR\n *\n * See XXH3_NEON_LANES for details on the pipsline.\n *\n * XXH3_64bits_withSeed, len == 256, Snapdragon 835\n * without hack: 2654.4 MB/s\n * with hack: 3202.9 MB/s\n */\n XXH_COMPILER_GUARD(kSecretPtr);\n#endif\n { int const nbRounds = XXH_SECRET_DEFAULT_SIZE / 16;\n int i;\n for (i=0; i < nbRounds; i++) {\n /*\n * The asm hack causes the compiler to assume that kSecretPtr aliases with\n * customSecret, and on aarch64, this prevented LDP from merging two\n * loads together for free. Putting the loads together before the stores\n * properly generates LDP.\n */\n xxh_u64 lo = XXH_readLE64(kSecretPtr + 16*i) + seed64;\n xxh_u64 hi = XXH_readLE64(kSecretPtr + 16*i + 8) - seed64;\n XXH_writeLE64((xxh_u8*)customSecret + 16*i, lo);\n XXH_writeLE64((xxh_u8*)customSecret + 16*i + 8, hi);\n } }\n}\n\n\ntypedef void (*XXH3_f_accumulate)(xxh_u64* XXH_RESTRICT, const xxh_u8* XXH_RESTRICT, const xxh_u8* XXH_RESTRICT, size_t);\ntypedef void (*XXH3_f_scrambleAcc)(void* XXH_RESTRICT, const void*);\ntypedef void (*XXH3_f_initCustomSecret)(void* XXH_RESTRICT, xxh_u64);\n\n\n#if (XXH_VECTOR == XXH_AVX512)\n\n#define XXH3_accumulate_512 XXH3_accumulate_512_avx512\n#define XXH3_accumulate XXH3_accumulate_avx512\n#define XXH3_scrambleAcc XXH3_scrambleAcc_avx512\n#define XXH3_initCustomSecret XXH3_initCustomSecret_avx512\n\n#elif (XXH_VECTOR == XXH_AVX2)\n\n#define XXH3_accumulate_512 XXH3_accumulate_512_avx2\n#define XXH3_accumulate XXH3_accumulate_avx2\n#define XXH3_scrambleAcc XXH3_scrambleAcc_avx2\n#define XXH3_initCustomSecret XXH3_initCustomSecret_avx2\n\n#elif (XXH_VECTOR == XXH_SSE2)\n\n#define XXH3_accumulate_512 XXH3_accumulate_512_sse2\n#define XXH3_accumulate XXH3_accumulate_sse2\n#define XXH3_scrambleAcc XXH3_scrambleAcc_sse2\n#define XXH3_initCustomSecret XXH3_initCustomSecret_sse2\n\n#elif (XXH_VECTOR == XXH_NEON)\n\n#define XXH3_accumulate_512 XXH3_accumulate_512_neon\n#define XXH3_accumulate XXH3_accumulate_neon\n#define XXH3_scrambleAcc XXH3_scrambleAcc_neon\n#define XXH3_initCustomSecret XXH3_initCustomSecret_scalar\n\n#elif (XXH_VECTOR == XXH_VSX)\n\n#define XXH3_accumulate_512 XXH3_accumulate_512_vsx\n#define XXH3_accumulate XXH3_accumulate_vsx\n#define XXH3_scrambleAcc XXH3_scrambleAcc_vsx\n#define XXH3_initCustomSecret XXH3_initCustomSecret_scalar\n\n#elif (XXH_VECTOR == XXH_SVE)\n#define XXH3_accumulate_512 XXH3_accumulate_512_sve\n#define XXH3_accumulate XXH3_accumulate_sve\n#define XXH3_scrambleAcc XXH3_scrambleAcc_scalar\n#define XXH3_initCustomSecret XXH3_initCustomSecret_scalar\n\n#elif (XXH_VECTOR == XXH_LSX)\n#define XXH3_accumulate_512 XXH3_accumulate_512_lsx\n#define XXH3_accumulate XXH3_accumulate_lsx\n#define XXH3_scrambleAcc XXH3_scrambleAcc_lsx\n#define XXH3_initCustomSecret XXH3_initCustomSecret_scalar\n\n#else /* scalar */\n\n#define XXH3_accumulate_512 XXH3_accumulate_512_scalar\n#define XXH3_accumulate XXH3_accumulate_scalar\n#define XXH3_scrambleAcc XXH3_scrambleAcc_scalar\n#define XXH3_initCustomSecret XXH3_initCustomSecret_scalar\n\n#endif\n\n#if XXH_SIZE_OPT >= 1 /* don't do SIMD for initialization */\n# undef XXH3_initCustomSecret\n# define XXH3_initCustomSecret XXH3_initCustomSecret_scalar\n#endif\n\nXXH_FORCE_INLINE void\nXXH3_hashLong_internal_loop(xxh_u64* XXH_RESTRICT acc,\n const xxh_u8* XXH_RESTRICT input, size_t len,\n const xxh_u8* XXH_RESTRICT secret, size_t secretSize,\n XXH3_f_accumulate f_acc,\n XXH3_f_scrambleAcc f_scramble)\n{\n size_t const nbStripesPerBlock = (secretSize - XXH_STRIPE_LEN) / XXH_SECRET_CONSUME_RATE;\n size_t const block_len = XXH_STRIPE_LEN * nbStripesPerBlock;\n size_t const nb_blocks = (len - 1) / block_len;\n\n size_t n;\n\n XXH_ASSERT(secretSize >= XXH3_SECRET_SIZE_MIN);\n\n for (n = 0; n < nb_blocks; n++) {\n f_acc(acc, input + n*block_len, secret, nbStripesPerBlock);\n f_scramble(acc, secret + secretSize - XXH_STRIPE_LEN);\n }\n\n /* last partial block */\n XXH_ASSERT(len > XXH_STRIPE_LEN);\n { size_t const nbStripes = ((len - 1) - (block_len * nb_blocks)) / XXH_STRIPE_LEN;\n XXH_ASSERT(nbStripes <= (secretSize / XXH_SECRET_CONSUME_RATE));\n f_acc(acc, input + nb_blocks*block_len, secret, nbStripes);\n\n /* last stripe */\n { const xxh_u8* const p = input + len - XXH_STRIPE_LEN;\n#define XXH_SECRET_LASTACC_START 7 /* not aligned on 8, last secret is different from acc & scrambler */\n XXH3_accumulate_512(acc, p, secret + secretSize - XXH_STRIPE_LEN - XXH_SECRET_LASTACC_START);\n } }\n}\n\nXXH_FORCE_INLINE xxh_u64\nXXH3_mix2Accs(const xxh_u64* XXH_RESTRICT acc, const xxh_u8* XXH_RESTRICT secret)\n{\n return XXH3_mul128_fold64(\n acc[0] ^ XXH_readLE64(secret),\n acc[1] ^ XXH_readLE64(secret+8) );\n}\n\nstatic XXH_PUREF XXH64_hash_t\nXXH3_mergeAccs(const xxh_u64* XXH_RESTRICT acc, const xxh_u8* XXH_RESTRICT secret, xxh_u64 start)\n{\n xxh_u64 result64 = start;\n size_t i = 0;\n\n for (i = 0; i < 4; i++) {\n result64 += XXH3_mix2Accs(acc+2*i, secret + 16*i);\n#if defined(__clang__) /* Clang */ \\\n && (defined(__arm__) || defined(__thumb__)) /* ARMv7 */ \\\n && (defined(__ARM_NEON) || defined(__ARM_NEON__)) /* NEON */ \\\n && !defined(XXH_ENABLE_AUTOVECTORIZE) /* Define to disable */\n /*\n * UGLY HACK:\n * Prevent autovectorization on Clang ARMv7-a. Exact same problem as\n * the one in XXH3_len_129to240_64b. Speeds up shorter keys > 240b.\n * XXH3_64bits, len == 256, Snapdragon 835:\n * without hack: 2063.7 MB/s\n * with hack: 2560.7 MB/s\n */\n XXH_COMPILER_GUARD(result64);\n#endif\n }\n\n return XXH3_avalanche(result64);\n}\n\n/* do not align on 8, so that the secret is different from the accumulator */\n#define XXH_SECRET_MERGEACCS_START 11\n\nstatic XXH_PUREF XXH64_hash_t\nXXH3_finalizeLong_64b(const xxh_u64* XXH_RESTRICT acc, const xxh_u8* XXH_RESTRICT secret, xxh_u64 len)\n{\n return XXH3_mergeAccs(acc, secret + XXH_SECRET_MERGEACCS_START, len * XXH_PRIME64_1);\n}\n\n#define XXH3_INIT_ACC { XXH_PRIME32_3, XXH_PRIME64_1, XXH_PRIME64_2, XXH_PRIME64_3, \\\n XXH_PRIME64_4, XXH_PRIME32_2, XXH_PRIME64_5, XXH_PRIME32_1 }\n\nXXH_FORCE_INLINE XXH64_hash_t\nXXH3_hashLong_64b_internal(const void* XXH_RESTRICT input, size_t len,\n const void* XXH_RESTRICT secret, size_t secretSize,\n XXH3_f_accumulate f_acc,\n XXH3_f_scrambleAcc f_scramble)\n{\n XXH_ALIGN(XXH_ACC_ALIGN) xxh_u64 acc[XXH_ACC_NB] = XXH3_INIT_ACC;\n\n XXH3_hashLong_internal_loop(acc, (const xxh_u8*)input, len, (const xxh_u8*)secret, secretSize, f_acc, f_scramble);\n\n /* converge into final hash */\n XXH_STATIC_ASSERT(sizeof(acc) == 64);\n XXH_ASSERT(secretSize >= sizeof(acc) + XXH_SECRET_MERGEACCS_START);\n return XXH3_finalizeLong_64b(acc, (const xxh_u8*)secret, (xxh_u64)len);\n}\n\n/*\n * It's important for performance to transmit secret's size (when it's static)\n * so that the compiler can properly optimize the vectorized loop.\n * This makes a big performance difference for \"medium\" keys (<1 KB) when using AVX instruction set.\n * When the secret size is unknown, or on GCC 12 where the mix of NO_INLINE and FORCE_INLINE\n * breaks -Og, this is XXH_NO_INLINE.\n */\nXXH3_WITH_SECRET_INLINE XXH64_hash_t\nXXH3_hashLong_64b_withSecret(const void* XXH_RESTRICT input, size_t len,\n XXH64_hash_t seed64, const xxh_u8* XXH_RESTRICT secret, size_t secretLen)\n{\n (void)seed64;\n return XXH3_hashLong_64b_internal(input, len, secret, secretLen, XXH3_accumulate, XXH3_scrambleAcc);\n}\n\n/*\n * It's preferable for performance that XXH3_hashLong is not inlined,\n * as it results in a smaller function for small data, easier to the instruction cache.\n * Note that inside this no_inline function, we do inline the internal loop,\n * and provide a statically defined secret size to allow optimization of vector loop.\n */\nXXH_NO_INLINE XXH_PUREF XXH64_hash_t\nXXH3_hashLong_64b_default(const void* XXH_RESTRICT input, size_t len,\n XXH64_hash_t seed64, const xxh_u8* XXH_RESTRICT secret, size_t secretLen)\n{\n (void)seed64; (void)secret; (void)secretLen;\n return XXH3_hashLong_64b_internal(input, len, XXH3_kSecret, sizeof(XXH3_kSecret), XXH3_accumulate, XXH3_scrambleAcc);\n}\n\n/*\n * XXH3_hashLong_64b_withSeed():\n * Generate a custom key based on alteration of default XXH3_kSecret with the seed,\n * and then use this key for long mode hashing.\n *\n * This operation is decently fast but nonetheless costs a little bit of time.\n * Try to avoid it whenever possible (typically when seed==0).\n *\n * It's important for performance that XXH3_hashLong is not inlined. Not sure\n * why (uop cache maybe?), but the difference is large and easily measurable.\n */\nXXH_FORCE_INLINE XXH64_hash_t\nXXH3_hashLong_64b_withSeed_internal(const void* input, size_t len,\n XXH64_hash_t seed,\n XXH3_f_accumulate f_acc,\n XXH3_f_scrambleAcc f_scramble,\n XXH3_f_initCustomSecret f_initSec)\n{\n#if XXH_SIZE_OPT <= 0\n if (seed == 0)\n return XXH3_hashLong_64b_internal(input, len,\n XXH3_kSecret, sizeof(XXH3_kSecret),\n f_acc, f_scramble);\n#endif\n { XXH_ALIGN(XXH_SEC_ALIGN) xxh_u8 secret[XXH_SECRET_DEFAULT_SIZE];\n f_initSec(secret, seed);\n return XXH3_hashLong_64b_internal(input, len, secret, sizeof(secret),\n f_acc, f_scramble);\n }\n}\n\n/*\n * It's important for performance that XXH3_hashLong is not inlined.\n */\nXXH_NO_INLINE XXH64_hash_t\nXXH3_hashLong_64b_withSeed(const void* XXH_RESTRICT input, size_t len,\n XXH64_hash_t seed, const xxh_u8* XXH_RESTRICT secret, size_t secretLen)\n{\n (void)secret; (void)secretLen;\n return XXH3_hashLong_64b_withSeed_internal(input, len, seed,\n XXH3_accumulate, XXH3_scrambleAcc, XXH3_initCustomSecret);\n}\n\n\ntypedef XXH64_hash_t (*XXH3_hashLong64_f)(const void* XXH_RESTRICT, size_t,\n XXH64_hash_t, const xxh_u8* XXH_RESTRICT, size_t);\n\nXXH_FORCE_INLINE XXH64_hash_t\nXXH3_64bits_internal(const void* XXH_RESTRICT input, size_t len,\n XXH64_hash_t seed64, const void* XXH_RESTRICT secret, size_t secretLen,\n XXH3_hashLong64_f f_hashLong)\n{\n XXH_ASSERT(secretLen >= XXH3_SECRET_SIZE_MIN);\n /*\n * If an action is to be taken if `secretLen` condition is not respected,\n * it should be done here.\n * For now, it's a contract pre-condition.\n * Adding a check and a branch here would cost performance at every hash.\n * Also, note that function signature doesn't offer room to return an error.\n */\n if (len <= 16)\n return XXH3_len_0to16_64b((const xxh_u8*)input, len, (const xxh_u8*)secret, seed64);\n if (len <= 128)\n return XXH3_len_17to128_64b((const xxh_u8*)input, len, (const xxh_u8*)secret, secretLen, seed64);\n if (len <= XXH3_MIDSIZE_MAX)\n return XXH3_len_129to240_64b((const xxh_u8*)input, len, (const xxh_u8*)secret, secretLen, seed64);\n return f_hashLong(input, len, seed64, (const xxh_u8*)secret, secretLen);\n}\n\n\n/* === Public entry point === */\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH64_hash_t XXH3_64bits(XXH_NOESCAPE const void* input, size_t length)\n{\n return XXH3_64bits_internal(input, length, 0, XXH3_kSecret, sizeof(XXH3_kSecret), XXH3_hashLong_64b_default);\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH64_hash_t\nXXH3_64bits_withSecret(XXH_NOESCAPE const void* input, size_t length, XXH_NOESCAPE const void* secret, size_t secretSize)\n{\n return XXH3_64bits_internal(input, length, 0, secret, secretSize, XXH3_hashLong_64b_withSecret);\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH64_hash_t\nXXH3_64bits_withSeed(XXH_NOESCAPE const void* input, size_t length, XXH64_hash_t seed)\n{\n return XXH3_64bits_internal(input, length, seed, XXH3_kSecret, sizeof(XXH3_kSecret), XXH3_hashLong_64b_withSeed);\n}\n\nXXH_PUBLIC_API XXH64_hash_t\nXXH3_64bits_withSecretandSeed(XXH_NOESCAPE const void* input, size_t length, XXH_NOESCAPE const void* secret, size_t secretSize, XXH64_hash_t seed)\n{\n if (length <= XXH3_MIDSIZE_MAX)\n return XXH3_64bits_internal(input, length, seed, XXH3_kSecret, sizeof(XXH3_kSecret), NULL);\n return XXH3_hashLong_64b_withSecret(input, length, seed, (const xxh_u8*)secret, secretSize);\n}\n\n\n/* === XXH3 streaming === */\n#ifndef XXH_NO_STREAM\n/*\n * Malloc's a pointer that is always aligned to @align.\n *\n * This must be freed with `XXH_alignedFree()`.\n *\n * malloc typically guarantees 16 byte alignment on 64-bit systems and 8 byte\n * alignment on 32-bit. This isn't enough for the 32 byte aligned loads in AVX2\n * or on 32-bit, the 16 byte aligned loads in SSE2 and NEON.\n *\n * This underalignment previously caused a rather obvious crash which went\n * completely unnoticed due to XXH3_createState() not actually being tested.\n * Credit to RedSpah for noticing this bug.\n *\n * The alignment is done manually: Functions like posix_memalign or _mm_malloc\n * are avoided: To maintain portability, we would have to write a fallback\n * like this anyways, and besides, testing for the existence of library\n * functions without relying on external build tools is impossible.\n *\n * The method is simple: Overallocate, manually align, and store the offset\n * to the original behind the returned pointer.\n *\n * Align must be a power of 2 and 8 <= align <= 128.\n */\nstatic XXH_MALLOCF void* XXH_alignedMalloc(size_t s, size_t align)\n{\n XXH_ASSERT(align <= 128 && align >= 8); /* range check */\n XXH_ASSERT((align & (align-1)) == 0); /* power of 2 */\n XXH_ASSERT(s != 0 && s < (s + align)); /* empty/overflow */\n { /* Overallocate to make room for manual realignment and an offset byte */\n xxh_u8* base = (xxh_u8*)XXH_malloc(s + align);\n if (base != NULL) {\n /*\n * Get the offset needed to align this pointer.\n *\n * Even if the returned pointer is aligned, there will always be\n * at least one byte to store the offset to the original pointer.\n */\n size_t offset = align - ((size_t)base & (align - 1)); /* base % align */\n /* Add the offset for the now-aligned pointer */\n xxh_u8* ptr = base + offset;\n\n XXH_ASSERT((size_t)ptr % align == 0);\n\n /* Store the offset immediately before the returned pointer. */\n ptr[-1] = (xxh_u8)offset;\n return ptr;\n }\n return NULL;\n }\n}\n/*\n * Frees an aligned pointer allocated by XXH_alignedMalloc(). Don't pass\n * normal malloc'd pointers, XXH_alignedMalloc has a specific data layout.\n */\nstatic void XXH_alignedFree(void* p)\n{\n if (p != NULL) {\n xxh_u8* ptr = (xxh_u8*)p;\n /* Get the offset byte we added in XXH_malloc. */\n xxh_u8 offset = ptr[-1];\n /* Free the original malloc'd pointer */\n xxh_u8* base = ptr - offset;\n XXH_free(base);\n }\n}\n/*! @ingroup XXH3_family */\n/*!\n * @brief Allocate an @ref XXH3_state_t.\n *\n * @return An allocated pointer of @ref XXH3_state_t on success.\n * @return `NULL` on failure.\n *\n * @note Must be freed with XXH3_freeState().\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH3_state_t* XXH3_createState(void)\n{\n XXH3_state_t* const state = (XXH3_state_t*)XXH_alignedMalloc(sizeof(XXH3_state_t), 64);\n if (state==NULL) return NULL;\n XXH3_INITSTATE(state);\n return state;\n}\n\n/*! @ingroup XXH3_family */\n/*!\n * @brief Frees an @ref XXH3_state_t.\n *\n * @param statePtr A pointer to an @ref XXH3_state_t allocated with @ref XXH3_createState().\n *\n * @return @ref XXH_OK.\n *\n * @note Must be allocated with XXH3_createState().\n *\n * @see @ref streaming_example \"Streaming Example\"\n */\nXXH_PUBLIC_API XXH_errorcode XXH3_freeState(XXH3_state_t* statePtr)\n{\n XXH_alignedFree(statePtr);\n return XXH_OK;\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API void\nXXH3_copyState(XXH_NOESCAPE XXH3_state_t* dst_state, XXH_NOESCAPE const XXH3_state_t* src_state)\n{\n XXH_memcpy(dst_state, src_state, sizeof(*dst_state));\n}\n\nstatic void\nXXH3_reset_internal(XXH3_state_t* statePtr,\n XXH64_hash_t seed,\n const void* secret, size_t secretSize)\n{\n size_t const initStart = offsetof(XXH3_state_t, bufferedSize);\n size_t const initLength = offsetof(XXH3_state_t, nbStripesPerBlock) - initStart;\n XXH_ASSERT(offsetof(XXH3_state_t, nbStripesPerBlock) > initStart);\n XXH_ASSERT(statePtr != NULL);\n /* set members from bufferedSize to nbStripesPerBlock (excluded) to 0 */\n memset((char*)statePtr + initStart, 0, initLength);\n statePtr->acc[0] = XXH_PRIME32_3;\n statePtr->acc[1] = XXH_PRIME64_1;\n statePtr->acc[2] = XXH_PRIME64_2;\n statePtr->acc[3] = XXH_PRIME64_3;\n statePtr->acc[4] = XXH_PRIME64_4;\n statePtr->acc[5] = XXH_PRIME32_2;\n statePtr->acc[6] = XXH_PRIME64_5;\n statePtr->acc[7] = XXH_PRIME32_1;\n statePtr->seed = seed;\n statePtr->useSeed = (seed != 0);\n statePtr->extSecret = (const unsigned char*)secret;\n XXH_ASSERT(secretSize >= XXH3_SECRET_SIZE_MIN);\n statePtr->secretLimit = secretSize - XXH_STRIPE_LEN;\n statePtr->nbStripesPerBlock = statePtr->secretLimit / XXH_SECRET_CONSUME_RATE;\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH_errorcode\nXXH3_64bits_reset(XXH_NOESCAPE XXH3_state_t* statePtr)\n{\n if (statePtr == NULL) return XXH_ERROR;\n XXH3_reset_internal(statePtr, 0, XXH3_kSecret, XXH_SECRET_DEFAULT_SIZE);\n return XXH_OK;\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH_errorcode\nXXH3_64bits_reset_withSecret(XXH_NOESCAPE XXH3_state_t* statePtr, XXH_NOESCAPE const void* secret, size_t secretSize)\n{\n if (statePtr == NULL) return XXH_ERROR;\n XXH3_reset_internal(statePtr, 0, secret, secretSize);\n if (secret == NULL) return XXH_ERROR;\n if (secretSize < XXH3_SECRET_SIZE_MIN) return XXH_ERROR;\n return XXH_OK;\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH_errorcode\nXXH3_64bits_reset_withSeed(XXH_NOESCAPE XXH3_state_t* statePtr, XXH64_hash_t seed)\n{\n if (statePtr == NULL) return XXH_ERROR;\n if (seed==0) return XXH3_64bits_reset(statePtr);\n if ((seed != statePtr->seed) || (statePtr->extSecret != NULL))\n XXH3_initCustomSecret(statePtr->customSecret, seed);\n XXH3_reset_internal(statePtr, seed, NULL, XXH_SECRET_DEFAULT_SIZE);\n return XXH_OK;\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH_errorcode\nXXH3_64bits_reset_withSecretandSeed(XXH_NOESCAPE XXH3_state_t* statePtr, XXH_NOESCAPE const void* secret, size_t secretSize, XXH64_hash_t seed64)\n{\n if (statePtr == NULL) return XXH_ERROR;\n if (secret == NULL) return XXH_ERROR;\n if (secretSize < XXH3_SECRET_SIZE_MIN) return XXH_ERROR;\n XXH3_reset_internal(statePtr, seed64, secret, secretSize);\n statePtr->useSeed = 1; /* always, even if seed64==0 */\n return XXH_OK;\n}\n\n/*!\n * @internal\n * @brief Processes a large input for XXH3_update() and XXH3_digest_long().\n *\n * Unlike XXH3_hashLong_internal_loop(), this can process data that overlaps a block.\n *\n * @param acc Pointer to the 8 accumulator lanes\n * @param nbStripesSoFarPtr In/out pointer to the number of leftover stripes in the block*\n * @param nbStripesPerBlock Number of stripes in a block\n * @param input Input pointer\n * @param nbStripes Number of stripes to process\n * @param secret Secret pointer\n * @param secretLimit Offset of the last block in @p secret\n * @param f_acc Pointer to an XXH3_accumulate implementation\n * @param f_scramble Pointer to an XXH3_scrambleAcc implementation\n * @return Pointer past the end of @p input after processing\n */\nXXH_FORCE_INLINE const xxh_u8 *\nXXH3_consumeStripes(xxh_u64* XXH_RESTRICT acc,\n size_t* XXH_RESTRICT nbStripesSoFarPtr, size_t nbStripesPerBlock,\n const xxh_u8* XXH_RESTRICT input, size_t nbStripes,\n const xxh_u8* XXH_RESTRICT secret, size_t secretLimit,\n XXH3_f_accumulate f_acc,\n XXH3_f_scrambleAcc f_scramble)\n{\n const xxh_u8* initialSecret = secret + *nbStripesSoFarPtr * XXH_SECRET_CONSUME_RATE;\n /* Process full blocks */\n if (nbStripes >= (nbStripesPerBlock - *nbStripesSoFarPtr)) {\n /* Process the initial partial block... */\n size_t nbStripesThisIter = nbStripesPerBlock - *nbStripesSoFarPtr;\n\n do {\n /* Accumulate and scramble */\n f_acc(acc, input, initialSecret, nbStripesThisIter);\n f_scramble(acc, secret + secretLimit);\n input += nbStripesThisIter * XXH_STRIPE_LEN;\n nbStripes -= nbStripesThisIter;\n /* Then continue the loop with the full block size */\n nbStripesThisIter = nbStripesPerBlock;\n initialSecret = secret;\n } while (nbStripes >= nbStripesPerBlock);\n *nbStripesSoFarPtr = 0;\n }\n /* Process a partial block */\n if (nbStripes > 0) {\n f_acc(acc, input, initialSecret, nbStripes);\n input += nbStripes * XXH_STRIPE_LEN;\n *nbStripesSoFarPtr += nbStripes;\n }\n /* Return end pointer */\n return input;\n}\n\n#ifndef XXH3_STREAM_USE_STACK\n# if XXH_SIZE_OPT <= 0 && !defined(__clang__) /* clang doesn't need additional stack space */\n# define XXH3_STREAM_USE_STACK 1\n# endif\n#endif\n/*\n * Both XXH3_64bits_update and XXH3_128bits_update use this routine.\n */\nXXH_FORCE_INLINE XXH_errorcode\nXXH3_update(XXH3_state_t* XXH_RESTRICT const state,\n const xxh_u8* XXH_RESTRICT input, size_t len,\n XXH3_f_accumulate f_acc,\n XXH3_f_scrambleAcc f_scramble)\n{\n if (input==NULL) {\n XXH_ASSERT(len == 0);\n return XXH_OK;\n }\n\n XXH_ASSERT(state != NULL);\n { const xxh_u8* const bEnd = input + len;\n const unsigned char* const secret = (state->extSecret == NULL) ? state->customSecret : state->extSecret;\n#if defined(XXH3_STREAM_USE_STACK) && XXH3_STREAM_USE_STACK >= 1\n /* For some reason, gcc and MSVC seem to suffer greatly\n * when operating accumulators directly into state.\n * Operating into stack space seems to enable proper optimization.\n * clang, on the other hand, doesn't seem to need this trick */\n XXH_ALIGN(XXH_ACC_ALIGN) xxh_u64 acc[8];\n XXH_memcpy(acc, state->acc, sizeof(acc));\n#else\n xxh_u64* XXH_RESTRICT const acc = state->acc;\n#endif\n state->totalLen += len;\n XXH_ASSERT(state->bufferedSize <= XXH3_INTERNALBUFFER_SIZE);\n\n /* small input : just fill in tmp buffer */\n if (len <= XXH3_INTERNALBUFFER_SIZE - state->bufferedSize) {\n XXH_memcpy(state->buffer + state->bufferedSize, input, len);\n state->bufferedSize += (XXH32_hash_t)len;\n return XXH_OK;\n }\n\n /* total input is now > XXH3_INTERNALBUFFER_SIZE */\n #define XXH3_INTERNALBUFFER_STRIPES (XXH3_INTERNALBUFFER_SIZE / XXH_STRIPE_LEN)\n XXH_STATIC_ASSERT(XXH3_INTERNALBUFFER_SIZE % XXH_STRIPE_LEN == 0); /* clean multiple */\n\n /*\n * Internal buffer is partially filled (always, except at beginning)\n * Complete it, then consume it.\n */\n if (state->bufferedSize) {\n size_t const loadSize = XXH3_INTERNALBUFFER_SIZE - state->bufferedSize;\n XXH_memcpy(state->buffer + state->bufferedSize, input, loadSize);\n input += loadSize;\n XXH3_consumeStripes(acc,\n &state->nbStripesSoFar, state->nbStripesPerBlock,\n state->buffer, XXH3_INTERNALBUFFER_STRIPES,\n secret, state->secretLimit,\n f_acc, f_scramble);\n state->bufferedSize = 0;\n }\n XXH_ASSERT(input < bEnd);\n if (bEnd - input > XXH3_INTERNALBUFFER_SIZE) {\n size_t nbStripes = (size_t)(bEnd - 1 - input) / XXH_STRIPE_LEN;\n input = XXH3_consumeStripes(acc,\n &state->nbStripesSoFar, state->nbStripesPerBlock,\n input, nbStripes,\n secret, state->secretLimit,\n f_acc, f_scramble);\n XXH_memcpy(state->buffer + sizeof(state->buffer) - XXH_STRIPE_LEN, input - XXH_STRIPE_LEN, XXH_STRIPE_LEN);\n\n }\n /* Some remaining input (always) : buffer it */\n XXH_ASSERT(input < bEnd);\n XXH_ASSERT(bEnd - input <= XXH3_INTERNALBUFFER_SIZE);\n XXH_ASSERT(state->bufferedSize == 0);\n XXH_memcpy(state->buffer, input, (size_t)(bEnd-input));\n state->bufferedSize = (XXH32_hash_t)(bEnd-input);\n#if defined(XXH3_STREAM_USE_STACK) && XXH3_STREAM_USE_STACK >= 1\n /* save stack accumulators into state */\n XXH_memcpy(state->acc, acc, sizeof(acc));\n#endif\n }\n\n return XXH_OK;\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH_errorcode\nXXH3_64bits_update(XXH_NOESCAPE XXH3_state_t* state, XXH_NOESCAPE const void* input, size_t len)\n{\n return XXH3_update(state, (const xxh_u8*)input, len,\n XXH3_accumulate, XXH3_scrambleAcc);\n}\n\n\nXXH_FORCE_INLINE void\nXXH3_digest_long (XXH64_hash_t* acc,\n const XXH3_state_t* state,\n const unsigned char* secret)\n{\n xxh_u8 lastStripe[XXH_STRIPE_LEN];\n const xxh_u8* lastStripePtr;\n\n /*\n * Digest on a local copy. This way, the state remains unaltered, and it can\n * continue ingesting more input afterwards.\n */\n XXH_memcpy(acc, state->acc, sizeof(state->acc));\n if (state->bufferedSize >= XXH_STRIPE_LEN) {\n /* Consume remaining stripes then point to remaining data in buffer */\n size_t const nbStripes = (state->bufferedSize - 1) / XXH_STRIPE_LEN;\n size_t nbStripesSoFar = state->nbStripesSoFar;\n XXH3_consumeStripes(acc,\n &nbStripesSoFar, state->nbStripesPerBlock,\n state->buffer, nbStripes,\n secret, state->secretLimit,\n XXH3_accumulate, XXH3_scrambleAcc);\n lastStripePtr = state->buffer + state->bufferedSize - XXH_STRIPE_LEN;\n } else { /* bufferedSize < XXH_STRIPE_LEN */\n /* Copy to temp buffer */\n size_t const catchupSize = XXH_STRIPE_LEN - state->bufferedSize;\n XXH_ASSERT(state->bufferedSize > 0); /* there is always some input buffered */\n XXH_memcpy(lastStripe, state->buffer + sizeof(state->buffer) - catchupSize, catchupSize);\n XXH_memcpy(lastStripe + catchupSize, state->buffer, state->bufferedSize);\n lastStripePtr = lastStripe;\n }\n /* Last stripe */\n XXH3_accumulate_512(acc,\n lastStripePtr,\n secret + state->secretLimit - XXH_SECRET_LASTACC_START);\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH64_hash_t XXH3_64bits_digest (XXH_NOESCAPE const XXH3_state_t* state)\n{\n const unsigned char* const secret = (state->extSecret == NULL) ? state->customSecret : state->extSecret;\n if (state->totalLen > XXH3_MIDSIZE_MAX) {\n XXH_ALIGN(XXH_ACC_ALIGN) XXH64_hash_t acc[XXH_ACC_NB];\n XXH3_digest_long(acc, state, secret);\n return XXH3_finalizeLong_64b(acc, secret, (xxh_u64)state->totalLen);\n }\n /* totalLen <= XXH3_MIDSIZE_MAX: digesting a short input */\n if (state->useSeed)\n return XXH3_64bits_withSeed(state->buffer, (size_t)state->totalLen, state->seed);\n return XXH3_64bits_withSecret(state->buffer, (size_t)(state->totalLen),\n secret, state->secretLimit + XXH_STRIPE_LEN);\n}\n#endif /* !XXH_NO_STREAM */\n\n\n/* ==========================================\n * XXH3 128 bits (a.k.a XXH128)\n * ==========================================\n * XXH3's 128-bit variant has better mixing and strength than the 64-bit variant,\n * even without counting the significantly larger output size.\n *\n * For example, extra steps are taken to avoid the seed-dependent collisions\n * in 17-240 byte inputs (See XXH3_mix16B and XXH128_mix32B).\n *\n * This strength naturally comes at the cost of some speed, especially on short\n * lengths. Note that longer hashes are about as fast as the 64-bit version\n * due to it using only a slight modification of the 64-bit loop.\n *\n * XXH128 is also more oriented towards 64-bit machines. It is still extremely\n * fast for a _128-bit_ hash on 32-bit (it usually clears XXH64).\n */\n\nXXH_FORCE_INLINE XXH_PUREF XXH128_hash_t\nXXH3_len_1to3_128b(const xxh_u8* input, size_t len, const xxh_u8* secret, XXH64_hash_t seed)\n{\n /* A doubled version of 1to3_64b with different constants. */\n XXH_ASSERT(input != NULL);\n XXH_ASSERT(1 <= len && len <= 3);\n XXH_ASSERT(secret != NULL);\n /*\n * len = 1: combinedl = { input[0], 0x01, input[0], input[0] }\n * len = 2: combinedl = { input[1], 0x02, input[0], input[1] }\n * len = 3: combinedl = { input[2], 0x03, input[0], input[1] }\n */\n { xxh_u8 const c1 = input[0];\n xxh_u8 const c2 = input[len >> 1];\n xxh_u8 const c3 = input[len - 1];\n xxh_u32 const combinedl = ((xxh_u32)c1 <<16) | ((xxh_u32)c2 << 24)\n | ((xxh_u32)c3 << 0) | ((xxh_u32)len << 8);\n xxh_u32 const combinedh = XXH_rotl32(XXH_swap32(combinedl), 13);\n xxh_u64 const bitflipl = (XXH_readLE32(secret) ^ XXH_readLE32(secret+4)) + seed;\n xxh_u64 const bitfliph = (XXH_readLE32(secret+8) ^ XXH_readLE32(secret+12)) - seed;\n xxh_u64 const keyed_lo = (xxh_u64)combinedl ^ bitflipl;\n xxh_u64 const keyed_hi = (xxh_u64)combinedh ^ bitfliph;\n XXH128_hash_t h128;\n h128.low64 = XXH64_avalanche(keyed_lo);\n h128.high64 = XXH64_avalanche(keyed_hi);\n return h128;\n }\n}\n\nXXH_FORCE_INLINE XXH_PUREF XXH128_hash_t\nXXH3_len_4to8_128b(const xxh_u8* input, size_t len, const xxh_u8* secret, XXH64_hash_t seed)\n{\n XXH_ASSERT(input != NULL);\n XXH_ASSERT(secret != NULL);\n XXH_ASSERT(4 <= len && len <= 8);\n seed ^= (xxh_u64)XXH_swap32((xxh_u32)seed) << 32;\n { xxh_u32 const input_lo = XXH_readLE32(input);\n xxh_u32 const input_hi = XXH_readLE32(input + len - 4);\n xxh_u64 const input_64 = input_lo + ((xxh_u64)input_hi << 32);\n xxh_u64 const bitflip = (XXH_readLE64(secret+16) ^ XXH_readLE64(secret+24)) + seed;\n xxh_u64 const keyed = input_64 ^ bitflip;\n\n /* Shift len to the left to ensure it is even, this avoids even multiplies. */\n XXH128_hash_t m128 = XXH_mult64to128(keyed, XXH_PRIME64_1 + (len << 2));\n\n m128.high64 += (m128.low64 << 1);\n m128.low64 ^= (m128.high64 >> 3);\n\n m128.low64 = XXH_xorshift64(m128.low64, 35);\n m128.low64 *= PRIME_MX2;\n m128.low64 = XXH_xorshift64(m128.low64, 28);\n m128.high64 = XXH3_avalanche(m128.high64);\n return m128;\n }\n}\n\nXXH_FORCE_INLINE XXH_PUREF XXH128_hash_t\nXXH3_len_9to16_128b(const xxh_u8* input, size_t len, const xxh_u8* secret, XXH64_hash_t seed)\n{\n XXH_ASSERT(input != NULL);\n XXH_ASSERT(secret != NULL);\n XXH_ASSERT(9 <= len && len <= 16);\n { xxh_u64 const bitflipl = (XXH_readLE64(secret+32) ^ XXH_readLE64(secret+40)) - seed;\n xxh_u64 const bitfliph = (XXH_readLE64(secret+48) ^ XXH_readLE64(secret+56)) + seed;\n xxh_u64 const input_lo = XXH_readLE64(input);\n xxh_u64 input_hi = XXH_readLE64(input + len - 8);\n XXH128_hash_t m128 = XXH_mult64to128(input_lo ^ input_hi ^ bitflipl, XXH_PRIME64_1);\n /*\n * Put len in the middle of m128 to ensure that the length gets mixed to\n * both the low and high bits in the 128x64 multiply below.\n */\n m128.low64 += (xxh_u64)(len - 1) << 54;\n input_hi ^= bitfliph;\n /*\n * Add the high 32 bits of input_hi to the high 32 bits of m128, then\n * add the long product of the low 32 bits of input_hi and XXH_PRIME32_2 to\n * the high 64 bits of m128.\n *\n * The best approach to this operation is different on 32-bit and 64-bit.\n */\n if (sizeof(void *) < sizeof(xxh_u64)) { /* 32-bit */\n /*\n * 32-bit optimized version, which is more readable.\n *\n * On 32-bit, it removes an ADC and delays a dependency between the two\n * halves of m128.high64, but it generates an extra mask on 64-bit.\n */\n m128.high64 += (input_hi & 0xFFFFFFFF00000000ULL) + XXH_mult32to64((xxh_u32)input_hi, XXH_PRIME32_2);\n } else {\n /*\n * 64-bit optimized (albeit more confusing) version.\n *\n * Uses some properties of addition and multiplication to remove the mask:\n *\n * Let:\n * a = input_hi.lo = (input_hi & 0x00000000FFFFFFFF)\n * b = input_hi.hi = (input_hi & 0xFFFFFFFF00000000)\n * c = XXH_PRIME32_2\n *\n * a + (b * c)\n * Inverse Property: x + y - x == y\n * a + (b * (1 + c - 1))\n * Distributive Property: x * (y + z) == (x * y) + (x * z)\n * a + (b * 1) + (b * (c - 1))\n * Identity Property: x * 1 == x\n * a + b + (b * (c - 1))\n *\n * Substitute a, b, and c:\n * input_hi.hi + input_hi.lo + ((xxh_u64)input_hi.lo * (XXH_PRIME32_2 - 1))\n *\n * Since input_hi.hi + input_hi.lo == input_hi, we get this:\n * input_hi + ((xxh_u64)input_hi.lo * (XXH_PRIME32_2 - 1))\n */\n m128.high64 += input_hi + XXH_mult32to64((xxh_u32)input_hi, XXH_PRIME32_2 - 1);\n }\n /* m128 ^= XXH_swap64(m128 >> 64); */\n m128.low64 ^= XXH_swap64(m128.high64);\n\n { /* 128x64 multiply: h128 = m128 * XXH_PRIME64_2; */\n XXH128_hash_t h128 = XXH_mult64to128(m128.low64, XXH_PRIME64_2);\n h128.high64 += m128.high64 * XXH_PRIME64_2;\n\n h128.low64 = XXH3_avalanche(h128.low64);\n h128.high64 = XXH3_avalanche(h128.high64);\n return h128;\n } }\n}\n\n/*\n * Assumption: `secret` size is >= XXH3_SECRET_SIZE_MIN\n */\nXXH_FORCE_INLINE XXH_PUREF XXH128_hash_t\nXXH3_len_0to16_128b(const xxh_u8* input, size_t len, const xxh_u8* secret, XXH64_hash_t seed)\n{\n XXH_ASSERT(len <= 16);\n { if (len > 8) return XXH3_len_9to16_128b(input, len, secret, seed);\n if (len >= 4) return XXH3_len_4to8_128b(input, len, secret, seed);\n if (len) return XXH3_len_1to3_128b(input, len, secret, seed);\n { XXH128_hash_t h128;\n xxh_u64 const bitflipl = XXH_readLE64(secret+64) ^ XXH_readLE64(secret+72);\n xxh_u64 const bitfliph = XXH_readLE64(secret+80) ^ XXH_readLE64(secret+88);\n h128.low64 = XXH64_avalanche(seed ^ bitflipl);\n h128.high64 = XXH64_avalanche( seed ^ bitfliph);\n return h128;\n } }\n}\n\n/*\n * A bit slower than XXH3_mix16B, but handles multiply by zero better.\n */\nXXH_FORCE_INLINE XXH128_hash_t\nXXH128_mix32B(XXH128_hash_t acc, const xxh_u8* input_1, const xxh_u8* input_2,\n const xxh_u8* secret, XXH64_hash_t seed)\n{\n acc.low64 += XXH3_mix16B (input_1, secret+0, seed);\n acc.low64 ^= XXH_readLE64(input_2) + XXH_readLE64(input_2 + 8);\n acc.high64 += XXH3_mix16B (input_2, secret+16, seed);\n acc.high64 ^= XXH_readLE64(input_1) + XXH_readLE64(input_1 + 8);\n return acc;\n}\n\n\nXXH_FORCE_INLINE XXH_PUREF XXH128_hash_t\nXXH3_len_17to128_128b(const xxh_u8* XXH_RESTRICT input, size_t len,\n const xxh_u8* XXH_RESTRICT secret, size_t secretSize,\n XXH64_hash_t seed)\n{\n XXH_ASSERT(secretSize >= XXH3_SECRET_SIZE_MIN); (void)secretSize;\n XXH_ASSERT(16 < len && len <= 128);\n\n { XXH128_hash_t acc;\n acc.low64 = len * XXH_PRIME64_1;\n acc.high64 = 0;\n\n#if XXH_SIZE_OPT >= 1\n {\n /* Smaller, but slightly slower. */\n unsigned int i = (unsigned int)(len - 1) / 32;\n do {\n acc = XXH128_mix32B(acc, input+16*i, input+len-16*(i+1), secret+32*i, seed);\n } while (i-- != 0);\n }\n#else\n if (len > 32) {\n if (len > 64) {\n if (len > 96) {\n acc = XXH128_mix32B(acc, input+48, input+len-64, secret+96, seed);\n }\n acc = XXH128_mix32B(acc, input+32, input+len-48, secret+64, seed);\n }\n acc = XXH128_mix32B(acc, input+16, input+len-32, secret+32, seed);\n }\n acc = XXH128_mix32B(acc, input, input+len-16, secret, seed);\n#endif\n { XXH128_hash_t h128;\n h128.low64 = acc.low64 + acc.high64;\n h128.high64 = (acc.low64 * XXH_PRIME64_1)\n + (acc.high64 * XXH_PRIME64_4)\n + ((len - seed) * XXH_PRIME64_2);\n h128.low64 = XXH3_avalanche(h128.low64);\n h128.high64 = (XXH64_hash_t)0 - XXH3_avalanche(h128.high64);\n return h128;\n }\n }\n}\n\nXXH_NO_INLINE XXH_PUREF XXH128_hash_t\nXXH3_len_129to240_128b(const xxh_u8* XXH_RESTRICT input, size_t len,\n const xxh_u8* XXH_RESTRICT secret, size_t secretSize,\n XXH64_hash_t seed)\n{\n XXH_ASSERT(secretSize >= XXH3_SECRET_SIZE_MIN); (void)secretSize;\n XXH_ASSERT(128 < len && len <= XXH3_MIDSIZE_MAX);\n\n { XXH128_hash_t acc;\n unsigned i;\n acc.low64 = len * XXH_PRIME64_1;\n acc.high64 = 0;\n /*\n * We set as `i` as offset + 32. We do this so that unchanged\n * `len` can be used as upper bound. This reaches a sweet spot\n * where both x86 and aarch64 get simple agen and good codegen\n * for the loop.\n */\n for (i = 32; i < 160; i += 32) {\n acc = XXH128_mix32B(acc,\n input + i - 32,\n input + i - 16,\n secret + i - 32,\n seed);\n }\n acc.low64 = XXH3_avalanche(acc.low64);\n acc.high64 = XXH3_avalanche(acc.high64);\n /*\n * NB: `i <= len` will duplicate the last 32-bytes if\n * len % 32 was zero. This is an unfortunate necessity to keep\n * the hash result stable.\n */\n for (i=160; i <= len; i += 32) {\n acc = XXH128_mix32B(acc,\n input + i - 32,\n input + i - 16,\n secret + XXH3_MIDSIZE_STARTOFFSET + i - 160,\n seed);\n }\n /* last bytes */\n acc = XXH128_mix32B(acc,\n input + len - 16,\n input + len - 32,\n secret + XXH3_SECRET_SIZE_MIN - XXH3_MIDSIZE_LASTOFFSET - 16,\n (XXH64_hash_t)0 - seed);\n\n { XXH128_hash_t h128;\n h128.low64 = acc.low64 + acc.high64;\n h128.high64 = (acc.low64 * XXH_PRIME64_1)\n + (acc.high64 * XXH_PRIME64_4)\n + ((len - seed) * XXH_PRIME64_2);\n h128.low64 = XXH3_avalanche(h128.low64);\n h128.high64 = (XXH64_hash_t)0 - XXH3_avalanche(h128.high64);\n return h128;\n }\n }\n}\n\nstatic XXH_PUREF XXH128_hash_t\nXXH3_finalizeLong_128b(const xxh_u64* XXH_RESTRICT acc, const xxh_u8* XXH_RESTRICT secret, size_t secretSize, xxh_u64 len)\n{\n XXH128_hash_t h128;\n h128.low64 = XXH3_finalizeLong_64b(acc, secret, len);\n h128.high64 = XXH3_mergeAccs(acc, secret + secretSize\n - XXH_STRIPE_LEN - XXH_SECRET_MERGEACCS_START,\n ~(len * XXH_PRIME64_2));\n return h128;\n}\n\nXXH_FORCE_INLINE XXH128_hash_t\nXXH3_hashLong_128b_internal(const void* XXH_RESTRICT input, size_t len,\n const xxh_u8* XXH_RESTRICT secret, size_t secretSize,\n XXH3_f_accumulate f_acc,\n XXH3_f_scrambleAcc f_scramble)\n{\n XXH_ALIGN(XXH_ACC_ALIGN) xxh_u64 acc[XXH_ACC_NB] = XXH3_INIT_ACC;\n\n XXH3_hashLong_internal_loop(acc, (const xxh_u8*)input, len, secret, secretSize, f_acc, f_scramble);\n\n /* converge into final hash */\n XXH_STATIC_ASSERT(sizeof(acc) == 64);\n XXH_ASSERT(secretSize >= sizeof(acc) + XXH_SECRET_MERGEACCS_START);\n return XXH3_finalizeLong_128b(acc, secret, secretSize, (xxh_u64)len);\n}\n\n/*\n * It's important for performance that XXH3_hashLong() is not inlined.\n */\nXXH_NO_INLINE XXH_PUREF XXH128_hash_t\nXXH3_hashLong_128b_default(const void* XXH_RESTRICT input, size_t len,\n XXH64_hash_t seed64,\n const void* XXH_RESTRICT secret, size_t secretLen)\n{\n (void)seed64; (void)secret; (void)secretLen;\n return XXH3_hashLong_128b_internal(input, len, XXH3_kSecret, sizeof(XXH3_kSecret),\n XXH3_accumulate, XXH3_scrambleAcc);\n}\n\n/*\n * It's important for performance to pass @p secretLen (when it's static)\n * to the compiler, so that it can properly optimize the vectorized loop.\n *\n * When the secret size is unknown, or on GCC 12 where the mix of NO_INLINE and FORCE_INLINE\n * breaks -Og, this is XXH_NO_INLINE.\n */\nXXH3_WITH_SECRET_INLINE XXH128_hash_t\nXXH3_hashLong_128b_withSecret(const void* XXH_RESTRICT input, size_t len,\n XXH64_hash_t seed64,\n const void* XXH_RESTRICT secret, size_t secretLen)\n{\n (void)seed64;\n return XXH3_hashLong_128b_internal(input, len, (const xxh_u8*)secret, secretLen,\n XXH3_accumulate, XXH3_scrambleAcc);\n}\n\nXXH_FORCE_INLINE XXH128_hash_t\nXXH3_hashLong_128b_withSeed_internal(const void* XXH_RESTRICT input, size_t len,\n XXH64_hash_t seed64,\n XXH3_f_accumulate f_acc,\n XXH3_f_scrambleAcc f_scramble,\n XXH3_f_initCustomSecret f_initSec)\n{\n if (seed64 == 0)\n return XXH3_hashLong_128b_internal(input, len,\n XXH3_kSecret, sizeof(XXH3_kSecret),\n f_acc, f_scramble);\n { XXH_ALIGN(XXH_SEC_ALIGN) xxh_u8 secret[XXH_SECRET_DEFAULT_SIZE];\n f_initSec(secret, seed64);\n return XXH3_hashLong_128b_internal(input, len, (const xxh_u8*)secret, sizeof(secret),\n f_acc, f_scramble);\n }\n}\n\n/*\n * It's important for performance that XXH3_hashLong is not inlined.\n */\nXXH_NO_INLINE XXH128_hash_t\nXXH3_hashLong_128b_withSeed(const void* input, size_t len,\n XXH64_hash_t seed64, const void* XXH_RESTRICT secret, size_t secretLen)\n{\n (void)secret; (void)secretLen;\n return XXH3_hashLong_128b_withSeed_internal(input, len, seed64,\n XXH3_accumulate, XXH3_scrambleAcc, XXH3_initCustomSecret);\n}\n\ntypedef XXH128_hash_t (*XXH3_hashLong128_f)(const void* XXH_RESTRICT, size_t,\n XXH64_hash_t, const void* XXH_RESTRICT, size_t);\n\nXXH_FORCE_INLINE XXH128_hash_t\nXXH3_128bits_internal(const void* input, size_t len,\n XXH64_hash_t seed64, const void* XXH_RESTRICT secret, size_t secretLen,\n XXH3_hashLong128_f f_hl128)\n{\n XXH_ASSERT(secretLen >= XXH3_SECRET_SIZE_MIN);\n /*\n * If an action is to be taken if `secret` conditions are not respected,\n * it should be done here.\n * For now, it's a contract pre-condition.\n * Adding a check and a branch here would cost performance at every hash.\n */\n if (len <= 16)\n return XXH3_len_0to16_128b((const xxh_u8*)input, len, (const xxh_u8*)secret, seed64);\n if (len <= 128)\n return XXH3_len_17to128_128b((const xxh_u8*)input, len, (const xxh_u8*)secret, secretLen, seed64);\n if (len <= XXH3_MIDSIZE_MAX)\n return XXH3_len_129to240_128b((const xxh_u8*)input, len, (const xxh_u8*)secret, secretLen, seed64);\n return f_hl128(input, len, seed64, secret, secretLen);\n}\n\n\n/* === Public XXH128 API === */\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH128_hash_t XXH3_128bits(XXH_NOESCAPE const void* input, size_t len)\n{\n return XXH3_128bits_internal(input, len, 0,\n XXH3_kSecret, sizeof(XXH3_kSecret),\n XXH3_hashLong_128b_default);\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH128_hash_t\nXXH3_128bits_withSecret(XXH_NOESCAPE const void* input, size_t len, XXH_NOESCAPE const void* secret, size_t secretSize)\n{\n return XXH3_128bits_internal(input, len, 0,\n (const xxh_u8*)secret, secretSize,\n XXH3_hashLong_128b_withSecret);\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH128_hash_t\nXXH3_128bits_withSeed(XXH_NOESCAPE const void* input, size_t len, XXH64_hash_t seed)\n{\n return XXH3_128bits_internal(input, len, seed,\n XXH3_kSecret, sizeof(XXH3_kSecret),\n XXH3_hashLong_128b_withSeed);\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH128_hash_t\nXXH3_128bits_withSecretandSeed(XXH_NOESCAPE const void* input, size_t len, XXH_NOESCAPE const void* secret, size_t secretSize, XXH64_hash_t seed)\n{\n if (len <= XXH3_MIDSIZE_MAX)\n return XXH3_128bits_internal(input, len, seed, XXH3_kSecret, sizeof(XXH3_kSecret), NULL);\n return XXH3_hashLong_128b_withSecret(input, len, seed, secret, secretSize);\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH128_hash_t\nXXH128(XXH_NOESCAPE const void* input, size_t len, XXH64_hash_t seed)\n{\n return XXH3_128bits_withSeed(input, len, seed);\n}\n\n\n/* === XXH3 128-bit streaming === */\n#ifndef XXH_NO_STREAM\n/*\n * All initialization and update functions are identical to 64-bit streaming variant.\n * The only difference is the finalization routine.\n */\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH_errorcode\nXXH3_128bits_reset(XXH_NOESCAPE XXH3_state_t* statePtr)\n{\n return XXH3_64bits_reset(statePtr);\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH_errorcode\nXXH3_128bits_reset_withSecret(XXH_NOESCAPE XXH3_state_t* statePtr, XXH_NOESCAPE const void* secret, size_t secretSize)\n{\n return XXH3_64bits_reset_withSecret(statePtr, secret, secretSize);\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH_errorcode\nXXH3_128bits_reset_withSeed(XXH_NOESCAPE XXH3_state_t* statePtr, XXH64_hash_t seed)\n{\n return XXH3_64bits_reset_withSeed(statePtr, seed);\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH_errorcode\nXXH3_128bits_reset_withSecretandSeed(XXH_NOESCAPE XXH3_state_t* statePtr, XXH_NOESCAPE const void* secret, size_t secretSize, XXH64_hash_t seed)\n{\n return XXH3_64bits_reset_withSecretandSeed(statePtr, secret, secretSize, seed);\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH_errorcode\nXXH3_128bits_update(XXH_NOESCAPE XXH3_state_t* state, XXH_NOESCAPE const void* input, size_t len)\n{\n return XXH3_64bits_update(state, input, len);\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH128_hash_t XXH3_128bits_digest (XXH_NOESCAPE const XXH3_state_t* state)\n{\n const unsigned char* const secret = (state->extSecret == NULL) ? state->customSecret : state->extSecret;\n if (state->totalLen > XXH3_MIDSIZE_MAX) {\n XXH_ALIGN(XXH_ACC_ALIGN) XXH64_hash_t acc[XXH_ACC_NB];\n XXH3_digest_long(acc, state, secret);\n XXH_ASSERT(state->secretLimit + XXH_STRIPE_LEN >= sizeof(acc) + XXH_SECRET_MERGEACCS_START);\n return XXH3_finalizeLong_128b(acc, secret, state->secretLimit + XXH_STRIPE_LEN, (xxh_u64)state->totalLen);\n }\n /* len <= XXH3_MIDSIZE_MAX : short code */\n if (state->useSeed)\n return XXH3_128bits_withSeed(state->buffer, (size_t)state->totalLen, state->seed);\n return XXH3_128bits_withSecret(state->buffer, (size_t)(state->totalLen),\n secret, state->secretLimit + XXH_STRIPE_LEN);\n}\n#endif /* !XXH_NO_STREAM */\n/* 128-bit utility functions */\n\n#include /* memcmp, memcpy */\n\n/* return : 1 is equal, 0 if different */\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API int XXH128_isEqual(XXH128_hash_t h1, XXH128_hash_t h2)\n{\n /* note : XXH128_hash_t is compact, it has no padding byte */\n return !(memcmp(&h1, &h2, sizeof(h1)));\n}\n\n/* This prototype is compatible with stdlib's qsort().\n * @return : >0 if *h128_1 > *h128_2\n * <0 if *h128_1 < *h128_2\n * =0 if *h128_1 == *h128_2 */\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API int XXH128_cmp(XXH_NOESCAPE const void* h128_1, XXH_NOESCAPE const void* h128_2)\n{\n XXH128_hash_t const h1 = *(const XXH128_hash_t*)h128_1;\n XXH128_hash_t const h2 = *(const XXH128_hash_t*)h128_2;\n int const hcmp = (h1.high64 > h2.high64) - (h2.high64 > h1.high64);\n /* note : bets that, in most cases, hash values are different */\n if (hcmp) return hcmp;\n return (h1.low64 > h2.low64) - (h2.low64 > h1.low64);\n}\n\n\n/*====== Canonical representation ======*/\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API void\nXXH128_canonicalFromHash(XXH_NOESCAPE XXH128_canonical_t* dst, XXH128_hash_t hash)\n{\n XXH_STATIC_ASSERT(sizeof(XXH128_canonical_t) == sizeof(XXH128_hash_t));\n if (XXH_CPU_LITTLE_ENDIAN) {\n hash.high64 = XXH_swap64(hash.high64);\n hash.low64 = XXH_swap64(hash.low64);\n }\n XXH_memcpy(dst, &hash.high64, sizeof(hash.high64));\n XXH_memcpy((char*)dst + sizeof(hash.high64), &hash.low64, sizeof(hash.low64));\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH128_hash_t\nXXH128_hashFromCanonical(XXH_NOESCAPE const XXH128_canonical_t* src)\n{\n XXH128_hash_t h;\n h.high64 = XXH_readBE64(src);\n h.low64 = XXH_readBE64(src->digest + 8);\n return h;\n}\n\n\n\n/* ==========================================\n * Secret generators\n * ==========================================\n */\n#define XXH_MIN(x, y) (((x) > (y)) ? (y) : (x))\n\nXXH_FORCE_INLINE void XXH3_combine16(void* dst, XXH128_hash_t h128)\n{\n XXH_writeLE64( dst, XXH_readLE64(dst) ^ h128.low64 );\n XXH_writeLE64( (char*)dst+8, XXH_readLE64((char*)dst+8) ^ h128.high64 );\n}\n\n/*! @ingroup XXH3_family */\nXXH_PUBLIC_API XXH_errorcode\nXXH3_generateSecret(XXH_NOESCAPE void* secretBuffer, size_t secretSize, XXH_NOESCAPE const void* customSeed, size_t customSeedSize)\n{\n#if (XXH_DEBUGLEVEL >= 1)\n XXH_ASSERT(secretBuffer != NULL);\n XXH_ASSERT(secretSize >= XXH3_SECRET_SIZE_MIN);\n#else\n /* production mode, assert() are disabled */\n if (secretBuffer == NULL) return XXH_ERROR;\n if (secretSize < XXH3_SECRET_SIZE_MIN) return XXH_ERROR;\n#endif\n\n if (customSeedSize == 0) {\n customSeed = XXH3_kSecret;\n customSeedSize = XXH_SECRET_DEFAULT_SIZE;\n }\n#if (XXH_DEBUGLEVEL >= 1)\n XXH_ASSERT(customSeed != NULL);\n#else\n if (customSeed == NULL) return XXH_ERROR;\n#endif\n\n /* Fill secretBuffer with a copy of customSeed - repeat as needed */\n { size_t pos = 0;\n while (pos < secretSize) {\n size_t const toCopy = XXH_MIN((secretSize - pos), customSeedSize);\n memcpy((char*)secretBuffer + pos, customSeed, toCopy);\n pos += toCopy;\n } }\n\n { size_t const nbSeg16 = secretSize / 16;\n size_t n;\n XXH128_canonical_t scrambler;\n XXH128_canonicalFromHash(&scrambler, XXH128(customSeed, customSeedSize, 0));\n for (n=0; n\n\n Permission is hereby granted, free of charge, to any person\n obtaining a copy of this software and associated documentation\n files (the \"Software\"), to deal in the Software without\n restriction, including without limitation the rights to use, copy,\n modify, merge, publish, distribute, sublicense, and/or sell copies\n of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n\n The above copyright notice and this permission notice shall be\n included in all copies or substantial portions of the Software.\n\n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\n MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\n BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\n ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\n CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.\n*/\n\n/* Pseudo Random Number Generator (PRNG) based on MUM hash function.\n It is not a crypto level PRNG.\n\n To use a generator call `init_mum_prng` first, then call\n `get_mum_prn` as much as you want to get a new PRN. At the end of\n the PRNG use, call `finish_mum_prng`. You can change the default\n seed by calling set_mum_seed.\n\n The PRNG passes NIST Statistical Test Suite for Random and\n Pseudorandom Number Generators for Cryptographic Applications\n (version 2.2.1) with 1000 bitstreams each containing 1M bits.\n\n The generation of a new number takes about 3.5 CPU cycles on x86_64\n (Intel 4.2GHz i7-4790K), or speed of the generation is about 1120M\n numbers per sec. So it is very fast. */\n\n#ifndef __MUM_PRNG__\n#define __MUM_PRNG__\n\n#include \"mum.h\"\n\n#ifndef MUM_PRNG_UNROLL\n#define MUM_PRNG_UNROLL 16\n#endif\n\n#if MUM_PRNG_UNROLL < 1 || MUM_PRNG_UNROLL > 16\n#error \"wrong MUM_PRNG_UNROLL value\"\n#endif\n\n#ifdef __GNUC__\n#define EXPECT(cond, v) __builtin_expect((cond), (v))\n#else\n#define EXPECT(cond, v) (cond)\n#endif\n\n#if defined(__GNUC__) && ((__GNUC__ == 4) && (__GNUC_MINOR__ >= 9) || (__GNUC__ > 4))\n#define _MUM_PRNG_FRESH_GCC\n#endif\n\nstatic struct {\n int count; \n void (*update_func) (void);\n /* MUM PRNG state */\n uint64_t state[MUM_PRNG_UNROLL]; \n} _mum_prng_state;\n\n#if defined(__x86_64__) && defined(_MUM_PRNG_FRESH_GCC)\n/* This code specialized for Haswell generates MULX insns. */\nstatic inline uint64_t _MUM_TARGET(\"arch=haswell\")\n_mum_avx2 (uint64_t v, uint64_t p) {\n uint64_t hi, lo;\n __uint128_t r = (__uint128_t) v * (__uint128_t) p;\n hi = (uint64_t) (r >> 64);\n lo = (uint64_t) r;\n return hi + lo;\n}\n\nstatic void _MUM_TARGET(\"arch=haswell\")\n_mum_prng_update_avx2 (void) {\n int i;\n\n _mum_prng_state.count = 0;\n for (i = 0; i < MUM_PRNG_UNROLL - 1; i++)\n _mum_prng_state.state[i] ^= _mum_avx2 (_mum_prng_state.state[i + 1], _mum_primes[i]);\n _mum_prng_state.state[MUM_PRNG_UNROLL - 1] ^= _mum_avx2 (_mum_prng_state.state[0], _mum_primes[MUM_PRNG_UNROLL - 1]);\n}\n#endif\n\nstatic void __attribute__ ((noinline))\n_mum_prng_update (void) {\n int i;\n\n _mum_prng_state.count = 0;\n for (i = 0; i < MUM_PRNG_UNROLL - 1; i++)\n _mum_prng_state.state[i] ^= _mum (_mum_prng_state.state[i + 1], _mum_primes[i]);\n _mum_prng_state.state[MUM_PRNG_UNROLL - 1] ^= _mum (_mum_prng_state.state[0], _mum_primes[MUM_PRNG_UNROLL - 1]);\n}\n\n#if defined(__x86_64__) && defined(_MUM_PRNG_FRESH_GCC)\nstatic inline void\n_mum_prng_setup_avx2 (void) {\n __builtin_cpu_init ();\n if (__builtin_cpu_supports (\"avx2\"))\n _mum_prng_state.update_func = _mum_prng_update_avx2;\n else\n _mum_prng_state.update_func = _mum_prng_update;\n\n}\n#endif\n\nstatic inline void\n_start_mum_prng (uint32_t seed) {\n int i;\n\n _mum_prng_state.count = MUM_PRNG_UNROLL;\n for (i = 0; i < MUM_PRNG_UNROLL; i++)\n _mum_prng_state.state[i] = seed + 1;\n#if defined(__x86_64__) && defined(_MUM_PRNG_FRESH_GCC)\n _mum_prng_setup_avx2 ();\n#else\n _mum_prng_state.update_func = _mum_prng_update;\n#endif\n}\n\nstatic inline void\ninit_mum_prng (void) {\n _start_mum_prng (0);\n}\n\nstatic inline void\nset_mum_prng_seed (uint32_t seed) {\n _start_mum_prng (seed);\n}\n\nstatic inline uint64_t\nget_mum_prn (void) {\n if (EXPECT (_mum_prng_state.count == MUM_PRNG_UNROLL, 0)) {\n _mum_prng_state.update_func ();\n _mum_prng_state.count = 1;\n return _mum_prng_state.state[0];\n }\n return _mum_prng_state.state[_mum_prng_state.count++];\n}\n\nstatic inline void\nfinish_mum_prng (void) {\n}\n\n#endif\n" }, { "path": "mum.h", "content": "/* Copyright (c) 2016-2025\n Vladimir Makarov \n\n Permission is hereby granted, free of charge, to any person\n obtaining a copy of this software and associated documentation\n files (the \"Software\"), to deal in the Software without\n restriction, including without limitation the rights to use, copy,\n modify, merge, publish, distribute, sublicense, and/or sell copies\n of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n\n The above copyright notice and this permission notice shall be\n included in all copies or substantial portions of the Software.\n\n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\n MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\n BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\n ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\n CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.\n*/\n\n/* This file implements MUM (MUltiply and Mix) hashing. We randomize input data by 64x64-bit\n multiplication and mixing hi- and low-parts of the multiplication result by using an addition and\n then mix it into the current state. We use prime numbers randomly generated with the equal\n probability of their bit values for the multiplication. When all primes are used once, the state\n is randomized and the same prime numbers are used again for data randomization.\n\n The MUM hashing passes all SMHasher tests. Pseudo Random Number Generator based on MUM also\n passes NIST Statistical Test Suite for Random and Pseudorandom Number Generators for\n Cryptographic Applications (version 2.2.1) with 1000 bitstreams each containing 1M bits. MUM\n hashing is also faster Spooky64 and City64 on small strings (at least upto 512-bit) on Haswell\n and Power7. The MUM bulk speed (speed on very long data) is bigger than Spooky and City on\n Power7. On Haswell the bulk speed is bigger than Spooky one and close to City speed. */\n\n#ifndef __MUM_HASH__\n#define __MUM_HASH__\n\n#include \n#include \n#include \n#include \n\n#ifdef _MSC_VER\ntypedef unsigned __int16 uint16_t;\ntypedef unsigned __int32 uint32_t;\ntypedef unsigned __int64 uint64_t;\n#else\n#include \n#endif\n\n#ifdef __GNUC__\n#define _MUM_ATTRIBUTE_UNUSED __attribute__ ((unused))\n#define _MUM_INLINE inline __attribute__ ((always_inline))\n#else\n#define _MUM_ATTRIBUTE_UNUSED\n#define _MUM_INLINE inline\n#endif\n\n#if defined(MUM_QUALITY) && !defined(MUM_TARGET_INDEPENDENT_HASH)\n#define MUM_TARGET_INDEPENDENT_HASH\n#endif\n\n/* Macro saying to use 128-bit integers implemented by GCC for some targets. */\n#ifndef _MUM_USE_INT128\n/* In GCC uint128_t is defined if HOST_BITS_PER_WIDE_INT >= 64. HOST_WIDE_INT is long if\n HOST_BITS_PER_LONG > HOST_BITS_PER_INT, otherwise int. */\n#if defined(__GNUC__) && UINT_MAX != ULONG_MAX\n#define _MUM_USE_INT128 1\n#else\n#define _MUM_USE_INT128 0\n#endif\n#endif\n\n/* Here are different primes randomly generated with the equal probability of their bit values. They\n are used to randomize input values. */\nstatic uint64_t _mum_hash_step_prime = 0x2e0bb864e9ea7df5ULL;\nstatic uint64_t _mum_key_step_prime = 0xcdb32970830fcaa1ULL;\nstatic uint64_t _mum_block_start_prime = 0xc42b5e2e6480b23bULL;\nstatic uint64_t _mum_unroll_prime = 0x7b51ec3d22f7096fULL;\nstatic uint64_t _mum_tail_prime = 0xaf47d47c99b1461bULL;\nstatic uint64_t _mum_finish_prime1 = 0xa9a7ae7ceff79f3fULL;\nstatic uint64_t _mum_finish_prime2 = 0xaf47d47c99b1461bULL;\n\nstatic uint64_t _mum_primes[] = {\n 0X9ebdcae10d981691, 0X32b9b9b97a27ac7d, 0X29b5584d83d35bbd, 0X4b04e0e61401255f,\n 0X25e8f7b1f1c9d027, 0X80d4c8c000f3e881, 0Xbd1255431904b9dd, 0X8a3bd4485eee6d81,\n 0X3bc721b2aad05197, 0X71b1a19b907d6e33, 0X525e6c1084a8534b, 0X9e4c2cd340c1299f,\n 0Xde3add92e94caa37, 0X7e14eadb1f65311d, 0X3f5aa40f89812853, 0X33b15a3b587d15c9,\n};\n\n/* Multiply 64-bit V and P and return sum of high and low parts of the result. */\nstatic _MUM_INLINE uint64_t _mum (uint64_t v, uint64_t p) {\n uint64_t hi, lo;\n#if _MUM_USE_INT128\n __uint128_t r = (__uint128_t) v * (__uint128_t) p;\n hi = (uint64_t) (r >> 64);\n lo = (uint64_t) r;\n#else\n /* Implementation of 64x64->128-bit multiplication by four 32x32->64 bit multiplication. */\n uint64_t hv = v >> 32, hp = p >> 32;\n uint64_t lv = (uint32_t) v, lp = (uint32_t) p;\n uint64_t rh = hv * hp;\n uint64_t rm_0 = hv * lp;\n uint64_t rm_1 = hp * lv;\n uint64_t rl = lv * lp;\n uint64_t t, carry = 0;\n\n /* We could ignore a carry bit here if we did not care about the same hash for 32-bit and 64-bit\n targets. */\n t = rl + (rm_0 << 32);\n#ifdef MUM_TARGET_INDEPENDENT_HASH\n carry = t < rl;\n#endif\n lo = t + (rm_1 << 32);\n#ifdef MUM_TARGET_INDEPENDENT_HASH\n carry += lo < t;\n#endif\n hi = rh + (rm_0 >> 32) + (rm_1 >> 32) + carry;\n#endif\n /* We could use XOR here too but, for some reasons, on Haswell and Power7 using an addition\n improves hashing performance by 10% for small strings. */\n return hi + lo;\n}\n\n#if defined(_MSC_VER)\n#define _mum_bswap32(x) _byteswap_uint32_t (x)\n#define _mum_bswap64(x) _byteswap_uint64_t (x)\n#elif defined(__APPLE__)\n#include \n#define _mum_bswap32(x) OSSwapInt32 (x)\n#define _mum_bswap64(x) OSSwapInt64 (x)\n#elif defined(__GNUC__)\n#define _mum_bswap32(x) __builtin_bswap32 (x)\n#define _mum_bswap64(x) __builtin_bswap64 (x)\n#else\n#include \n#define _mum_bswap32(x) bswap32 (x)\n#define _mum_bswap64(x) bswap64 (x)\n#endif\n\nstatic _MUM_INLINE uint64_t _mum_le (uint64_t v) {\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ || !defined(MUM_TARGET_INDEPENDENT_HASH)\n return v;\n#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n return _mum_bswap64 (v);\n#else\n#error \"Unknown endianess\"\n#endif\n}\n\nstatic _MUM_INLINE uint32_t _mum_le32 (uint32_t v) {\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ || !defined(MUM_TARGET_INDEPENDENT_HASH)\n return v;\n#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n return _mum_bswap32 (v);\n#else\n#error \"Unknown endianess\"\n#endif\n}\n\nstatic _MUM_INLINE uint64_t _mum_le16 (uint16_t v) {\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ || !defined(MUM_TARGET_INDEPENDENT_HASH)\n return v;\n#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n return (v >> 8) | ((v & 0xff) << 8);\n#else\n#error \"Unknown endianess\"\n#endif\n}\n\n/* Macro defining how many times the most nested loop in _mum_hash_aligned will be unrolled by the\n compiler (although it can make an own decision:). Use only a constant here to help a compiler to\n unroll a major loop.\n\n The macro value affects the result hash for strings > 128 bit. The unroll factor greatly affects\n the hashing speed. We prefer the speed. */\n#ifndef _MUM_UNROLL_FACTOR_POWER\n#if defined(__PPC64__) && !defined(MUM_TARGET_INDEPENDENT_HASH)\n#define _MUM_UNROLL_FACTOR_POWER 3\n#elif defined(__aarch64__) && !defined(MUM_TARGET_INDEPENDENT_HASH)\n#define _MUM_UNROLL_FACTOR_POWER 4\n#elif defined(MUM_V1) || defined(MUM_V2)\n#define _MUM_UNROLL_FACTOR_POWER 2\n#else\n#define _MUM_UNROLL_FACTOR_POWER 3\n#endif\n#endif\n\n#if _MUM_UNROLL_FACTOR_POWER < 1\n#error \"too small unroll factor\"\n#elif _MUM_UNROLL_FACTOR_POWER > 4\n#error \"We have not enough primes for such unroll factor\"\n#endif\n\n#define _MUM_UNROLL_FACTOR (1 << _MUM_UNROLL_FACTOR_POWER)\n\n/* Rotate V left by SH. */\nstatic _MUM_INLINE uint64_t _mum_rotl (uint64_t v, int sh) { return v << sh | v >> (64 - sh); }\n\nstatic _MUM_INLINE uint64_t _mum_xor (uint64_t a, uint64_t b) {\n#ifdef MUM_V3\n return a ^ b;\n#else\n return (a ^ b) != 0 ? a ^ b : b;\n#endif\n}\n\n#if defined(MUM_V1) || defined(MUM_V2) || !defined(MUM_QUALITY)\n#define _MUM_TAIL_START(v) 0\n#else\n#define _MUM_TAIL_START(v) v\n#endif\nstatic _MUM_INLINE uint64_t\n#if defined(__GNUC__) && !defined(__clang__)\n __attribute__ ((__optimize__ (\"unroll-loops\")))\n#endif\n _mum_hash_aligned (uint64_t start, const void *key, size_t len) {\n uint64_t result = start;\n const unsigned char *str = (const unsigned char *) key;\n uint64_t u64;\n size_t i;\n size_t n;\n\n#ifndef MUM_V2\n result = _mum (result, _mum_block_start_prime);\n#endif\n while (len > _MUM_UNROLL_FACTOR * sizeof (uint64_t)) {\n /* This loop could be vectorized when we have vector insns for 64x64->128-bit multiplication.\n AVX2 currently only have vector insns for 4 32x32->64-bit multiplication and for 1\n 64x64->128-bit multiplication (pclmulqdq). */\n#if defined(MUM_V1) || defined(MUM_V2)\n for (i = 0; i < _MUM_UNROLL_FACTOR; i++)\n result ^= _mum (_mum_le (((uint64_t *) str)[i]), _mum_primes[i]);\n#else\n for (i = 0; i < _MUM_UNROLL_FACTOR; i += 2)\n result ^= _mum (_mum_xor (_mum_le (((uint64_t *) str)[i]), _mum_primes[i]),\n _mum_xor (_mum_le (((uint64_t *) str)[i + 1]), _mum_primes[i + 1]));\n#endif\n len -= _MUM_UNROLL_FACTOR * sizeof (uint64_t);\n str += _MUM_UNROLL_FACTOR * sizeof (uint64_t);\n /* We will use the same prime numbers on the next iterations -- randomize the state. */\n result = _mum (result, _mum_unroll_prime);\n }\n n = len / sizeof (uint64_t);\n#if defined(MUM_V1) || defined(MUM_V2) || !defined(MUM_QUALITY)\n for (i = 0; i < n; i++) result ^= _mum (_mum_le (((uint64_t *) str)[i]), _mum_primes[i]);\n#else\n for (i = 0; i < n; i++)\n result ^= _mum (_mum_le (((uint64_t *) str)[i]) + _mum_primes[i], _mum_primes[i]);\n#endif\n len -= n * sizeof (uint64_t);\n str += n * sizeof (uint64_t);\n switch (len) {\n case 7:\n u64 = _MUM_TAIL_START (_mum_primes[0]) + _mum_le32 (*(uint32_t *) str);\n u64 += _mum_le16 (*(uint16_t *) (str + 4)) << 32;\n u64 += (uint64_t) str[6] << 48;\n return result ^ _mum (u64, _mum_tail_prime);\n case 6:\n u64 = _MUM_TAIL_START (_mum_primes[1]) + _mum_le32 (*(uint32_t *) str);\n u64 += _mum_le16 (*(uint16_t *) (str + 4)) << 32;\n return result ^ _mum (u64, _mum_tail_prime);\n case 5:\n u64 = _MUM_TAIL_START (_mum_primes[2]) + _mum_le32 (*(uint32_t *) str);\n u64 += (uint64_t) str[4] << 32;\n return result ^ _mum (u64, _mum_tail_prime);\n case 4:\n u64 = _MUM_TAIL_START (_mum_primes[3]) + _mum_le32 (*(uint32_t *) str);\n return result ^ _mum (u64, _mum_tail_prime);\n case 3:\n u64 = _MUM_TAIL_START (_mum_primes[4]) + _mum_le16 (*(uint16_t *) str);\n u64 += (uint64_t) str[2] << 16;\n return result ^ _mum (u64, _mum_tail_prime);\n case 2:\n u64 = _MUM_TAIL_START (_mum_primes[5]) + _mum_le16 (*(uint16_t *) str);\n return result ^ _mum (u64, _mum_tail_prime);\n case 1:\n u64 = _MUM_TAIL_START (_mum_primes[6]) + str[0];\n return result ^ _mum (u64, _mum_tail_prime);\n }\n return result;\n}\n\n/* Final randomization of H. */\nstatic _MUM_INLINE uint64_t _mum_final (uint64_t h) {\n#if defined(MUM_V1)\n h ^= _mum (h, _mum_finish_prime1);\n h ^= _mum (h, _mum_finish_prime2);\n#elif defined(MUM_V2)\n h ^= _mum_rotl (h, 33);\n h ^= _mum (h, _mum_finish_prime1);\n#else\n h = _mum (h, h);\n#endif\n return h;\n}\n\n#ifndef _MUM_UNALIGNED_ACCESS\n#if defined(__x86_64__) || defined(__i386__) || defined(__PPC64__) || defined(__s390__) \\\n || defined(__m32c__) || defined(cris) || defined(__CR16__) || defined(__vax__) \\\n || defined(__m68k__) || defined(__aarch64__) || defined(_M_AMD64) || defined(_M_IX86)\n#define _MUM_UNALIGNED_ACCESS 1\n#else\n#define _MUM_UNALIGNED_ACCESS 0\n#endif\n#endif\n\n/* When we need an aligned access to data being hashed we move part of the unaligned data to an\n aligned block of given size and then process it, repeating processing the data by the block. */\n#ifndef _MUM_BLOCK_LEN\n#define _MUM_BLOCK_LEN 1024\n#endif\n\n#if _MUM_BLOCK_LEN < 8\n#error \"too small block length\"\n#endif\n\nstatic _MUM_INLINE uint64_t\n#if defined(__x86_64__) && defined(__GNUC__) && !defined(__clang__)\n __attribute__ ((__target__ (\"inline-all-stringops\")))\n#endif\n _mum_hash_default (const void *key, size_t len, uint64_t seed) {\n uint64_t result;\n const unsigned char *str = (const unsigned char *) key;\n size_t block_len;\n uint64_t buf[_MUM_BLOCK_LEN / sizeof (uint64_t)];\n\n result = seed + len;\n if (((size_t) str & 0x7) == 0)\n result = _mum_hash_aligned (result, key, len);\n else {\n while (len != 0) {\n block_len = len < _MUM_BLOCK_LEN ? len : _MUM_BLOCK_LEN;\n memcpy (buf, str, block_len);\n result = _mum_hash_aligned (result, buf, block_len);\n len -= block_len;\n str += block_len;\n }\n }\n return _mum_final (result);\n}\n\nstatic _MUM_INLINE uint64_t _mum_next_factor (void) {\n uint64_t start = 0;\n int i;\n\n for (i = 0; i < 8; i++) start = (start << 8) | rand () % 256;\n return start;\n}\n\n/* ++++++++++++++++++++++++++ Interface functions: +++++++++++++++++++ */\n\n/* Set random multiplicators depending on SEED. */\nstatic _MUM_INLINE void mum_hash_randomize (uint64_t seed) {\n size_t i;\n\n srand (seed);\n _mum_hash_step_prime = _mum_next_factor ();\n _mum_key_step_prime = _mum_next_factor ();\n _mum_finish_prime1 = _mum_next_factor ();\n _mum_finish_prime2 = _mum_next_factor ();\n _mum_block_start_prime = _mum_next_factor ();\n _mum_unroll_prime = _mum_next_factor ();\n _mum_tail_prime = _mum_next_factor ();\n for (i = 0; i < sizeof (_mum_primes) / sizeof (uint64_t); i++)\n _mum_primes[i] = _mum_next_factor ();\n}\n\n/* Start hashing data with SEED. Return the state. */\nstatic _MUM_INLINE uint64_t mum_hash_init (uint64_t seed) { return seed; }\n\n/* Process data KEY with the state H and return the updated state. */\nstatic _MUM_INLINE uint64_t mum_hash_step (uint64_t h, uint64_t key) {\n return _mum (h, _mum_hash_step_prime) ^ _mum (key, _mum_key_step_prime);\n}\n\n/* Return the result of hashing using the current state H. */\nstatic _MUM_INLINE uint64_t mum_hash_finish (uint64_t h) { return _mum_final (h); }\n\n/* Fast hashing of KEY with SEED. The hash is always the same for the same key on any target. */\nstatic _MUM_INLINE size_t mum_hash64 (uint64_t key, uint64_t seed) {\n return mum_hash_finish (mum_hash_step (mum_hash_init (seed), key));\n}\n\n/* Hash data KEY of length LEN and SEED. The hash depends on the target endianess and the unroll\n factor. */\nstatic _MUM_INLINE uint64_t mum_hash (const void *key, size_t len, uint64_t seed) {\n#if _MUM_UNALIGNED_ACCESS\n return _mum_final (_mum_hash_aligned (seed + len, key, len));\n#else\n return _mum_hash_default (key, len, seed);\n#endif\n}\n\n#endif\n" }, { "path": "mum512.h", "content": "/* Copyright (c) 2016, 2017, 2018\n Vladimir Makarov \n\n Permission is hereby granted, free of charge, to any person\n obtaining a copy of this software and associated documentation\n files (the \"Software\"), to deal in the Software without\n restriction, including without limitation the rights to use, copy,\n modify, merge, publish, distribute, sublicense, and/or sell copies\n of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n\n The above copyright notice and this permission notice shall be\n included in all copies or substantial portions of the Software.\n\n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\n MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\n BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\n ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\n CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.\n*/\n\n/* This file implements MUM512 (MUltiply and Mix) hash function. It\n is a candidate for a crypto-level hash function and keyed hash\n function.\n\n The key size is 256-bit. The size of the state, block, and output\n (digest) is 512-bit.\n\n The input data by 128x128-bit multiplication and mixing hi- and\n low-parts of the multiplication result by using an addition and\n then mix it into the current state. We use prime numbers randomly\n generated with the equal probability of their bit values for the\n multiplication. When all primes are used once, the state is\n randomized and the same prime numbers are used again for data\n randomization.\n\n The MUM512 hashing passes all SMHasher tests and NIST Statistical\n Test Suite for Random and Pseudorandom Number Generators for\n Cryptographic Applications (v. 2.2.1). MUM512 is also faster SHA-2\n (512) and SHA-3 (512). */\n\n#ifndef __MC_HASH__\n#define __MC_HASH__\n\n#include \n#include \n#include \n\n#if defined(_MSC_VER)\ntypedef unsigned __int32 uint32_t;\ntypedef unsigned __int64 uint64_t;\n#else\n#include \n#endif\n\n#if defined(_MSC_VER)\n#define _mc_bswap_64(x) _byteswap_uint64_t (x)\n#elif defined(__APPLE__)\n#include \n#define _mc_bswap_64(x) OSSwapInt64 (x)\n#elif defined(__GNUC__)\n#define _mc_bswap64(x) __builtin_bswap64 (x)\n#else\n#include \n#define _mc_bswap64(x) bswap64 (x)\n#endif\n\n#ifndef _MC_USE_INT128\n/* In GCC if uint128_t is defined if HOST_BITS_PER_WIDE_INT >= 64.\n HOST_WIDE_INT is long if HOST_BITS_PER_LONG > HOST_BITS_PER_IN,\n otherwise int. */\n#if defined(__GNUC__) && UINT_MAX != ULONG_MAX\n#define _MC_USE_INT128 1\n#else\n#define _MC_USE_INT128 0\n#endif\n#endif\n\n#ifdef __GNUC__\n#define _MC_ATTRIBUTE_UNUSED __attribute__((unused))\n#define _MC_OPTIMIZE(opts) __attribute__((__optimize__ (opts)))\n#define _MC_TARGET(opts) __attribute__((__target__ (opts)))\n#else\n#define _MC_ATTRIBUTE_UNUSED\n#define _MC_OPTIMIZE(opts)\n#define _MC_TARGET(opts)\n#endif\n\n\n#if _MC_USE_INT128\ntypedef __uint128_t _mc_ti;\n#else\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\ntypedef struct {uint64_t lo, hi;} _mc_ti;\n#else\ntypedef struct {uint64_t hi, lo;} _mc_ti;\n#endif\n#endif\n\n#define inline\n\n#if _MC_USE_INT128\nstatic inline uint64_t _mc_lo64 (_mc_ti a) {return a;}\nstatic inline uint64_t _mc_hi64 (_mc_ti a) {return a >> 64;}\nstatic inline _mc_ti _mc_lo2hi (_mc_ti a) {return a << 64;}\nstatic inline _mc_ti _mc_hi2lo (_mc_ti a) {return a >> 64;}\nstatic inline _mc_ti _mc_add (_mc_ti a, _mc_ti b) {return a + b;}\nstatic inline _mc_ti _mc_xor (_mc_ti a, _mc_ti b) {return a ^ b;}\nstatic inline uint32_t _mc_lt (_mc_ti a, _mc_ti b) { return a < b;}\nstatic inline _mc_ti _mc_const (uint64_t hi, uint64_t lo) {\n return (_mc_ti) hi << 64 | lo;\n}\nstatic inline _mc_ti _mc_rotr (_mc_ti a, int sh) {\n return (a >> sh) | (a << (128 - sh));\n}\n\nstatic inline _mc_ti _mc_mul64 (uint64_t a, uint64_t b) {\n#if defined(__aarch64__)\n /* AARCH64 needs 2 insns to calculate 128-bit result of the\n multiplication. If we use a generic code we actually call a\n function doing 128x128->128 bit multiplication. The function is\n very slow. */\n uint64_t lo = a * b, hi;\n\n asm (\"umulh %0, %1, %2\" : \"=r\" (hi) : \"r\" (a), \"r\" (b));\n return (_mc_ti) hi << 64 | lo;\n#else\n return (_mc_ti) a * (_mc_ti) b;\n#endif\n}\n\nstatic inline _mc_ti _mc_swap (_mc_ti v) {\n return _mc_const (_mc_bswap64 (_mc_lo64 (v)), _mc_bswap64 (_mc_hi64 (v)));\n}\n\n#else /* #if _MC_USE_INT128 */\n\nstatic inline uint64_t _mc_lo64 (_mc_ti a) {return a.lo;}\nstatic inline uint64_t _mc_hi64 (_mc_ti a) {return a.hi;}\nstatic inline _mc_ti _mc_lo2hi (_mc_ti a) {a.hi = a.lo; a.lo = 0; return a;}\nstatic inline _mc_ti _mc_hi2lo (_mc_ti a) {a.lo = a.hi; a.hi = 0; return a;}\nstatic inline _mc_ti _mc_const (uint64_t hi, uint64_t lo) {\n _mc_ti r;\n r.hi = hi; r.lo = lo; return r;\n}\nstatic inline _mc_ti _mc_add (_mc_ti a, _mc_ti b) {\n a.lo += b.lo, a.hi += b.hi + (a.lo < b.lo); return a;\n}\nstatic inline uint32_t _mc_lt (_mc_ti a, _mc_ti b) {\n return a.hi < b.hi || (a.hi == b.hi && a.lo < b.lo);\n}\nstatic inline _mc_ti _mc_xor (_mc_ti a, _mc_ti b) {\n a.lo ^= b.lo; a.hi ^= b.hi; return a;\n}\nstatic inline _mc_ti _mc_rotr (_mc_ti a, int sh) {\n _mc_ti res;\n\n if (sh <= 64) {\n res.lo = a.lo >> sh | a.hi << (64 - sh);\n res.hi = a.hi >> sh | a.lo << (64 - sh);\n } else {\n sh -= 64;\n res.lo = a.hi >> sh | a.lo << (64 - sh);\n res.hi = a.lo >> sh | a.hi << (64 - sh);\n }\n return res;\n}\n\nstatic inline _mc_ti _mc_mul64 (uint64_t a, uint64_t b) {\n /* Implementation of 64x64->128-bit multiplication by four 32x32->64\n bit multiplication. */\n uint64_t ha = a >> 32, hb = b >> 32;\n uint64_t la = (uint32_t) a, lb = (uint32_t) b;\n uint64_t rhi = ha * hb;\n uint64_t rm_0 = ha * lb;\n uint64_t rm_1 = hb * la;\n uint64_t rlo = la * lb;\n uint64_t lo, carry;\n _mc_ti res;\n \n lo = rlo + (rm_0 << 32);\n carry = lo < rlo;\n res.lo = lo + (rm_1 << 32);\n carry += res.lo < lo;\n res.hi = rhi + (rm_0 >> 32) + (rm_1 >> 32) + carry;\n return res;\n}\n\nstatic inline _mc_ti _mc_swap (_mc_ti v) {\n _mc_ti r;\n r.lo = _mc_bswap64 (v.hi); r.hi = _mc_bswap64 (v.lo); return r;\n}\n\n#endif /* #if _MC_USE_INT128 */\n\nstatic inline _mc_ti _mc_mum (_mc_ti a, _mc_ti b) {\n /* Implementation of 128x128->256-bit multiplication by four\n 64x64->128 bit multiplication. */\n uint64_t ha = _mc_hi64 (a), hb = _mc_hi64 (b);\n uint64_t la = _mc_lo64 (a), lb = _mc_lo64 (b);\n _mc_ti rhi = _mc_mul64 (ha, hb);\n _mc_ti rm_0 = _mc_mul64 (ha, lb);\n _mc_ti rm_1 = _mc_mul64 (hb, la);\n _mc_ti rlo = _mc_mul64 (la, lb);\n _mc_ti hi, lo, t;\n uint32_t carry;\n \n t = _mc_add (_mc_lo2hi (rm_0), rlo); carry = _mc_lt (t, rlo);\n lo = _mc_add (_mc_lo2hi (rm_1), t); carry += _mc_lt (lo, t);\n hi = _mc_add (_mc_add (_mc_add (rhi, _mc_hi2lo (rm_0)), _mc_hi2lo (rm_1)),\n\t\t_mc_const (0, carry));\n return _mc_add (hi, lo);\n}\n\nstatic inline _mc_ti _mc_2le (_mc_ti v) {\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n return v;\n#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n return _mc_swap (v);\n#else\n#error \"Unknown endianess\"\n#endif\n}\n\nstatic inline _mc_ti _mc_get (const uint64_t a[2]) {return _mc_const (a[0], a[1]);}\n\n/* Here are different primes randomly generated with the equal\n probability of their bit values. They are used to randomize input\n values. */\nstatic const uint64_t _mc_block_start_primes[4][2] = {\n {0Xcdc037073edf8fd4ULL, 0X9e2a1f46e21930f3ULL},\n {0X128281cfcc981531ULL, 0Xae5a66b7a27ba90fULL},\n {0X676f0ed409f0019aULL, 0X4bd8035682ec0095ULL},\n {0X6fae98c0d49ed3ffULL, 0X5268b2b43337690bULL},\n};\nstatic const uint64_t _mc_unroll_primes[4][2] = {\n {0X5cc2a100bf7b7c05ULL, 0Xd5ce2499d1844187ULL},\n {0X2fdc8fe047488edaULL, 0Xd4c8739663e296e9ULL},\n {0Xd36d0a4431717b61ULL, 0X10cb847ca52a3e25ULL},\n {0Xf011560af9264d79ULL, 0Xc9d48bb40afed5d3ULL},\n};\nstatic const uint64_t _mc_tail_primes[4][2] = {\n {0X270957969935d145ULL, 0Xcb75144659dbd40dULL},\n {0X12db17d4be72d52cULL, 0X3b0cbf3954138dfdULL},\n {0X5022d90d6ccc2372ULL, 0X889add17a2618f99ULL},\n {0Xe282c1104925025bULL, 0Xfdf6cd06e47a8c85ULL},\n};\nstatic const uint64_t _mc_primes [16][2] = {\n {0Xfc35b01709510063ULL, 0Xec2fdbeaade88605ULL},\n {0Xd2b356ae579f47a6ULL, 0X65862b62bdf5ef4dULL},\n {0X288eea216831e6a7ULL, 0Xa79128dc46b0173fULL},\n {0Xb737b72062479c04ULL, 0X5f6c924506e59f99ULL},\n {0X92a4d7c15a574735ULL, 0X5d1d4782c102eadfULL},\n {0Xf9c1acb5076cf7c7ULL, 0Xb9ecfe43117ae249ULL},\n {0X10ee204d93f26f50ULL, 0Xb70b8f530f15aa6bULL},\n {0X181da242743a22dcULL, 0Xff14e8e58f6e1721ULL},\n {0X3e431aaaac0f869eULL, 0X9feebb1f9c386fe3ULL},\n {0X817292a324fa17b0ULL, 0X2c37dcc844bc3d37ULL},\n {0Xcdf95a3e7c6fcb0dULL, 0Xcd2aa0073fbeb7b9ULL},\n {0Xfc67b9a71847832aULL, 0X640c8f5ed3777417ULL},\n {0X6cb4dcd475f71ae5ULL, 0X51f0328a65fc65bbULL},\n {0Xdcab58b43c79dee4ULL, 0X745bd64098a26981ULL},\n {0X1a8f800285c50091ULL, 0X2afffc9213aa8a5fULL},\n {0X532e0082221804dbULL, 0X20c941f9b512df11ULL},\n};\n\nstatic const uint64_t _mc_finish_primes [64][2] = {\n {0X1e0055fb724474e8ULL, 0Xff29de49c378e459ULL},\n {0X83e8be07b5531ec3ULL, 0X4266dedbf60f5523ULL},\n {0X25d03dacd0a5c8e5ULL, 0Xdbbcac5c7cda883fULL},\n {0X9984f048ee48645eULL, 0X12c928afdf625a47ULL},\n {0X8100dab7b00755d3ULL, 0X12a2f064b6def9fULL},\n {0X2a4936c9ef78297cULL, 0Xa9c902ba35037359ULL},\n {0X833ac9a368619006ULL, 0X16fc020d9f40323ULL},\n {0X56f880bf9ae924acULL, 0X87262cbecf623dc5ULL},\n {0Xf92080d63124cc29ULL, 0X94da893165805cfbULL},\n {0X852b1ea7d206adfaULL, 0Xded5f478baf4f65bULL},\n {0X6c090f99eb7da834ULL, 0X8619a705f355b9d1ULL},\n {0Xc24556cc93aff883ULL, 0X5df87f6b5076a2cfULL},\n {0X24fa93e0575eb1f6ULL, 0X3d7063f15281ff03ULL},\n {0X47ee8f94f08cb813ULL, 0X8128602ec25d6a19ULL},\n {0X3625bd5ebdd10f45ULL, 0Xe43054d209066c7bULL},\n {0X120db6d8b3382a5ULL, 0X314fb5d22fced3f9ULL},\n {0X78276dc95fb85e7dULL, 0X57bfc5460f873c9dULL},\n {0X96d099814184b5b0ULL, 0Xd89f836c35b9b0c9ULL},\n {0X6d79572ccf590f33ULL, 0Xab8030fdeffd48f5ULL},\n {0X19c5d2c51e451eddULL, 0Xc675c5e74f1c2c45ULL},\n {0X63b09e2290211695ULL, 0X494b134c5d324f3ULL},\n {0Xed2cda11f978bb7dULL, 0X827e3d4edf1475e1ULL},\n {0X9a5e00a94de13a14ULL, 0X55667b0f406987abULL},\n {0Xcbcabf81b1338e26ULL, 0X2dd53df2ae66d191ULL},\n {0X13c880d6b45a6c9cULL, 0Xe0a28ed2c8049f5fULL},\n {0X55279312bf2ed6feULL, 0Xc6c19c752e4bcb2dULL},\n {0Xf138de98d83585caULL, 0Xc90ad41770782dd1ULL},\n {0X81f28fded1813f57ULL, 0X94900f7877c93a37ULL},\n {0X473457080f9406d2ULL, 0Xc5989b97426594afULL},\n {0Xe5799a1363835a22ULL, 0Xce4ca5f532d3980bULL},\n {0Xfd7b3c358f596498ULL, 0X70fd7270db8949b5ULL},\n {0Xfd6a0c5c43a6c653ULL, 0Xeb3aa98686ffa645ULL},\n {0X3df53625c3afc40aULL, 0Xdf3bffec06a23541ULL},\n {0Xef491e13aa526595ULL, 0Xf6688ada28d7e81fULL},\n {0X8419c0511b7e4242ULL, 0X7ba8ddb66b20a5d7ULL},\n {0X11b3da4f4f90fb72ULL, 0X71de3984d3c23847ULL},\n {0X6616fb20c0231403ULL, 0Xfa06db687a1bc2ffULL},\n {0X6e661bbd3bd6e950ULL, 0X6e45cbd289a77c23ULL},\n {0X599d1220a03da949ULL, 0Xff568091f9c59349ULL},\n {0X4464d21d20da203bULL, 0X352006ad6fb6de5ULL},\n {0X6413c7b09fc3dcc8ULL, 0X55fc3da745579871ULL},\n {0X160111a644959396ULL, 0Xd12e059acb048b01ULL},\n {0Xc66e719ac9a96714ULL, 0X832bb4f760cb0c0dULL},\n {0X92b777a80cfd2a59ULL, 0X883dc6537f8d2a6bULL},\n {0Xf40a3463bd3220d7ULL, 0Xadd624b1439d6507ULL},\n {0X2d11c5c5e6c3bf75ULL, 0Xfc027148e7ed66c9ULL},\n {0Xe9437c1b3494cb46ULL, 0Xacd35405f3d2f4b7ULL},\n {0X838f551f688bd046ULL, 0X3e87440f1a4642e9ULL},\n {0Xce20a8f09e985b88ULL, 0Xa98931ac95adde85ULL},\n {0Xe3580702cf5a1c80ULL, 0X8e76e05a566733cdULL},\n {0X15ff369908c7d67dULL, 0X11bc44ed911d2aa7ULL},\n {0Xab2694c16fe59e02ULL, 0Xc1f5ba02d82d5e85ULL},\n {0X7c59a06636a7edddULL, 0X684643ed7e49b619ULL},\n {0X40b6270f7f9fa251ULL, 0X9d8b29eef5b17a8dULL},\n {0X4044ad0f26d7e774ULL, 0Xf3d1fd1ab3ca5037ULL},\n {0X5ec3d199e0063437ULL, 0X12fb529ef8cc3a73ULL},\n {0X6b5f89cb3017e3e2ULL, 0Xfdb9b50566a2626dULL},\n {0X76ad12644718202cULL, 0X967fbb370d71862fULL},\n {0Xb3b7cb5a21f3eaeaULL, 0X18f9fe203ad955ffULL},\n {0Xfd28dc38cbec1a6cULL, 0X8e1920c5a0132445ULL},\n {0X23f9cf9b2158b106ULL, 0Xd9a70361d96b31f9ULL},\n {0X224fa274d6ea69f7ULL, 0X6316130d5d925dc3ULL},\n {0X14b09aa58cb268d5ULL, 0Xf34d372c6f503cc9ULL},\n {0X6289be577a4ae0a6ULL, 0X3ba5692b1eafeb15ULL},\n};\n\n/* Macro defining how many times the most nested loop in\n _mc_hash_aligned will be unrolled by the compiler (although it can\n make an own decision:). Use only a constant here to help a\n compiler to unroll a major loop. Don't change the value. Changing\n it changes the hash. If you increase it you need more numberss in\n _mc_primes. */\n#define _MC_UNROLL_FACTOR 4\n\nstatic inline void _MC_OPTIMIZE (\"unroll-all-loops\")\n_mc_hash_aligned (_mc_ti state[4], const void *data, size_t len) {\n _mc_ti result[4];\n const unsigned char *str = (const unsigned char *) data;\n union {_mc_ti i; unsigned char s[sizeof (_mc_ti)];} p;\n int j;\n size_t i, n;\n \n for (i = 0; i < 4; i++)\n result[i] = _mc_mum (state[i], _mc_get (_mc_block_start_primes[i]));\n while (len > _MC_UNROLL_FACTOR * sizeof (_mc_ti)) {\n for (i = 0; i < _MC_UNROLL_FACTOR; i++)\n for (j = 0; j < 4; j++)\n\tresult[j] = _mc_xor (result[j], _mc_mum (_mc_2le (((_mc_ti *) str)[i]),\n\t\t\t\t\t\t _mc_get (_mc_primes[4 * i + j])));\n /* We might use the same prime numbers on the next iterations --\n randomize the state. */\n for (i = 0; i < 4; i++)\n result[i] = _mc_mum (result[i], _mc_get (_mc_unroll_primes[i]));\n len -= _MC_UNROLL_FACTOR * sizeof (_mc_ti);\n str += _MC_UNROLL_FACTOR * sizeof (_mc_ti);\n }\n n = len / sizeof (_mc_ti);\n for (i = 0; i < n; i++)\n for (j = 0; j < 4; j++)\n result[j] = _mc_xor (result[j], _mc_mum (_mc_2le (((_mc_ti *) str)[i]),\n\t\t\t\t\t _mc_get (_mc_primes[4 * i + j])));\n len -= n * sizeof (_mc_ti); str += n * sizeof (_mc_ti);\n if (len) {\n p.i = _mc_const (0, 0); memcpy (p.s, str, len);\n for (i = 0; i < 4; i++)\n result[i] = _mc_xor (result[i],\n\t\t\t _mc_mum (_mc_2le (((_mc_ti *) p.s)[0]),\n\t\t\t\t _mc_get (_mc_tail_primes[i])));\n }\n for (i = 0; i < 4; i++)\n state[i] = _mc_xor (state[i], result[i]);\n}\n\n#ifndef MUM512_ROUNDS\n#define MUM512_ROUNDS 8\n#endif\n\n#if MUM512_ROUNDS < 2\n#error \"too few final rounds\"\n#elif MUM512_ROUNDS > 16\n#error \"too many final rounds\"\n#endif\n\nstatic inline void\n_mc_permute (_mc_ti t[4]) {\n /* Row */\n t[0] = _mc_rotr (_mc_xor (t[0], t[1]), 17);\n t[1] = _mc_rotr (_mc_xor (t[1], t[0]), 33);\n t[3] = _mc_rotr (_mc_xor (t[3], t[2]), 49);\n t[2] = _mc_rotr (_mc_xor (t[2], t[3]), 65);\n /* Column */\n t[0] = _mc_rotr (_mc_xor (t[0], t[2]), 9);\n t[2] = _mc_rotr (_mc_xor (t[2], t[0]), 25);\n t[3] = _mc_rotr (_mc_xor (t[3], t[1]), 41);\n t[1] = _mc_rotr (_mc_xor (t[1], t[3]), 57);\n /* Diagonal */\n t[0] = _mc_rotr (_mc_xor (t[0], t[3]), 13);\n t[3] = _mc_rotr (_mc_xor (t[3], t[0]), 29);\n t[2] = _mc_rotr (_mc_xor (t[2], t[1]), 45);\n t[1] = _mc_rotr (_mc_xor (t[1], t[2]), 61);\n}\n\nstatic inline void\n_mc_mix (_mc_ti state[4], uint64_t out[8]) {\n int i, j;\n _mc_ti t[4];\n \n for (i = 0; i < 4; i++)\n t[i] = state[i];\n for (i = 0; i < MUM512_ROUNDS; i++) {\n for (j = 0; j < 4; j++)\n t[j] = _mc_xor (t[j], _mc_mum (t[j], _mc_get (_mc_finish_primes[i * 4 + j])));\n _mc_permute (t);\n }\n for (i = 0; i < 4; i++) {\n t[i] = _mc_2le (t[i]);\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n out[2 * i] = _mc_lo64 (t[i]);\n out[2 * i + 1] = _mc_hi64 (t[i]);\n#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n out[2 * i] = _mc_hi64 (t[i]);\n out[2 * i + 1] = _mc_lo64 (t[i]);\n#endif\n }\n}\n\n/* Random prime numbers for the initialization. */\n#define SEED_PRIME0 0x2e0bb864e9ea7df5ULL\n#define SEED_PRIME1 0xcdb32970830fcaa1ULL\n#define SEED_PRIME2 0xc42b5e2e6480b23bULL\n#define SEED_PRIME3 0x746addee2093e7e1ULL\n#define SEED_PRIME4 0xa9a7ae7ceff79f3fULL\n#define SEED_PRIME5 0xaf47d47c99b1461bULL\n#define SEED_PRIME6 0x7b51ec3d22f7096fULL\n\nstatic inline void\n_mc_init_state (_mc_ti state[4], const uint64_t seed[4], size_t len) {\n state[0] = _mc_const (seed[0] ^ SEED_PRIME0, seed[1] ^ SEED_PRIME1);\n state[1] = _mc_const (seed[2] ^ SEED_PRIME2, seed[3] ^ SEED_PRIME3);\n state[2] = _mc_const (len, SEED_PRIME4);\n state[3] = _mc_const (SEED_PRIME5, SEED_PRIME6);\n}\n\n#ifndef _MC_UNALIGNED_ACCESS\n#if defined(__x86_64__) || defined(__i386__) || defined(__PPC64__) \\\n || defined(__s390__) || defined(__m32c__) || defined(cris) \\\n || defined(__CR16__) || defined(__vax__) || defined(__m68k__) \\\n || defined(__aarch64__) || defined(_M_AMD64) || defined(_M_IX86)\n#define _MC_UNALIGNED_ACCESS 1\n#else\n#define _MC_UNALIGNED_ACCESS 0\n#endif\n#endif\n\n/* When we need an aligned access to data being hashed we move part of\n the unaligned data to an aligned block of given size and then\n process it, repeating processing the data by the block. */\n#ifndef _MC_BLOCK_LEN_POW2\n#define _MC_BLOCK_LEN_POW2 10\n#endif\n\n#if _MC_BLOCK_LEN_POW2 < 4\n/* We should process by full 128-bit unless it is a tail. */\n#error \"too small block length\"\n#endif\n\nstatic inline void\n#if defined(__x86_64__)\n_MC_TARGET(\"inline-all-stringops\")\n#endif\n_mc_hash_default (const void *data, size_t len, const uint64_t seed[4], unsigned char *out) {\n _mc_ti state[4];\n const unsigned char *str = (const unsigned char *) data;\n size_t block_len;\n _mc_ti buf[_MC_BLOCK_LEN_POW2 / sizeof (_mc_ti)];\n \n _mc_init_state (state, seed, len);\n if (((size_t) str & 0x7) == 0)\n _mc_hash_aligned (state, data, len);\n else {\n while (len != 0) {\n block_len\n\t= len < (1 << _MC_BLOCK_LEN_POW2) ? len : (1 << _MC_BLOCK_LEN_POW2);\n memcpy (buf, str, block_len);\n _mc_hash_aligned (state, buf, block_len);\n len -= block_len;\n str += block_len;\n }\n }\n if (((size_t) out & 0x7) == 0)\n _mc_mix (state, (uint64_t *) out);\n else {\n uint64_t ui64[8];\n \n _mc_mix (state, ui64);\n memmove (out, ui64, 64);\n }\n}\n\n/* ++++++++++++++++++++++++++ Interface functions: +++++++++++++++++++ */\n\n/* Hash DATA of length LEN and SEED. */\nstatic inline void\nmum512_keyed_hash (const void *data, size_t len, const uint64_t key[4], unsigned char *out) {\n#if _MC_UNALIGNED_ACCESS\n _mc_ti state[4];\n \n _mc_init_state (state, key, len);\n _mc_hash_aligned (state, data, len);\n _mc_mix (state, (uint64_t *) out);\n return;\n#else\n _mc_hash_default (data, len, key, out);\n#endif\n}\n\n/* Hash DATA of length LEN. */\nstatic inline void\nmum512_hash (const void *data, size_t len, unsigned char *out) {\n static const uint64_t key[4] = {0, 0, 0, 0};\n\n mum512_keyed_hash (data, len, key, out);\n}\n\n#endif\n" }, { "path": "vmum.h", "content": "/* Copyright (c) 2025\n Vladimir Makarov \n\n Permission is hereby granted, free of charge, to any person\n obtaining a copy of this software and associated documentation\n files (the \"Software\"), to deal in the Software without\n restriction, including without limitation the rights to use, copy,\n modify, merge, publish, distribute, sublicense, and/or sell copies\n of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n\n The above copyright notice and this permission notice shall be\n included in all copies or substantial portions of the Software.\n\n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\n MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\n BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\n ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\n CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.\n*/\n\n/* This file implements VMUM (Vector MUltiply and Mix) hashing. We randomize\n input data by 64x64-bit multiplication and mixing hi- and low-parts of the\n multiplication result by using an addition and then mix it into the current\n state. We use random numbers generated with sha3sum for the multiplication.\n When all the numbers are used once, the state is randomized and the same\n numbers are used again for data randomization.\n\n The VMUM hashing passes all rurban SMHasher tests. */\n\n#ifndef __VMUM_HASH__\n#define __VMUM_HASH__\n\n#include \n#include \n#include \n#include \n\n#if defined(__STDC_VERSION__) && __STDC_VERSION__ >= 201112L\n#include \n#else\n#define static_assert(a)\n#endif\n\n#ifdef _MSC_VER\ntypedef unsigned __int16 uint16_t;\ntypedef unsigned __int32 uint32_t;\ntypedef unsigned __int64 uint64_t;\n#else\n#include \n#endif\n\n#if defined(__GNUC__)\n#define _VMUM_ATTRIBUTE_UNUSED __attribute__ ((unused))\n#define _VMUM_INLINE inline __attribute__ ((always_inline))\n#else\n#define _VMUM_ATTRIBUTE_UNUSED\n#define _VMUM_INLINE inline\n#endif\n\n/* Macro saying to use 128-bit integers implemented by GCC for some targets. */\n#ifndef _VMUM_USE_INT128\n/* In GCC uint128_t is defined if HOST_BITS_PER_WIDE_INT >= 64. HOST_WIDE_INT\n is long if HOST_BITS_PER_LONG > HOST_BITS_PER_INT, otherwise int. */\n#if defined(__GNUC__) && UINT_MAX != ULONG_MAX\n#define _VMUM_USE_INT128 1\n#else\n#define _VMUM_USE_INT128 0\n#endif\n#endif\n\n/* Here are different random numbers generated with the equal probability of\n their bit values. They are used to randomize input values. */\nstatic uint64_t _vmum_hash_step_factor = 0x7fc65e8a22c2f74bull;\nstatic uint64_t _vmum_key_step_factor = 0x9b307d68270e94e5ull;\nstatic uint64_t _vmum_block_start_factor = 0x6608b54dafbc797cull;\nstatic uint64_t _vmum_tail_factor = 0xc6f58747253b0e84ull;\nstatic uint64_t _vmum_finish_factor1 = 0xbc4bb29ce739b5d9ull;\nstatic uint64_t _vmum_finish_factor2 = 0x7007946aa4fdb987ull;\n\nstatic uint64_t _vmum_unroll_factors[] = {\n 0x191fb5fc4a9bf2deull,\n 0xd9a09a0a2c4eb3ebull,\n 0x90f15ee96deb1eecull,\n 0x1a970df0a79d09baull,\n};\n\nstatic const uint64_t _vmum_zero[] = {0ull, 0ull, 0ull, 0ll};\nstatic uint64_t _vmum_factors[] = {\n 0x0c3ec9639d3a203full, 0x898368b5fb060422ull, 0xfe3c08767c1e5068ull, 0x4535ac3cb182d3caull,\n 0xba6ba8dcc8a2d978ull, 0x9f1221df37b27ca1ull, 0x57b1b40817cde05eull, 0xadb5c6075e5dd1c3ull,\n 0x0f91abf611686bc3ull, 0x72fc850fbe9023f4ull, 0x4922ec730400d7e1ull, 0x7452d927d9970eb2ull,\n 0x607ad8c0dbe89ff9ull, 0x463bf8a99a222448ull, 0x6a0c8e5c2171d499ull, 0x88a9af1a70175e0full,\n\n 0x50c07b9351011316ull, 0xce26a1f4e66e3d1full, 0x396d79c90c86b1edull, 0x3739b72e11fc4684ull,\n 0x7fe4adc8400af08eull, 0xfa2e47475b11a3ceull, 0xd519635f1d7b9242ull, 0x866d04bd866130fcull,\n 0x6132e913fd0ae2c9ull, 0xaeacc8d99a02880dull, 0xf196fbab2ef62dbbull, 0x62a6a4ae6d3f5fddull,\n 0x3147dbedb6618cf4ull, 0x18dc2a4e6e46b7f5ull, 0x98ce62d3e346f804ull, 0x3e507b6626b9c9c6ull,\n\n 0x08d2d6aba49b53a7ull, 0x64f897cf54cc0984ull, 0xe6568e3081d4dc31ull, 0x936eff54d90e79bdull,\n 0x45ecee4cb55e5529ull, 0xe7729fe4b83bd1f0ull, 0x8ced03855b3a62aeull, 0xc29ee17b601d5ea6ull,\n 0xfadd87cd819b8b23ull, 0x5ff6b6787a02da34ull, 0xcc0f997728759ba9ull, 0x06a51956916c45ffull,\n 0xd31923282290a41full, 0x9e2fcda93feb36aaull, 0xfdc11827f9bb4b63ull, 0xe11b2c099cf17325ull,\n\n 0xd4efd1919316bb4eull, 0x7749918c7756d897ull, 0x32eff42dc1c4f41eull, 0x4171f5e422bdf4faull,\n 0xdd5c7f50a7977e65ull, 0xc52dc937487cc197ull, 0xbff445a1c9b19184ull, 0xb2fd8206a0325cb9ull,\n 0xbf32bb70e354a83cull, 0x587313d37681ffc2ull, 0x687c0b98ef7ad731ull, 0x2c09a37608248f89ull,\n 0x20c8d9b1b6e28faaull, 0x747008703cbc9483ull, 0x0bdcacce877cb231ull, 0xf740c0ac1dc79a0cull,\n\n 0x8e843baef228089dull, 0xc379d4c3b6e28c1bull, 0xb5d44eee257f1206ull, 0xb5dfee44ef6b05adull,\n 0xaab0edc0f566f359ull, 0x21883ede514d45ddull, 0xcdaeea0482c73a46ull, 0xb3751631e5645c4full,\n 0xd22ac81254aad1acull, 0x59c0f8f5c93121dfull, 0x3b343a848e3097c3ull, 0x87c65a3a866ed9edull,\n 0xf112c880c63ddafdull, 0xff98b9575bfce057ull, 0xd6e8f6ad6f4da8aeull, 0xba0379ba20645a51ull,\n};\n\n/* Multiply 64-bit V and P and return sum of high and low parts of the result.\n */\nstatic _VMUM_INLINE uint64_t _vmum (uint64_t v, uint64_t p) {\n uint64_t hi, lo;\n#if _VMUM_USE_INT128\n __uint128_t r = (__uint128_t) v * (__uint128_t) p;\n hi = (uint64_t) (r >> 64);\n lo = (uint64_t) r;\n#else\n /* Implementation of 64x64->128-bit multiplication by four 32x32->64 bit\n * multiplication. */\n uint64_t hv = v >> 32, hp = p >> 32;\n uint64_t lv = (uint32_t) v, lp = (uint32_t) p;\n uint64_t rh = hv * hp;\n uint64_t rm_0 = hv * lp;\n uint64_t rm_1 = hp * lv;\n uint64_t rl = lv * lp;\n return rh + rl + rm_0 + rm_1;\n uint64_t t, carry = 0;\n\n /* We could ignore a carry bit here if we did not care about the same hash\n for 32-bit and 64-bit targets. */\n t = rl + (rm_0 << 32);\n#ifdef VMUM_TARGET_INDEPENDENT_HASH\n carry = t < rl;\n#endif\n lo = t + (rm_1 << 32);\n#ifdef VMUM_TARGET_INDEPENDENT_HASH\n carry += lo < t;\n#endif\n hi = rh + (rm_0 >> 32) + (rm_1 >> 32) + carry;\n#endif\n return hi + lo;\n}\n\n#if defined(_MSC_VER)\n#define _vmum_bswap32(x) _byteswap_uint32_t (x)\n#define _vmum_bswap64(x) _byteswap_uint64_t (x)\n#elif defined(__APPLE__)\n#include \n#define _vmum_bswap32(x) OSSwapInt32 (x)\n#define _vmum_bswap64(x) OSSwapInt64 (x)\n#elif defined(__GNUC__)\n#define _vmum_bswap32(x) __builtin_bswap32 (x)\n#define _vmum_bswap64(x) __builtin_bswap64 (x)\n#else\n#include \n#define _vmum_bswap32(x) bswap32 (x)\n#define _vmum_bswap64(x) bswap64 (x)\n#endif\n\nstatic _VMUM_INLINE uint64_t _vmum_le (uint64_t v) {\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ || !defined(VMUM_TARGET_INDEPENDENT_HASH)\n return v;\n#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n return _vmum_bswap64 (v);\n#else\n#error \"Unknown endianess\"\n#endif\n}\n\nstatic _VMUM_INLINE uint32_t _vmum_le32 (uint32_t v) {\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ || !defined(VMUM_TARGET_INDEPENDENT_HASH)\n return v;\n#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n return _vmum_bswap32 (v);\n#else\n#error \"Unknown endianess\"\n#endif\n}\n\nstatic _VMUM_INLINE uint64_t _vmum_le16 (uint16_t v) {\n#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ || !defined(VMUM_TARGET_INDEPENDENT_HASH)\n return v;\n#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__\n return (v >> 8) | ((v & 0xff) << 8);\n#else\n#error \"Unknown endianess\"\n#endif\n}\n\nstatic _VMUM_INLINE uint64_t _vmum_xor (uint64_t a, uint64_t b) {\n#ifdef VMUM_V1\n return a ^ b;\n#else\n return (a ^ b) != 0 ? a ^ b : b;\n#endif\n}\n\nstatic _VMUM_INLINE uint64_t _vmum_plus (uint64_t a, uint64_t b) {\n#ifdef VMUM_V1\n return a + b;\n#else\n return a + b != 0 ? a + b : b;\n#endif\n}\n\n/* Here is the vector code for different targets. We process blocks of 256-bit\n length. Vector consists of 64-bit elements. The vector block update uses\n 32-bit x 32-bit usigned multiplication to 64-bit result. The understand what\n the update implements see scalar version below. */\n#if defined(__AVX2__)\ntypedef long long __attribute__ ((vector_size (32), aligned (1))) _vmum_block_t;\ntypedef int __attribute__ ((vector_size (32), aligned (1))) _vmum_v8si;\nstatic _VMUM_INLINE _vmum_block_t _vmum_block (_vmum_block_t v, _vmum_block_t p) {\n _vmum_block_t hv = v >> 32, hp = p >> 32;\n _vmum_block_t r\n = (_vmum_block_t) (__builtin_ia32_pmuludq256 ((_vmum_v8si) v, (_vmum_v8si) p)\n + __builtin_ia32_pmuludq256 ((_vmum_v8si) hv, (_vmum_v8si) hp));\n return r;\n}\nstatic _VMUM_INLINE _vmum_block_t _vmum_nonzero (_vmum_block_t v) {\n _vmum_block_t r\n = (_vmum_block_t) __builtin_ia32_pcmpeqd256 ((_vmum_v8si) v, *(_vmum_v8si *) _vmum_zero);\n return v | r;\n}\nstatic _VMUM_INLINE void _vmum_update_block (_vmum_block_t *s, const _vmum_block_t *v,\n const _vmum_block_t *p) {\n#ifdef VMUM_V1\n *s ^= _vmum_block (v[0] ^ p[0], v[1] ^ p[1]);\n#else\n *s ^= _vmum_block (_vmum_nonzero (v[0] ^ p[0]), _vmum_nonzero (v[1] ^ p[1]));\n#endif\n}\nstatic _VMUM_INLINE void _vmum_factor_block (_vmum_block_t *s, const _vmum_block_t *p) {\n *s = _vmum_block (*s, *p);\n}\nstatic _VMUM_INLINE void _vmum_zero_block (_vmum_block_t *b) { *b = (_vmum_block_t) {0, 0, 0, 0}; }\nstatic _VMUM_INLINE uint64_t _vmum_fold_block (const _vmum_block_t *b) {\n return (*b)[0] ^ (*b)[1] ^ (*b)[2] ^ (*b)[3];\n}\n#elif defined(__ARM_NEON)\n#include \ntypedef struct {\n uint64x2_t v[2];\n} _vmum_block_t;\nstatic _VMUM_INLINE uint64x2_t _vmum_val (uint64x2_t v, uint64x2_t p) {\n uint32x4_t v32 = (uint32x4_t) v, p32 = (uint32x4_t) p;\n uint64x2_t r = vmull_u32 (vget_low_u32 (v32), vget_low_u32 (p32)), r2 = vmull_high_u32 (v32, p32);\n return vpaddq_u64 (r, r2);\n}\nstatic _VMUM_INLINE uint64x2_t _vmum_nonzero (uint64x2_t v) {\n uint64x2_t r = vceqzq_u64 (v);\n return r | v;\n}\nstatic _VMUM_INLINE void _vmum_update_block (_vmum_block_t *s, const _vmum_block_t *v,\n const _vmum_block_t *p) {\n const _vmum_block_t *v2 = &v[1], *p2 = &p[1];\n#ifdef VMUM_V1\n s->v[0] ^= _vmum_val (v->v[0] ^ p->v[0], v2->v[0] ^ p2->v[0]);\n s->v[1] ^= _vmum_val (v->v[1] ^ p->v[1], v2->v[1] ^ p2->v[1]);\n#else\n s->v[0] ^= _vmum_val (_vmum_nonzero (v->v[0] ^ p->v[0]), _vmum_nonzero (v2->v[0] ^ p2->v[0]));\n s->v[1] ^= _vmum_val (_vmum_nonzero (v->v[1] ^ p->v[1]), _vmum_nonzero (v2->v[1] ^ p2->v[1]));\n#endif\n}\nstatic _VMUM_INLINE void _vmum_factor_block (_vmum_block_t *s, const _vmum_block_t *p) {\n s->v[0] = _vmum_val (s->v[0], p->v[0]);\n s->v[1] = _vmum_val (s->v[1], p->v[1]);\n}\nstatic _VMUM_INLINE void _vmum_zero_block (_vmum_block_t *b) { *b = (_vmum_block_t) {0, 0, 0, 0}; }\nstatic _VMUM_INLINE uint64_t _vmum_fold_block (_vmum_block_t *b) {\n return b->v[0][0] ^ b->v[0][1] ^ b->v[1][0] ^ b->v[1][1];\n}\n#elif defined(__ALTIVEC__)\n#include \ntypedef unsigned long long __attribute__ ((vector_size (16))) _vmum_v2di;\ntypedef struct {\n _vmum_v2di v[2];\n} _vmum_block_t;\nstatic _VMUM_INLINE _vmum_v2di _vmum_val (_vmum_v2di v, _vmum_v2di p) {\n typedef unsigned int __attribute__ ((vector_size (16))) v4si;\n return (_vmum_v2di) (vec_mule ((v4si) v, (v4si) p) + vec_mulo ((v4si) v, (v4si) p));\n}\nstatic _VMUM_INLINE void _vmum_zero_block (_vmum_block_t *b) { *b = (_vmum_block_t) {0, 0, 0, 0}; }\nstatic _VMUM_INLINE _vmum_v2di _vmum_nonzero (_vmum_v2di v) {\n __vector __bool long mask = vec_cmpne (v, (_vmum_v2di) {0ull, 0ull});\n _vmum_v2di r = vec_sel ((_vmum_v2di) {0xffffffffffffffffull, 0xffffffffffffffffull}, v, mask);\n return r;\n}\nstatic _VMUM_INLINE void _vmum_update_block (_vmum_block_t *s, const _vmum_block_t *v,\n const _vmum_block_t *p) {\n const _vmum_block_t *v2 = &v[1], *p2 = &p[1];\n#ifdef VMUM_V1\n s->v[0] ^= _vmum_val (v->v[0] ^ p->v[0], v2->v[0] ^ p2->v[0]);\n s->v[1] ^= _vmum_val (v->v[1] ^ p->v[1], v2->v[1] ^ p2->v[1]);\n#else\n s->v[0] ^= _vmum_val (_vmum_nonzero (v->v[0] ^ p->v[0]), _vmum_nonzero (v2->v[0] ^ p2->v[0]));\n s->v[1] ^= _vmum_val (_vmum_nonzero (v->v[1] ^ p->v[1]), _vmum_nonzero (v2->v[1] ^ p2->v[1]));\n#endif\n}\nstatic _VMUM_INLINE void _vmum_factor_block (_vmum_block_t *s, const _vmum_block_t *p) {\n s->v[0] = _vmum_val (s->v[0], p->v[0]);\n s->v[1] = _vmum_val (s->v[1], p->v[1]);\n}\nstatic _VMUM_INLINE uint64_t _vmum_fold_block (_vmum_block_t *b) {\n return b->v[0][0] ^ b->v[0][1] ^ b->v[1][0] ^ b->v[1][1];\n}\n#else /* scalar version: */\ntypedef struct {\n uint64_t v[4];\n} _vmum_block_t;\nstatic _VMUM_INLINE uint64_t _vmum_val (uint64_t v, uint64_t p) {\n uint64_t lv = (uint32_t) v, lp = (uint32_t) p, hv = v >> 32, hp = p >> 32;\n return lv * lp + hv * hp;\n}\nstatic _VMUM_INLINE void _vmum_update_block (_vmum_block_t *s, const _vmum_block_t *v,\n const _vmum_block_t *p) {\n const _vmum_block_t *v2 = &v[1], *p2 = &p[1];\n s->v[0] ^= _vmum_val (_vmum_xor (_vmum_le (v->v[0]), p->v[0]),\n _vmum_xor (_vmum_le (v2->v[0]), p2->v[0]));\n s->v[1] ^= _vmum_val (_vmum_xor (_vmum_le (v->v[1]), p->v[1]),\n _vmum_xor (_vmum_le (v2->v[1]), p2->v[1]));\n s->v[2] ^= _vmum_val (_vmum_xor (_vmum_le (v->v[2]), p->v[2]),\n _vmum_xor (_vmum_le (v2->v[2]), p2->v[2]));\n s->v[3] ^= _vmum_val (_vmum_xor (_vmum_le (v->v[3]), p->v[3]),\n _vmum_xor (_vmum_le (v2->v[3]), p2->v[3]));\n}\nstatic _VMUM_INLINE void _vmum_factor_block (_vmum_block_t *s, const _vmum_block_t *p) {\n s->v[0] = _vmum_val (s->v[0], p->v[0]);\n s->v[1] = _vmum_val (s->v[1], p->v[1]);\n s->v[2] = _vmum_val (s->v[2], p->v[2]);\n s->v[3] = _vmum_val (s->v[3], p->v[3]);\n}\nstatic _VMUM_INLINE void _vmum_zero_block (_vmum_block_t *b) { *b = (_vmum_block_t) {0, 0, 0, 0}; }\nstatic _VMUM_INLINE uint64_t _vmum_fold_block (_vmum_block_t *b) {\n return b->v[0] ^ b->v[1] ^ b->v[2] ^ b->v[3];\n}\n#endif\n\n/* Macro defining how many vectors the most the 1st nested loop in\n _vmum_hash_aligned will be unrolled by the compiler (although it can make an\n own decision:). Use only a constant here to help a compiler to unroll a\n major loop. The unroll factor greatly affects the hashing speed. We prefer\n the speed. */\n#define _VMUM_UNROLL_BLOCK_FACTOR 16\n/* Macro defining how many uint64 the most the 2st nested loop in\n _vmum_hash_aligned will be unrolled by the compiler. */\n#define _VMUM_UNROLL_FACTOR 16\n\nstatic _VMUM_INLINE uint64_t\n#if defined(__GNUC__) && !defined(__clang__)\n __attribute__ ((__optimize__ (\"unroll-loops\")))\n#endif\n _vmum_hash_aligned (uint64_t start, const void *key, size_t len) {\n uint64_t state = start;\n const unsigned char *str = (const unsigned char *) key;\n uint64_t u64, w;\n size_t i, j;\n size_t n;\n\n state = _vmum (state, _vmum_block_start_factor);\n if (len >= _VMUM_UNROLL_BLOCK_FACTOR * sizeof (_vmum_block_t)) {\n _vmum_block_t block_state;\n _vmum_zero_block (&block_state);\n do {\n static_assert (_VMUM_UNROLL_BLOCK_FACTOR <= sizeof (_vmum_factors) / sizeof (uint64_t));\n for (i = 0; i < _VMUM_UNROLL_BLOCK_FACTOR; i += 2)\n _vmum_update_block (&block_state, &((_vmum_block_t *) str)[i],\n &((_vmum_block_t *) _vmum_factors)[i]);\n len -= _VMUM_UNROLL_BLOCK_FACTOR * sizeof (_vmum_block_t);\n str += _VMUM_UNROLL_BLOCK_FACTOR * sizeof (_vmum_block_t);\n /* We will use the same factor numbers on the next iterations --\n * randomize the state. */\n _vmum_factor_block (&block_state, (_vmum_block_t *) _vmum_unroll_factors);\n } while (len >= _VMUM_UNROLL_BLOCK_FACTOR * sizeof (_vmum_block_t));\n state += _vmum_fold_block (&block_state);\n }\n /* Here we have enough factors to do hashing w/o state randominzation: */\n static_assert (_VMUM_UNROLL_BLOCK_FACTOR * sizeof (_vmum_block_t) / sizeof (uint64_t) + 7\n <= sizeof (_vmum_factors) / sizeof (uint64_t));\n i = 0;\n while (len >= _VMUM_UNROLL_FACTOR * sizeof (uint64_t)) {\n for (j = 0; j < _VMUM_UNROLL_FACTOR; j += 2, i += 2)\n state ^= _vmum (_vmum_xor (_vmum_le (((uint64_t *) str)[i]), _vmum_factors[i]),\n _vmum_xor (_vmum_le (((uint64_t *) str)[i + 1]), _vmum_factors[i + 1]));\n len -= _VMUM_UNROLL_FACTOR * sizeof (uint64_t);\n }\n n = len / sizeof (uint64_t) & ~(size_t) 1;\n for (j = 0; j < n; j += 2, i += 2)\n state ^= _vmum (_vmum_plus (_vmum_le (((uint64_t *) str)[i]), _vmum_factors[i]),\n _vmum_plus (_vmum_le (((uint64_t *) str)[i + 1]), _vmum_factors[i + 1]));\n len -= n * sizeof (uint64_t);\n str += i * sizeof (uint64_t);\n uint64_t p;\n switch (len) {\n case 15:\n w = _vmum_plus (_vmum_le (*(uint64_t *) str), _vmum_factors[i]);\n u64 = _vmum_factors[i + 1] + _vmum_le32 (*(uint32_t *) (str + 8));\n u64 += (_vmum_le16 (*(uint16_t *) (str + 12)) << 32) + ((uint64_t) str[14] << 48);\n return state ^ _vmum (u64 ^ _vmum_tail_factor, w);\n case 14:\n w = _vmum_plus (_vmum_le (*(uint64_t *) str), _vmum_factors[i]);\n u64 = _vmum_factors[i + 2] + _vmum_le32 (*(uint32_t *) (str + 8));\n u64 += _vmum_le16 (*(uint16_t *) (str + 12)) << 32;\n return state ^ _vmum (u64 ^ _vmum_tail_factor, w);\n case 13:\n w = _vmum_plus (_vmum_le (*(uint64_t *) str), _vmum_factors[i]);\n u64 = _vmum_factors[i + 3] + _vmum_le32 (*(uint32_t *) (str + 8));\n u64 += (uint64_t) str[12] << 32;\n return state ^ _vmum (u64 ^ _vmum_tail_factor, w);\n case 12:\n w = _vmum_plus (_vmum_le (*(uint64_t *) str), _vmum_factors[i]);\n u64 = _vmum_factors[i + 4] + _vmum_le32 (*(uint32_t *) (str + 8));\n return state ^ _vmum (u64 ^ _vmum_tail_factor, w);\n case 11:\n w = _vmum_plus (_vmum_le (*(uint64_t *) str), _vmum_factors[i]);\n u64 = _vmum_factors[i + 5] + _vmum_le16 (*(uint16_t *) (str + 8));\n u64 += (uint64_t) str[10] << 16;\n return state ^ _vmum (u64 ^ _vmum_tail_factor, w);\n case 10:\n w = _vmum_plus (_vmum_le (*(uint64_t *) str), _vmum_factors[i]);\n u64 = _vmum_factors[i + 6] + _vmum_le16 (*(uint16_t *) (str + 8));\n return state ^ _vmum (u64 ^ _vmum_tail_factor, w);\n case 9:\n w = _vmum_plus (_vmum_le (*(uint64_t *) str), _vmum_factors[i]);\n u64 = _vmum_factors[i + 7] + str[8];\n return state ^ _vmum (u64 ^ _vmum_tail_factor, w);\n case 8: return state ^ _vmum (_vmum_le (*(uint64_t *) str) + _vmum_factors[i], _vmum_factors[i]);\n case 7:\n u64 = _vmum_factors[i + 1] + _vmum_le32 (*(uint32_t *) str);\n u64 += (_vmum_le16 (*(uint16_t *) (str + 4)) << 32) + ((uint64_t) str[6] << 48);\n return state ^ _vmum (u64, _vmum_tail_factor);\n case 6:\n u64 = _vmum_factors[i + 2] + _vmum_le32 (*(uint32_t *) str);\n u64 += _vmum_le16 (*(uint16_t *) (str + 4)) << 32;\n return state ^ _vmum (u64, _vmum_tail_factor);\n case 5:\n u64 = _vmum_factors[i + 3] + _vmum_le32 (*(uint32_t *) str);\n u64 += (uint64_t) str[4] << 32;\n return state ^ _vmum (u64, _vmum_tail_factor);\n case 4:\n u64 = _vmum_factors[i + 4] + _vmum_le32 (*(uint32_t *) str);\n return state ^ _vmum (u64, _vmum_tail_factor);\n case 3:\n u64 = _vmum_factors[i + 5] + _vmum_le16 (*(uint16_t *) str);\n u64 += (uint64_t) str[2] << 16;\n return state ^ _vmum (u64, _vmum_tail_factor);\n case 2:\n u64 = _vmum_factors[i + 6] + _vmum_le16 (*(uint16_t *) str);\n return state ^ _vmum (u64, _vmum_tail_factor);\n case 1: u64 = _vmum_factors[i + 7] + str[0]; return state ^ _vmum (u64, _vmum_tail_factor);\n }\n return state;\n}\n\n/* Final randomization of H. */\nstatic _VMUM_INLINE uint64_t _vmum_final (uint64_t h) {\n h = _vmum (h, h);\n return h;\n}\n\n#ifndef _VMUM_UNALIGNED_ACCESS\n#if defined(__x86_64__) || defined(__i386__) || defined(__PPC64__) || defined(__s390__) \\\n || defined(__m32c__) || defined(cris) || defined(__CR16__) || defined(__vax__) \\\n || defined(__m68k__) || defined(__aarch64__) || defined(_M_AMD64) || defined(_M_IX86)\n#define _VMUM_UNALIGNED_ACCESS 1\n#else\n#define _VMUM_UNALIGNED_ACCESS 0\n#endif\n#endif\n\n/* When we need an aligned access to data being hashed we move part of the\n unaligned data to an aligned block of given size and then process it,\n repeating processing the data by the block. */\n#ifndef _VMUM_BLOCK_LEN\n#define _VMUM_BLOCK_LEN 1024\n#endif\n\n#if _VMUM_BLOCK_LEN < 8\n#error \"too small block length\"\n#endif\n\nstatic _VMUM_INLINE uint64_t\n#if defined(__x86_64__) && defined(__GNUC__) && !defined(__clang__)\n __attribute__ ((__target__ (\"inline-all-stringops\")))\n#endif\n _vmum_hash_default (const void *key, size_t len, uint64_t seed) {\n uint64_t result;\n const unsigned char *str = (const unsigned char *) key;\n size_t block_len;\n uint64_t buf[_VMUM_BLOCK_LEN / sizeof (uint64_t)];\n\n result = seed + len;\n if (((size_t) str & 0x7) == 0)\n result = _vmum_hash_aligned (result, key, len);\n else {\n while (len != 0) {\n block_len = len < _VMUM_BLOCK_LEN ? len : _VMUM_BLOCK_LEN;\n memmove (buf, str, block_len);\n result = _vmum_hash_aligned (result, buf, block_len);\n len -= block_len;\n str += block_len;\n }\n }\n return _vmum_final (result);\n}\n\nstatic _VMUM_INLINE uint64_t _vmum_next_factor (void) {\n uint64_t start = 0;\n int i;\n\n for (i = 0; i < 8; i++) start = (start << 8) | rand () % 256;\n return start;\n}\n\n/* ++++++++++++++++++++++++++ Interface functions: +++++++++++++++++++ */\n\n/* Set random multiplicators depending on SEED. */\nstatic _VMUM_INLINE void vmum_hash_randomize (uint64_t seed) {\n size_t i;\n\n srand (seed);\n _vmum_hash_step_factor = _vmum_next_factor ();\n _vmum_key_step_factor = _vmum_next_factor ();\n _vmum_finish_factor1 = _vmum_next_factor ();\n _vmum_finish_factor2 = _vmum_next_factor ();\n _vmum_block_start_factor = _vmum_next_factor ();\n _vmum_tail_factor = _vmum_next_factor ();\n for (i = 0; i < sizeof (_vmum_unroll_factors) / sizeof (uint64_t); i++)\n _vmum_unroll_factors[i] = _vmum_next_factor ();\n for (i = 0; i < sizeof (_vmum_factors) / sizeof (uint64_t); i++)\n _vmum_factors[i] = _vmum_next_factor ();\n}\n\n/* Start hashing data with SEED. Return the state. */\nstatic _VMUM_INLINE uint64_t vmum_hash_init (uint64_t seed) { return seed; }\n\n/* Process data KEY with the state H and return the updated state. */\nstatic _VMUM_INLINE uint64_t vmum_hash_step (uint64_t h, uint64_t key) {\n return _vmum (h, _vmum_hash_step_factor) ^ _vmum (key, _vmum_key_step_factor);\n}\n\n/* Return the result of hashing using the current state H. */\nstatic _VMUM_INLINE uint64_t vmum_hash_finish (uint64_t h) { return _vmum_final (h); }\n\n/* Fast hashing of KEY with SEED. The hash is always the same for the same key\n * on any target. */\nstatic _VMUM_INLINE size_t vmum_hash64 (uint64_t key, uint64_t seed) {\n return vmum_hash_finish (vmum_hash_step (vmum_hash_init (seed + 8), key));\n}\n\n/* Hash data KEY of length LEN and SEED. The hash depends on the target\n endianess and the unroll factor. */\nstatic _VMUM_INLINE uint64_t vmum_hash (const void *key, size_t len, uint64_t seed) {\n#if _VMUM_UNALIGNED_ACCESS\n return _vmum_final (_vmum_hash_aligned (seed + len, key, len));\n#else\n return _vmum_hash_default (key, len, seed);\n#endif\n}\n\n#endif\n" } ]