[
  {
    "path": "LICENSE",
    "content": "MIT License\n\nCopyright (c) 2017 Jung-Sang Ahn\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n"
  },
  {
    "path": "Makefile",
    "content": "LDFLAGS = -pthread\nCFLAGS = \\\n\t-g -D_GNU_SOURCE \\\n\t-I. -I./src -I./debug -I./include -I./examples -I./tests \\\n\t-fPIC \\\n\nCFLAGS += -Wall\nCFLAGS += -O3\n#CFLAGS += -fsanitize=address -fuse-ld=gold\n\nCXXFLAGS = $(CFLAGS) \\\n\t--std=c++11 \\\n\n\nSKIPLIST = src/skiplist.o\nSHARED_LIB = libskiplist.so\nSTATIC_LIB = libskiplist.a\n\nTEST = \\\n\ttests/skiplist_test.o \\\n\t$(STATIC_LIB) \\\n\nMT_TEST = \\\n\ttests/mt_test.o \\\n\t$(STATIC_LIB) \\\n\nSTL_MAP_COMPARE = \\\n\ttests/stl_map_compare.o \\\n\t$(STATIC_LIB) \\\n\nCONTAINER_TEST = \\\n\ttests/container_test.o \\\n\t$(STATIC_LIB) \\\n\nPURE_C_EXAMPLE = \\\n\texamples/pure_c_example.o \\\n\t$(STATIC_LIB) \\\n\nCPP_MAP_EXAMPLE = \\\n\texamples/cpp_map_example.o \\\n\t$(STATIC_LIB) \\\n\nCPP_SET_EXAMPLE = \\\n\texamples/cpp_set_example.o \\\n\t$(STATIC_LIB) \\\n\nPROGRAMS = \\\n\ttests/skiplist_test \\\n\ttests/mt_test \\\n\ttests/container_test \\\n\ttests/stl_map_compare \\\n\texamples/pure_c_example \\\n\texamples/cpp_set_example \\\n\texamples/cpp_map_example \\\n\tlibskiplist.so \\\n\tlibskiplist.a \\\n\nall: $(PROGRAMS)\n\nlibskiplist.so: $(SKIPLIST)\n\t$(CXX) $(CXXFLAGS) -shared $(LDBFALGS) -o $(SHARED_LIB) $(SKIPLIST)\n\nlibskiplist.a: $(SKIPLIST)\n\tar rcs $(STATIC_LIB) $(SKIPLIST)\n\ntests/skiplist_test: $(TEST)\n\t$(CXX) $(CXXFLAGS) $^ -o $@ $(LDFLAGS)\n\ntests/mt_test: $(MT_TEST)\n\t$(CXX) $(CXXFLAGS) $^ -o $@ $(LDFLAGS)\n\ntests/container_test: $(CONTAINER_TEST)\n\t$(CXX) $(CXXFLAGS) $^ -o $@ $(LDFLAGS)\n\ntests/stl_map_compare: $(STL_MAP_COMPARE)\n\t$(CXX) $(CXXFLAGS) $^ -o $@ $(LDFLAGS)\n\nexamples/pure_c_example: $(PURE_C_EXAMPLE)\n\t$(CC) $(CFLAGS) $^ -o $@ $(LDFLAGS)\n\nexamples/cpp_map_example: $(CPP_MAP_EXAMPLE)\n\t$(CXX) $(CXXFLAGS) $^ -o $@ $(LDFLAGS)\n\nexamples/cpp_set_example: $(CPP_SET_EXAMPLE)\n\t$(CXX) $(CXXFLAGS) $^ -o $@ $(LDFLAGS)\n\nclean:\n\trm -rf $(PROGRAMS) ./*.o ./*.so ./*/*.o ./*/*.so\n"
  },
  {
    "path": "README.md",
    "content": "Skiplist\n--------\nA generic [Skiplist](https://en.wikipedia.org/wiki/Skip_list) container C implementation, lock-free for both multiple readers and writers. It can be used as a set or a map, containing any type of data.\n\nIt basically uses STL atomic variables with C++ compiler, but they can be switched to built-in GCC atomic operations when we compile it with pure C compiler.\n\nThis repository also includes STL-style lock-free `set` and `map` containers, based on Skiplist implementation. \n\n\nAuthor\n------\nJung-Sang Ahn <jungsang.ahn@gmail.com>\n\n\nBuild\n-----\n```sh\n$ make\n```\n\nHow to use\n----------\n\nCopy [`skiplist.cc`](src/skiplist.cc) file and [`include`](include) files to your source repository.\n\nOr, use library file (`libskiplist.so` or `libskiplist.a`).\n\n* Pure C\n\n[examples/pure_c_example.c](examples/pure_c_example.c)\n\n* C++ (STL-style `set` and `map`)\n\n[examples/cpp_set_example.cc](examples/cpp_set_example.cc)\n\n[examples/cpp_map_example.cc](examples/cpp_map_example.cc)\n\n\nBenchmark results\n-----------------\n* Skiplist vs. STL set + STL mutex\n* Single writer and multiple readers\n* Randomly insert and read 100K integers\n\n![alt text](https://github.com/greensky00/skiplist/blob/master/docs/swmr_graph.png \"Throughput\")\n"
  },
  {
    "path": "debug/skiplist_debug.h",
    "content": "/**\n * Copyright (C) 2017-present Jung-Sang Ahn <jungsang.ahn@gmail.com>\n * All rights reserved.\n *\n * https://github.com/greensky00\n *\n * Skiplist\n * Version: 0.2.5\n *\n * Permission is hereby granted, free of charge, to any person\n * obtaining a copy of this software and associated documentation\n * files (the \"Software\"), to deal in the Software without\n * restriction, including without limitation the rights to use,\n * copy, modify, merge, publish, distribute, sublicense, and/or sell\n * copies of the Software, and to permit persons to whom the\n * Software is furnished to do so, subject to the following\n * conditions:\n *\n * The above copyright notice and this permission notice shall be\n * included in all copies or substantial portions of the Software.\n *\n * THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES\n * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT\n * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,\n * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\n * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR\n * OTHER DEALINGS IN THE SOFTWARE.\n */\n\n#pragma once\n\n#include <assert.h>\n#include <stdio.h>\n\n#include \"skiplist.h\"\n\nstruct dbg_node {\n    skiplist_node snode;\n    int value;\n};\n\n#if __SL_DEBUG >= 1\n    #undef __SLD_ASSERT\n    #undef __SLD_\n    #define __SLD_ASSERT(cond) assert(cond)\n    #define __SLD_(b) b\n#endif\n#if __SL_DEBUG >= 2\n    #undef __SLD_P\n    #define __SLD_P(args...) printf(args)\n#endif\n#if __SL_DEBUG >= 3\n    #undef __SLD_RT_INS\n    #undef __SLD_NC_INS\n    #undef __SLD_RT_RMV\n    #undef __SLD_NC_RMV\n    #undef __SLD_BM\n    #define __SLD_RT_INS(e, n, t, c) __sld_rt_ins(e, n, t, c)\n    #define __SLD_NC_INS(n, nn, t, c) __sld_nc_ins(n, nn, t, c)\n    #define __SLD_RT_RMV(e, n, t, c) __sld_rt_rmv(e, n, t, c)\n    #define __SLD_NC_RMV(n, nn, t, c) __sld_nc_rmv(n, nn, t, c)\n    #define __SLD_BM(n) __sld_bm(n)\n#endif\n#if __SL_DEBUG >= 4\n    #error \"unknown debugging level\"\n#endif\n\n\ninline void __sld_rt_ins(int error_code,\n                         skiplist_node *node,\n                         int top_layer,\n                         int cur_layer)\n{\n    dbg_node *ddd = _get_entry(node, dbg_node, snode);\n    printf(\"[INS] retry (code %d) \"\n           \"%p (top %d, cur %d) %d\\n\",\n           error_code, node,\n           top_layer, cur_layer, ddd->value);\n}\n\ninline void __sld_nc_ins(skiplist_node *node,\n                         skiplist_node *next_node,\n                         int top_layer,\n                         int cur_layer)\n{\n    dbg_node *ddd = _get_entry(node, dbg_node, snode);\n    dbg_node *ddd_next = _get_entry(next_node, dbg_node, snode);\n\n    printf(\"[INS] next node changed, \"\n           \"%p %p (top %d, cur %d) %d %d\\n\",\n           node, next_node,\n           top_layer, cur_layer, ddd->value, ddd_next->value);\n}\n\ninline void __sld_rt_rmv(int error_code,\n                         skiplist_node *node,\n                         int top_layer,\n                         int cur_layer)\n{\n    dbg_node *ddd = _get_entry(node, dbg_node, snode);\n    printf(\"[RMV] retry (code %d) \"\n           \"%p (top %d, cur %d) %d\\n\",\n           error_code, node,\n           top_layer, cur_layer, ddd->value);\n}\n\ninline void __sld_nc_rmv(skiplist_node *node,\n                         skiplist_node *next_node,\n                         int top_layer,\n                         int cur_layer)\n{\n    dbg_node *ddd = _get_entry(node, dbg_node, snode);\n    dbg_node *ddd_next = _get_entry(next_node, dbg_node, snode);\n\n    printf(\"[RMV] next node changed, \"\n           \"%p %p (top %d, cur %d) %d %d\\n\",\n           node, next_node,\n           top_layer, cur_layer, ddd->value, ddd_next->value);\n}\n\ninline void __sld_bm(skiplist_node *node) {\n    dbg_node *ddd = _get_entry(node, dbg_node, snode);\n    printf(\"[RMV] node is being modified %d\\n\", ddd->value);\n}\n\n"
  },
  {
    "path": "examples/cpp_map_example.cc",
    "content": "#include \"sl_map.h\"\n\n#include <stdio.h>\n\nint main() {\n    // sl_map: Busy-waiting implementation.\n    //         erase() API may be blocked by concurrent\n    //         operations dealing with iterator on the\n    //         same key.\n    //\n    // sl_map_gc: Lazy reclaiming implementation.\n    //            erase() API will not be blocked by\n    //            any concurrent operations, but may\n    //            consume more memory.\n\n    // sl_map<int, int> slist;\n    sl_map_gc<int, int> slist;\n\n    //   << Insertion >>\n    // Insert 3 KV pairs: {0, 0}, {1, 10}, {2, 20}.\n    for (int i=0; i<3; ++i) {\n        slist.insert(std::make_pair(i, i*10));\n    }\n\n    //   << Point lookup >>\n    for (int i=0; i<3; ++i) {\n        auto itr = slist.find(i);\n        if (itr == slist.end()) continue; // Not found.\n        printf(\"[point lookup] key: %d, value: %d\\n\", itr->first, itr->second);\n\n        // Note: In `sl_map`, while `itr` is alive and holding a node\n        //       in skiplist, other thread cannot erase and free the node.\n        //       Same as `shared_ptr`, `itr` will automatically release\n        //       the node when it is not referred anymore.\n        //       But if you want to release the node before that,\n        //       you can do it as follows:\n        // itr = slist.end();\n    }\n\n    //   << Erase >>\n    // Erase the KV pair for key 1: {1, 10}.\n    slist.erase(1);\n\n    //   << Iteration >>\n    for (auto& entry: slist) {\n        printf(\"[iteration] key: %d, value: %d\\n\", entry.first, entry.second);\n    }\n\n    return 0;\n}\n\n"
  },
  {
    "path": "examples/cpp_set_example.cc",
    "content": "#include \"sl_set.h\"\n\n#include <stdio.h>\n\nint main() {\n    // sl_set: Busy-waiting implementation.\n    //         erase() API may be blocked by concurrent\n    //         operations dealing with iterator on the\n    //         same key.\n    //\n    // sl_set_gc: Lazy reclaiming implementation.\n    //            erase() API will not be blocked by\n    //            any concurrent operations, but may\n    //            consume more memory.\n\n    // sl_set<int> slist;\n    sl_set_gc<int> slist;\n\n    //   << Insertion >>\n    // Insert 3 integers: 0, 1, and 2.\n    for (int i=0; i<3; ++i) {\n        slist.insert(i);\n    }\n\n    //   << Point lookup >>\n    for (int i=0; i<3; ++i) {\n        auto itr = slist.find(i);\n        if (itr == slist.end()) continue; // Not found.\n        printf(\"[point lookup] %d\\n\", *itr);\n\n        // Note: In `sl_set`, while `itr` is alive and holding a node\n        //       in skiplist, other thread cannot erase and free the node.\n        //       Same as `shared_ptr`, `itr` will automatically release\n        //       the node when it is not referred anymore.\n        //       But if you want to release the node before that,\n        //       you can do it as follows:\n        // itr = slist.end();\n    }\n\n    //   << Erase >>\n    // Erase 1.\n    slist.erase(1);\n\n    //   << Iteration >>\n    for (auto& entry: slist) {\n        printf(\"[iteration] %d\\n\", entry);\n    }\n\n    return 0;\n}\n\n"
  },
  {
    "path": "examples/pure_c_example.c",
    "content": "#include \"skiplist.h\"\n\n#include <stdio.h>\n#include <stdlib.h>\n\n// Define a node that contains key and value pair.\nstruct my_node {\n    // Metadata for skiplist node.\n    skiplist_node snode;\n    // My data here: {int, int} pair.\n    int key;\n    int value;\n};\n\n// Define a comparison function for `my_node`.\nstatic int my_cmp(skiplist_node* a, skiplist_node* b, void* aux) {\n    // Get `my_node` from skiplist node `a` and `b`.\n    struct my_node *aa, *bb;\n    aa = _get_entry(a, struct my_node, snode);\n    bb = _get_entry(b, struct my_node, snode);\n\n    // aa  < bb: return neg\n    // aa == bb: return 0\n    // aa  > bb: return pos\n    if (aa->key < bb->key) return -1;\n    if (aa->key > bb->key) return 1;\n    return 0;\n}\n\n\nint main() {\n    skiplist_raw slist;\n\n    // Initialize skiplist.\n    skiplist_init(&slist, my_cmp);\n\n    //   << Insertion >>\n    // Allocate & insert 3 KV pairs: {0, 0}, {1, 10}, {2, 20}.\n    struct my_node* nodes[3];\n    for (int i=0; i<3; ++i) {\n        // Allocate memory.\n        nodes[i] = (struct my_node*)malloc(sizeof(struct my_node));\n        // Initialize node.\n        skiplist_init_node(&nodes[i]->snode);\n        // Assign key and value.\n        nodes[i]->key = i;\n        nodes[i]->value = i*10;\n        // Insert into skiplist.\n        skiplist_insert(&slist, &nodes[i]->snode);\n    }\n\n    //   << Point lookup >>\n    for (int i=0; i<3; ++i) {\n        // Define a query.\n        struct my_node query;\n        query.key = i;\n        // Find a skiplist node `cursor`.\n        skiplist_node* cursor = skiplist_find(&slist, &query.snode);\n        // If `cursor` is NULL, key doesn't exist.\n        if (!cursor) continue;\n        // Get `my_node` from `cursor`.\n        // Note: found->snode == *cursor\n        struct my_node* found = _get_entry(cursor, struct my_node, snode);\n        printf(\"[point lookup] key: %d, value: %d\\n\", found->key, found->value);\n        // Release `cursor` (== &found->snode).\n        // Other thread cannot free `cursor` until `cursor` is released.\n        skiplist_release_node(cursor);\n    }\n\n    //   << Erase >>\n    // Erase the KV pair for key 1: {1, 10}.\n    {\n        // Define a query.\n        struct my_node query;\n        query.key = 1;\n        // Find a skiplist node `cursor`.\n        skiplist_node* cursor = skiplist_find(&slist, &query.snode);\n        // Get `my_node` from `cursor`.\n        // Note: found->snode == *cursor\n        struct my_node* found = _get_entry(cursor, struct my_node, snode);\n        printf(\"[erase] key: %d, value: %d\\n\", found->key, found->value);\n\n        // Detach `found` from skiplist.\n        skiplist_erase_node(&slist, &found->snode);\n        // Release `found`, to free its memory.\n        skiplist_release_node(&found->snode);\n        // Free `found` after it becomes safe.\n        skiplist_wait_for_free(&found->snode);\n        skiplist_free_node(&found->snode);\n        free(found);\n    }\n\n    //   << Iteration >>\n    {\n        // Get the first cursor.\n        skiplist_node* cursor = skiplist_begin(&slist);\n        while (cursor) {\n            // Get `entry` from `cursor`.\n            // Note: entry->snode == *cursor\n            struct my_node* entry = _get_entry(cursor, struct my_node, snode);\n            printf(\"[iteration] key: %d, value: %d\\n\", entry->key, entry->value);\n            // Get next `cursor`.\n            cursor = skiplist_next(&slist, cursor);\n            // Release `entry`.\n            skiplist_release_node(&entry->snode);\n        }\n    }\n\n    //   << Destroy >>\n    {\n        // Iterate and free all nodes.\n        skiplist_node* cursor = skiplist_begin(&slist);\n        while (cursor) {\n            struct my_node* entry = _get_entry(cursor, struct my_node, snode);\n            printf(\"[destroy] key: %d, value: %d\\n\", entry->key, entry->value);\n            // Get next `cursor`.\n            cursor = skiplist_next(&slist, cursor);\n\n            // Detach `entry` from skiplist.\n            skiplist_erase_node(&slist, &entry->snode);\n            // Release `entry`, to free its memory.\n            skiplist_release_node(&entry->snode);\n            // Free `entry` after it becomes safe.\n            skiplist_wait_for_free(&entry->snode);\n            skiplist_free_node(&entry->snode);\n            free(entry);\n        }\n    }\n\n    // Free skiplist.\n    skiplist_free(&slist);\n\n    return 0;\n}\n\n"
  },
  {
    "path": "include/skiplist.h",
    "content": "/**\n * Copyright (C) 2017-present Jung-Sang Ahn <jungsang.ahn@gmail.com>\n * All rights reserved.\n *\n * https://github.com/greensky00\n *\n * Skiplist\n * Version: 0.2.9\n *\n * Permission is hereby granted, free of charge, to any person\n * obtaining a copy of this software and associated documentation\n * files (the \"Software\"), to deal in the Software without\n * restriction, including without limitation the rights to use,\n * copy, modify, merge, publish, distribute, sublicense, and/or sell\n * copies of the Software, and to permit persons to whom the\n * Software is furnished to do so, subject to the following\n * conditions:\n *\n * The above copyright notice and this permission notice shall be\n * included in all copies or substantial portions of the Software.\n *\n * THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES\n * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT\n * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,\n * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\n * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR\n * OTHER DEALINGS IN THE SOFTWARE.\n */\n\n#ifndef _JSAHN_SKIPLIST_H\n#define _JSAHN_SKIPLIST_H (1)\n\n#include <stddef.h>\n#include <stdint.h>\n\n#define SKIPLIST_MAX_LAYER (64)\n\nstruct _skiplist_node;\n\n//#define _STL_ATOMIC (1)\n#ifdef __APPLE__\n    #define _STL_ATOMIC (1)\n#endif\n#if defined(_STL_ATOMIC) && defined(__cplusplus)\n    #include <atomic>\n    typedef std::atomic<_skiplist_node*>   atm_node_ptr;\n    typedef std::atomic<bool>              atm_bool;\n    typedef std::atomic<uint8_t>           atm_uint8_t;\n    typedef std::atomic<uint16_t>          atm_uint16_t;\n    typedef std::atomic<uint32_t>          atm_uint32_t;\n#else\n    typedef struct _skiplist_node*         atm_node_ptr;\n    typedef uint8_t                        atm_bool;\n    typedef uint8_t                        atm_uint8_t;\n    typedef uint16_t                       atm_uint16_t;\n    typedef uint32_t                       atm_uint32_t;\n#endif\n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\ntypedef struct _skiplist_node {\n    atm_node_ptr *next;\n    atm_bool is_fully_linked;\n    atm_bool being_modified;\n    atm_bool removed;\n    uint8_t top_layer; // 0: bottom\n    atm_uint16_t ref_count;\n    atm_uint32_t accessing_next;\n} skiplist_node;\n\n// *a  < *b : return neg\n// *a == *b : return 0\n// *a  > *b : return pos\ntypedef int skiplist_cmp_t(skiplist_node *a, skiplist_node *b, void *aux);\n\ntypedef struct {\n    size_t fanout;\n    size_t maxLayer;\n    void *aux;\n} skiplist_raw_config;\n\ntypedef struct {\n    skiplist_node head;\n    skiplist_node tail;\n    skiplist_cmp_t *cmp_func;\n    void *aux;\n    atm_uint32_t num_entries;\n    atm_uint32_t* layer_entries;\n    atm_uint8_t top_layer;\n    uint8_t fanout;\n    uint8_t max_layer;\n} skiplist_raw;\n\n#ifndef _get_entry\n#define _get_entry(ELEM, STRUCT, MEMBER)                              \\\n        ((STRUCT *) ((uint8_t *) (ELEM) - offsetof (STRUCT, MEMBER)))\n#endif\n\nvoid skiplist_init(skiplist_raw* slist,\n                   skiplist_cmp_t* cmp_func);\nvoid skiplist_free(skiplist_raw* slist);\n\nvoid skiplist_init_node(skiplist_node* node);\nvoid skiplist_free_node(skiplist_node* node);\n\nsize_t skiplist_get_size(skiplist_raw* slist);\n\nskiplist_raw_config skiplist_get_default_config();\nskiplist_raw_config skiplist_get_config(skiplist_raw* slist);\n\nvoid skiplist_set_config(skiplist_raw* slist,\n                         skiplist_raw_config config);\n\nint skiplist_insert(skiplist_raw* slist,\n                    skiplist_node* node);\nint skiplist_insert_nodup(skiplist_raw *slist,\n                          skiplist_node *node);\n\nskiplist_node* skiplist_find(skiplist_raw* slist,\n                             skiplist_node* query);\nskiplist_node* skiplist_find_smaller_or_equal(skiplist_raw* slist,\n                                              skiplist_node* query);\nskiplist_node* skiplist_find_greater_or_equal(skiplist_raw* slist,\n                                              skiplist_node* query);\n\nint skiplist_erase_node_passive(skiplist_raw* slist,\n                                skiplist_node* node);\nint skiplist_erase_node(skiplist_raw *slist,\n                        skiplist_node *node);\nint skiplist_erase(skiplist_raw* slist,\n                   skiplist_node* query);\n\nint skiplist_is_valid_node(skiplist_node* node);\nint skiplist_is_safe_to_free(skiplist_node* node);\nvoid skiplist_wait_for_free(skiplist_node* node);\n\nvoid skiplist_grab_node(skiplist_node* node);\nvoid skiplist_release_node(skiplist_node* node);\n\nskiplist_node* skiplist_next(skiplist_raw* slist,\n                             skiplist_node* node);\nskiplist_node* skiplist_prev(skiplist_raw* slist,\n                             skiplist_node* node);\nskiplist_node* skiplist_begin(skiplist_raw* slist);\nskiplist_node* skiplist_end(skiplist_raw* slist);\n\n#ifdef __cplusplus\n}\n#endif\n\n#endif  // _JSAHN_SKIPLIST_H\n"
  },
  {
    "path": "include/sl_map.h",
    "content": "/**\n * Copyright (C) 2017-present Jung-Sang Ahn <jungsang.ahn@gmail.com>\n * All rights reserved.\n *\n * https://github.com/greensky00\n *\n * Skiplist map container\n * Version: 0.2.0\n *\n * Permission is hereby granted, free of charge, to any person\n * obtaining a copy of this software and associated documentation\n * files (the \"Software\"), to deal in the Software without\n * restriction, including without limitation the rights to use,\n * copy, modify, merge, publish, distribute, sublicense, and/or sell\n * copies of the Software, and to permit persons to whom the\n * Software is furnished to do so, subject to the following\n * conditions:\n *\n * The above copyright notice and this permission notice shall be\n * included in all copies or substantial portions of the Software.\n *\n * THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES\n * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT\n * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,\n * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\n * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR\n * OTHER DEALINGS IN THE SOFTWARE.\n */\n\n#pragma once\n\n#include \"skiplist.h\"\n\n#include <atomic>\n#include <cstdlib>\n#include <mutex>\n#include <thread>\n#include <utility>\n#include <vector>\n\ntemplate<typename K, typename V>\nstruct map_node {\n    map_node() {\n        skiplist_init_node(&snode);\n    }\n    ~map_node() {\n        skiplist_free_node(&snode);\n    }\n    static int cmp(skiplist_node* a, skiplist_node* b, void* aux) {\n        map_node *aa, *bb;\n        aa = _get_entry(a, map_node, snode);\n        bb = _get_entry(b, map_node, snode);\n        if (aa->kv.first < bb->kv.first) return -1;\n        if (aa->kv.first > bb->kv.first) return 1;\n        return 0;\n    }\n\n    skiplist_node snode;\n    std::pair<K, V> kv;\n};\n\ntemplate<typename K, typename V> class sl_map;\ntemplate<typename K, typename V> class sl_map_gc;\n\ntemplate<typename K, typename V>\nclass map_iterator {\n    friend class sl_map<K, V>;\n    friend class sl_map_gc<K, V>;\n\nprivate:\n    using T = std::pair<K, V>;\n    using Node = map_node<K, V>;\n\npublic:\n    map_iterator() : slist(nullptr), cursor(nullptr) {}\n\n    map_iterator(map_iterator&& src)\n        : slist(src.slist), cursor(src.cursor)\n    {\n        // Mimic perfect forwarding.\n        src.slist = nullptr;\n        src.cursor = nullptr;\n    }\n\n    ~map_iterator() {\n        if (cursor) skiplist_release_node(cursor);\n    }\n\n    void operator=(const map_iterator& src) {\n        // This reference counting is similar to that of shared_ptr.\n        skiplist_node* tmp = cursor;\n        if (src.cursor)\n            skiplist_grab_node(src.cursor);\n        cursor = src.cursor;\n        if (tmp)\n            skiplist_release_node(tmp);\n    }\n\n    bool operator==(const map_iterator& src) const { return (cursor == src.cursor); }\n    bool operator!=(const map_iterator& src) const { return !operator==(src); }\n\n    T* operator->() const {\n        Node* node = _get_entry(cursor, Node, snode);\n        return &node->kv;\n    }\n    T& operator*() const {\n        Node* node = _get_entry(cursor, Node, snode);\n        return node->kv;\n    }\n\n    // ++A\n    map_iterator& operator++() {\n        if (!slist || !cursor) {\n            cursor = nullptr;\n            return *this;\n        }\n        skiplist_node* next = skiplist_next(slist, cursor);\n        skiplist_release_node(cursor);\n        cursor = next;\n        return *this;\n    }\n    // A++\n    map_iterator& operator++(int) { return operator++(); }\n    // --A\n    map_iterator& operator--() {\n        if (!slist || !cursor) {\n            cursor = nullptr;\n            return *this;\n        }\n        skiplist_node* prev = skiplist_prev(slist, cursor);\n        skiplist_release_node(cursor);\n        cursor = prev;\n        return *this;\n    }\n    // A--\n    map_iterator operator--(int) { return operator--(); }\n\nprivate:\n    map_iterator(skiplist_raw* _slist,\n                 skiplist_node* _cursor)\n        : slist(_slist), cursor(_cursor) {}\n\n    skiplist_raw* slist;\n    skiplist_node* cursor;\n};\n\n\n\ntemplate<typename K, typename V>\nclass sl_map {\nprivate:\n    using T = std::pair<K, V>;\n    using Node = map_node<K, V>;\n\npublic:\n    using iterator = map_iterator<K, V>;\n    using reverse_iterator = map_iterator<K, V>;\n\n    sl_map() {\n        skiplist_init(&slist, Node::cmp);\n    }\n\n    virtual\n    ~sl_map() {\n        skiplist_node* cursor = skiplist_begin(&slist);\n        while (cursor) {\n            Node* node = _get_entry(cursor, Node, snode);\n            cursor = skiplist_next(&slist, cursor);\n            // Don't need to care about release.\n            delete node;\n        }\n        skiplist_free(&slist);\n    }\n\n    bool empty() {\n        skiplist_node* cursor = skiplist_begin(&slist);\n        if (cursor) {\n            skiplist_release_node(cursor);\n            return false;\n        }\n        return true;\n    }\n\n    size_t size() { return skiplist_get_size(&slist); }\n\n    std::pair<iterator, bool> insert(const std::pair<K, V>& kv) {\n        do {\n            Node* node = new Node();\n            node->kv = kv;\n\n            int rc = skiplist_insert_nodup(&slist, &node->snode);\n            if (rc == 0) {\n                skiplist_grab_node(&node->snode);\n                return std::pair<iterator, bool>\n                       ( iterator(&slist, &node->snode), true );\n            }\n            delete node;\n\n            Node query;\n            query.kv.first = kv.first;\n            skiplist_node* cursor = skiplist_find(&slist, &query.snode);\n            if (cursor) {\n                return std::pair<iterator, bool>\n                       ( iterator(&slist, cursor), false );\n            }\n        } while (true);\n\n        // NOTE: Should not reach here.\n        return std::pair<iterator, bool>(iterator(), false);\n    }\n\n    iterator find(K key) {\n        Node query;\n        query.kv.first = key;\n        skiplist_node* cursor = skiplist_find(&slist, &query.snode);\n        return iterator(&slist, cursor);\n    }\n\n    virtual\n    iterator erase(iterator& position) {\n        skiplist_node* cursor = position.cursor;\n        skiplist_node* next = skiplist_next(&slist, cursor);\n\n        skiplist_erase_node(&slist, cursor);\n        skiplist_release_node(cursor);\n        skiplist_wait_for_free(cursor);\n        Node* node = _get_entry(cursor, Node, snode);\n        delete node;\n\n        position.cursor = nullptr;\n        return iterator(&slist, next);\n    }\n\n    virtual\n    size_t erase(const K& key) {\n        size_t count = 0;\n        Node query;\n        query.kv.first = key;\n        skiplist_node* cursor = skiplist_find(&slist, &query.snode);\n        while (cursor) {\n            Node* node = _get_entry(cursor, Node, snode);\n            if (node->kv.first != key) break;\n\n            cursor = skiplist_next(&slist, cursor);\n\n            skiplist_erase_node(&slist, &node->snode);\n            skiplist_release_node(&node->snode);\n            skiplist_wait_for_free(&node->snode);\n            delete node;\n        }\n        if (cursor) skiplist_release_node(cursor);\n        return count;\n    }\n\n    iterator begin() {\n        skiplist_node* cursor = skiplist_begin(&slist);\n        return iterator(&slist, cursor);\n    }\n    iterator end() { return iterator(); }\n\n    reverse_iterator rbegin() {\n        skiplist_node* cursor = skiplist_end(&slist);\n        return reverse_iterator(&slist, cursor);\n    }\n    reverse_iterator rend() { return reverse_iterator(); }\n\nprotected:\n    skiplist_raw slist;\n};\n\n\ntemplate<typename K, typename V>\nclass sl_map_gc : public sl_map<K, V> {\nprivate:\n    using T = std::pair<K, V>;\n    using Node = map_node<K, V>;\n\npublic:\n    using iterator = map_iterator<K, V>;\n    using reverse_iterator = map_iterator<K, V>;\n\n    sl_map_gc()\n        : sl_map<K, V>()\n        , gcVector( std::max( (size_t)4,\n                              (size_t)std::thread::hardware_concurrency() ) )\n    {\n        for (auto& entry: gcVector) {\n            entry = new std::atomic<Node*>(nullptr);\n        }\n    }\n\n    ~sl_map_gc() {\n        execGc();\n        for (std::atomic<Node*>*& a_node: gcVector) {\n            Node* node = a_node->load();\n            delete node;\n            delete a_node;\n        }\n    }\n\n    iterator erase(iterator& position) {\n        skiplist_node* cursor = position.cursor;\n        skiplist_node* next = skiplist_next(&this->slist, cursor);\n\n        skiplist_erase_node(&this->slist, cursor);\n\n        Node* node = _get_entry(cursor, Node, snode);\n        gcPush(node);\n        skiplist_release_node(cursor);\n        execGc();\n\n        position.cursor = nullptr;\n        return iterator(&this->slist, next);\n    }\n\n    size_t erase(const K& key) {\n        size_t count = 0;\n        Node query;\n        query.kv.first = key;\n        skiplist_node* cursor = skiplist_find(&this->slist, &query.snode);\n        while (cursor) {\n            Node* node = _get_entry(cursor, Node, snode);\n            if (node->kv.first != key) break;\n\n            cursor = skiplist_next(&this->slist, cursor);\n\n            skiplist_erase_node(&this->slist, &node->snode);\n            gcPush(node);\n            skiplist_release_node(&node->snode);\n        }\n        if (cursor) skiplist_release_node(cursor);\n\n        execGc();\n        return count;\n    }\n\nprivate:\n    void gcPush(Node* node) {\n        size_t v_len = gcVector.size();\n\n        do {\n            size_t rr = std::rand() % v_len;\n            for (size_t ii = rr; ii < rr + v_len; ++ii) {\n                std::atomic<Node*>& a_node = *gcVector[ii % v_len];\n\n                Node* exp = nullptr;\n                if ( a_node.compare_exchange_strong\n                            ( exp, node, std::memory_order_relaxed ) ) {\n                    return;\n                }\n            }\n\n            std::this_thread::yield();\n            execGc();\n        } while (true);\n    }\n\n    void execGc() {\n        std::unique_lock<std::mutex> l(gcVectorLock, std::try_to_lock);\n        if (!l.owns_lock()) return;\n\n        size_t v_len = gcVector.size();\n        for (size_t ii = 0; ii < v_len; ++ii) {\n            std::atomic<Node*>& a_node = *gcVector[ii];\n            Node* node = a_node.load();\n            if (!node) continue;\n\n            if (skiplist_is_safe_to_free(&node->snode)) {\n                Node* exp = node;\n                Node* val = nullptr;\n                a_node.compare_exchange_strong\n                       ( exp, val, std::memory_order_relaxed );\n\n                delete node;\n            }\n        }\n    }\n\n    std::mutex gcVectorLock;\n    std::vector< std::atomic<Node*>* > gcVector;\n};\n\n\n"
  },
  {
    "path": "include/sl_set.h",
    "content": "/**\n * Copyright (C) 2017-present Jung-Sang Ahn <jungsang.ahn@gmail.com>\n * All rights reserved.\n *\n * https://github.com/greensky00\n *\n * Skiplist set container\n * Version: 0.2.0\n *\n * Permission is hereby granted, free of charge, to any person\n * obtaining a copy of this software and associated documentation\n * files (the \"Software\"), to deal in the Software without\n * restriction, including without limitation the rights to use,\n * copy, modify, merge, publish, distribute, sublicense, and/or sell\n * copies of the Software, and to permit persons to whom the\n * Software is furnished to do so, subject to the following\n * conditions:\n *\n * The above copyright notice and this permission notice shall be\n * included in all copies or substantial portions of the Software.\n *\n * THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES\n * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT\n * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,\n * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\n * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR\n * OTHER DEALINGS IN THE SOFTWARE.\n */\n\n#pragma once\n\n#include \"skiplist.h\"\n\n#include <atomic>\n#include <cstdlib>\n#include <mutex>\n#include <thread>\n#include <utility>\n#include <vector>\n\ntemplate<typename K>\nstruct set_node {\n    set_node() {\n        skiplist_init_node(&snode);\n    }\n    ~set_node() {\n        skiplist_free_node(&snode);\n    }\n    static int cmp(skiplist_node* a, skiplist_node* b, void* aux) {\n        set_node *aa, *bb;\n        aa = _get_entry(a, set_node, snode);\n        bb = _get_entry(b, set_node, snode);\n        if (aa->key < bb->key) return -1;\n        if (aa->key > bb->key) return 1;\n        return 0;\n    }\n\n    skiplist_node snode;\n    K key;\n};\n\ntemplate<typename K> class sl_set;\ntemplate<typename K> class sl_set_gc;\n\ntemplate<typename K>\nclass set_iterator {\n    friend class sl_set<K>;\n    friend class sl_set_gc<K>;\npublic:\n    using Node = set_node<K>;\n\n    set_iterator() : slist(nullptr), cursor(nullptr) {}\n\n    set_iterator(set_iterator&& src)\n        : slist(src.slist), cursor(src.cursor)\n    {\n        // Mimic perfect forwarding.\n        src.slist = nullptr;\n        src.cursor = nullptr;\n    }\n\n    ~set_iterator() {\n        if (cursor) skiplist_release_node(cursor);\n    }\n\n    void operator=(const set_iterator& src) {\n        // This reference counting is similar to that of shared_ptr.\n        skiplist_node* tmp = cursor;\n        if (src.cursor)\n            skiplist_grab_node(src.cursor);\n        cursor = src.cursor;\n        if (tmp)\n            skiplist_release_node(tmp);\n    }\n\n    bool operator==(const set_iterator& src) const {\n        return (cursor == src.cursor);\n    }\n    bool operator!=(const set_iterator& src) const {\n        return (cursor != src.cursor);\n    }\n\n    K& operator*() const {\n        Node* node = _get_entry(cursor, Node, snode);\n        return node->key;\n    }\n\n    // ++A\n    set_iterator& operator++() {\n        if (!slist || !cursor) {\n            cursor = nullptr;\n            return *this;\n        }\n        skiplist_node* next = skiplist_next(slist, cursor);\n        skiplist_release_node(cursor);\n        cursor = next;\n        return *this;\n    }\n\n    // A++\n    set_iterator& operator++(int) { return operator++(); }\n\n    // --A\n    set_iterator& operator--() {\n        if (!slist || !cursor) {\n            cursor = nullptr;\n            return *this;\n        }\n        skiplist_node* prev = skiplist_prev(slist, cursor);\n        skiplist_release_node(cursor);\n        cursor = prev;\n        return *this;\n    }\n\n    // A--\n    set_iterator operator--(int) { return operator--(); }\n\nprivate:\n    set_iterator(skiplist_raw* _slist,\n                 skiplist_node* _cursor)\n        : slist(_slist), cursor(_cursor) {}\n\n    skiplist_raw* slist;\n    skiplist_node* cursor;\n};\n\n\ntemplate<typename K>\nclass sl_set {\nprivate:\n    using Node = set_node<K>;\n\npublic:\n    using iterator = set_iterator<K>;\n    using reverse_iterator = set_iterator<K>;\n\n    sl_set() {\n        skiplist_init(&slist, Node::cmp);\n    }\n\n    virtual\n    ~sl_set() {\n        skiplist_node* cursor = skiplist_begin(&slist);\n        while (cursor) {\n            Node* node = _get_entry(cursor, Node, snode);\n            cursor = skiplist_next(&slist, cursor);\n            // Don't need to care about release.\n            delete node;\n        }\n        skiplist_free(&slist);\n    }\n\n    bool empty() {\n        skiplist_node* cursor = skiplist_begin(&slist);\n        if (cursor) {\n            skiplist_release_node(cursor);\n            return false;\n        }\n        return true;\n    }\n\n    size_t size() { return skiplist_get_size(&slist); }\n\n    std::pair<iterator, bool> insert(const K& key) {\n        do {\n            Node* node = new Node();\n            node->key = key;\n\n            int rc = skiplist_insert_nodup(&slist, &node->snode);\n            if (rc == 0) {\n                skiplist_grab_node(&node->snode);\n                return std::pair<iterator, bool>\n                       ( iterator(&slist, &node->snode), true );\n            }\n            delete node;\n\n            Node query;\n            query.key = key;\n            skiplist_node* cursor = skiplist_find(&slist, &query.snode);\n            if (cursor) {\n                return std::pair<iterator, bool>\n                       ( iterator(&slist, cursor), false );\n            }\n        } while (true);\n\n        // NOTE: Should not reach here.\n        return std::pair<iterator, bool>(iterator(), false);\n    }\n\n    iterator find(const K& key) {\n        Node query;\n        query.key = key;\n        skiplist_node* cursor = skiplist_find(&slist, &query.snode);\n        return iterator(&slist, cursor);\n    }\n\n    virtual\n    iterator erase(iterator& position) {\n        skiplist_node* cursor = position.cursor;\n        skiplist_node* next = skiplist_next(&slist, cursor);\n\n        skiplist_erase_node(&slist, cursor);\n        skiplist_release_node(cursor);\n        skiplist_wait_for_free(cursor);\n        Node* node = _get_entry(cursor, Node, snode);\n        delete node;\n\n        position.cursor = nullptr;\n        return iterator(&slist, next);\n    }\n\n    virtual\n    size_t erase(const K& key) {\n        size_t count = 0;\n        Node query;\n        query.key = key;\n        skiplist_node* cursor = skiplist_find(&slist, &query.snode);\n        while (cursor) {\n            Node* node = _get_entry(cursor, Node, snode);\n            if (node->key != key) break;\n\n            cursor = skiplist_next(&slist, cursor);\n\n            skiplist_erase_node(&slist, &node->snode);\n            skiplist_release_node(&node->snode);\n            skiplist_wait_for_free(&node->snode);\n            delete node;\n        }\n        if (cursor) skiplist_release_node(cursor);\n        return count;\n    }\n\n    iterator begin() {\n        skiplist_node* cursor = skiplist_begin(&slist);\n        return iterator(&slist, cursor);\n    }\n    iterator end() { return iterator(); }\n\n    reverse_iterator rbegin() {\n        skiplist_node* cursor = skiplist_end(&slist);\n        return reverse_iterator(&slist, cursor);\n    }\n    reverse_iterator rend() { return reverse_iterator(); }\n\nprotected:\n    skiplist_raw slist;\n};\n\ntemplate<typename K>\nclass sl_set_gc : public sl_set<K> {\nprivate:\n    using Node = set_node<K>;\n\npublic:\n    using iterator = set_iterator<K>;\n    using reverse_iterator = set_iterator<K>;\n\n    sl_set_gc()\n        : sl_set<K>()\n        , gcVector( std::max( (size_t)16,\n                              (size_t)std::thread::hardware_concurrency() ) )\n    {\n        for (auto& entry: gcVector) {\n            entry = new std::atomic<Node*>(nullptr);\n        }\n    }\n\n    ~sl_set_gc() {\n        execGc();\n        for (std::atomic<Node*>*& a_node: gcVector) {\n            Node* node = a_node->load();\n            delete node;\n            delete a_node;\n        }\n    }\n\n    iterator erase(iterator& position) {\n        skiplist_node* cursor = position.cursor;\n        skiplist_node* next = skiplist_next(&this->slist, cursor);\n\n        skiplist_erase_node(&this->slist, cursor);\n\n        Node* node = _get_entry(cursor, Node, snode);\n        gcPush(node);\n        skiplist_release_node(cursor);\n        execGc();\n\n        position.cursor = nullptr;\n        return iterator(&this->slist, next);\n    }\n\n    size_t erase(const K& key) {\n        size_t count = 0;\n        Node query;\n        query.key = key;\n        skiplist_node* cursor = skiplist_find(&this->slist, &query.snode);\n        while (cursor) {\n            Node* node = _get_entry(cursor, Node, snode);\n            if (node->key != key) break;\n\n            cursor = skiplist_next(&this->slist, cursor);\n\n            skiplist_erase_node(&this->slist, &node->snode);\n            gcPush(node);\n            skiplist_release_node(&node->snode);\n        }\n        if (cursor) skiplist_release_node(cursor);\n\n        execGc();\n        return count;\n    }\n\nprivate:\n    void gcPush(Node* node) {\n        size_t v_len = gcVector.size();\n\n        do {\n            size_t rr = std::rand() % v_len;\n            for (size_t ii = rr; ii < rr + v_len; ++ii) {\n                std::atomic<Node*>& a_node = *gcVector[ii % v_len];\n\n                Node* exp = nullptr;\n                if ( a_node.compare_exchange_strong\n                            ( exp, node, std::memory_order_relaxed ) ) {\n                    return;\n                }\n            }\n\n            std::this_thread::yield();\n            execGc();\n        } while (true);\n    }\n\n    void execGc() {\n        std::unique_lock<std::mutex> l(gcVectorLock, std::try_to_lock);\n        if (!l.owns_lock()) return;\n\n        size_t v_len = gcVector.size();\n        for (size_t ii = 0; ii < v_len; ++ii) {\n            std::atomic<Node*>& a_node = *gcVector[ii];\n            Node* node = a_node.load();\n            if (!node) continue;\n\n            if (skiplist_is_safe_to_free(&node->snode)) {\n                Node* exp = node;\n                Node* val = nullptr;\n                a_node.compare_exchange_strong\n                       ( exp, val, std::memory_order_relaxed );\n\n                delete node;\n            }\n        }\n    }\n\n    std::mutex gcVectorLock;\n    std::vector< std::atomic<Node*>* > gcVector;\n};\n\n"
  },
  {
    "path": "src/skiplist.cc",
    "content": "/**\n * Copyright (C) 2017-present Jung-Sang Ahn <jungsang.ahn@gmail.com>\n * All rights reserved.\n *\n * https://github.com/greensky00\n *\n * Skiplist\n * Version: 0.2.9\n *\n * Permission is hereby granted, free of charge, to any person\n * obtaining a copy of this software and associated documentation\n * files (the \"Software\"), to deal in the Software without\n * restriction, including without limitation the rights to use,\n * copy, modify, merge, publish, distribute, sublicense, and/or sell\n * copies of the Software, and to permit persons to whom the\n * Software is furnished to do so, subject to the following\n * conditions:\n *\n * The above copyright notice and this permission notice shall be\n * included in all copies or substantial portions of the Software.\n *\n * THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES\n * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT\n * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,\n * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\n * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR\n * OTHER DEALINGS IN THE SOFTWARE.\n */\n\n#include \"skiplist.h\"\n\n#include <stdlib.h>\n\n#define __SLD_RT_INS(e, n, t, c)\n#define __SLD_NC_INS(n, nn, t, c)\n#define __SLD_RT_RMV(e, n, t, c)\n#define __SLD_NC_RMV(n, nn, t, c)\n#define __SLD_BM(n)\n#define __SLD_ASSERT(cond)\n#define __SLD_P(args...)\n#define __SLD_(b)\n\n//#define __SL_DEBUG (1)\n#ifdef __SL_DEBUG\n    #ifndef __cplusplus\n        #error \"Debug mode is available with C++ compiler only.\"\n    #endif\n    #include \"skiplist_debug.h\"\n#endif\n\n#define __SL_YIELD (1)\n#ifdef __SL_YIELD\n    #ifdef __cplusplus\n        #include <thread>\n        #define YIELD() std::this_thread::yield()\n    #else\n        #include <sched.h>\n        #define YIELD() sched_yield()\n    #endif\n#else\n    #define YIELD()\n#endif\n\n#if defined(_STL_ATOMIC) && defined(__cplusplus)\n    // C++ (STL) atomic operations\n    #define MOR                         std::memory_order_relaxed\n    #define ATM_GET(var)                (var).load(MOR)\n    #define ATM_LOAD(var, val)          (val) = (var).load(MOR)\n    #define ATM_STORE(var, val)         (var).store((val), MOR)\n    #define ATM_CAS(var, exp, val)      (var).compare_exchange_weak((exp), (val))\n    #define ATM_FETCH_ADD(var, val)     (var).fetch_add(val, MOR)\n    #define ATM_FETCH_SUB(var, val)     (var).fetch_sub(val, MOR)\n    #define ALLOC_(type, var, count)    (var) = new type[count]\n    #define FREE_(var)                  delete[] (var)\n#else\n    // C-style atomic operations\n    #ifndef __cplusplus\n        typedef uint8_t bool;\n        #ifndef true\n            #define true 1\n        #endif\n        #ifndef false\n            #define false 0\n        #endif\n    #endif\n\n    #ifndef __cplusplus\n        #define thread_local /*_Thread_local*/\n    #endif\n\n    #define MOR                         __ATOMIC_RELAXED\n    #define ATM_GET(var)                (var)\n    #define ATM_LOAD(var, val)          __atomic_load(&(var), &(val), MOR)\n    #define ATM_STORE(var, val)         __atomic_store(&(var), &(val), MOR)\n    #define ATM_CAS(var, exp, val)      \\\n            __atomic_compare_exchange(&(var), &(exp), &(val), 1, MOR, MOR)\n    #define ATM_FETCH_ADD(var, val)     __atomic_fetch_add(&(var), (val), MOR)\n    #define ATM_FETCH_SUB(var, val)     __atomic_fetch_sub(&(var), (val), MOR)\n    #define ALLOC_(type, var, count)    \\\n            (var) = (type*)calloc(count, sizeof(type))\n    #define FREE_(var)                  free(var)\n#endif\n\nstatic inline void _sl_node_init(skiplist_node *node,\n                                 size_t top_layer)\n{\n    if (top_layer > UINT8_MAX) top_layer = UINT8_MAX;\n\n    __SLD_ASSERT(node->is_fully_linked == false);\n    __SLD_ASSERT(node->being_modified == false);\n\n    bool bool_val = false;\n    ATM_STORE(node->is_fully_linked, bool_val);\n    ATM_STORE(node->being_modified, bool_val);\n    ATM_STORE(node->removed, bool_val);\n\n    if (node->top_layer != top_layer ||\n        node->next == NULL) {\n\n        node->top_layer = top_layer;\n\n        if (node->next) FREE_(node->next);\n        ALLOC_(atm_node_ptr, node->next, top_layer+1);\n    }\n}\n\nvoid skiplist_init(skiplist_raw *slist,\n                   skiplist_cmp_t *cmp_func) {\n\n    slist->cmp_func = NULL;\n    slist->aux = NULL;\n\n    // fanout 4 + layer 12: 4^12 ~= upto 17M items under O(lg n) complexity.\n    // for +17M items, complexity will grow linearly: O(k lg n).\n    slist->fanout = 4;\n    slist->max_layer = 12;\n    slist->num_entries = 0;\n\n    ALLOC_(atm_uint32_t, slist->layer_entries, slist->max_layer);\n    slist->top_layer = 0;\n\n    skiplist_init_node(&slist->head);\n    skiplist_init_node(&slist->tail);\n\n    _sl_node_init(&slist->head, slist->max_layer);\n    _sl_node_init(&slist->tail, slist->max_layer);\n\n    size_t layer;\n    for (layer = 0; layer < slist->max_layer; ++layer) {\n        slist->head.next[layer] = &slist->tail;\n        slist->tail.next[layer] = NULL;\n    }\n\n    bool bool_val = true;\n    ATM_STORE(slist->head.is_fully_linked, bool_val);\n    ATM_STORE(slist->tail.is_fully_linked, bool_val);\n    slist->cmp_func = cmp_func;\n}\n\nvoid skiplist_free(skiplist_raw *slist)\n{\n    skiplist_free_node(&slist->head);\n    skiplist_free_node(&slist->tail);\n\n    FREE_(slist->layer_entries);\n    slist->layer_entries = NULL;\n\n    slist->aux = NULL;\n    slist->cmp_func = NULL;\n}\n\nvoid skiplist_init_node(skiplist_node *node)\n{\n    node->next = NULL;\n\n    bool bool_false = false;\n    ATM_STORE(node->is_fully_linked, bool_false);\n    ATM_STORE(node->being_modified, bool_false);\n    ATM_STORE(node->removed, bool_false);\n\n    node->accessing_next = 0;\n    node->top_layer = 0;\n    node->ref_count = 0;\n}\n\nvoid skiplist_free_node(skiplist_node *node)\n{\n    FREE_(node->next);\n    node->next = NULL;\n}\n\nsize_t skiplist_get_size(skiplist_raw* slist) {\n    uint32_t val;\n    ATM_LOAD(slist->num_entries, val);\n    return val;\n}\n\nskiplist_raw_config skiplist_get_default_config()\n{\n    skiplist_raw_config ret;\n    ret.fanout = 4;\n    ret.maxLayer = 12;\n    ret.aux = NULL;\n    return ret;\n}\n\nskiplist_raw_config skiplist_get_config(skiplist_raw *slist)\n{\n    skiplist_raw_config ret;\n    ret.fanout = slist->fanout;\n    ret.maxLayer = slist->max_layer;\n    ret.aux = slist->aux;\n    return ret;\n}\n\nvoid skiplist_set_config(skiplist_raw *slist,\n                         skiplist_raw_config config)\n{\n    slist->fanout = config.fanout;\n\n    slist->max_layer = config.maxLayer;\n    if (slist->layer_entries) FREE_(slist->layer_entries);\n    ALLOC_(atm_uint32_t, slist->layer_entries, slist->max_layer);\n\n    slist->aux = config.aux;\n}\n\nstatic inline int _sl_cmp(skiplist_raw *slist,\n                          skiplist_node *a,\n                          skiplist_node *b)\n{\n    if (a == b) return 0;\n    if (a == &slist->head || b == &slist->tail) return -1;\n    if (a == &slist->tail || b == &slist->head) return 1;\n    return slist->cmp_func(a, b, slist->aux);\n}\n\nstatic inline bool _sl_valid_node(skiplist_node *node) {\n    bool is_fully_linked = false;\n    ATM_LOAD(node->is_fully_linked, is_fully_linked);\n    return is_fully_linked;\n}\n\nstatic inline void _sl_read_lock_an(skiplist_node* node) {\n    for(;;) {\n        // Wait for active writer to release the lock\n        uint32_t accessing_next = 0;\n        ATM_LOAD(node->accessing_next, accessing_next);\n        while (accessing_next & 0xfff00000) {\n            YIELD();\n            ATM_LOAD(node->accessing_next, accessing_next);\n        }\n\n        ATM_FETCH_ADD(node->accessing_next, 0x1);\n        ATM_LOAD(node->accessing_next, accessing_next);\n        if ((accessing_next & 0xfff00000) == 0) {\n            return;\n        }\n\n        ATM_FETCH_SUB(node->accessing_next, 0x1);\n    }\n}\n\nstatic inline void _sl_read_unlock_an(skiplist_node* node) {\n    ATM_FETCH_SUB(node->accessing_next, 0x1);\n}\n\nstatic inline void _sl_write_lock_an(skiplist_node* node) {\n    for(;;) {\n        // Wait for active writer to release the lock\n        uint32_t accessing_next = 0;\n        ATM_LOAD(node->accessing_next, accessing_next);\n        while (accessing_next & 0xfff00000) {\n            YIELD();\n            ATM_LOAD(node->accessing_next, accessing_next);\n        }\n\n        ATM_FETCH_ADD(node->accessing_next, 0x100000);\n        ATM_LOAD(node->accessing_next, accessing_next);\n        if((accessing_next & 0xfff00000) == 0x100000) {\n            // Wait until there's no more readers\n            while (accessing_next & 0x000fffff) {\n                YIELD();\n                ATM_LOAD(node->accessing_next, accessing_next);\n            }\n            return;\n        }\n\n        ATM_FETCH_SUB(node->accessing_next, 0x100000);\n    }\n}\n\nstatic inline void _sl_write_unlock_an(skiplist_node* node) {\n    ATM_FETCH_SUB(node->accessing_next, 0x100000);\n}\n\n// Note: it increases the `ref_count` of returned node.\n//       Caller is responsible to decrease it.\nstatic inline skiplist_node* _sl_next(skiplist_raw* slist,\n                                      skiplist_node* cur_node,\n                                      int layer,\n                                      skiplist_node* node_to_find,\n                                      bool* found)\n{\n    skiplist_node *next_node = NULL;\n\n    // Turn on `accessing_next`:\n    // now `cur_node` is not removable from skiplist,\n    // which means that `cur_node->next` will be consistent\n    // until clearing `accessing_next`.\n    _sl_read_lock_an(cur_node); {\n        if (!_sl_valid_node(cur_node)) {\n            _sl_read_unlock_an(cur_node);\n            return NULL;\n        }\n        ATM_LOAD(cur_node->next[layer], next_node);\n        // Increase ref count of `next_node`:\n        // now `next_node` is not destroyable.\n\n        //   << Remaining issue >>\n        // 1) initially: A -> B\n        // 2) T1: call _sl_next(A):\n        //        A.accessing_next := true;\n        //        next_node := B;\n        // ----- context switch happens here -----\n        // 3) T2: insert C:\n        //        A -> C -> B\n        // 4) T2: and then erase B, and free B.\n        //        A -> C    B(freed)\n        // ----- context switch back again -----\n        // 5) T1: try to do something with B,\n        //        but crash happens.\n        //\n        // ... maybe resolved using RW spinlock (Aug 21, 2017).\n        __SLD_ASSERT(next_node);\n        ATM_FETCH_ADD(next_node->ref_count, 1);\n        __SLD_ASSERT(next_node->top_layer >= layer);\n    } _sl_read_unlock_an(cur_node);\n\n    size_t num_nodes = 0;\n    skiplist_node* nodes[256];\n\n    while ( (next_node && !_sl_valid_node(next_node)) ||\n             next_node == node_to_find ) {\n        if (found && node_to_find == next_node) *found = true;\n\n        skiplist_node* temp = next_node;\n        _sl_read_lock_an(temp); {\n            __SLD_ASSERT(next_node);\n            if (!_sl_valid_node(temp)) {\n                _sl_read_unlock_an(temp);\n                ATM_FETCH_SUB(temp->ref_count, 1);\n                next_node = NULL;\n                break;\n            }\n            ATM_LOAD(temp->next[layer], next_node);\n            ATM_FETCH_ADD(next_node->ref_count, 1);\n            nodes[num_nodes++] = temp;\n            __SLD_ASSERT(next_node->top_layer >= layer);\n        } _sl_read_unlock_an(temp);\n    }\n\n    for (size_t ii=0; ii<num_nodes; ++ii) {\n        ATM_FETCH_SUB(nodes[ii]->ref_count, 1);\n    }\n\n    return next_node;\n}\n\nstatic inline size_t _sl_decide_top_layer(skiplist_raw *slist)\n{\n    size_t layer = 0;\n    while (layer+1 < slist->max_layer) {\n        // coin filp\n        if (rand() % slist->fanout == 0) {\n            // grow: 1/fanout probability\n            layer++;\n        } else {\n            // stop: 1 - 1/fanout probability\n            break;\n        }\n    }\n    return layer;\n}\n\nstatic inline void _sl_clr_flags(skiplist_node** node_arr,\n                                 int start_layer,\n                                 int top_layer)\n{\n    int layer;\n    for (layer = start_layer; layer <= top_layer; ++layer) {\n        if ( layer == top_layer ||\n             node_arr[layer] != node_arr[layer+1] ) {\n\n            bool exp = true;\n            bool bool_false = false;\n            if (!ATM_CAS(node_arr[layer]->being_modified, exp, bool_false)) {\n                __SLD_ASSERT(0);\n            }\n        }\n    }\n}\n\nstatic inline bool _sl_valid_prev_next(skiplist_node *prev,\n                                       skiplist_node *next) {\n    return _sl_valid_node(prev) && _sl_valid_node(next);\n}\n\nstatic inline int _skiplist_insert(skiplist_raw *slist,\n                                   skiplist_node *node,\n                                   bool no_dup)\n{\n    __SLD_(\n        thread_local std::thread::id tid = std::this_thread::get_id();\n        thread_local size_t tid_hash = std::hash<std::thread::id>{}(tid) % 256;\n        (void)tid_hash;\n    )\n\n    int top_layer = _sl_decide_top_layer(slist);\n    bool bool_true = true;\n\n    // init node before insertion\n    _sl_node_init(node, top_layer);\n    _sl_write_lock_an(node);\n\n    skiplist_node* prevs[SKIPLIST_MAX_LAYER];\n    skiplist_node* nexts[SKIPLIST_MAX_LAYER];\n\n    __SLD_P(\"%02x ins %p begin\\n\", (int)tid_hash, node);\n\ninsert_retry:\n    // in pure C, a label can only be part of a stmt.\n    (void)top_layer;\n\n    int cmp = 0, cur_layer = 0, layer;\n    skiplist_node *cur_node = &slist->head;\n    ATM_FETCH_ADD(cur_node->ref_count, 1);\n\n    __SLD_(size_t nh = 0);\n    __SLD_(thread_local skiplist_node* history[1024]; (void)history);\n\n    int sl_top_layer = slist->top_layer;\n    if (top_layer > sl_top_layer) sl_top_layer = top_layer;\n    for (cur_layer = sl_top_layer; cur_layer >= 0; --cur_layer) {\n        do {\n            __SLD_( history[nh++] = cur_node );\n\n            skiplist_node *next_node = _sl_next(slist, cur_node, cur_layer,\n                                                NULL, NULL);\n            if (!next_node) {\n                _sl_clr_flags(prevs, cur_layer+1, top_layer);\n                ATM_FETCH_SUB(cur_node->ref_count, 1);\n                YIELD();\n                goto insert_retry;\n            }\n            cmp = _sl_cmp(slist, node, next_node);\n            if (cmp > 0) {\n                // cur_node < next_node < node\n                // => move to next node\n                skiplist_node* temp = cur_node;\n                cur_node = next_node;\n                ATM_FETCH_SUB(temp->ref_count, 1);\n                continue;\n            } else {\n                // otherwise: cur_node < node <= next_node\n                ATM_FETCH_SUB(next_node->ref_count, 1);\n            }\n\n            if (no_dup && cmp == 0) {\n                // Duplicate key is not allowed.\n                _sl_clr_flags(prevs, cur_layer+1, top_layer);\n                ATM_FETCH_SUB(cur_node->ref_count, 1);\n                return -1;\n            }\n\n            if (cur_layer <= top_layer) {\n                prevs[cur_layer] = cur_node;\n                nexts[cur_layer] = next_node;\n                // both 'prev' and 'next' should be fully linked before\n                // insertion, and no other thread should not modify 'prev'\n                // at the same time.\n\n                int error_code = 0;\n                int locked_layer = cur_layer + 1;\n\n                // check if prev node is duplicated with upper layer\n                if (cur_layer < top_layer &&\n                    prevs[cur_layer] == prevs[cur_layer+1]) {\n                    // duplicate\n                    // => which means that 'being_modified' flag is already true\n                    // => do nothing\n                } else {\n                    bool expected = false;\n                    if (ATM_CAS(prevs[cur_layer]->being_modified,\n                                expected, bool_true)) {\n                        locked_layer = cur_layer;\n                    } else {\n                        error_code = -1;\n                    }\n                }\n\n                if (error_code == 0 &&\n                    !_sl_valid_prev_next(prevs[cur_layer], nexts[cur_layer])) {\n                    error_code = -2;\n                }\n\n                if (error_code != 0) {\n                    __SLD_RT_INS(error_code, node, top_layer, cur_layer);\n                    _sl_clr_flags(prevs, locked_layer, top_layer);\n                    ATM_FETCH_SUB(cur_node->ref_count, 1);\n                    YIELD();\n                    goto insert_retry;\n                }\n\n                // set current node's pointers\n                ATM_STORE(node->next[cur_layer], nexts[cur_layer]);\n\n                // check if `cur_node->next` has been changed from `next_node`.\n                skiplist_node* next_node_again =\n                    _sl_next(slist, cur_node, cur_layer, NULL, NULL);\n                ATM_FETCH_SUB(next_node_again->ref_count, 1);\n                if (next_node_again != next_node) {\n                    __SLD_NC_INS(cur_node, next_node, top_layer, cur_layer);\n                    // clear including the current layer\n                    // as we already set modification flag above.\n                    _sl_clr_flags(prevs, cur_layer, top_layer);\n                    ATM_FETCH_SUB(cur_node->ref_count, 1);\n                    YIELD();\n                    goto insert_retry;\n                }\n            }\n\n            if (cur_layer) {\n                // non-bottom layer => go down\n                break;\n            }\n\n            // bottom layer => insertion succeeded\n            // change prev/next nodes' prev/next pointers from 0 ~ top_layer\n            for (layer = 0; layer <= top_layer; ++layer) {\n                // `accessing_next` works as a spin-lock.\n                _sl_write_lock_an(prevs[layer]);\n                skiplist_node* exp = nexts[layer];\n                if ( !ATM_CAS(prevs[layer]->next[layer], exp, node) ) {\n                    __SLD_P(\"%02x ASSERT ins %p[%d] -> %p (expected %p)\\n\",\n                            (int)tid_hash, prevs[layer], cur_layer,\n                            ATM_GET(prevs[layer]->next[layer]), nexts[layer] );\n                    __SLD_ASSERT(0);\n                }\n                __SLD_P(\"%02x ins %p[%d] -> %p -> %p\\n\",\n                        (int)tid_hash, prevs[layer], layer,\n                        node, ATM_GET(node->next[layer]) );\n                _sl_write_unlock_an(prevs[layer]);\n            }\n\n            // now this node is fully linked\n            ATM_STORE(node->is_fully_linked, bool_true);\n\n            // allow removing next nodes\n            _sl_write_unlock_an(node);\n\n            __SLD_P(\"%02x ins %p done\\n\", (int)tid_hash, node);\n\n            ATM_FETCH_ADD(slist->num_entries, 1);\n            ATM_FETCH_ADD(slist->layer_entries[node->top_layer], 1);\n            for (int ii=slist->max_layer-1; ii>=0; --ii) {\n                if (slist->layer_entries[ii] > 0) {\n                    slist->top_layer = ii;\n                    break;\n                }\n            }\n\n            // modification is done for all layers\n            _sl_clr_flags(prevs, 0, top_layer);\n            ATM_FETCH_SUB(cur_node->ref_count, 1);\n\n            return 0;\n        } while (cur_node != &slist->tail);\n    }\n    return 0;\n}\n\nint skiplist_insert(skiplist_raw *slist,\n                    skiplist_node *node)\n{\n    return _skiplist_insert(slist, node, false);\n}\n\nint skiplist_insert_nodup(skiplist_raw *slist,\n                          skiplist_node *node)\n{\n    return _skiplist_insert(slist, node, true);\n}\n\ntypedef enum {\n    SM   = -2,\n    SMEQ = -1,\n    EQ   =  0,\n    GTEQ =  1,\n    GT   =  2\n} _sl_find_mode;\n\n// Note: it increases the `ref_count` of returned node.\n//       Caller is responsible to decrease it.\nstatic inline skiplist_node* _sl_find(skiplist_raw *slist,\n                                      skiplist_node *query,\n                                      _sl_find_mode mode)\n{\n    // mode:\n    //  SM   -2: smaller\n    //  SMEQ -1: smaller or equal\n    //  EQ    0: equal\n    //  GTEQ  1: greater or equal\n    //  GT    2: greater\nfind_retry:\n    (void)mode;\n    int cmp = 0;\n    int cur_layer = 0;\n    skiplist_node *cur_node = &slist->head;\n    ATM_FETCH_ADD(cur_node->ref_count, 1);\n\n    __SLD_(size_t nh = 0);\n    __SLD_(thread_local skiplist_node* history[1024]; (void)history);\n\n    uint8_t sl_top_layer = slist->top_layer;\n    for (cur_layer = sl_top_layer; cur_layer >= 0; --cur_layer) {\n        do {\n            __SLD_(history[nh++] = cur_node);\n\n            skiplist_node *next_node = _sl_next(slist, cur_node, cur_layer,\n                                                NULL, NULL);\n            if (!next_node) {\n                ATM_FETCH_SUB(cur_node->ref_count, 1);\n                YIELD();\n                goto find_retry;\n            }\n            cmp = _sl_cmp(slist, query, next_node);\n            if (cmp > 0) {\n                // cur_node < next_node < query\n                // => move to next node\n                skiplist_node* temp = cur_node;\n                cur_node = next_node;\n                ATM_FETCH_SUB(temp->ref_count, 1);\n                continue;\n            } else if (-1 <= mode && mode <= 1 && cmp == 0) {\n                // cur_node < query == next_node .. return\n                ATM_FETCH_SUB(cur_node->ref_count, 1);\n                return next_node;\n            }\n\n            // otherwise: cur_node < query < next_node\n            if (cur_layer) {\n                // non-bottom layer => go down\n                ATM_FETCH_SUB(next_node->ref_count, 1);\n                break;\n            }\n\n            // bottom layer\n            if (mode < 0 && cur_node != &slist->head) {\n                // smaller mode\n                ATM_FETCH_SUB(next_node->ref_count, 1);\n                return cur_node;\n            } else if (mode > 0 && next_node != &slist->tail) {\n                // greater mode\n                ATM_FETCH_SUB(cur_node->ref_count, 1);\n                return next_node;\n            }\n            // otherwise: exact match mode OR not found\n            ATM_FETCH_SUB(cur_node->ref_count, 1);\n            ATM_FETCH_SUB(next_node->ref_count, 1);\n            return NULL;\n        } while (cur_node != &slist->tail);\n    }\n\n    return NULL;\n}\n\nskiplist_node* skiplist_find(skiplist_raw *slist,\n                             skiplist_node *query)\n{\n    return _sl_find(slist, query, EQ);\n}\n\nskiplist_node* skiplist_find_smaller_or_equal(skiplist_raw *slist,\n                                              skiplist_node *query)\n{\n    return _sl_find(slist, query, SMEQ);\n}\n\nskiplist_node* skiplist_find_greater_or_equal(skiplist_raw *slist,\n                                              skiplist_node *query)\n{\n    return _sl_find(slist, query, GTEQ);\n}\n\nint skiplist_erase_node_passive(skiplist_raw *slist,\n                                skiplist_node *node)\n{\n    __SLD_(\n        thread_local std::thread::id tid = std::this_thread::get_id();\n        thread_local size_t tid_hash = std::hash<std::thread::id>{}(tid) % 256;\n        (void)tid_hash;\n    )\n\n    int top_layer = node->top_layer;\n    bool bool_true = true, bool_false = false;\n    bool removed = false;\n    bool is_fully_linked = false;\n\n    ATM_LOAD(node->removed, removed);\n    if (removed) {\n        // already removed\n        return -1;\n    }\n\n    skiplist_node* prevs[SKIPLIST_MAX_LAYER];\n    skiplist_node* nexts[SKIPLIST_MAX_LAYER];\n\n    bool expected = false;\n    if (!ATM_CAS(node->being_modified, expected, bool_true)) {\n        // already being modified .. cannot work on this node for now.\n        __SLD_BM(node);\n        return -2;\n    }\n\n    // set removed flag first, so that reader cannot read this node.\n    ATM_STORE(node->removed, bool_true);\n\n    __SLD_P(\"%02x rmv %p begin\\n\", (int)tid_hash, node);\n\nerase_node_retry:\n    ATM_LOAD(node->is_fully_linked, is_fully_linked);\n    if (!is_fully_linked) {\n        // already unlinked .. remove is done by other thread\n        ATM_STORE(node->removed, bool_false);\n        ATM_STORE(node->being_modified, bool_false);\n        return -3;\n    }\n\n    int cmp = 0;\n    bool found_node_to_erase = false;\n    (void)found_node_to_erase;\n    skiplist_node *cur_node = &slist->head;\n    ATM_FETCH_ADD(cur_node->ref_count, 1);\n\n    __SLD_(size_t nh = 0);\n    __SLD_(thread_local skiplist_node* history[1024]; (void)history);\n\n    int cur_layer = slist->top_layer;\n    for (; cur_layer >= 0; --cur_layer) {\n        do {\n            __SLD_( history[nh++] = cur_node );\n\n            bool node_found = false;\n            skiplist_node *next_node = _sl_next(slist, cur_node, cur_layer,\n                                                node, &node_found);\n            if (!next_node) {\n                _sl_clr_flags(prevs, cur_layer+1, top_layer);\n                ATM_FETCH_SUB(cur_node->ref_count, 1);\n                YIELD();\n                goto erase_node_retry;\n            }\n\n            // Note: unlike insert(), we should find exact position of `node`.\n            cmp = _sl_cmp(slist, node, next_node);\n            if (cmp > 0 || (cur_layer <= top_layer && !node_found) ) {\n                // cur_node <= next_node < node\n                // => move to next node\n                skiplist_node* temp = cur_node;\n                cur_node = next_node;\n                __SLD_( if (cmp > 0) {\n                    int cmp2 = _sl_cmp(slist, cur_node, node);\n                    if (cmp2 > 0) {\n                        // node < cur_node <= next_node: not found.\n                        _sl_clr_flags(prevs, cur_layer+1, top_layer);\n                        ATM_FETCH_SUB(temp->ref_count, 1);\n                        ATM_FETCH_SUB(next_node->ref_count, 1);\n                        __SLD_ASSERT(0);\n                    }\n                } )\n                ATM_FETCH_SUB(temp->ref_count, 1);\n                continue;\n            } else {\n                // otherwise: cur_node <= node <= next_node\n                ATM_FETCH_SUB(next_node->ref_count, 1);\n            }\n\n            if (cur_layer <= top_layer) {\n                prevs[cur_layer] = cur_node;\n                // note: 'next_node' and 'node' should not be the same,\n                //       as 'removed' flag is already set.\n                __SLD_ASSERT(next_node != node);\n                nexts[cur_layer] = next_node;\n\n                // check if prev node duplicates with upper layer\n                int error_code = 0;\n                int locked_layer = cur_layer + 1;\n                if (cur_layer < top_layer &&\n                    prevs[cur_layer] == prevs[cur_layer+1]) {\n                    // duplicate with upper layer\n                    // => which means that 'being_modified' flag is already true\n                    // => do nothing.\n                } else {\n                    expected = false;\n                    if (ATM_CAS(prevs[cur_layer]->being_modified,\n                                expected, bool_true)) {\n                        locked_layer = cur_layer;\n                    } else {\n                        error_code = -1;\n                    }\n                }\n\n                if (error_code == 0 &&\n                    !_sl_valid_prev_next(prevs[cur_layer], nexts[cur_layer])) {\n                    error_code = -2;\n                }\n\n                if (error_code != 0) {\n                    __SLD_RT_RMV(error_code, node, top_layer, cur_layer);\n                    _sl_clr_flags(prevs, locked_layer, top_layer);\n                    ATM_FETCH_SUB(cur_node->ref_count, 1);\n                    YIELD();\n                    goto erase_node_retry;\n                }\n\n                skiplist_node* next_node_again =\n                    _sl_next(slist, cur_node, cur_layer, node, NULL);\n                ATM_FETCH_SUB(next_node_again->ref_count, 1);\n                if (next_node_again != nexts[cur_layer]) {\n                    // `next` pointer has been changed, retry.\n                    __SLD_NC_RMV(cur_node, nexts[cur_layer], top_layer, cur_layer);\n                    _sl_clr_flags(prevs, cur_layer, top_layer);\n                    ATM_FETCH_SUB(cur_node->ref_count, 1);\n                    YIELD();\n                    goto erase_node_retry;\n                }\n            }\n            if (cur_layer == 0) found_node_to_erase = true;\n            // go down\n            break;\n        } while (cur_node != &slist->tail);\n    }\n    // Not exist in the skiplist, should not happen.\n    __SLD_ASSERT(found_node_to_erase);\n    // bottom layer => removal succeeded.\n    // mark this node unlinked\n    _sl_write_lock_an(node); {\n        ATM_STORE(node->is_fully_linked, bool_false);\n    } _sl_write_unlock_an(node);\n\n    // change prev nodes' next pointer from 0 ~ top_layer\n    for (cur_layer = 0; cur_layer <= top_layer; ++cur_layer) {\n        _sl_write_lock_an(prevs[cur_layer]);\n        skiplist_node* exp = node;\n        __SLD_ASSERT(exp != nexts[cur_layer]);\n        __SLD_ASSERT(nexts[cur_layer]->is_fully_linked);\n        if ( !ATM_CAS(prevs[cur_layer]->next[cur_layer],\n                      exp, nexts[cur_layer]) ) {\n            __SLD_P(\"%02x ASSERT rmv %p[%d] -> %p (node %p)\\n\",\n                    (int)tid_hash, prevs[cur_layer], cur_layer,\n                    ATM_GET(prevs[cur_layer]->next[cur_layer]), node );\n            __SLD_ASSERT(0);\n        }\n        __SLD_ASSERT(nexts[cur_layer]->top_layer >= cur_layer);\n        __SLD_P(\"%02x rmv %p[%d] -> %p (node %p)\\n\",\n                (int)tid_hash, prevs[cur_layer], cur_layer,\n                nexts[cur_layer], node);\n        _sl_write_unlock_an(prevs[cur_layer]);\n    }\n\n    __SLD_P(\"%02x rmv %p done\\n\", (int)tid_hash, node);\n\n    ATM_FETCH_SUB(slist->num_entries, 1);\n    ATM_FETCH_SUB(slist->layer_entries[node->top_layer], 1);\n    for (int ii=slist->max_layer-1; ii>=0; --ii) {\n        if (slist->layer_entries[ii] > 0) {\n            slist->top_layer = ii;\n            break;\n        }\n    }\n\n    // modification is done for all layers\n    _sl_clr_flags(prevs, 0, top_layer);\n    ATM_FETCH_SUB(cur_node->ref_count, 1);\n\n    ATM_STORE(node->being_modified, bool_false);\n\n    return 0;\n}\n\nint skiplist_erase_node(skiplist_raw *slist,\n                        skiplist_node *node)\n{\n    int ret = 0;\n    do {\n        ret = skiplist_erase_node_passive(slist, node);\n        // if ret == -2, other thread is accessing the same node\n        // at the same time. try again.\n    } while (ret == -2);\n    return ret;\n}\n\nint skiplist_erase(skiplist_raw *slist,\n                   skiplist_node *query)\n{\n    skiplist_node *found = skiplist_find(slist, query);\n    if (!found) {\n        // key not found\n        return -4;\n    }\n\n    int ret = 0;\n    do {\n        ret = skiplist_erase_node_passive(slist, found);\n        // if ret == -2, other thread is accessing the same node\n        // at the same time. try again.\n    } while (ret == -2);\n\n    ATM_FETCH_SUB(found->ref_count, 1);\n    return ret;\n}\n\nint skiplist_is_valid_node(skiplist_node* node) {\n    return _sl_valid_node(node);\n}\n\nint skiplist_is_safe_to_free(skiplist_node* node) {\n    if (node->accessing_next) return 0;\n    if (node->being_modified) return 0;\n    if (!node->removed) return 0;\n\n    uint16_t ref_count = 0;\n    ATM_LOAD(node->ref_count, ref_count);\n    if (ref_count) return 0;\n    return 1;\n}\n\nvoid skiplist_wait_for_free(skiplist_node* node) {\n    while (!skiplist_is_safe_to_free(node)) {\n        YIELD();\n    }\n}\n\nvoid skiplist_grab_node(skiplist_node* node) {\n    ATM_FETCH_ADD(node->ref_count, 1);\n}\n\nvoid skiplist_release_node(skiplist_node* node) {\n    __SLD_ASSERT(node->ref_count);\n    ATM_FETCH_SUB(node->ref_count, 1);\n}\n\nskiplist_node* skiplist_next(skiplist_raw *slist,\n                             skiplist_node *node) {\n    //   << Issue >>\n    // If `node` is already removed and its next node is also removed\n    // and then released, the link update will not be applied to `node`\n    // as it is already unrechable from skiplist. `node` still points to\n    // the released node so that `_sl_next(node)` may return corrupted\n    // memory region.\n    //\n    // 0) initial:\n    //    A -> B -> C -> D\n    //\n    // 1) B is `node`, which is removed but not yet released:\n    //    B --+-> C -> D\n    //        |\n    //    A --+\n    //\n    // 2) remove C, and then release:\n    //    B -> !C!  +-> D\n    //              |\n    //    A --------+\n    //\n    // 3) skiplist_next(B):\n    //    will fetch C, which is already released so that\n    //    may contain garbage data.\n    //\n    // In this case, start over from the top layer,\n    // to find valid link (same as in prev()).\n\n    skiplist_node *next = _sl_next(slist, node, 0, NULL, NULL);\n    if (!next) next = _sl_find(slist, node, GT);\n\n    if (next == &slist->tail) return NULL;\n    return next;\n}\n\nskiplist_node* skiplist_prev(skiplist_raw *slist,\n                             skiplist_node *node) {\n    skiplist_node *prev = _sl_find(slist, node, SM);\n    if (prev == &slist->head) return NULL;\n    return prev;\n}\n\nskiplist_node* skiplist_begin(skiplist_raw *slist) {\n    skiplist_node *next = NULL;\n    while (!next) {\n        next = _sl_next(slist, &slist->head, 0, NULL, NULL);\n    }\n    if (next == &slist->tail) return NULL;\n    return next;\n}\n\nskiplist_node* skiplist_end(skiplist_raw *slist) {\n    return skiplist_prev(slist, &slist->tail);\n}\n\n"
  },
  {
    "path": "tests/container_test.cc",
    "content": "#include \"sl_map.h\"\n#include \"sl_set.h\"\n\n#include \"test_common.h\"\n\n#include <chrono>\n#include <thread>\n#include <vector>\n\n#include <stdio.h>\n#include <unistd.h>\n\nint _map_basic(sl_map<int, int>& sl) {\n    for (int i=0; i<10; ++i) {\n        auto itr = sl.insert( std::make_pair(i, i*10) );\n        CHK_TRUE(itr.second);\n        CHK_EQ(i, itr.first->first);\n        CHK_EQ(i*10, itr.first->second);\n    }\n\n    // Duplicate insert should not be allowed.\n    for (int i=0; i<10; ++i) {\n        auto itr = sl.insert( std::make_pair(i, i*20) );\n        CHK_FALSE(itr.second);\n        CHK_EQ(i, itr.first->first);\n        CHK_EQ(i*10, itr.first->second);\n    }\n\n    for (int i=0; i<10; ++i) {\n        auto entry = sl.find(i);\n        if (entry != sl.end()) {\n            CHK_EQ(i, entry->first);\n            CHK_EQ(i*10, entry->second);\n        }\n    }\n\n    auto ii = sl.find(5);\n    sl.erase(ii);\n\n    int count = 0;\n    for (auto& entry: sl) {\n        CHK_EQ(count, entry.first);\n        CHK_EQ(count*10, entry.second);\n        if (count == 4) count += 2;\n        else count++;\n    }\n\n    ii = sl.begin();\n    while (ii != sl.end()) {\n        if (ii->first % 2 == 0) {\n            ii = sl.erase(ii);\n        } else {\n            ii++;\n        }\n    }\n\n    count = 1;\n    for (auto& entry: sl) {\n        CHK_EQ(count, entry.first);\n        CHK_EQ(count*10, entry.second);\n        if (count == 3) count += 4;\n        else count += 2;\n    }\n    CHK_EQ(4, (int)sl.size());\n\n    return 0;\n}\n\nint map_basic_wait() {\n    sl_map<int, int> sl_wait;\n    return _map_basic(sl_wait);\n}\n\nint map_basic_gc() {\n    sl_map_gc<int, int> sl_gc;\n    return _map_basic(sl_gc);\n}\n\nint _set_basic(sl_set<int>& sl) {\n    for (int i=0; i<10; ++i) {\n        auto itr = sl.insert(i);\n        CHK_TRUE(itr.second);\n        CHK_EQ(i, *itr.first);\n    }\n\n    // Duplicate insert should not be allowed.\n    for (int i=0; i<10; ++i) {\n        auto itr = sl.insert(i);\n        CHK_FALSE(itr.second);\n        CHK_EQ(i, *itr.first);\n    }\n\n    for (int i=0; i<10; ++i) {\n        auto entry = sl.find(i);\n        if (entry != sl.end()) {\n            CHK_EQ(i, *entry);\n        }\n    }\n\n    auto ii = sl.find(5);\n    sl.erase(ii);\n\n    int count = 0;\n    for (auto& entry: sl) {\n        int val = entry;\n        CHK_EQ(count, val);\n        if (count == 4) count += 2;\n        else count++;\n    }\n\n    ii = sl.begin();\n    while (ii != sl.end()) {\n        int val = *ii;\n        if (val % 2 == 0) {\n            ii = sl.erase(ii);\n        } else {\n            ii++;\n        }\n    }\n\n    count = 1;\n    for (auto& entry: sl) {\n        int val = entry;\n        CHK_EQ(count, val);\n        if (count == 3) count += 4;\n        else count += 2;\n    }\n    CHK_EQ(4, (int)sl.size());\n\n    return 0;\n}\n\nint set_basic_wait() {\n    sl_set<int> sl_wait;\n    return _set_basic(sl_wait);\n}\n\nint set_basic_gc() {\n    sl_set_gc<int> sl_gc;\n    return _set_basic(sl_gc);\n}\n\nint map_self_refer_test() {\n    sl_map_gc<int, int> sl;\n    for (int i=0; i<10; ++i) {\n        auto itr = sl.insert( std::make_pair(i, i*10) );\n        CHK_TRUE(itr.second);\n        CHK_EQ(i, itr.first->first);\n        CHK_EQ(i*10, itr.first->second);\n    }\n\n    for(;;) {\n        auto entry = sl.begin();\n        if (entry == sl.end()) break;\n        sl.erase(entry->first);\n    }\n    return 0;\n}\n\nint set_self_refer_test() {\n    sl_set_gc<int> sl;\n    for (int i=0; i<10; ++i) {\n        auto itr = sl.insert(i);\n        CHK_TRUE(itr.second);\n        CHK_EQ(i, *itr.first);\n    }\n\n    for(;;) {\n        auto entry = sl.begin();\n        if (entry == sl.end()) break;\n        sl.erase(*entry);\n    }\n    return 0;\n}\n\nint main(int argc, char** argv) {\n    TestSuite tt(argc, argv);\n\n    tt.doTest(\"container map test (busy wait)\", map_basic_wait);\n    tt.doTest(\"container map test (lazy gc)\", map_basic_gc);\n    tt.doTest(\"container map self refer test\", map_self_refer_test);\n    tt.doTest(\"container set test (busy wait)\", set_basic_wait);\n    tt.doTest(\"container set test (lazy gc)\", set_basic_gc);\n    tt.doTest(\"container set self refer test\", set_self_refer_test);\n\n    return 0;\n}\n\n"
  },
  {
    "path": "tests/mt_test.cc",
    "content": "#include \"skiplist.h\"\n\n#include \"test_common.h\"\n\n#include <chrono>\n#include <thread>\n#include <vector>\n\n#include <stdio.h>\n#include <unistd.h>\n\nstruct TestNode {\n    TestNode() : value(0) {\n        skiplist_init_node(&snode);\n    }\n    ~TestNode() {\n        skiplist_free_node(&snode);\n    }\n\n    static int cmp(skiplist_node* a, skiplist_node* b, void* aux) {\n        TestNode *aa, *bb;\n        aa = _get_entry(a, TestNode, snode);\n        bb = _get_entry(b, TestNode, snode);\n        if (aa->value < bb->value) return -1;\n        if (aa->value > bb->value) return 1;\n        return 0;\n    }\n\n    skiplist_node snode;\n    int value;\n};\n\nstruct ThreadArgs {\n    skiplist_raw* slist;\n    int max_num;\n    int duration_ms;\n    int ret;\n};\n\nint _itr_thread(ThreadArgs* args) {\n    TestSuite::Timer timer(args->duration_ms);\n    do {\n        int num_walks = 10;\n        int count = 0;\n        int r = rand() % args->max_num;\n\n        TestNode query;\n        query.value = r;\n        skiplist_node* cursor = skiplist_find(args->slist, &query.snode);\n        while (cursor) {\n            TestNode* node = _get_entry(cursor, TestNode, snode);\n            cursor = skiplist_next(args->slist, cursor);\n            usleep(10);\n            (void)node;\n            skiplist_release_node(&node->snode);\n            if (++count > num_walks) break;\n        }\n        if (cursor) skiplist_release_node(cursor);\n    } while (!timer.timeover());\n\n    return 0;\n}\n\nvoid itr_thread(ThreadArgs* args) {\n    args->ret = _itr_thread(args);\n}\n\nint _writer_thread(ThreadArgs* args) {\n    TestSuite::Timer timer(args->duration_ms);\n    do {\n        int r;\n        TestNode* node;\n        TestNode query;\n        skiplist_node* cursor;\n\n        r = rand() % args->max_num;\n        query.value = r;\n        cursor = skiplist_find(args->slist, &query.snode);\n        if (!cursor) {\n            node = new TestNode();\n            node->value = r;\n            skiplist_insert(args->slist, &node->snode);\n        } else {\n            skiplist_release_node(cursor);\n        }\n\n        r = rand() % args->max_num;\n        query.value = r;\n        cursor = skiplist_find(args->slist, &query.snode);\n        if (cursor) {\n            node = _get_entry(cursor, TestNode, snode);\n            skiplist_erase_node(args->slist, &node->snode);\n            if (node->snode.being_modified ||\n                !node->snode.removed)\n                printf(\"%d %d\\n\", (int)node->snode.being_modified,\n                    (int)node->snode.removed);\n            skiplist_release_node(&node->snode);\n            skiplist_wait_for_free(&node->snode);\n            delete node;\n        }\n    } while (!timer.timeover());\n\n    uint64_t c_check = 0;\n    skiplist_node* cursor = skiplist_begin(args->slist);\n    while (cursor) {\n        skiplist_node* temp_node = cursor;\n        cursor = skiplist_next(args->slist, cursor);\n        skiplist_release_node(temp_node);\n        c_check++;\n    }\n    if (cursor) skiplist_release_node(cursor);\n\n    CHK_EQ(c_check, skiplist_get_size(args->slist));\n    return 0;\n}\n\nvoid writer_thread(ThreadArgs* args) {\n    args->ret = _writer_thread(args);\n}\n\nint itr_write_erase() {\n    std::thread iterator;\n    std::thread writer;\n    int num = 30000;\n\n    skiplist_raw slist;\n    skiplist_init(&slist, TestNode::cmp);\n    TestNode* node[num];\n\n    for (int ii=0; ii<num; ii+=1) {\n        node[ii] = new TestNode();\n        node[ii]->value = ii;\n        skiplist_insert(&slist, &node[ii]->snode);\n    }\n    CHK_EQ(num, (int)skiplist_get_size(&slist));\n\n    for (int ii=0; ii<num; ii+=1) {\n        CHK_EQ(0, node[ii]->snode.ref_count);\n    }\n\n    for (int ii=0; ii<num; ii+=2) {\n        skiplist_erase_node(&slist, &node[ii]->snode);\n        delete node[ii];\n    }\n    for (int ii=1; ii<num; ii+=2) {\n        CHK_EQ(0, node[ii]->snode.ref_count);\n    }\n\n    ThreadArgs args_itr;\n    ThreadArgs args_writer;\n    args_itr.slist = &slist;\n    args_itr.duration_ms = 1000;\n    args_itr.max_num = num;\n    args_itr.ret = 0;\n    args_writer = args_itr;\n    iterator = std::thread(itr_thread, &args_itr);\n    writer = std::thread(writer_thread, &args_writer);\n\n    iterator.join();\n    writer.join();\n\n    CHK_EQ(0, args_itr.ret);\n    CHK_EQ(0, args_writer.ret);\n\n    skiplist_node* cursor = skiplist_begin(&slist);\n    while(cursor) {\n        TestNode* cur_node = _get_entry(cursor, TestNode, snode);\n        if (cur_node->snode.ref_count != 1)\n            printf(\"%d %d\\n\", (int)cur_node->value, (int)cur_node->snode.ref_count);\n        CHK_EQ(1, cur_node->snode.ref_count);\n        cursor = skiplist_next(&slist, cursor);\n        skiplist_release_node(&cur_node->snode);\n        delete cur_node;\n    }\n    if (cursor) skiplist_release_node(cursor);\n\n    skiplist_free(&slist);\n    return 0;\n}\n\nint itr_erase_deterministic() {\n    int num = 10;\n    skiplist_raw slist;\n    skiplist_init(&slist, TestNode::cmp);\n\n    TestNode* node[num];\n    for (int ii=0; ii<num; ++ii) {\n        node[ii] = new TestNode();\n        node[ii]->value = ii;\n        skiplist_insert(&slist, &node[ii]->snode);\n    }\n\n    skiplist_node* cursor = skiplist_begin(&slist);\n    while (cursor) {\n        TestNode* cur_node = _get_entry(cursor, TestNode, snode);\n        if (cur_node->value == 2) {\n            skiplist_erase_node(&slist, &cur_node->snode);\n            CHK_NOT(skiplist_is_safe_to_free(&cur_node->snode));\n        }\n        cursor = skiplist_next(&slist, cursor);\n        skiplist_release_node(&cur_node->snode);\n        if (cur_node->value == 2) {\n            CHK_OK(skiplist_is_safe_to_free(&cur_node->snode));\n            delete cur_node;\n        }\n    }\n    if (cursor) skiplist_release_node(cursor);\n\n    int count = 0;\n    cursor = skiplist_begin(&slist);\n    while (cursor) {\n        TestNode* cur_node = _get_entry(cursor, TestNode, snode);\n        cursor = skiplist_next(&slist, cursor);\n        skiplist_release_node(&cur_node->snode);\n        count++;\n        delete cur_node;\n    }\n    if (cursor) skiplist_release_node(cursor);\n    CHK_EQ(num-1, count);\n\n    skiplist_free(&slist);\n    return 0;\n}\n\nint main(int argc, char** argv) {\n    TestSuite tt(argc, argv);\n\n    tt.doTest(\"iterator write erase test\", itr_write_erase);\n    tt.doTest(\"iterator write erase deterministic test\", itr_erase_deterministic);\n\n    return 0;\n}\n\n"
  },
  {
    "path": "tests/skiplist_test.cc",
    "content": "/**\n * Copyright (C) 2017-present Jung-Sang Ahn <jungsang.ahn@gmail.com>\n * All rights reserved.\n *\n * Permission is hereby granted, free of charge, to any person\n * obtaining a copy of this software and associated documentation\n * files (the \"Software\"), to deal in the Software without\n * restriction, including without limitation the rights to use,\n * copy, modify, merge, publish, distribute, sublicense, and/or sell\n * copies of the Software, and to permit persons to whom the\n * Software is furnished to do so, subject to the following\n * conditions:\n *\n * The above copyright notice and this permission notice shall be\n * included in all copies or substantial portions of the Software.\n *\n * THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES\n * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT\n * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,\n * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\n * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR\n * OTHER DEALINGS IN THE SOFTWARE.\n */\n\n#include \"skiplist.h\"\n\n#include \"test_common.h\"\n\n#include <chrono>\n#include <ctime>\n#include <mutex>\n#include <set>\n#include <thread>\n#include <vector>\n\n#include <stdio.h>\n#include <stdlib.h>\n\nstruct IntNode {\n    IntNode() {\n        skiplist_init_node(&snode);\n    }\n    ~IntNode() {\n        skiplist_free_node(&snode);\n    }\n\n    skiplist_node snode;\n    int value;\n};\n\nint _cmp_IntNode(skiplist_node *a, skiplist_node *b, void *aux)\n{\n    IntNode *aa, *bb;\n    aa = _get_entry(a, IntNode, snode);\n    bb = _get_entry(b, IntNode, snode);\n    if (aa->value < bb->value) {\n        return -1;\n    } else if (aa->value == bb->value) {\n        return 0;\n    } else {\n        return 1;\n    }\n}\n\nint basic_insert_and_erase()\n{\n    skiplist_raw list;\n    skiplist_init(&list, _cmp_IntNode);\n\n    int i, j, temp;\n    int n = 16;\n    std::vector<int> key(n);\n    std::vector<IntNode> arr(n);\n\n    // key assign\n    for (i=0; i<n; ++i) {\n        key[i] = i;\n    }\n    // shuffle\n    for (i=0; i<n; ++i) {\n        j = rand() % n;\n        temp = key[i];\n        key[i] = key[j];\n        key[j] = temp;\n    }\n\n    // random insert\n    for (i=0; i<n; ++i) {\n        arr[i].value = key[i];\n        skiplist_insert(&list, &arr[i].snode);\n    }\n\n    // forward iteration\n    int count = 0;\n    skiplist_node *cur = skiplist_begin(&list);\n    while (cur) {\n        IntNode *node = _get_entry(cur, IntNode, snode);\n        CHK_EQ(count, node->value);\n        cur = skiplist_next(&list, cur);\n        count++;\n    }\n    CHK_EQ(n, count);\n\n    // backward iteration\n    count = n;\n    cur = skiplist_end(&list);\n    while (cur) {\n        count--;\n        IntNode *node = _get_entry(cur, IntNode, snode);\n        CHK_EQ(count, node->value);\n        cur = skiplist_prev(&list, cur);\n    }\n    CHK_EQ(0, count);\n\n    // remove even numbers\n    IntNode query;\n    for (i=0; i<n; i+=2) {\n        query.value = i;\n        cur = skiplist_find(&list, &query.snode);\n        CHK_NONNULL(cur);\n        int ret_value = skiplist_erase_node(&list, cur);\n        CHK_EQ(0, ret_value);\n    }\n\n    // forward iteration\n    count = 0;\n    cur = skiplist_begin(&list);\n    while (cur) {\n        IntNode *node = _get_entry(cur, IntNode, snode);\n        CHK_EQ(count*2 + 1, node->value);\n        cur = skiplist_next(&list, cur);\n        count++;\n    }\n    CHK_EQ(n/2, count);\n\n    // backward iteration\n    count = n/2;\n    cur = skiplist_end(&list);\n    while (cur) {\n        count--;\n        IntNode *node = _get_entry(cur, IntNode, snode);\n        CHK_EQ(count*2 + 1, node->value);\n        cur = skiplist_prev(&list, cur);\n    }\n    CHK_EQ(0, count);\n\n    skiplist_free(&list);\n\n    return 0;\n}\n\nint find_test()\n{\n    TestSuite::Timer tt;\n    TestSuite::appendResultMessage(\"\\n\");\n\n    double elapsed_sec = 1;\n    char msg[1024];\n\n    skiplist_raw list;\n    skiplist_init(&list, _cmp_IntNode);\n\n    int i, j, temp;\n    int n = 10000;\n    std::vector<int> key(n);\n    std::vector<IntNode> arr(n);\n\n    // assign\n    for (i=0; i<n; ++i) {\n        key[i] = i*10;\n    }\n    // shuffle\n    for (i=0; i<n; ++i) {\n        j = rand() % n;\n        temp = key[i];\n        key[i] = key[j];\n        key[j] = temp;\n    }\n\n    tt.reset();\n    for (i=0; i<n; ++i) {\n        arr[i].value = key[i];\n        skiplist_insert(&list, &arr[i].snode);\n    }\n    elapsed_sec = tt.getTimeUs() / 1000000.0;\n    sprintf(msg, \"insert %.4f (%.1f ops/sec)\\n\",\n            elapsed_sec, n / elapsed_sec);\n    TestSuite::appendResultMessage(msg);\n\n    // ==== find exact match key\n    IntNode query, *item;\n    skiplist_node *ret;\n\n    tt.reset();\n    for (i=0; i<n; ++i) {\n        query.value = i*10;\n        ret = skiplist_find(&list, &query.snode);\n        CHK_NONNULL(ret);\n        item = _get_entry(ret, IntNode, snode);\n        CHK_EQ(query.value, item->value);\n    }\n    elapsed_sec = tt.getTimeUs() / 1000000.0;\n    sprintf(msg, \"find (existing key) done: %.4f (%.1f ops/sec)\\n\",\n            elapsed_sec, n / elapsed_sec);\n    TestSuite::appendResultMessage(msg);\n\n    // ==== find smaller key\n    tt.reset();\n\n    // smaller than smallest key\n    query.value = -5;\n    ret = skiplist_find_smaller_or_equal(&list, &query.snode);\n    CHK_NULL(ret);\n\n    for (i=0; i<n; ++i) {\n        query.value = i*10 + 5;\n        ret = skiplist_find_smaller_or_equal(&list, &query.snode);\n        CHK_NONNULL(ret);\n        item = _get_entry(ret, IntNode, snode);\n        CHK_EQ(i*10, item->value);\n    }\n    elapsed_sec = tt.getTimeUs() / 1000000.0;\n    sprintf(msg, \"find (smaller key) done: %.4f (%.1f ops/sec)\\n\",\n            elapsed_sec, n / elapsed_sec);\n    TestSuite::appendResultMessage(msg);\n\n    // ==== find greater key\n    tt.reset();\n\n    for (i=0; i<n; ++i) {\n        query.value = i*10 - 5;\n        ret = skiplist_find_greater_or_equal(&list, &query.snode);\n        CHK_NONNULL(ret);\n        item = _get_entry(ret, IntNode, snode);\n        CHK_EQ(i*10, item->value);\n    }\n\n    // greater than greatest key\n    query.value = i*10;\n    ret = skiplist_find_greater_or_equal(&list, &query.snode);\n    CHK_NULL(ret);\n\n    elapsed_sec = tt.getTimeUs() / 1000000.0;\n    sprintf(msg, \"find (greater key) done: %.4f (%.1f ops/sec)\\n\",\n            elapsed_sec, n / elapsed_sec);\n    TestSuite::appendResultMessage(msg);\n\n    // ==== find non-existing key\n    tt.reset();\n    for (i=0; i<n; ++i) {\n        query.value = i*10 + 1;\n        ret = skiplist_find(&list, &query.snode);\n        CHK_NULL(ret);\n    }\n    elapsed_sec = tt.getTimeUs() / 1000000.0;\n    sprintf(msg, \"find (non-existing key) done: %.4f (%.1f ops/sec)\\n\",\n            elapsed_sec, n / elapsed_sec);\n    TestSuite::appendResultMessage(msg);\n\n    skiplist_free(&list);\n\n    return 0;\n}\n\nstruct thread_args : TestSuite::ThreadArgs {\n    skiplist_raw* list;\n    std::set<int>* stl_set;\n    std::mutex *lock;\n    bool use_skiplist;\n    std::vector<int>* key;\n    std::vector<IntNode>* arr;\n    int range_begin;\n    int range_end;\n    double elapsed_sec;\n};\n\nstruct test_args {\n    int n_keys;\n    int n_writers;\n    int n_erasers;\n    int n_readers;\n    bool random_order;\n    bool use_skiplist;\n};\n\nint writer_thread(void *voidargs)\n{\n    thread_args *args = (thread_args*)voidargs;\n    TestSuite::Timer tt;\n\n    int i;\n    for (i=args->range_begin; i<=args->range_end; ++i) {\n        IntNode& node = args->arr->at(i);\n        if (args->use_skiplist) {\n            skiplist_insert(args->list, &node.snode);\n        } else {\n            args->lock->lock();\n            args->stl_set->insert(node.value);\n            args->lock->unlock();\n        }\n    }\n    args->elapsed_sec = tt.getTimeUs() / 1000000.0;\n\n    return 0;\n}\n\nint eraser_thread(void *voidargs)\n{\n    thread_args *args = (thread_args*)voidargs;\n    TestSuite::Timer tt;\n\n    int i;\n    for (i=args->range_begin; i<=args->range_end; ++i) {\n        IntNode query;\n        if (args->use_skiplist) {\n            query.value = args->key->at(i);\n            int ret = skiplist_erase(args->list, &query.snode);\n            (void)ret;\n        } else {\n            args->lock->lock();\n            args->stl_set->erase(args->key->at(i));\n            args->lock->unlock();\n        }\n    }\n    args->elapsed_sec = tt.getTimeUs() / 1000000.0;\n\n    return 0;\n}\n\nint reader_thread(void *voidargs)\n{\n    thread_args *args = (thread_args*)voidargs;\n    TestSuite::Timer tt;\n\n    int i;\n    for (i=args->range_begin; i<=args->range_end; ++i) {\n        IntNode query;\n        if (args->use_skiplist) {\n            query.value = args->key->at(i);\n            skiplist_node *ret = skiplist_find(args->list, &query.snode);\n            (void)ret;\n        } else {\n            args->lock->lock();\n            auto ret = args->stl_set->find(args->key->at(i));\n            (void)ret;\n            args->lock->unlock();\n        }\n    }\n    args->elapsed_sec = tt.getTimeUs() / 1000000.0;\n\n    return 0;\n}\n\nint concurrent_write_test(test_args t_args)\n{\n    char msg[1024];\n    TestSuite::appendResultMessage(\"\\n\");\n\n    skiplist_raw list;\n    std::mutex lock;\n    std::set<int> stl_set;\n\n    skiplist_init(&list, _cmp_IntNode);\n\n    int i, j, temp;\n    int n = t_args.n_keys;\n    std::vector<int> key(n);\n    std::vector<IntNode> arr(n);\n    // assign\n    for (i=0; i<n; ++i) {\n        key[i] = i;\n    }\n\n    // shuffle\n    if (t_args.random_order) {\n        for (i=0; i<n; ++i) {\n            j = rand() % n;\n            temp = key[i];\n            key[i] = key[j];\n            key[j] = temp;\n        }\n    }\n\n    for (i=0; i<n; ++i) {\n        arr[i].value = key[i];\n    }\n\n    int n_threads = t_args.n_writers;\n    int n_keys_per_thread = n / n_threads;\n\n    std::vector<TestSuite::ThreadHolder*> t_holder(n_threads);\n    std::vector<thread_args> args(n_threads);\n\n    for (i=0; i<n_threads; ++i) {\n        args[i].list = &list;\n        args[i].stl_set = &stl_set;\n        args[i].lock = &lock;\n        args[i].key = &key;\n        args[i].arr = &arr;\n        args[i].range_begin = i*n_keys_per_thread;\n        args[i].range_end = (i+1)*n_keys_per_thread - 1;\n        args[i].use_skiplist = t_args.use_skiplist;\n\n        t_holder[i] = new TestSuite::ThreadHolder(&args[i], writer_thread, nullptr);\n    }\n\n    TestSuite::Timer tt;\n    for (i=0; i<n_threads; ++i){\n        t_holder[i]->join();\n        delete t_holder[i];\n    }\n    double elapsed_sec = tt.getTimeUs() / 1000000.0;\n    sprintf(msg, \"insert %.4f (%d threads, %.1f ops/sec)\\n\",\n            elapsed_sec, n_threads, n/elapsed_sec);\n    TestSuite::appendResultMessage(msg);\n\n    if (!t_args.use_skiplist) return 0;\n\n    // integrity check (forward iteration, skiplist only)\n    tt.reset();\n\n    int count = 0;\n    bool corruption = false;\n    skiplist_node *cur = skiplist_begin(&list);\n    while (cur) {\n        IntNode *node = _get_entry(cur, IntNode, snode);\n        if (node->value != count) {\n            skiplist_node *missing = &arr[count].snode;\n            sprintf(msg, \"idx %d is missing, %lx\\n\", count, (uint64_t)missing);\n            TestSuite::appendResultMessage(msg);\n\n            skiplist_node *prev = skiplist_prev(&list, missing);\n            skiplist_node *next = skiplist_next(&list, missing);\n            IntNode *prev_node = _get_entry(prev, IntNode, snode);\n            IntNode *next_node = _get_entry(next, IntNode, snode);\n            sprintf(msg, \"%d %d\\n\", prev_node->value, next_node->value);\n            TestSuite::appendResultMessage(msg);\n\n            count = node->value;\n            corruption = true;\n        }\n        CHK_EQ(count, node->value);\n        cur = skiplist_next(&list, cur);\n        count++;\n    }\n    CHK_EQ(n, count);\n    CHK_NOT(corruption);\n\n    elapsed_sec = tt.getTimeUs() / 1000000.0;\n\n    sprintf(msg, \"iteration %.4f (%.1f ops/sec)\\n\",\n            elapsed_sec, n/elapsed_sec);\n    TestSuite::appendResultMessage(msg);\n\n    skiplist_free(&list);\n\n    return 0;\n}\n\nint concurrent_write_erase_test(struct test_args t_args)\n{\n    char msg[1024];\n    TestSuite::appendResultMessage(\"\\n\");\n\n    skiplist_raw list;\n    std::mutex lock;\n    std::set<int> stl_set;\n\n    skiplist_init(&list, _cmp_IntNode);\n\n    int i, j, temp;\n    int n = t_args.n_keys;\n    std::vector<int> key_add(n);\n    std::vector<int> key_del(n);\n    std::vector<IntNode> arr_add(n);\n    std::vector<IntNode> arr_del(n);\n    std::vector<IntNode*> arr_add_dbgref(n);\n\n    // initial list state: 0, 10, 20, ...\n    //  => writer threads: adding 5, 15, 25, ...\n    //  => eraser threads: erasing 0, 10, 20, ...\n    // final list state: 5, 15, 25, ...\n\n    // initial insert\n    for (i=0; i<n; ++i) {\n        // 0, 10, 20, 30 ...\n        key_del[i] = i * 10;\n        arr_del[i].value = key_del[i];\n        skiplist_insert(&list, &arr_del[i].snode);\n    }\n\n    // assign keys to add\n    for (i=0; i<n; ++i) {\n        // 5, 15, 25, 35, ...\n        key_add[i] = i*10 + 5;\n    }\n\n    if (t_args.random_order) {\n        // shuffle keys to add\n        for (i=0; i<n; ++i) {\n            j = rand() % n;\n            temp = key_add[i];\n            key_add[i] = key_add[j];\n            key_add[j] = temp;\n        }\n    }\n    for (i=0; i<n; ++i) {\n        int ori_idx = (key_add[i] - 5) / 10;\n        arr_add[i].value = key_add[i];\n        arr_add_dbgref[ori_idx] = &arr_add[i];\n    }\n\n    if (t_args.random_order) {\n        // also shuffle keys to delete\n        for (i=0; i<n; ++i) {\n            j = rand() % n;\n            temp = key_del[i];\n            key_del[i] = key_del[j];\n            key_del[j] = temp;\n        }\n    }\n\n    int n_threads_add = t_args.n_writers;\n    int n_threads_del = t_args.n_erasers;\n\n    int n_keys_per_thread_add = n / n_threads_add;\n    int n_keys_per_thread_del = n / n_threads_del;\n\n    std::vector<TestSuite::ThreadHolder*> t_holder_add(n_threads_add);\n    std::vector<thread_args> args_add(n_threads_add);\n\n    for (i=0; i<n_threads_add; ++i) {\n        args_add[i].list = &list;\n        args_add[i].stl_set = &stl_set;\n        args_add[i].lock = &lock;\n        args_add[i].key = &key_add;\n        args_add[i].arr = &arr_add;\n        args_add[i].range_begin = i*n_keys_per_thread_add;\n        args_add[i].range_end = (i+1)*n_keys_per_thread_add - 1;\n        args_add[i].use_skiplist = t_args.use_skiplist;\n\n        t_holder_add[i] = new TestSuite::ThreadHolder\n                              (&args_add[i], writer_thread, nullptr);\n    }\n\n    std::vector<TestSuite::ThreadHolder*> t_holder_del(n_threads_del);\n    std::vector<thread_args> args_del(n_threads_del);\n\n    for (i=0; i<n_threads_del; ++i) {\n        args_del[i].list = &list;\n        args_del[i].stl_set = &stl_set;\n        args_del[i].lock = &lock;\n        args_del[i].key = &key_del;\n        args_del[i].arr = &arr_del;\n        args_del[i].range_begin = i*n_keys_per_thread_del;\n        args_del[i].range_end = (i+1)*n_keys_per_thread_del - 1;\n        args_del[i].use_skiplist = t_args.use_skiplist;\n\n        t_holder_del[i] = new TestSuite::ThreadHolder\n                              (&args_del[i], eraser_thread, nullptr);\n    }\n\n\n    for (i=0; i<n_threads_add; ++i){\n        t_holder_add[i]->join();\n        delete t_holder_add[i];\n    }\n    for (i=0; i<n_threads_del; ++i){\n        t_holder_del[i]->join();\n        delete t_holder_del[i];\n    }\n\n    double max_seconds_add = 0;\n    double max_seconds_del = 0;\n\n    for (i=0; i<n_threads_add; ++i) {\n        if (args_add[i].elapsed_sec > max_seconds_add) {\n            max_seconds_add = args_add[i].elapsed_sec;\n        }\n    }\n\n    for (i=0; i<n_threads_del; ++i) {\n        if (args_del[i].elapsed_sec > max_seconds_del) {\n            max_seconds_del = args_del[i].elapsed_sec;\n        }\n    }\n\n    sprintf(msg, \"insertion %.4f (%d threads, %.1f ops/sec)\\n\",\n            max_seconds_add, n_threads_add, n / max_seconds_add);\n    TestSuite::appendResultMessage(msg);\n\n    sprintf(msg, \"deletion %.4f (%d threads, %.1f ops/sec)\\n\",\n            max_seconds_del, n_threads_del, n / max_seconds_del);\n    TestSuite::appendResultMessage(msg);\n\n    sprintf(msg, \"mutation total %.4f (%d threads, %.1f ops/sec)\\n\",\n            std::max(max_seconds_add, max_seconds_del),\n            n_threads_add + n_threads_del,\n            (n*2) / std::max(max_seconds_add, max_seconds_del));\n    TestSuite::appendResultMessage(msg);\n\n    if (!t_args.use_skiplist) {\n        return 0;\n    }\n\n    // integrity check (forward iteration, skiplist only)\n    std::chrono::time_point<std::chrono::system_clock> start, end;\n    std::chrono::duration<double> elapsed_seconds;\n    start = std::chrono::system_clock::now();\n\n    int count = 0;\n    bool corruption = false;\n    skiplist_node *cur = skiplist_begin(&list);\n    std::vector<skiplist_node*> dbg_node(n);\n    std::vector<IntNode*> dbg_int(n);\n\n    while (cur) {\n        IntNode *node = _get_entry(cur, IntNode, snode);\n        dbg_node[count] = cur;\n        dbg_int[count] = node;\n        // 5, 15, 25, 35 ...\n        int idx = count * 10 + 5;\n        if (node->value != idx) {\n            skiplist_node *missing = &arr_add_dbgref[count]->snode;\n            sprintf(msg, \"count %d, idx %d is missing %lx\\n\",\n                    count, idx, (uint64_t)missing);\n            TestSuite::appendResultMessage(msg);\n\n            skiplist_node *prev = skiplist_prev(&list, missing);\n            skiplist_node *next = skiplist_next(&list, missing);\n            IntNode *prev_node = _get_entry(prev, IntNode, snode);\n            IntNode *next_node = _get_entry(next, IntNode, snode);\n            sprintf(msg, \"%d %d\\n\", prev_node->value, next_node->value);\n            TestSuite::appendResultMessage(msg);\n\n            corruption = true;\n        }\n        CHK_EQ(idx, node->value);\n        cur = skiplist_next(&list, cur);\n        count++;\n    }\n    CHK_EQ(n, count);\n    CHK_NOT(corruption);\n\n    end = std::chrono::system_clock::now();\n    elapsed_seconds = end-start;\n\n    sprintf(msg, \"iteration %.4f (%.1f ops/sec)\\n\",\n            elapsed_seconds.count(), n/elapsed_seconds.count());\n    TestSuite::appendResultMessage(msg);\n\n    skiplist_free(&list);\n\n    return 0;\n}\n\nint concurrent_write_read_test(struct test_args t_args)\n{\n    char msg[1024];\n    TestSuite::appendResultMessage(\"\\n\");\n\n    skiplist_raw list;\n    std::mutex lock;\n    std::set<int> stl_set;\n\n    skiplist_init(&list, _cmp_IntNode);\n\n    int i, j, temp;\n    int n = t_args.n_keys;\n    std::vector<int> key_add(n);\n    std::vector<int> key_read(n);\n    std::vector<IntNode> arr_add(n);\n    std::vector<IntNode> arr_find(n);\n    std::vector<IntNode*> arr_add_dbgref(n);\n\n    // initial list state: 0, 10, 20, ...\n    //  => writer threads: adding 5, 15, 25, ...\n    //  => reader threads: reading 0, 10, 20, ...\n    // final list state: 0, 5, 10, 15, 20, ...\n\n    // initial insert\n    for (i=0; i<n; ++i) {\n        // 0, 10, 20, 30 ...\n        key_read[i] = i * 10;\n        arr_find[i].value = key_read[i];\n        skiplist_insert(&list, &arr_find[i].snode);\n    }\n\n    // assign keys to add\n    for (i=0; i<n; ++i) {\n        // 5, 15, 25, 35, ...\n        key_add[i] = i*10 + 5;\n    }\n\n    if (t_args.random_order) {\n        // shuffle keys to add\n        for (i=0; i<n; ++i) {\n            j = rand() % n;\n            temp = key_add[i];\n            key_add[i] = key_add[j];\n            key_add[j] = temp;\n        }\n    }\n    for (i=0; i<n; ++i) {\n        int ori_idx = (key_add[i] - 5) / 10;\n        arr_add[i].value = key_add[i];\n        arr_add_dbgref[ori_idx] = &arr_add[i];\n    }\n\n    if (t_args.random_order) {\n        // also shuffle keys to find\n        for (i=0; i<n; ++i) {\n            j = rand() % n;\n            temp = key_read[i];\n            key_read[i] = key_read[j];\n            key_read[j] = temp;\n        }\n    }\n\n    int n_threads_add = t_args.n_writers;\n    int n_threads_find = t_args.n_readers;\n\n    std::vector<TestSuite::ThreadHolder*> t_holder_add(n_threads_add);\n    std::vector<TestSuite::ThreadHolder*> t_holder_find(n_threads_find);\n    std::vector<thread_args> args_add(n_threads_add);\n    std::vector<thread_args> args_find(n_threads_find);\n\n    if (n_threads_add) {\n        int n_keys_per_thread_add = n / n_threads_add;\n\n        for (i=0; i<n_threads_add; ++i) {\n            args_add[i].list = &list;\n            args_add[i].stl_set = &stl_set;\n            args_add[i].lock = &lock;\n            args_add[i].key = &key_add;\n            args_add[i].arr = &arr_add;\n            args_add[i].range_begin = i*n_keys_per_thread_add;\n            args_add[i].range_end = (i+1)*n_keys_per_thread_add - 1;\n            args_add[i].use_skiplist = t_args.use_skiplist;\n\n            t_holder_add[i] = new TestSuite::ThreadHolder\n                              (&args_add[i], writer_thread, nullptr);\n        }\n    }\n\n    if (n_threads_find) {\n        int n_keys_per_thread_find = n / n_threads_find;\n\n        for (i=0; i<n_threads_find; ++i) {\n            args_find[i].list = &list;\n            args_find[i].stl_set = &stl_set;\n            args_find[i].lock = &lock;\n            args_find[i].key = &key_read;\n            args_find[i].arr = &arr_find;\n            args_find[i].range_begin = i*n_keys_per_thread_find;\n            args_find[i].range_end = (i+1)*n_keys_per_thread_find - 1;\n            args_find[i].use_skiplist = t_args.use_skiplist;\n\n            t_holder_find[i] = new TestSuite::ThreadHolder\n                               (&args_find[i], reader_thread, nullptr);\n        }\n    }\n\n    for (i=0; i<n_threads_add; ++i){\n        t_holder_add[i]->join();\n        delete t_holder_add[i];\n    }\n    for (i=0; i<n_threads_find; ++i){\n        t_holder_find[i]->join();\n        delete t_holder_find[i];\n    }\n\n    if (n_threads_add) {\n        double max_seconds_add = 0;\n        for (i=0; i<n_threads_add; ++i) {\n            if (args_add[i].elapsed_sec > max_seconds_add) {\n                max_seconds_add = args_add[i].elapsed_sec;\n            }\n        }\n        sprintf(msg, \"insertion %.4f (%d threads, %.1f ops/sec)\\n\",\n                max_seconds_add, n_threads_add, n / max_seconds_add);\n        TestSuite::appendResultMessage(msg);\n    }\n\n    if (n_threads_find) {\n        double max_seconds_find = 0;\n        for (i=0; i<n_threads_find; ++i) {\n            if (args_find[i].elapsed_sec > max_seconds_find) {\n                max_seconds_find = args_find[i].elapsed_sec;\n            }\n        }\n        sprintf(msg, \"retrieval %.4f (%d threads, %.1f ops/sec)\\n\",\n                max_seconds_find, n_threads_find, n / max_seconds_find);\n        TestSuite::appendResultMessage(msg);\n    }\n\n    skiplist_free(&list);\n\n    return 0;\n}\n\nint main(int argc, char** argv) {\n    TestSuite ts(argc, argv);\n    srand(0xabcd);\n\n    struct test_args args;\n    args.n_keys = 40000;\n    args.random_order = true;\n    args.use_skiplist = true;\n\n    //ts.options.printTestMessage = true;\n    ts.doTest(\"basic insert and erase\", basic_insert_and_erase);\n    ts.doTest(\"find test\", find_test);\n\n    args.n_writers = 8;\n    ts.doTest(\"concurrent write test\", concurrent_write_test, args);\n\n    args.n_writers = 4;\n    args.n_erasers = 4;\n    ts.doTest(\"concurrent write erase test\", concurrent_write_erase_test, args);\n\n    args.n_writers = 1;\n    args.n_readers = 7;\n    ts.doTest(\"concurrent write read test\", concurrent_write_read_test, args);\n\n    return 0;\n}\n\n\n"
  },
  {
    "path": "tests/stl_map_compare.cc",
    "content": "#include \"sl_map.h\"\n\n#include \"test_common.h\"\n\n#include <map>\n#include <mutex>\n#include <thread>\n#include <vector>\n\n#include <stdio.h>\n\nstruct thread_args {\n    thread_args() : mode(SKIPLIST), num(0), id(0), modulo(0),\n                    duration_ms(0), op_count(0), temp(0),\n                    sl(nullptr),\n                    stdmap(nullptr), lock(nullptr) {}\n\n    enum Mode {\n        SKIPLIST = 0,\n        MAP_MUTEX = 1,\n        MAP_ONLY = 2\n    } mode;\n    int num;\n    int id;\n    int modulo;\n    int duration_ms;\n    int op_count;\n    volatile uint64_t temp;\n    sl_map<int, int>* sl;\n    std::map<int, int>* stdmap;\n    std::mutex* lock;\n};\n\nsize_t num_primes(uint64_t number, size_t max_prime) {\n    size_t ret = 0;\n    for (size_t ii=2; ii<=max_prime; ++ii) {\n        if (number % ii == 0) {\n            number /= ii;\n            ret++;\n        }\n    }\n    return ret;\n}\n\nvoid reader(thread_args* args) {\n    TestSuite::Timer timer(args->duration_ms);\n    while (!timer.timeover()) {\n        int r = rand() % args->num;\n        int max_walks = 3;\n        int walks = 0;\n\n        if (args->mode == thread_args::SKIPLIST) {\n            auto itr = args->sl->find(r);\n            while (itr != args->sl->end()) {\n                uint64_t number = itr->second;\n                itr++;\n                args->temp += num_primes(number, 10000);\n                if (++walks >= max_walks) break;\n            }\n\n        } else if (args->mode == thread_args::MAP_MUTEX) {\n            std::lock_guard<std::mutex> l(*args->lock);\n            auto itr = args->stdmap->find(r);\n            while (itr != args->stdmap->end()) {\n                uint64_t number = itr->second;\n                itr++;\n                args->temp += num_primes(number, 10000);\n                if (++walks >= max_walks) break;\n            }\n\n        } else  {\n            auto itr = args->stdmap->find(r);\n            while (itr != args->stdmap->end()) {\n                uint64_t number = itr->second;\n                itr++;\n                args->temp += num_primes(number, 10000);\n                if (++walks >= max_walks) break;\n            }\n        }\n        args->op_count += max_walks;\n    }\n}\n\nvoid writer(thread_args* args) {\n    TestSuite::Timer timer(args->duration_ms);\n    while (!timer.timeover()) {\n        int r = rand() % (args->num / args->modulo);\n        r *= args->modulo;\n        r += args->id;\n\n        if (args->mode == thread_args::SKIPLIST) {\n            auto itr = args->sl->find(r);\n            if (itr == args->sl->end()) {\n                args->sl->insert(std::make_pair(r, r));\n            } else {\n                args->sl->erase(itr);\n            }\n\n        } else if (args->mode == thread_args::MAP_MUTEX) {\n            std::lock_guard<std::mutex> l(*args->lock);\n            auto itr = args->stdmap->find(r);\n            if (itr == args->stdmap->end()) {\n                args->stdmap->insert(std::make_pair(r, r));\n            } else {\n                args->stdmap->erase(itr);\n            }\n\n        } else  {\n            auto itr = args->stdmap->find(r);\n            if (itr == args->stdmap->end()) {\n                args->stdmap->insert(std::make_pair(r, r));\n            } else {\n                args->stdmap->erase(itr);\n            }\n        }\n        args->op_count++;\n    }\n}\n\nint concurrent_test(int mode) {\n    sl_map_gc<int, int> sl;\n    std::map<int, int> stdmap;\n    std::mutex lock;\n    int num = 10000000;\n    int duration_ms = 5000;\n\n    int num_readers = 4;\n    thread_args r_args[num_readers];\n    std::thread readers[num_readers];\n    for (int i=0; i<num_readers; ++i) {\n        r_args[i].mode = static_cast<thread_args::Mode>(mode);\n        r_args[i].num = num;\n        r_args[i].duration_ms = duration_ms;\n        r_args[i].sl = &sl;\n        r_args[i].stdmap = &stdmap;\n        r_args[i].lock = &lock;\n        readers[i] = std::thread(reader, &r_args[i]);\n    }\n    int num_writers = 4;\n    thread_args w_args[num_writers];\n    std::thread writers[num_writers];\n    for (int i=0; i<num_writers; ++i) {\n        w_args[i].mode = static_cast<thread_args::Mode>(mode);\n        w_args[i].num = num;\n        w_args[i].id = i;\n        w_args[i].modulo = num_writers;\n        w_args[i].duration_ms = duration_ms;\n        w_args[i].sl = &sl;\n        w_args[i].stdmap = &stdmap;\n        w_args[i].lock = &lock;\n        writers[i] = std::thread(writer, &w_args[i]);\n    }\n\n    int r_total = 0, w_total = 0;\n    for (int i=0; i<num_readers; ++i) {\n        readers[i].join();\n        r_total += r_args[i].op_count;\n    }\n    for (int i=0; i<num_writers; ++i) {\n        writers[i].join();\n        w_total += w_args[i].op_count;\n    }\n\n    printf(\"read: %.1f ops/sec\\n\"\n           \"write: %.1f ops/sec\\n\"\n           \"total: %.1f ops/sec\\n\",\n           (double)r_total * 1000.0 / duration_ms,\n           (double)w_total * 1000.0 / duration_ms,\n           (double)(r_total + w_total) * 1000.0 / duration_ms);\n\n    return 0;\n}\n\nint main(int argc, char** argv) {\n    TestSuite tt(argc, argv);\n\n    std::vector<int> params = {0, 1/*, 2*/};\n    tt.options.printTestMessage = true;\n    tt.doTest(\"concurrent access comparison test\", concurrent_test, TestRange<int>(params));\n\n    return 0;\n}\n"
  },
  {
    "path": "tests/test_common.h",
    "content": "/**\n * Copyright (C) 2017-present Jung-Sang Ahn <jungsang.ahn@gmail.com>\n * All rights reserved.\n *\n * https://github.com/greensky00\n *\n * Test Suite\n * Version: 0.1.65\n *\n * Permission is hereby granted, free of charge, to any person\n * obtaining a copy of this software and associated documentation\n * files (the \"Software\"), to deal in the Software without\n * restriction, including without limitation the rights to use,\n * copy, modify, merge, publish, distribute, sublicense, and/or sell\n * copies of the Software, and to permit persons to whom the\n * Software is furnished to do so, subject to the following\n * conditions:\n *\n * The above copyright notice and this permission notice shall be\n * included in all copies or substantial portions of the Software.\n *\n * THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES\n * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT\n * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,\n * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\n * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR\n * OTHER DEALINGS IN THE SOFTWARE.\n */\n\n#pragma once\n\n#include <cassert>\n#include <chrono>\n#include <cmath>\n#include <functional>\n#include <iostream>\n#include <iomanip>\n#include <memory>\n#include <mutex>\n#include <sstream>\n#include <string>\n#include <thread>\n#include <tuple>\n#include <type_traits>\n#include <vector>\n\n#include <stdio.h>\n#include <stdlib.h>\n#include <stdarg.h>\n#include <string.h>\n#include <sys/stat.h>\n#include <sys/types.h>\n\n#ifndef _CLM_DEFINED\n#define _CLM_DEFINED (1)\n\n#ifdef TESTSUITE_NO_COLOR\n    #define _CLM_D_GRAY     \"\"\n    #define _CLM_GREEN      \"\"\n    #define _CLM_B_GREEN    \"\"\n    #define _CLM_RED        \"\"\n    #define _CLM_B_RED      \"\"\n    #define _CLM_BROWN      \"\"\n    #define _CLM_B_BROWN    \"\"\n    #define _CLM_BLUE       \"\"\n    #define _CLM_B_BLUE     \"\"\n    #define _CLM_MAGENTA    \"\"\n    #define _CLM_B_MAGENTA  \"\"\n    #define _CLM_CYAN       \"\"\n    #define _CLM_END        \"\"\n\n    #define _CLM_WHITE_FG_RED_BG    \"\"\n#else\n    #define _CLM_D_GRAY     \"\\033[1;30m\"\n    #define _CLM_GREEN      \"\\033[32m\"\n    #define _CLM_B_GREEN    \"\\033[1;32m\"\n    #define _CLM_RED        \"\\033[31m\"\n    #define _CLM_B_RED      \"\\033[1;31m\"\n    #define _CLM_BROWN      \"\\033[33m\"\n    #define _CLM_B_BROWN    \"\\033[1;33m\"\n    #define _CLM_BLUE       \"\\033[34m\"\n    #define _CLM_B_BLUE     \"\\033[1;34m\"\n    #define _CLM_MAGENTA    \"\\033[35m\"\n    #define _CLM_B_MAGENTA  \"\\033[1;35m\"\n    #define _CLM_CYAN       \"\\033[36m\"\n    #define _CLM_B_GREY     \"\\033[1;37m\"\n    #define _CLM_END        \"\\033[0m\"\n\n    #define _CLM_WHITE_FG_RED_BG    \"\\033[37;41m\"\n#endif\n\n#define _CL_D_GRAY(str)     _CLM_D_GRAY     str _CLM_END\n#define _CL_GREEN(str)      _CLM_GREEN      str _CLM_END\n#define _CL_RED(str)        _CLM_RED        str _CLM_END\n#define _CL_B_RED(str)      _CLM_B_RED      str _CLM_END\n#define _CL_MAGENTA(str)    _CLM_MAGENTA    str _CLM_END\n#define _CL_BROWN(str)      _CLM_BROWN      str _CLM_END\n#define _CL_B_BROWN(str)    _CLM_B_BROWN    str _CLM_END\n#define _CL_B_BLUE(str)     _CLM_B_BLUE     str _CLM_END\n#define _CL_B_MAGENTA(str)  _CLM_B_MAGENTA  str _CLM_END\n#define _CL_CYAN(str)       _CLM_CYAN       str _CLM_END\n#define _CL_B_GRAY(str)     _CLM_B_GREY     str _CLM_END\n\n#define _CL_WHITE_FG_RED_BG(str)    _CLM_WHITE_FG_RED_BG    str _CLM_END\n\n#endif\n\n#define __COUT_STACK_INFO__                                                     \\\n       std::endl                                                                \\\n    << \"        time: \" << _CLM_D_GRAY <<                                       \\\n       TestSuite::getTimeString() << _CLM_END << \"\\n\"                           \\\n    << \"      thread: \" << _CLM_BROWN                                           \\\n    << std::hex << std::setw(4) << std::setfill('0') <<                         \\\n       (std::hash<std::thread::id>{}( std::this_thread::get_id() ) & 0xffff)    \\\n    << std::dec << _CLM_END << \"\\n\"                                             \\\n    << \"          in: \" << _CLM_CYAN << __func__ << \"()\" _CLM_END << \"\\n\"       \\\n    << \"          at: \" << _CLM_GREEN << __FILE__ << _CLM_END \":\"               \\\n    << _CLM_B_MAGENTA << __LINE__ << _CLM_END << \"\\n\"                           \\\n\n// exp_value == value\n#define CHK_EQ(exp_value, value)                                        \\\n{                                                                       \\\n    auto _ev = (exp_value);                                             \\\n    decltype(_ev) _v = (decltype(_ev))(value);                          \\\n    if (_ev != _v) {                                                    \\\n        std::cout                                                       \\\n        << __COUT_STACK_INFO__                                          \\\n        << \"    value of: \" _CLM_B_BLUE #value _CLM_END \"\\n\"            \\\n        << \"    expected: \" _CLM_B_GREEN << _ev << _CLM_END \"\\n\"        \\\n        << \"      actual: \" _CLM_B_RED << _v << _CLM_END \"\\n\";          \\\n        TestSuite::failHandler();                                       \\\n        return -1;                                                      \\\n    }                                                                   \\\n}\n\n// exp_value != value\n#define CHK_NEQ(exp_value, value)                                       \\\n{                                                                       \\\n    auto _ev = (exp_value);                                             \\\n    decltype(_ev) _v = (decltype(_ev))(value);                          \\\n    if (_ev == _v) {                                                    \\\n        std::cout                                                       \\\n        << __COUT_STACK_INFO__                                          \\\n        << \"    value of: \" _CLM_B_BLUE #value _CLM_END \"\\n\"            \\\n        << \"    expected: not \" _CLM_B_GREEN << _ev << _CLM_END \"\\n\"    \\\n        << \"      actual: \" _CLM_B_RED << _v << _CLM_END \"\\n\";          \\\n        TestSuite::failHandler();                                       \\\n        return -1;                                                      \\\n    }                                                                   \\\n}\n\n// value == true\n#define CHK_OK(value)                                                   \\\n    if (!(value)) {                                                     \\\n        std::cout                                                       \\\n        << __COUT_STACK_INFO__                                          \\\n        << \"    value of: \" _CLM_B_BLUE #value _CLM_END \"\\n\"            \\\n        << \"    expected: \" _CLM_B_GREEN << \"true\" << _CLM_END \"\\n\"     \\\n        << \"      actual: \" _CLM_B_RED << \"false\" << _CLM_END \"\\n\";     \\\n        TestSuite::failHandler();                                       \\\n        return -1;                                                      \\\n    }\n\n#define CHK_TRUE(value) CHK_OK(value)\n\n// value == false\n#define CHK_NOT(value)                                                  \\\n    if (value) {                                                        \\\n        std::cout                                                       \\\n        << __COUT_STACK_INFO__                                          \\\n        << \"    value of: \" _CLM_B_BLUE #value _CLM_END \"\\n\"            \\\n        << \"    expected: \" _CLM_B_GREEN << \"false\" << _CLM_END \"\\n\"    \\\n        << \"      actual: \" _CLM_B_RED << \"true\" << _CLM_END \"\\n\";      \\\n        TestSuite::failHandler();                                       \\\n        return -1;                                                      \\\n    }\n\n#define CHK_FALSE(value) CHK_NOT(value)\n\n// value == NULL\n#define CHK_NULL(value)                                                 \\\n{                                                                       \\\n    auto _v = (value);                                                  \\\n    if (_v) {                                                           \\\n        std::cout                                                       \\\n        << __COUT_STACK_INFO__                                          \\\n        << \"    value of: \" _CLM_B_BLUE #value _CLM_END \"\\n\"            \\\n        << \"    expected: \" _CLM_B_GREEN << \"NULL\" << _CLM_END \"\\n\";    \\\n        printf(\"      actual: \" _CLM_B_RED \"%p\" _CLM_END \"\\n\", _v);     \\\n        TestSuite::failHandler();                                       \\\n        return -1;                                                      \\\n    }                                                                   \\\n}\n\n// value != NULL\n#define CHK_NONNULL(value)                                              \\\n    if (!(value)) {                                                     \\\n        std::cout                                                       \\\n        << __COUT_STACK_INFO__                                          \\\n        << \"    value of: \" _CLM_B_BLUE #value _CLM_END \"\\n\"            \\\n        << \"    expected: \" _CLM_B_GREEN << \"non-NULL\" << _CLM_END \"\\n\" \\\n        << \"      actual: \" _CLM_B_RED << \"NULL\" << _CLM_END \"\\n\";      \\\n        TestSuite::failHandler();                                       \\\n        return -1;                                                      \\\n    }\n\n// value == 0\n#define CHK_Z(value)                                                    \\\n{                                                                       \\\n    auto _v = (value);                                                  \\\n    if ((0) != _v) {                                                    \\\n        std::cout                                                       \\\n        << __COUT_STACK_INFO__                                          \\\n        << \"    value of: \" _CLM_B_BLUE #value _CLM_END \"\\n\"            \\\n        << \"    expected: \" _CLM_B_GREEN << \"0\" << _CLM_END \"\\n\"        \\\n        << \"      actual: \" _CLM_B_RED << _v << _CLM_END \"\\n\";          \\\n        TestSuite::failHandler();                                       \\\n        return -1;                                                      \\\n    }                                                                   \\\n}\n\n// smaller < greater\n#define CHK_SM(smaller, greater)                                \\\n{                                                               \\\n    auto _sm = (smaller);                                       \\\n    decltype(_sm) _gt = (decltype(_sm))(greater);               \\\n    if (!(_sm < _gt)) {                                         \\\n        std::cout                                               \\\n        << __COUT_STACK_INFO__                                  \\\n        << \"    expected: \"                                     \\\n        << _CLM_B_BLUE #smaller \" < \" #greater _CLM_END \"\\n\"    \\\n        << \"    value of \"                                      \\\n        << _CLM_B_GREEN #smaller _CLM_END \": \"                  \\\n        << _CLM_B_RED << _sm << _CLM_END \"\\n\"                   \\\n        << \"    value of \"                                      \\\n        << _CLM_B_GREEN #greater _CLM_END \": \"                  \\\n        << _CLM_B_RED << _gt << _CLM_END \"\\n\";                  \\\n        TestSuite::failHandler();                               \\\n        return -1;                                              \\\n    }                                                           \\\n}\n\n// smaller <= greater\n#define CHK_SMEQ(smaller , greater)                             \\\n{                                                               \\\n    auto _sm = (smaller);                                       \\\n    decltype(_sm) _gt = (decltype(_sm))(greater);               \\\n    if (!(_sm <= _gt)) {                                        \\\n        std::cout                                               \\\n        << __COUT_STACK_INFO__                                  \\\n        << \"    expected: \"                                     \\\n        << _CLM_B_BLUE #smaller \" <= \" #greater _CLM_END \"\\n\"   \\\n        << \"    value of \"                                      \\\n        << _CLM_B_GREEN #smaller _CLM_END \": \"                  \\\n        << _CLM_B_RED << _sm << _CLM_END \"\\n\"                   \\\n        << \"    value of \"                                      \\\n        << _CLM_B_GREEN #greater _CLM_END \": \"                  \\\n        << _CLM_B_RED << _gt << _CLM_END \"\\n\";                  \\\n        TestSuite::failHandler();                               \\\n        return -1;                                              \\\n    }                                                           \\\n}\n\n// greater > smaller\n#define CHK_GT(greater, smaller)                                \\\n{                                                               \\\n    auto _sm = (smaller);                                       \\\n    decltype(_sm) _gt = (decltype(_sm))(greater);               \\\n    if (!(_gt > _sm)) {                                         \\\n        std::cout                                               \\\n        << __COUT_STACK_INFO__                                  \\\n        << \"    expected: \"                                     \\\n        << _CLM_B_BLUE #greater \" > \" #smaller _CLM_END \"\\n\"    \\\n        << \"    value of \"                                      \\\n        << _CLM_B_GREEN #greater _CLM_END \": \"                  \\\n        << _CLM_B_RED << _gt << _CLM_END \"\\n\"                   \\\n        << \"    value of \"                                      \\\n        << _CLM_B_GREEN #smaller _CLM_END \": \"                  \\\n        << _CLM_B_RED << _sm << _CLM_END \"\\n\";                  \\\n        TestSuite::failHandler();                               \\\n        return -1;                                              \\\n    }                                                           \\\n}\n\n// greater >= smaller\n#define CHK_GTEQ(greater, smaller)                              \\\n{                                                               \\\n    auto _sm = (smaller);                                       \\\n    decltype(_sm) _gt = (decltype(_sm))(greater);               \\\n    if (!(_gt >= _sm)) {                                        \\\n        std::cout                                               \\\n        << __COUT_STACK_INFO__                                  \\\n        << \"    expected: \"                                     \\\n        << _CLM_B_BLUE #greater \" >= \" #smaller _CLM_END \"\\n\"   \\\n        << \"    value of \"                                      \\\n        << _CLM_B_GREEN #greater _CLM_END \": \"                  \\\n        << _CLM_B_RED << _gt << _CLM_END \"\\n\"                   \\\n        << \"    value of \"                                      \\\n        << _CLM_B_GREEN #smaller _CLM_END \": \"                  \\\n        << _CLM_B_RED << _sm << _CLM_END \"\\n\";                  \\\n        TestSuite::failHandler();                               \\\n        return -1;                                              \\\n    }                                                           \\\n}\n\n\nusing test_func = std::function<int()>;\n\nclass TestArgsBase;\nusing test_func_args = std::function<int(TestArgsBase*)>;\n\nclass TestSuite;\nclass TestArgsBase {\npublic:\n    virtual ~TestArgsBase() { }\n    void setCallback(std::string test_name,\n                     test_func_args func,\n                     TestSuite* test_instance) {\n        testName = test_name;\n        testFunction = func;\n        testInstance = test_instance;\n    }\n    void testAll() { testAllInternal(0); }\n    virtual void setParam(size_t param_no, size_t param_idx) = 0;\n    virtual size_t getNumSteps(size_t param_no) = 0;\n    virtual size_t getNumParams() = 0;\n    virtual std::string toString() = 0;\n\nprivate:\n    inline void testAllInternal(size_t depth);\n    std::string testName;\n    test_func_args testFunction;\n    TestSuite* testInstance;\n};\n\nclass TestArgsWrapper {\npublic:\n    TestArgsWrapper(TestArgsBase* _test_args) : test_args(_test_args) {}\n    ~TestArgsWrapper() { delete test_args; }\n    TestArgsBase* getArgs() const { return test_args; }\n    operator TestArgsBase*() const { return getArgs(); }\nprivate:\n    TestArgsBase* test_args;\n};\n\nenum class StepType {\n    LINEAR,\n    EXPONENTIAL\n};\n\ntemplate<typename T>\nclass TestRange {\npublic:\n    TestRange() : type(RangeType::NONE), begin(), end(), step() {}\n\n    // Constructor for given values\n    TestRange(const std::vector<T>& _array)\n        : type(RangeType::ARRAY), array(_array)\n        , begin(), end(), step() {}\n\n    // Constructor for regular steps\n    TestRange(T _begin, T _end, T _step, StepType _type)\n        : begin(_begin), end(_end), step(_step)\n    {\n        if (_type == StepType::LINEAR) {\n            type = RangeType::LINEAR;\n        } else {\n            type = RangeType::EXPONENTIAL;\n        }\n    }\n\n    T getEntry(size_t idx) {\n        if (type == RangeType::ARRAY) {\n            return array[idx];\n        } else if (type == RangeType::LINEAR) {\n            return begin + step * idx;\n        } else if (type == RangeType::EXPONENTIAL) {\n            ssize_t _begin = begin;\n            ssize_t _step = step;\n            ssize_t _ret = _begin * std::pow(_step, idx);\n            return (T)(_ret);\n        }\n\n        return begin;\n    }\n\n    size_t getSteps() {\n        if (type == RangeType::ARRAY) {\n            return array.size();\n        } else if (type == RangeType::LINEAR) {\n            return ((end - begin) / step) + 1;\n        } else if (type == RangeType::EXPONENTIAL) {\n            ssize_t coe = ((ssize_t)end) / ((ssize_t)begin);\n            double steps_double = (double)std::log(coe) / std::log(step);\n            return steps_double + 1;\n        }\n\n        return 0;\n    }\n\nprivate:\n    enum class RangeType {\n        NONE,\n        ARRAY,\n        LINEAR,\n        EXPONENTIAL\n    };\n\n    RangeType type;\n    std::vector<T> array;\n    T begin;\n    T end;\n    T step;\n};\n\nstruct TestOptions {\n    TestOptions()\n        : printTestMessage(false)\n        , abortOnFailure(false)\n        , preserveTestFiles(false)\n        {}\n    bool printTestMessage;\n    bool abortOnFailure;\n    bool preserveTestFiles;\n};\n\nclass TestSuite {\n    friend TestArgsBase;\nprivate:\n    static std::mutex& getResMsgLock() {\n        static std::mutex res_msg_lock;\n        return res_msg_lock;\n    }\n    static std::string& getResMsg() {\n        static std::string res_msg;\n        return res_msg;\n    }\n    static std::string& getInfoMsg() {\n        thread_local std::string info_msg;\n        return info_msg;\n    }\n    static std::string& getTestName() {\n        static std::string test_name;\n        return test_name;\n    }\n    static TestSuite*& getCurTest() {\n        static TestSuite* cur_test;\n        return cur_test;\n    }\npublic:\n    static bool& globalMsgFlag() {\n        static bool global_msg_flag = false;\n        return global_msg_flag;\n    }\n    static std::string getCurrentTestName() {\n        return getTestName();\n    }\n    static bool isMsgAllowed() {\n        TestSuite* cur_test = TestSuite::getCurTest();\n        if ( cur_test &&\n             (cur_test->options.printTestMessage || cur_test->displayMsg) &&\n             !cur_test->suppressMsg ) {\n            return true;\n        }\n        if (globalMsgFlag()) return true;\n        return false;\n    }\n\n    static void setInfo(const char* format, ...) {\n        thread_local char info_buf[4096];\n        size_t len = 0;\n        va_list args;\n        va_start(args, format);\n        len += vsnprintf(info_buf + len, 4096 - len, format, args);\n        va_end(args);\n        getInfoMsg() = info_buf;\n    }\n    static void clearInfo() {\n        getInfoMsg().clear();\n    }\n\n    static void failHandler() {\n        if (!getInfoMsg().empty()) {\n            std::cout << \"        info: \" << getInfoMsg() << std::endl;\n        }\n    }\n\n    static void usage(int argc, char** argv) {\n        printf(\"\\n\");\n        printf(\"Usage: %s [-f <keyword>] [-r <parameter>] [-p]\\n\", argv[0]);\n        printf(\"\\n\");\n        printf(\"    -f, --filter\\n\");\n        printf(\"        Run specific tests matching the given keyword.\\n\");\n        printf(\"    -r, --range\\n\");\n        printf(\"        Run TestRange-based tests using given parameter value.\\n\");\n        printf(\"    -p, --preserve\\n\");\n        printf(\"        Do not clean up test files.\\n\");\n        printf(\"    --abort-on-failure\\n\");\n        printf(\"        Immediately abort the test if failure happens.\\n\");\n        printf(\"    --suppress-msg\\n\");\n        printf(\"        Suppress test messages.\\n\");\n        printf(\"    --display-msg\\n\");\n        printf(\"        Display test messages.\\n\");\n        printf(\"\\n\");\n    }\n\n    static std::string usToString(uint64_t us) {\n        std::stringstream ss;\n        if (us < 1000) {\n            // us\n            ss << std::fixed << std::setprecision(0) << us << \" us\";\n        } else if (us < 1000000) {\n            // ms\n            double tmp = static_cast<double>(us / 1000.0);\n            ss << std::fixed << std::setprecision(1) << tmp << \" ms\";\n        } else if (us < (uint64_t)600 * 1000000) {\n            // second: 1 s -- 600 s (10 mins)\n            double tmp = static_cast<double>(us / 1000000.0);\n            ss << std::fixed << std::setprecision(1) << tmp << \" s\";\n        } else {\n            // minute\n            double tmp = static_cast<double>(us / 60.0 / 1000000.0);\n            ss << std::fixed << std::setprecision(0) << tmp << \" m\";\n        }\n        return ss.str();\n    }\n\n    static std::string countToString(uint64_t count) {\n        std::stringstream ss;\n        if (count < 1000) {\n            ss << count;\n        } else if (count < 1000000) {\n            // K\n            double tmp = static_cast<double>(count / 1000.0);\n            ss << std::fixed << std::setprecision(1) << tmp << \"K\";\n        } else if (count < (uint64_t)1000000000) {\n            // M\n            double tmp = static_cast<double>(count / 1000000.0);\n            ss << std::fixed << std::setprecision(1) << tmp << \"M\";\n        } else {\n            // B\n            double tmp = static_cast<double>(count / 1000000000.0);\n            ss << std::fixed << std::setprecision(1) << tmp << \"B\";\n        }\n        return ss.str();\n    }\n\n    static std::string sizeToString(uint64_t size) {\n        std::stringstream ss;\n        if (size < 1024) {\n            ss << size << \" B\";\n        } else if (size < 1024*1024) {\n            // K\n            double tmp = static_cast<double>(size / 1024.0);\n            ss << std::fixed << std::setprecision(1) << tmp << \" KiB\";\n        } else if (size < (uint64_t)1024*1024*1024) {\n            // M\n            double tmp = static_cast<double>(size / 1024.0 / 1024.0);\n            ss << std::fixed << std::setprecision(1) << tmp << \" MiB\";\n        } else {\n            // B\n            double tmp = static_cast<double>(size / 1024.0 / 1024.0 / 1024.0);\n            ss << std::fixed << std::setprecision(1) << tmp << \" GiB\";\n        }\n        return ss.str();\n    }\n\nprivate:\n    struct TimeInfo {\n        TimeInfo(std::tm* src)\n            : year(src->tm_year + 1900)\n            , month(src->tm_mon + 1)\n            , day(src->tm_mday)\n            , hour(src->tm_hour)\n            , min(src->tm_min)\n            , sec(src->tm_sec)\n            , msec(0)\n            , usec(0) {}\n        TimeInfo(std::chrono::system_clock::time_point now) {\n            std::time_t raw_time = std::chrono::system_clock::to_time_t(now);\n            std::tm* lt_tm = std::localtime(&raw_time);\n            year = lt_tm->tm_year + 1900;\n            month = lt_tm->tm_mon + 1;\n            day = lt_tm->tm_mday;\n            hour = lt_tm->tm_hour;\n            min = lt_tm->tm_min;\n            sec = lt_tm->tm_sec;\n\n            size_t us_epoch = std::chrono::duration_cast\n                              < std::chrono::microseconds >\n                              ( now.time_since_epoch() ).count();\n            msec = (us_epoch / 1000) % 1000;\n            usec = us_epoch % 1000;\n        }\n        int year;\n        int month;\n        int day;\n        int hour;\n        int min;\n        int sec;\n        int msec;\n        int usec;\n    };\n\npublic:\n    TestSuite(int argc = 0, char **argv = nullptr)\n        : cntPass(0)\n        , cntFail(0)\n        , useGivenRange(false)\n        , preserveTestFiles(false)\n        , forceAbortOnFailure(false)\n        , suppressMsg(false)\n        , displayMsg(false)\n        , givenRange(0)\n        , startTimeGlobal(std::chrono::system_clock::now())\n    {\n        for (int ii=1; ii<argc; ++ii) {\n            // Filter.\n            if ( ii < argc-1 &&\n                 (!strcmp(argv[ii], \"-f\") ||\n                  !strcmp(argv[ii], \"--filter\")) ) {\n                filter = argv[++ii];\n            }\n\n            // Range.\n            if ( ii < argc-1 &&\n                 (!strcmp(argv[ii], \"-r\") ||\n                  !strcmp(argv[ii], \"--range\")) ) {\n                givenRange = atoi(argv[++ii]);\n                useGivenRange = true;\n            }\n\n            // Do not clean up test files after test.\n            if ( !strcmp(argv[ii], \"-p\") ||\n                 !strcmp(argv[ii], \"--preserve\") ) {\n                preserveTestFiles = true;\n            }\n\n            // Force abort on failure.\n            if ( !strcmp(argv[ii], \"--abort-on-failure\") ) {\n                forceAbortOnFailure = true;\n            }\n\n            // Suppress test messages.\n            if ( !strcmp(argv[ii], \"--suppress-msg\") ) {\n                suppressMsg = true;\n            }\n\n            // Display test messages.\n            if ( !strcmp(argv[ii], \"--display-msg\") && !suppressMsg ) {\n                displayMsg = true;\n            }\n\n            // Help\n            if ( !strcmp(argv[ii], \"-h\") ||\n                 !strcmp(argv[ii], \"--help\") ) {\n                usage(argc, argv);\n                exit(0);\n            }\n        }\n    }\n\n    ~TestSuite() {\n        std::chrono::time_point<std::chrono::system_clock> cur_time =\n                std::chrono::system_clock::now();;\n        std::chrono::duration<double> elapsed = cur_time - startTimeGlobal;\n        std::string time_str = usToString(elapsed.count() * 1000000);\n\n        printf(_CL_GREEN(\"%zu\") \" tests passed\", cntPass);\n        if (cntFail) {\n            printf(\", \" _CL_RED(\"%zu\") \" tests failed\", cntFail);\n        }\n        printf(\" out of \" _CL_CYAN(\"%zu\") \" (\" _CL_BROWN(\"%s\") \")\\n\",\n               cntPass+cntFail, time_str.c_str());\n    }\n\n    // === Helper functions ====================================\n    static std::string getTestFileName(const std::string& prefix) {\n        TimeInfo lt(std::chrono::system_clock::now());\n        (void)lt;\n\n        char time_char[64];\n        sprintf(time_char, \"%04d%02d%02d_%02d%02d%02d\",\n                lt.year, lt.month, lt.day, lt.hour, lt.min, lt.sec);\n\n        std::string ret = prefix;\n        ret += \"_\";\n        ret += time_char;\n        return ret;\n    }\n\n    static std::string getTimeString() {\n        TimeInfo lt(std::chrono::system_clock::now());\n        char time_char[64];\n        sprintf(time_char, \"%04d-%02d-%02d %02d:%02d:%02d.%03d%03d\",\n                lt.year, lt.month, lt.day, lt.hour, lt.min, lt.sec, lt.msec, lt.usec);\n        return time_char;\n    }\n    static std::string getTimeStringShort() {\n        TimeInfo lt(std::chrono::system_clock::now());\n        char time_char[64];\n        sprintf(time_char, \"%02d:%02d.%03d %03d\",\n                lt.min, lt.sec, lt.msec, lt.usec);\n        return time_char;\n    }\n    static std::string getTimeStringPlain() {\n        TimeInfo lt(std::chrono::system_clock::now());\n        char time_char[64];\n        sprintf(time_char, \"%02d%02d_%02d%02d%02d\",\n                lt.month, lt.day, lt.hour, lt.min, lt.sec);\n        return time_char;\n    }\n\n    static int mkdir(const std::string& path) {\n        struct stat st;\n        if (stat(path.c_str(), &st) != 0) {\n            return ::mkdir(path.c_str(), 0755);\n        }\n        return 0;\n    }\n    static int copyfile(const std::string& src,\n                        const std::string& dst) {\n        std::string cmd = \"cp -R \" + src + \" \" + dst;\n        int rc = ::system(cmd.c_str());\n        return rc;\n    }\n    static int remove(const std::string& path) {\n        int rc = ::remove(path.c_str());\n        return rc;\n    }\n\n    enum TestPosition {\n        BEGINNING_OF_TEST   = 0,\n        MIDDLE_OF_TEST      = 1,\n        END_OF_TEST         = 2,\n    };\n    static void clearTestFile( const std::string& prefix,\n                               TestPosition test_pos = MIDDLE_OF_TEST ) {\n        TestSuite*& cur_test = TestSuite::getCurTest();\n        if ( test_pos == END_OF_TEST &&\n             ( cur_test->preserveTestFiles ||\n               cur_test->options.preserveTestFiles ) ) return;\n\n        int r;\n        std::string command = \"rm -rf \";\n        command += prefix;\n        command += \"*\";\n        r = system(command.c_str());\n        (void)r;\n    }\n\n    static void setResultMessage(const std::string& msg) {\n        TestSuite::getResMsg() = msg;\n    }\n\n    static void appendResultMessage(const std::string& msg) {\n        std::lock_guard<std::mutex> l(TestSuite::getResMsgLock());\n        TestSuite::getResMsg() += msg;\n    }\n\n    static size_t _msg(const char* format, ...) {\n        size_t cur_len = 0;\n        TestSuite* cur_test = TestSuite::getCurTest();\n        if ( ( cur_test &&\n               (cur_test->options.printTestMessage || cur_test->displayMsg) &&\n               !cur_test->suppressMsg ) ||\n             globalMsgFlag() ) {\n            va_list args;\n            va_start(args, format);\n            cur_len += vprintf(format, args);\n            va_end(args);\n        }\n        return cur_len;\n    }\n    static size_t _msgt(const char* format, ...) {\n        size_t cur_len = 0;\n        TestSuite* cur_test = TestSuite::getCurTest();\n        if ( ( cur_test &&\n               (cur_test->options.printTestMessage || cur_test->displayMsg) &&\n               !cur_test->suppressMsg ) ||\n             globalMsgFlag() ) {\n            std::cout << _CLM_D_GRAY\n                      << getTimeStringShort() << _CLM_END << \"] \";\n            va_list args;\n            va_start(args, format);\n            cur_len += vprintf(format, args);\n            va_end(args);\n        }\n        return cur_len;\n    }\n\n    class Msg {\n    public:\n        Msg() {}\n\n        template<typename T>\n        inline Msg& operator<<(const T& data) {\n            if (TestSuite::isMsgAllowed()) {\n                std::cout << data;\n            }\n            return *this;\n        }\n\n        using MyCout = std::basic_ostream< char, std::char_traits<char> >;\n        typedef MyCout& (*EndlFunc)(MyCout&);\n\n        Msg& operator<<(EndlFunc func) {\n            if (TestSuite::isMsgAllowed()) {\n                func(std::cout);\n            }\n            return *this;\n        }\n    };\n\n    static void sleep_us(size_t us, const std::string& msg = std::string()) {\n        if (!msg.empty()) TestSuite::_msg(\"%s (%zu us)\\n\", msg.c_str(), us);\n        std::this_thread::sleep_for(std::chrono::microseconds(us));\n    }\n    static void sleep_ms(size_t ms, const std::string& msg = std::string()) {\n        if (!msg.empty()) TestSuite::_msg(\"%s (%zu ms)\\n\", msg.c_str(), ms);\n        std::this_thread::sleep_for(std::chrono::milliseconds(ms));\n    }\n    static void sleep_sec(size_t sec, const std::string& msg = std::string()) {\n        if (!msg.empty()) TestSuite::_msg(\"%s (%zu s)\\n\", msg.c_str(), sec);\n        std::this_thread::sleep_for(std::chrono::seconds(sec));\n    }\n    static std::string lzStr(size_t digit, uint64_t num) {\n        std::stringstream ss;\n        ss << std::setw(digit) << std::setfill('0') << std::to_string(num);\n        return ss.str();\n    }\n    static double calcThroughput(uint64_t ops, uint64_t elapsed_us) {\n        return ops * 1000000.0 / elapsed_us;\n    }\n    static std::string throughputStr(uint64_t ops, uint64_t elapsed_us) {\n        return countToString(ops * 1000000.0 / elapsed_us);\n    }\n    static std::string sizeThroughputStr(uint64_t size_byte, uint64_t elapsed_us) {\n        return sizeToString(size_byte * 1000000.0 / elapsed_us);\n    }\n\n    // === Timer things ====================================\n    class Timer {\n    public:\n        Timer() : duration_ms(0) {\n            reset();\n        }\n        Timer(size_t _duration_ms) : duration_ms(_duration_ms) {\n            reset();\n        }\n        inline bool timeout() { return timeover(); }\n        bool timeover() {\n            auto cur = std::chrono::system_clock::now();\n            std::chrono::duration<double> elapsed = cur - start;\n            if (duration_ms < elapsed.count() * 1000) return true;\n            return false;\n        }\n        uint64_t getTimeSec() {\n            auto cur = std::chrono::system_clock::now();\n            std::chrono::duration<double> elapsed = cur - start;\n            return (uint64_t)(elapsed.count());\n        }\n        uint64_t getTimeMs() {\n            auto cur = std::chrono::system_clock::now();\n            std::chrono::duration<double> elapsed = cur - start;\n            return (uint64_t)(elapsed.count() * 1000);\n        }\n        uint64_t getTimeUs() {\n            auto cur = std::chrono::system_clock::now();\n            std::chrono::duration<double> elapsed = cur - start;\n            return (uint64_t)(elapsed.count() * 1000000);\n        }\n        void reset() {\n            start = std::chrono::system_clock::now();\n        }\n        void resetSec(size_t _duration_sec) {\n            duration_ms = _duration_sec * 1000;\n            reset();\n        }\n        void resetMs(size_t _duration_ms) {\n            duration_ms = _duration_ms;\n            reset();\n        }\n    private:\n        std::chrono::time_point<std::chrono::system_clock> start;\n        size_t duration_ms;\n    };\n\n    // === Workload generator things ====================================\n    class WorkloadGenerator {\n    public:\n        WorkloadGenerator(double ops_per_sec = 0.0, uint64_t max_ops_per_batch = 0)\n            : opsPerSec(ops_per_sec)\n            , maxOpsPerBatch(max_ops_per_batch)\n            , numOpsDone(0) {\n            reset();\n        }\n        void reset() {\n            start = std::chrono::system_clock::now();\n            numOpsDone = 0;\n        }\n        size_t getNumOpsToDo() {\n            if (opsPerSec <= 0) return 0;\n\n            auto cur = std::chrono::system_clock::now();\n            std::chrono::duration<double> elapsed = cur - start;\n\n            double exp = opsPerSec * elapsed.count();\n            if (numOpsDone < exp) {\n                if (maxOpsPerBatch) {\n                    return std::min(maxOpsPerBatch, (uint64_t)exp - numOpsDone);\n                }\n                return (uint64_t)exp - numOpsDone;\n            }\n            return 0;\n        }\n        void addNumOpsDone(size_t num) {\n            numOpsDone += num;\n        }\n    private:\n        std::chrono::time_point<std::chrono::system_clock> start;\n        double opsPerSec;\n        uint64_t maxOpsPerBatch;\n        uint64_t numOpsDone;\n    };\n\n    // === Progress things ==================================\n    // Progress that knows the maximum value.\n    class Progress {\n    public:\n        Progress(uint64_t _num,\n                 const std::string& _comment = std::string(),\n                 const std::string& _unit = std::string())\n            : curValue(0)\n            , num(_num)\n            , timer(0)\n            , lastPrintTimeUs(timer.getTimeUs())\n            , comment(_comment)\n            , unit(_unit) {}\n        void update(uint64_t cur) {\n            curValue = cur;\n            uint64_t curTimeUs = timer.getTimeUs();\n            if (curTimeUs - lastPrintTimeUs > 50000 ||\n                cur == 0 || curValue >= num) {\n                // Print every 0.05 sec (20 Hz).\n                lastPrintTimeUs = curTimeUs;\n                std::string _comment =\n                    (comment.empty()) ? \"\" : comment + \": \";\n                std::string _unit =\n                    (unit.empty()) ? \"\" : unit + \" \";\n\n                _msg(\"\\r%s%ld/%ld %s(%.1f%%)\",\n                     _comment.c_str(), curValue, num, _unit.c_str(),\n                     (double)curValue*100/num);\n                fflush(stdout);\n            }\n            if (curValue >= num) {\n                _msg(\"\\n\");\n                fflush(stdout);\n            }\n        }\n        void done() { if (curValue < num) update(num); }\n    private:\n        uint64_t curValue;\n        uint64_t num;\n        Timer timer;\n        uint64_t lastPrintTimeUs;\n        std::string comment;\n        std::string unit;\n    };\n\n    // Progress that doesn't know the maximum value.\n    class UnknownProgress {\n    public:\n        UnknownProgress(const std::string& _comment = std::string(),\n                        const std::string& _unit = std::string())\n            : curValue(0)\n            , timer(0)\n            , lastPrintTimeUs(timer.getTimeUs())\n            , comment(_comment)\n            , unit(_unit) {}\n        void update(uint64_t cur) {\n            curValue = cur;\n            uint64_t curTimeUs = timer.getTimeUs();\n            if ( curTimeUs - lastPrintTimeUs > 50000 ||\n                 cur == 0 ) {\n                // Print every 0.05 sec (20 Hz).\n                lastPrintTimeUs = curTimeUs;\n                std::string _comment =\n                    (comment.empty()) ? \"\" : comment + \": \";\n                std::string _unit =\n                    (unit.empty()) ? \"\" : unit + \" \";\n\n                _msg(\"\\r%s%ld %s\", _comment.c_str(), curValue, _unit.c_str());\n                fflush(stdout);\n            }\n        }\n        void done() {\n            _msg(\"\\n\");\n            fflush(stdout);\n        }\n    private:\n        uint64_t curValue;\n        Timer timer;\n        uint64_t lastPrintTimeUs;\n        std::string comment;\n        std::string unit;\n    };\n\n    // === Displayer things ==================================\n    class Displayer {\n    public:\n        Displayer(size_t num_raws, size_t num_cols)\n            : numRaws(num_raws)\n            , numCols(num_cols)\n            , colWidth(num_cols, 20)\n            , context(num_raws, std::vector<std::string>(num_cols)) {}\n        void init() {\n            for (size_t ii=0; ii<numRaws; ++ii) _msg(\"\\n\");\n        }\n        void setWidth(std::vector<size_t>& src) {\n            size_t num_src = src.size();\n            if (!num_src) return;\n\n            for (size_t ii=0; ii<num_src; ++ii) {\n                colWidth[ii] = src[ii];\n            }\n            for (size_t ii=num_src; ii<numCols; ++ii) {\n                colWidth[ii] = src[num_src-1];\n            }\n        }\n        void set(size_t raw_idx, size_t col_idx, const char* format, ...) {\n            if (raw_idx >= numRaws || col_idx >= numCols) return;\n\n            thread_local char info_buf[32];\n            size_t len = 0;\n            va_list args;\n            va_start(args, format);\n            len += vsnprintf(info_buf + len, 20 - len, format, args);\n            va_end(args);\n            context[raw_idx][col_idx] = info_buf;\n        }\n        void print() {\n            _msg(\"\\033[%zuA\", numRaws);\n            for (size_t ii=0; ii<numRaws; ++ii) {\n                std::stringstream ss;\n                for (size_t jj=0; jj<numCols; ++jj) {\n                    ss << std::setw(colWidth[jj]) << context[ii][jj];\n                }\n                _msg(\"\\r%s\\n\", ss.str().c_str());\n            }\n        }\n    private:\n        size_t numRaws;\n        size_t numCols;\n        std::vector<size_t> colWidth;\n        std::vector< std::vector< std::string > > context;\n    };\n\n    // === Gc things ====================================\n    template<typename T, typename T2 = T>\n    class GcVar {\n    public:\n        GcVar(T& _src, T2 _to)\n            : src(_src), to(_to) {}\n        ~GcVar() {\n            // GC by value.\n            src = to;\n        }\n    private:\n        T& src;\n        T2 to;\n    };\n\n    class GcFunc {\n    public:\n        GcFunc(std::function<void()> _func)\n            : func(_func) {}\n        ~GcFunc() {\n            // GC by function.\n            func();\n        }\n    private:\n        std::function<void()> func;\n    };\n\n    // === Thread things ====================================\n    struct ThreadArgs { /* Opaque. */ };\n    using ThreadFunc = std::function< int(ThreadArgs*) >;\n    using ThreadExitHandler = std::function< void(ThreadArgs*) >;\n\nprivate:\n    struct ThreadInternalArgs {\n        ThreadInternalArgs() : userArgs(nullptr), func(nullptr), rc(0) {}\n        ThreadArgs* userArgs;\n        ThreadFunc func;\n        int rc;\n    };\n\npublic:\n    struct ThreadHolder {\n        ThreadHolder() : tid(nullptr), handler(nullptr) {}\n        ThreadHolder(std::thread* _tid, ThreadExitHandler _handler)\n            : tid(_tid), handler(_handler) {}\n        ThreadHolder(ThreadArgs* u_args,\n                     ThreadFunc t_func,\n                     ThreadExitHandler t_handler)\n            : tid(nullptr), handler(nullptr)\n        { spawn(u_args, t_func, t_handler); }\n\n        ~ThreadHolder() { join(true); }\n\n        void spawn(ThreadArgs* u_args,\n                   ThreadFunc t_func,\n                   ThreadExitHandler t_handler) {\n            if (tid) return;\n            handler = t_handler;\n            args.userArgs = u_args;\n            args.func = t_func;\n            tid = new std::thread(spawnThread, &args);\n        }\n\n        void join(bool force = false) {\n            if (!tid) return;\n            if (tid->joinable()) {\n                if (force) {\n                    // Force kill.\n                    handler(args.userArgs);\n                }\n                tid->join();\n            }\n            delete tid;\n            tid = nullptr;\n        }\n        int getResult() const { return args.rc; }\n        std::thread* tid;\n        ThreadExitHandler handler;\n        ThreadInternalArgs args;\n    };\n\n\n    // === doTest things ====================================\n\n    // 1) Without parameter.\n    void doTest( const std::string& test_name,\n                 test_func func )\n    {\n        if (!matchFilter(test_name)) return;\n\n        readyTest(test_name);\n        TestSuite::getResMsg() = \"\";\n        TestSuite::getInfoMsg() = \"\";\n        TestSuite::getCurTest() = this;\n        int ret = func();\n        reportTestResult(test_name, ret);\n    }\n\n    // 2) Ranged parameter.\n    template<typename T, typename F>\n    void doTest( std::string test_name,\n                 F func,\n                 TestRange<T> range )\n    {\n        if (!matchFilter(test_name)) return;\n\n        size_t n = (useGivenRange) ? 1 : range.getSteps();\n        size_t i;\n\n        for (i=0; i<n; ++i) {\n            std::string actual_test_name = test_name;\n            std::stringstream ss;\n\n\n            T cur_arg = (useGivenRange)\n                        ? givenRange\n                        : range.getEntry(i);\n\n            ss << cur_arg;\n            actual_test_name += \" (\" + ss.str() + \")\";\n            readyTest(actual_test_name);\n\n            TestSuite::getResMsg() = \"\";\n            TestSuite::getInfoMsg() = \"\";\n            TestSuite::getCurTest() = this;\n\n            int ret = func(cur_arg);\n            reportTestResult(actual_test_name, ret);\n        }\n    }\n\n    // 3) Generic one-time parameters.\n    template<typename T1, typename... T2, typename F>\n    void doTest( const std::string& test_name,\n                 F func,\n                 T1 arg1,\n                 T2... args )\n    {\n        if (!matchFilter(test_name)) return;\n\n        readyTest(test_name);\n        TestSuite::getResMsg() = \"\";\n        TestSuite::getInfoMsg() = \"\";\n        TestSuite::getCurTest() = this;\n        int ret = func(arg1, args...);\n        reportTestResult(test_name, ret);\n    }\n\n    // 4) Multi composite parameters.\n    template<typename F>\n    void doTest( const std::string& test_name,\n                 F func,\n                 TestArgsWrapper& args_wrapper )\n    {\n        if (!matchFilter(test_name)) return;\n\n        TestArgsBase* args = args_wrapper.getArgs();\n        args->setCallback(test_name, func, this);\n        args->testAll();\n    }\n\n    TestOptions options;\n\nprivate:\n    void doTestCB( const std::string& test_name,\n                   test_func_args func,\n                   TestArgsBase* args )\n    {\n        readyTest(test_name);\n        TestSuite::getResMsg() = \"\";\n        TestSuite::getInfoMsg() = \"\";\n        TestSuite::getCurTest() = this;\n        int ret = func(args);\n        reportTestResult(test_name, ret);\n    }\n\n    static void spawnThread(ThreadInternalArgs* args) {\n        args->rc = args->func(args->userArgs);\n    }\n\n    bool matchFilter(const std::string& test_name) {\n        if (!filter.empty() &&\n            test_name.find(filter) == std::string::npos) {\n            // Doesn't match with the given filter.\n            return false;\n        }\n        return true;\n    }\n\n    void readyTest(const std::string& test_name) {\n        printf(\"[ \" \"....\" \" ] %s\\n\", test_name.c_str());\n        if ( (options.printTestMessage || displayMsg) &&\n             !suppressMsg ) {\n            printf(_CL_D_GRAY(\"   === TEST MESSAGE (BEGIN) ===\\n\"));\n        }\n        fflush(stdout);\n\n        getTestName() = test_name;\n        startTimeLocal = std::chrono::system_clock::now();\n    }\n\n    void reportTestResult(const std::string& test_name,\n                          int result)\n    {\n        std::chrono::time_point<std::chrono::system_clock> cur_time =\n                std::chrono::system_clock::now();;\n        std::chrono::duration<double> elapsed = cur_time - startTimeLocal;\n        std::string time_str = usToString(elapsed.count() * 1000000);\n\n        char msg_buf[1024];\n        std::string res_msg = TestSuite::getResMsg();\n        sprintf(msg_buf, \"%s (\" _CL_BROWN(\"%s\") \")%s%s\",\n                test_name.c_str(),\n                time_str.c_str(),\n                (res_msg.empty() ? \"\" : \": \"),\n                res_msg.c_str() );\n\n        if (result < 0) {\n            printf(\"[ \" _CL_RED(\"FAIL\") \" ] %s\\n\", msg_buf);\n            cntFail++;\n        } else {\n            if ( (options.printTestMessage || displayMsg) &&\n                 !suppressMsg ) {\n                printf(_CL_D_GRAY(\"   === TEST MESSAGE (END) ===\\n\"));\n            } else {\n                // Move a line up.\n                printf(\"\\033[1A\");\n                // Clear current line.\n                printf(\"\\r\");\n                // And then overwrite.\n            }\n            printf(\"[ \" _CL_GREEN(\"PASS\") \" ] %s\\n\", msg_buf);\n            cntPass++;\n        }\n\n        if ( result != 0 &&\n             (options.abortOnFailure || forceAbortOnFailure) ) {\n            abort();\n        }\n        getTestName().clear();\n    }\n\n    size_t cntPass;\n    size_t cntFail;\n    std::string filter;\n    bool useGivenRange;\n    bool preserveTestFiles;\n    bool forceAbortOnFailure;\n    bool suppressMsg;\n    bool displayMsg;\n    int64_t givenRange;\n    // Start time of each test.\n    std::chrono::time_point<std::chrono::system_clock> startTimeLocal;\n    // Start time of the entire test suite.\n    std::chrono::time_point<std::chrono::system_clock> startTimeGlobal;\n};\n\n// ===== Functor =====\n\nstruct TestArgsSetParamFunctor {\n    template<typename T>\n    void operator()(T* t, TestRange<T>& r, size_t param_idx) const {\n        *t = r.getEntry(param_idx);\n    }\n};\n\ntemplate<std::size_t I = 0,\n         typename FuncT,\n         typename... Tp>\ninline typename std::enable_if<I == sizeof...(Tp), void>::type\nTestArgsSetParamScan(int,\n                     std::tuple<Tp*...> &,\n                     std::tuple<TestRange<Tp>...> &,\n                     FuncT,\n                     size_t) { }\n\ntemplate<std::size_t I = 0,\n         typename FuncT,\n         typename... Tp>\ninline typename std::enable_if<I < sizeof...(Tp), void>::type\nTestArgsSetParamScan(int index,\n                     std::tuple<Tp*...>& t,\n                     std::tuple<TestRange<Tp>...>& r,\n                     FuncT f,\n                     size_t param_idx) {\n    if (index == 0) f(std::get<I>(t), std::get<I>(r), param_idx);\n    TestArgsSetParamScan<I + 1, FuncT, Tp...>(index-1, t, r, f, param_idx);\n}\nstruct TestArgsGetNumStepsFunctor {\n    template<typename T>\n    void operator()(T* t, TestRange<T>& r, size_t& steps_ret) const {\n        (void)t;\n        steps_ret = r.getSteps();\n    }\n};\n\ntemplate<std::size_t I = 0,\n         typename FuncT,\n         typename... Tp>\ninline typename std::enable_if<I == sizeof...(Tp), void>::type\nTestArgsGetStepsScan(int,\n                     std::tuple<Tp*...> &,\n                     std::tuple<TestRange<Tp>...> &,\n                     FuncT,\n                     size_t) { }\n\ntemplate<std::size_t I = 0,\n         typename FuncT,\n         typename... Tp>\ninline typename std::enable_if<I < sizeof...(Tp), void>::type\nTestArgsGetStepsScan(int index,\n                     std::tuple<Tp*...>& t,\n                     std::tuple<TestRange<Tp>...>& r,\n                     FuncT f,\n                     size_t& steps_ret) {\n    if (index == 0) f(std::get<I>(t), std::get<I>(r), steps_ret);\n    TestArgsGetStepsScan<I + 1, FuncT, Tp...>(index-1, t, r, f, steps_ret);\n}\n\n#define TEST_ARGS_CONTENTS()                               \\\n    void setParam(size_t param_no, size_t param_idx) {     \\\n        TestArgsSetParamScan(param_no, args, ranges,       \\\n                             TestArgsSetParamFunctor(),    \\\n                             param_idx); }                 \\\n    size_t getNumSteps(size_t param_no) {                  \\\n        size_t ret = 0;                                    \\\n        TestArgsGetStepsScan(param_no, args, ranges,       \\\n                             TestArgsGetNumStepsFunctor(), \\\n                             ret);                         \\\n        return ret; }                                      \\\n    size_t getNumParams() {                                \\\n        return std::tuple_size<decltype(args)>::value;     \\\n    }\n\n\n// ===== TestArgsBase =====\n\nvoid TestArgsBase::testAllInternal(size_t depth) {\n    size_t i;\n    size_t n_params = getNumParams();\n    size_t n_steps = getNumSteps(depth);\n\n    for (i=0; i<n_steps; ++i) {\n        setParam(depth, i);\n        if (depth+1 < n_params) {\n            testAllInternal(depth+1);\n        } else {\n            std::string test_name;\n            std::string args_name = toString();\n            if (!args_name.empty()) {\n                test_name = testName + \" (\" + args_name + \")\";\n            }\n            testInstance->doTestCB(test_name,\n                                   testFunction,\n                                   this);\n        }\n    }\n}\n\n// ===== Parameter macros =====\n\n#define DEFINE_PARAMS_2(name,                                  \\\n                        type1, param1, range1,                 \\\n                        type2, param2, range2)                 \\\n    class name ## _class : public TestArgsBase {               \\\n    public:                                                    \\\n        name ## _class() {                                     \\\n            args = std::make_tuple(&param1, &param2);          \\\n            ranges = std::make_tuple(                          \\\n                         TestRange<type1>range1,               \\\n                         TestRange<type2>range2 );             \\\n        }                                                      \\\n        std::string toString() {                               \\\n            std::stringstream ss;                              \\\n            ss << param1 << \", \" << param2;                    \\\n            return ss.str();                                   \\\n        }                                                      \\\n        TEST_ARGS_CONTENTS()                                   \\\n        type1 param1;                                          \\\n        type2 param2;                                          \\\n    private:                                                   \\\n        std::tuple<type1*, type2*> args;                       \\\n        std::tuple<TestRange<type1>, TestRange<type2>> ranges; \\\n    };\n\n#define DEFINE_PARAMS_3(name,                                  \\\n                        type1, param1, range1,                 \\\n                        type2, param2, range2,                 \\\n                        type3, param3, range3)                 \\\n    class name ## _class : public TestArgsBase {               \\\n    public:                                                    \\\n        name ## _class() {                                     \\\n            args = std::make_tuple(&param1, &param2, &param3); \\\n            ranges = std::make_tuple(                          \\\n                         TestRange<type1>range1,               \\\n                         TestRange<type2>range2,               \\\n                         TestRange<type3>range3 );             \\\n        }                                                      \\\n        std::string toString() {                               \\\n            std::stringstream ss;                              \\\n            ss << param1 << \", \" << param2 << \", \" << param3;  \\\n            return ss.str();                                   \\\n        }                                                      \\\n        TEST_ARGS_CONTENTS()                                   \\\n        type1 param1;                                          \\\n        type2 param2;                                          \\\n        type3 param3;                                          \\\n    private:                                                   \\\n        std::tuple<type1*, type2*, type3*> args;               \\\n        std::tuple<TestRange<type1>,                           \\\n                   TestRange<type2>,                           \\\n                   TestRange<type3>> ranges;                   \\\n    };\n\n#define SET_PARAMS(name) \\\n    TestArgsWrapper name(new name ## _class())\n\n#define GET_PARAMS(name) \\\n    name ## _class* name = static_cast<name ## _class*>(TEST_args_base__)\n\n#define PARAM_BASE TestArgsBase* TEST_args_base__\n\n#define TEST_SUITE_AUTO_PREFIX __func__\n\n#define TEST_SUITE_PREPARE_PATH(path)                               \\\n    const std::string _ts_auto_prefiix_ = TEST_SUITE_AUTO_PREFIX;   \\\n    TestSuite::clearTestFile(_ts_auto_prefiix_);                    \\\n    path = TestSuite::getTestFileName(_ts_auto_prefiix_);\n\n#define TEST_SUITE_CLEANUP_PATH()                       \\\n    TestSuite::clearTestFile( _ts_auto_prefiix_,        \\\n                              TestSuite::END_OF_TEST );\n\n"
  }
]